Zeng, Yue2023-03-062023-03-062023-03-06http://hdl.handle.net/10393/44678http://dx.doi.org/10.20381/ruor-28884In this thesis, we study the mathematics behind the information theory. The focus is on Shannon's notion of the asymptotic entropy (entropy rate) and its applications to natural languages. We discuss various approaches to estimation of the entropy rate and their ramifications.enasymptotic entropyentropy rateinformation theorynatural languagesThesis