Discover new theoretical connections between stochastic phenomena and the structure of natural language with this powerful volume!
Information Theory Meets Power Laws: Stochastic Processes and Language Models presents readers with a novel subtype of a probabilistic approach to language, which is based on statistical laws of texts and their analysis by means of information theory. The distinguished author insightfully and rigorously examines the linguistic and mathematical subject matter while eschewing needlessly abstract and superfluous constructions.
The book begins with a less formal treatment of its subjects in the first chapter, introducing its concepts to readers without mathematical training and allowing those unfamiliar with linguistics to learn the book's motivations. Despite its inherent complexity, Information Theory Meets Power Laws: Stochastic Processes and Language Models is a surprisingly approachable treatment of idealized mathematical models of human language.
The author succeeds in developing some of the theory underlying fundamental stochastic and semantic phenomena, like strong nonergodicity, in a way that has not previously been seriously attempted. In doing so, he covers topics including:
Zipf's and Herdan's laws for natural language
Power laws for information, repetitions, and correlations
Markov, finite-state,and Santa Fe processes
Bayesian and frequentist interpretations of probability
Ergodic decomposition, Kolmogorov complexity, and universal coding
Theorems about facts and words
Information measures for fields
Renyi entropies, recurrence times, and subword complexity
Asymptotically mean stationary processes
Written primarily for mathematics graduate students and professionals interested in information theory or discrete stochastic processes, Information Theory Meets Power Laws: Stochastic Processes and Language Models also belongs on the bookshelves of doctoral students and researchers in artificial intelligence, computational and quantitative linguistics as well as physics of complex systems.