Information Theory Meets Power Laws (Innbundet)

Stochastic Processes and Language Models


Innbinding: Innbundet
Utgivelsesår: 2021
Antall sider: 384
Forlag: John Wiley and Sons Ltd
Språk: Engelsk
ISBN/EAN: 9781119625278
Kategori: Teknologi, transport og landbruk
Omtale Information Theory Meets Power Laws
Discover new theoretical connections between stochastic phenomena and the structure of natural language with this powerful volume!

Information Theory Meets Power Laws: Stochastic Processes and Language Models presents readers with a novel subtype of a probabilistic approach to language, which is based on statistical laws of texts and their analysis by means of information theory. The distinguished author insightfully and rigorously examines the linguistic and mathematical subject matter while eschewing needlessly abstract and superfluous constructions.

The book begins with a less formal treatment of its subjects in the first chapter, introducing its concepts to readers without mathematical training and allowing those unfamiliar with linguistics to learn the book's motivations. Despite its inherent complexity, Information Theory Meets Power Laws: Stochastic Processes and Language Models is a surprisingly approachable treatment of idealized mathematical models of human language.

The author succeeds in developing some of the theory underlying fundamental stochastic and semantic phenomena, like strong nonergodicity, in a way that has not previously been seriously attempted. In doing so, he covers topics including:

Zipf's and Herdan's laws for natural language

Power laws for information, repetitions, and correlations

Markov, finite-state,and Santa Fe processes

Bayesian and frequentist interpretations of probability

Ergodic decomposition, Kolmogorov complexity, and universal coding

Theorems about facts and words

Information measures for fields

Renyi entropies, recurrence times, and subword complexity

Asymptotically mean stationary processes

Written primarily for mathematics graduate students and professionals interested in information theory or discrete stochastic processes, Information Theory Meets Power Laws: Stochastic Processes and Language Models also belongs on the bookshelves of doctoral students and researchers in artificial intelligence, computational and quantitative linguistics as well as physics of complex systems.

Til toppen