Modeling Statistical Properties of Written Text
M. Ángeles Serrano, Alessandro Flammini, Filippo Menczer · 2009
Read Modeling Statistical Properties of Written Text by M. Ángeles Serrano, Alessandro Flammini, Filippo Menczer (2009) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/plos-pone-0005372.txt.
Description
Written language follows statistical laws, of which Zipf's law about word frequency is the most famous. Serrano and colleagues modeled the broader statistical properties of written text in PLOS ONE in 2009, beyond that single law.
Universal regularities in text can reveal how brains process information and how societies organize shared knowledge. Their models aim to capture several such patterns at once. The work treats language as a complex system whose numbers can be explained, not just described. If text has deep laws, then search tools, code that sums up long works. And tests that spot fake text can all be built on those laws and work better than rules of thumb.
Catalog record
| CREATOR | M. Ángeles Serrano, Alessandro Flammini, Filippo Menczer |
|---|---|
| YEAR | 2009 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| LICENSE | Creative Commons Attribution License |
| RIGHTS | CC BY |
| SOURCE | PLOS ONE |
Source: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0005372