Information content and word frequency in natural language: Word length matters

For centuries, scientists have attempted to uncover commonalities that underlie the structure of human languages (1). In a recent issue of PNAS, Piantadosi et al. (2) reported an exciting finding with respect to one unique type of language universal. The authors empirically demonstrated that word length strongly correlated with information content across 11 distinct natural languages. This finding is remarkable in that it directly contravenes Zipf's principle of least effort, which holds that language optimizes communication by spontaneously truncating words as a function of their relative frequency (e.g., television → TV). Simply stated, more frequent words tend to be shorter, and this length–frequency relationship serves an adaptive function of communicative parsimony.

Piantadosi et al. (2) derived evidence for their hypothesis by mining a massive trillion-word corpus of cross-linguistic language samples from the world wide web. Piantadosi et al. (2) pose an alternative hypothesis to Zipf's widely held view, namely that the information content of a word is a stronger predictor of word length than its relative frequency of occurrence within a given language. For language researchers, the significance of this finding cannot be overstated. In a Nature News press release (3), linguist Roger Levy remarked that these results “may now supply the largest leap forward in 75 years in understanding how the evolution of words is governed by the efficiency with which they can be used to communicate” (3).

Information content is one aspect of a word. Two other aspects of word meaning pose obstacles for a clean interpretation of the length–content hypothesis advanced by Piantadosi and colleagues. These include the syntactic dimension of grammatical class (e.g., nouns vs. verbs) and the semantic dimension of word meaning (i.e., word concreteness). With respect to grammatical class, verbs tend to be longer than nouns across many natural languages (4). With respect to word concreteness, word length provides a cue for the sensory salience of a word. That is, longer words tend to denote more abstract ideas. Consider, for example, the concrete noun, friend. It is possible to inflect friend and transform its root into an abstract idea (e.g., friendliness) (5). This morphological transformation is also evident in other languages (e.g., the German root schade can be inflected to form an abstract concept such as schadenfreude). As such, word length can provide at least statistically reliable markers for distinctions such grammatical class and concreteness. Therefore, one difficulty with interpreting these findings regards construct validity of the dependent measure (information content). That is, it is unclear whether verbs (e.g., eat) and abstract nouns (e.g., honesty) convey more information content than their concrete noun counterparts (e.g., dog) and whether this trend holds across unrelated languages. Thus, there remains considerable ambiguity as to the nature of what constitutes the information content of a word and how this might be reliably measured. Despite such ambiguity, Piantadosi et al. (2) have potentially unveiled an intriguing domain for cross-linguistic research. Perhaps even more importantly, the authors have paved the way for a paradigmatic shift in how linguists view the relation between sound and meaning in natural languages.