A Rational Account of Categorization Based on Information Theory
Christopher J. MacLellan ([email protected]), Karthik Singaravadivelan, Xin Lian, Zekun Wang, Pat Langley · 2026
Read A Rational Account of Categorization Based on Information Theory by Christopher J. MacLellan ([email protected]), Karthik Singaravadivelan, Xin Lian, Zekun Wang, Pat Langley (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-29895.txt.
Introduction
MacLellan and co-authors offer an information-based theory of how people categorize. They test it against classic lab findings from three well-known studies. Those studies span work by Hayes-Roth, Medin and Schaffer, and Smith and Minda. The new theory matches human behavior as well as rival models. Rivals include the context model and Anderson's rational model. It also holds up against a hierarchical Dirichlet process model. The account frames sorting choices as rational information use.
Catalog record
| CREATOR | Christopher J. MacLellan ([email protected]), Karthik Singaravadivelan, Xin Lian, Zekun Wang, Pat Langley |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29337 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.29895
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Rational Account of Categorization Based on Information Theory Christopher J. MacLellan ([email protected]), Karthik Singaravadivelan, Xin Lian, Zekun Wang, Pat Langley Abstract We present a new theory of categorization based on an information-theoretic rational analysis. To evaluate this theory, we investigate how well it can account for key findings from classic categorization experiments conducted by 10 ; 13 ; 16 . We find that it explains the human categorization behavior at least as well (or better) than the independent cue and context models ( 13 ) , the rational model of categorization ( 1 ) , and a hierarchical Dirichlet process model ( 9 ) . Keywords: Rational analysis; Concepts and categories; Learning; Computational Modeling Introduction The underlying premise of a rational analysis is that humans have evolved cognitive mechanisms that maximize utility given the structure of their environment and their operating constraints ( 1 ; 11 ) . Thus, if we can devise artificial mechanisms that maximize a given human objective under the relevant environment and operating constraints, then it should have a close correspondence to the adapted human cognitive mechanisms—both should converge to isomorphic mechanisms. To apply this approach to human categorization requires making assumptions about (1) the objective people maximize during learning, (2) the key structural aspects of the environment, and (3) the relevant operating constraints. 1 proposes answers to all three. Specifically, he assumes that people learn categories that minimize the mean-squared feature predictive error within a Bayesian inference scheme, and that the environment is structured such that objects can be partitioned into disjoint categories that have independent, probabilistic features within each category. Lastly, he assumes people are constrained to maintaining a single hypothesis about category structure (they do not maintain all possible category structure hypotheses) and that this hypothesis is incrementally updated after each new experience. The current work builds on 1 ’s assumptions, but explores two changes. First, we propose a new objective based on information theory; mainly, that people learn categories that have maximal mutual information with the features of their experiences. This new objective is based on 4 ’s ( 4 ) category utility hypothesis , which states that a category has utility to the extent that it improves one’s ability to predict the features of an instance given knowledge of its category label. Second, we assume that the structure of categories in the environment is taxonomic, being organized hierarchically rather than in a flat partitioning. This assumption was suggested by 6 and later explored by 1 . Instantiating the Proposed Theory The next step in the rational analysis process is to devise artificial mechanisms that maximize this objective within the proposed environment and processing constraints. We instantiate our theory within 7 ’s ( 7 ) Cobweb system. Cobweb employs two distinct representations. First, instances are describe using attribute-value lists (e.g., {“color”: “green”, “shape”: “circle”, “size”: 2.4}). Second, concepts—which are organized in a hierarchical taxonomy—store probability distributions over the attribute values of training instances associated with them. Concepts maintain categorical distributions for discrete features and normal distributions for continuous features. The system has a single free parameter \alpha , which is a smoothing parameter representing uniform pseudocounts for every known attribute value within each categorical feature distribution. To predict missing features of an example, Cobweb sorts it into its hierarchy and uses the concepts it activates to generate predictions. Unlike 7 ’s original performance mechanism, our variant uses best-first search to expand the nodes with the highest pointwise mutual information with the example. The pointwise mutual information ( 3 ; 5 ) between a concept c and an example x is computed as PMI(c,x)=\log\frac{p(x,c)}{p(x)p(c)}=\log\frac{p(x|c)}{p(x)}=\log p(x|c)-\log p(x) . When deciding which node to expand, the p(x) term can be dropped as it is a constant that appears across all PMI scores; thus, this approach reduces to expanding the nodes with the highest \log p(x|c) . This measure, which quantifies the amount of information the concept label gives about the example’s features, directly operationalizes 4 ’s category utility hypothesis. Our system keeps expanding concept nodes up to a user-specified max_nodes value (we set this parameter to 100 for all our following simulations) or until there are no more concepts to expand. Finally, the system predicts the target missing features as a combination of the predictions from all activated nodes weighted by each node’s pointwise mutual information score. Note, we normalized the pointwise mutual information scores, so that the weights across expanded concepts sum to 1. This inference approach is analogous to an unsupervised version of the predictive equation used by 1 that assumes a uniform prior over each activated concept. To update its taxonomy, Cobweb sorts the examples into its hierarchy in a greedy fashion. At each branch it considers four possible update operations: (1) inserting the example into one of the children, (2) creating a new node to represent the example, (3) merging the two concepts most similar to the example and inserting into the new merged concept (the original concepts become the children of the new concept), and splitting the most similar concept and promoting its children. Our variant of Cobweb chooses the operation that maximizes the future expected pointwise mutual information across the children and all possible examples, which is equivalent to maximizing their mutual information: \mathbb{E}_{x,c}[PMI(x,c)]=\sum_{c}\sum_{x}p(x,c)\log\frac{p(x,c)}{p(x)p(c)}=I(X,C) . Similar to 1 , we assume that features are statistically independent within each category, which makes it possible to compute this value in a closed form for both categorical and normal feature distributions. When an example is incorporated into a concept, we incrementally update its feature probability distributions to reflect the new example. Given the space limitations, we refer the reader to a more detailed description of a Cobweb variant similar to ours by 12 ; 2 . The main difference between their approach and ours is use of collocation ( p(x|c)p(c|x) ) instead of pointwise mutual information. Preliminary Tests of the Theory Central Tendencies of Categories As a first test of our theory, we applied our Cobweb variant to simulate the data reported by 10 . Their experiment, which investigated human recognition and classification of old and new examples, showcases the central tendencies of human categorization—that classification accuracy increases the closer an example is to a target category’s prototype. Human Study Design In this study, 108 participants reviewed flashcards showing records of fictional individuals that were described by five features (last name, age, education, marital status, and hobby) and were asked to classify them into one of three classes: Club 1 , Club 2 , or Neither . The last name and hobby were distractor features; every card featured a different last name and one of three randomly assigned hobbies. The remaining features, which determined the class of the example, could take on one of four values. 1 1 1 See Table 1 of ( 10 ) for a full list of examples. During training, half the participants were randomly assigned to receive flashcards describing the classes and their prototypes, while the other half did not receive any such information. Next, participants classified 132 examples and received feedback after each classification. For the Club 1 and Club 2 classes, there were special training items with the following feature structure and frequency: •