Tversky Neural Networks: Psychologically Plausible Deep Learning with Differentiable Tversky Similarity

Moussa Koulako Bala Doumbouya Dan Jurafsky Christopher D. Manning · 2025

Read Tversky Neural Networks: Psychologically Plausible Deep Learning with Differentiable Tversky Similarity by Moussa Koulako Bala Doumbouya Dan Jurafsky Christopher D. Manning (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-11035.txt.

Introduction

Moussa Koulako Bala Doumbouya, Dan Jurafsky, and Christopher Manning present a 2025 computing paper that rebuilds neural networks on a psychology theory of similarity. Standard networks judge likeness by distance, which is symmetric. Human judgments are not: people rate some pairs as more alike in one direction than the other.

Amos Tversky's 1977 theory treats objects as sets of features and weighs shared and distinct features separately. The authors make that idea differentiable so it can train by gradient descent. Their Tversky projection layer can express bent functions, like XOR, that a plain linear layer cannot. Swapping the layer into image and language models improved both, including a large accuracy jump on a bird photo task. The work gives deep learning a more human notion of likeness.

Catalog record

CREATORMoussa Koulako Bala Doumbouya Dan Jurafsky Christopher D. Manning
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-11035

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tversky Neural Networks: Psychologically Plausible Deep Learning with Differentiable Tversky Similarity Moussa Koulako Bala Doumbouya Dan Jurafsky Christopher D. Manning Abstract Work in psychology has highlighted that the geometric model of similarity standard in deep learning is not psychologically plausible because its metric properties such as symmetry do not align with human perception of similarity. In contrast, Tversky 1977 proposed an axiomatic theory of similarity with psychological plausibility based on a representation of objects as sets of features, and their similarity as a function of their common and distinctive features. However, this model of similarity has not been used in deep learning before, in part because of the challenge of incorporating discrete set operations. In this paper, we develop a differentiable parameterization of Tversky’s similarity that is learnable through gradient descent, and derive basic neural network building blocks such as the Tversky projection layer , which unlike the linear projection layer can model non-linear functions such as xor . Through experiments with image recognition and language modeling neural networks, we show that the Tversky projection layer is a beneficial replacement for the linear projection layer (also known as fully connected layer ), which employs geometric similarity. For instance, on the NABirds image classification task, a frozen ResNet-50 adapted with a Tversky projection layer achieves a 24.7% relative accuracy improvement over the linear layer adapter baseline. With Tversky projection layers, GPT-2’s perplexity on PTB decreases by 7.5%, and its parameter count by 34.8%. Finally, we propose a unified interpretation of both types of projection layers as computing similarities of input stimuli to learned prototypes for which we also propose a novel visualization technique highlighting the interpretability of Tversky projection layers. Our work offers a new paradigm for thinking about the similarity model implicit in modern deep learning, and designing neural networks that are interpretable under an established theory of psychological similarity. 1 Introduction The geometric model of similarity is ubiquitously used in modern neural networks. Their architectures include operations that assess the similarity between objects using metric similarity functions such as the vector dot product or cosine similarity. For instance, linear projection layers, also known as dense , feed forward , or fully connected , output the dot-product similarity of the input vector with the columns of its weight matrix, which are concept prototypes. Classification modules typically normalize these similarities with softmax to form valid probability distributions ( Bridle 1990 ; LeCun et al. 1998 ) . Convolution layers of image recognition neural networks compute the dot product of image regions and convolution kernels, which represent prototypical patterns. LSTM gates compute dot product similarities of combinations of inputs, hidden states, and cell states with prototypical input , forget , and output gating patterns ( Hochreiter and Schmidhuber 1997 ; Graves et al. 2005 ; Graves 2013 ) . Semantic word embedding models such as word2vec ( Mikolov 2013 ) and GloVe ( Pennington et al. 2014 ) compute the dot-product and cosine similarity of word representations with prototypical embeddings to predict words. Neural language models apply the same classification mechanism to output discrete words or subwords at each time step ( Bengio et al. 2000 ; Sutskever 2014 ; Devlin 2018 ; Radford et al. 2018 ; Achiam et al. 2023 ) . Attention weights ( Bahdanau et al. 2014 ; Vaswani 2017 ) are the normalized dot-product similarity of queries and keys , which are themselves obtained using linear projection layers. Geometric similarity representations have also been widely used in machine learning objective function design for over 200 years, from least squares methods ( Legendre 1806 ) to modern approaches including L2 loss ( Hadsell et al. 2006 ) , Large Margin Cosine Loss ( Wang et al. 2018 ) , and Noise Contrastive Estimation objective ( van den Oord et al. 2018 ; Chen et al. 2020 ) . These objectives span both supervised and self-supervised learning paradigms. However, Tversky 1977 famously challenged this geometric representation of similarity, showing its psychological implausibility due to its fundamental inability to model phenomena such as humans’ asymmetric judgment of similarity. For instance, “We say that ‘the son resembles the father’ rather than ‘the father resembles the son’ .” He addressed this problem with his axiomatic theory based on a feature matching process in which objects are represented as sets of features and their similarity is measured as a linear combination of measures of their common and distinctive features. Tversky’s similarity has been shown to model human judgment more accurately than metric similarity measures in various empirical studies ( Gati and Tversky 1984 ; Ritov et al. 1990 ; Siegel et al. 1982 ; Rorissa 2007 ) . Moreover, over the past decade, several works have highlighted the limitations of the geometric model of similarity and related parallelogram model of semantic relations ( Garg et al. 2015 ; Chen et al. 2017 ; Peterson et al. 2020 ; Zhou et al. 2022 ) , and advocated for more psychologically principled similarity measures such as Tversky’s model, which has strong empirical support. Nevertheless, a differentiable expression of Tversky’s similarity function has not been proposed to date, which prevents its incorporation into neural networks trained using gradient-based methods ( Rumelhart et al. 1986 ; LeCun et al. 1998 ) . The formulation of a differentiable Tversky similarity function is non-trivial because it employs measures of set intersections and differences, which are not differentiable with respect to object features comprising those sets. To address this gap, we propose a novel representation of features as vectors of the same dimensionality as object vectors, and a dual representation of objects both as vectors and as sets, such that an object is the set of features with which it has a positive dot product. This representation of objects and features enables us to define differentiable measures of set intersections and differences, which we use to construct a differentiable Tversky similarity function suitable for deep learning. We also propose the Tversky Projection neural layer, which is analogous to the linear projection layer but is based on Tversky’s model of similarity. We introduce our proposed differentiable similarity function, and basic neural network modules, along with a novel method for interpreting and visualizing deep projection layers in Section 2 . Section 3 demonstrates the effectiveness of Tversky Projection layers in state-of-the-art architectures, including ResNet-50 and GPT-2, on image recognition and language modeling tasks. Our experiments show that the use of Tversky projection layers can lead to significant improvements in both accuracy and parameter efficiency. In particular, we show that the use of a Tversky Projection layer instead of the final linear projection layer in ResNet-50 can result in up to 24% accuracy improvement in image recognition tasks. We also show that replacing linear projection layers in the language modeling head, and attention blocks in GPT-2 can result in a 7.5% reduction in perplexity and 34.8% reduction in parameter count. Section 3.4 provides various qualitative analyses of Tversky projection layers, including results highlighting their principled explanability under Tversky’s theory of similarity. Overall the main contribution of this work is to provide a foundation for efficient and interpretable neural networks based on an established theory of psychological similarity. 2 Methods 2.1 Tversky’s Similarity Tversky’s

Read the complete text