Semantic Structure of Feature Space in Large Language Models

Austin C. Kozlowski, Andrei Boutyline · 2026

Read Semantic Structure of Feature Space in Large Language Models by Austin C. Kozlowski, Andrei Boutyline (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27169.txt.

Introduction

Kozlowski and Boutyline show that geometric links between meaning features in large language models mirror human psychological associations. They build feature vectors for 360 words and project them onto 32 meaning axes, such as beautiful-ugly and soft-hard. The projections track human ratings closely. Links between the axes themselves predict links between survey scales, and much of the variation sits in a low-dimensional space.

Catalog record

CREATORAustin C. Kozlowski, Andrei Boutyline
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30114
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.27169

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Semantic Structure of Feature Space in Large Language Models Austin C. Kozlowski, Andrei Boutyline Abstract We show that the geometric relations between semantic features in large language models’ hidden states closely mirror human psychological associations. We construct feature vectors corresponding to 360 words and project them on 32 semantic axes (e.g. beautiful-ugly , soft-hard ), and find that these projections correlate highly with human ratings of those words on the respective semantic scales. Second, we find that the cosine similarities between the semantic axes themselves are highly predictive of the correlations between these scales in the survey. Third, we show that substantial variance across the 32 semantic axes lies on a low-dimensional subspace, reproducing patterns typical of human semantic associations. Finally, we demonstrate that steering a word on one semantic axis causes spillover effects on the model’s rating of that word on other semantic scales proportionate to the cosine similarity between those semantic axes. These findings suggest that features should be understood not only in isolation but through their geometric relations and the meaningful subspaces they form. 1 Introduction A growing literature shows that many features, such as deception, refusal, and sycophancy, are encoded as linear directions in large language models’ (LLMs) internal representations ( 1 ; 6 ; 26 ) . This line of work has produced useful tools for auditing and controlling LLMs by isolating features of interest and then monitoring, amplifying, or suppressing them. Linear features thus at times serve as a vast collection of knobs that can be individually tweaked to modify LLM outputs ( 9 ; 19 ; 30 ) . However, considering features in isolation ignores a key characteristic of distributed representations – that many features are not rendered orthogonally, and that proximities and directions in the representation space encode meaningful relations between features. In the case of modeling language, this was first evident with early studies of word embedding models like word2vec ( 21 ; 22 ) . Cosine similarities between word vectors captured semantic and syntactic similarities, and directions in the space corresponded to general features like gender or nationality ( 3 ; 7 ; 12 ; 28 ) . Subsequent work combined these insights to show that, these general linear features, like masculine-feminine, good-bad, or rich-poor, are themselves meaningfully non-orthogonal, their cosine similarities reflecting cultural associations such as gender bias or markers of social class ( 4 ; 18 ) . Moreover, recent evidence shows that the first-layer token embeddings of LLMs preserve meaningful geometric feature structures much like classic word embeddings ( 17 ) . However, there remains a paucity of research investigating whether these semantic structures persist in LLMs’ middle layers, where features are most commonly monitored and steered ( 29 ) . Classic studies from psychology provide more reason to expect that important semantic features may be entangled in LLMs’ internal representations. Studies in the “semantic differential” tradition had respondents rate large sets of words on a variety of semantic scales, such as masculine-feminine, soft-hard, and fast-slow ( 16 ; 24 ; 23 ) , and consistently found these data reduced effectively to a 3-dimensional solution, with the most explanatory components corresponding to Evaluation (good-bad), Potency (strong-weak), and Activity (active-passive) ( 8 ; 14 ) . This core finding suggests that the apparent diversity of features that humans use to make sense of the world can be largely captured by a relatively low-dimensional subspace. Motivated by these lines of inquiry, we examine the geometric relations between semantic features in LLMs’ internal representations, then compare these relations to human psychological associations measured with a survey. We find that the geometry of the LLM feature space closely reproduces human associations in several respects. First, the projection of features along constructed “semantic axes” correlates highly with average human ratings of the corresponding words on respective semantic scales. Second, we show that the cosine similarities between the semantic axes themselves are substantial, and that these cosine similarities are highly predictive of the correlation between the corresponding scales in the semantic survey. Third, we apply principal components analysis (PCA) to these semantic axes, finding that a substantial share of their variance is explained by the first three components and that the subspace defined by the first three components is closely aligned with that derived from the survey data. Finally, we find that the proximities between semantic axes have practical implications for feature steering ( 9 ; 25 ) . Steering along one semantic axis induces spillover effects on how the model rates words on other semantic axes proportional to the cosine similarity between those axes. This implies that the cosine similarities between distinct semantic features are substantial enough to influence model behavior. These findings together suggest that the geometry of feature space is humanly meaningful, and that features need not be only considered in isolation; considering angles between features or the subspaces that preserve multiple features could offer valuable insight into a feature’s meaning and operation within an LLM. 2 Data & Methods Our analyses draw upon two forms of data: human responses to a survey of semantic associations, and values derived from LLM activations and outputs. 2.1 Survey of Semantic Associations To assess human associations between semantic features, we use data from a recent survey ( 5 ) . The survey was fielded on the online platform Prolific to a quota sample of 1,750 respondents approximately representative of the U.S. population along major demographic categories. Respondents each rated a subset of all word/scale pairs, and each word was ultimately rated on each scale by an average of 24 respondents. The survey is a modern replication of a classic semantic differential survey and reused the same set of 360 words and 20 scales. These scales were originally selected to “exhaust the semantic space most thoroughly” (p. 689) and the words were selected based on their high variance across several other psychological inventories ( 16 ) . The modern replication added 16 new scales to capture more social attributes (e.g. “free-unfree”, “equal-unequal”). We exclude four scales that were insufficiently distinct from other scales analyzed (e.g. human-machine with natural-artificial; trustworthy-untrustworthy with true-untrue). Our analytic sample therefore comprises 360 words rated on 32 semantic scales. 2.2 LLM Features We run our experiments on small and large models from the Llama family (Llama 3.2 3B and Llama 3.1 70B) ( 13 ) . The same analyses applied to Qwen3 1.7B and Qwen3 32B produce similar results and are included in the Appendix ( 34 ) . To facilitate direct comparison to human responses, we create a dataset of semantic associations derived from LLM activation patterns that matches the structure of the survey data. We constructed feature vectors for each of the 360 words rated in the survey and for 32 “semantic axes” corresponding to the survey’s semantic scales. To construct feature vectors for the 360 words, we followed a simple protocol based on the method described and proven effective in 20 . For each word, we passed the following four prompts through the LLMs and collected their activations: USER: Tell me about { word i }. USER: Tell me your associations with the word { word i }. USER: What do you think of when you hear

Read the complete text