Disentangling Linguistic Features with Dimension-Wise Analysis of Vector Embeddings

Saniya Karwa, Navpreet Singh · 2025

Read Disentangling Linguistic Features with Dimension-Wise Analysis of Vector Embeddings by Saniya Karwa, Navpreet Singh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-14766.txt.

Description

Saniya Karwa and Navpreet Singh probe a 2025 question: where inside a language model's word vectors do facts about language live? Models such as BERT turn words into long lists of numbers. Those lists work well but are hard to read, so it is unclear which numbers carry which traits, such as tense or negation.

The pair built a data set of sentence pairs that differ in just one of ten language traits, from sameness of meaning to quantity. They then used three analysis methods to find which vector slots matter most for each trait. They add a new score that rates each slot's impact on a trait. Some traits, such as negation and polarity, were found to sit firmly in set slots. The map could help make models leaner and clearer.

Catalog record

CREATORSaniya Karwa, Navpreet Singh
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-14766

Browse the full library · About the Tower of Babel