Interpretable Semantic Gradients in SSD: A PCA Sweep Approach and a Case Study on AI Discourse

Hubert Plisiecki, Maria Leniarska, Jan Piotrowski, Marcin Zajenkowski · 2026

Read Interpretable Semantic Gradients in SSD: A PCA Sweep Approach and a Case Study on AI Discourse by Hubert Plisiecki, Maria Leniarska, Jan Piotrowski, Marcin Zajenkowski (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13038.txt.

Introduction

Hubert Plisiecki and co-authors add a PCA sweep to Supervised Semantic Differential, a method that maps how text meaning shifts with personal traits. SSD cuts dimensions with PCA before regression, yet no rule set how many parts to keep. That free choice gave researchers too much slack. Their sweep picks the count by capacity, clarity, and stability together. On posts about AI, an Admiration gradient split warm, joint framings from distrustful ones. No robust Rivalry gradient appeared.

Catalog record

CREATORHubert Plisiecki, Maria Leniarska, Jan Piotrowski, Marcin Zajenkowski
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25212
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13038

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Interpretable Semantic Gradients in SSD: A PCA Sweep Approach and a Case Study on AI Discourse Hubert Plisiecki, Maria Leniarska, Jan Piotrowski, Marcin Zajenkowski Abstract Supervised Semantic Differential (SSD) is a mixed quantitative–interpretive method that models how text meaning varies with continuous individual-difference variables by estimating a semantic gradient in an embedding space and interpreting its poles through clustering and text retrieval. SSD applies PCA before regression, but currently no systematic method exists for choosing the number of retained components, introducing avoidable researcher degrees of freedom in the analysis pipeline. We propose a PCA sweep procedure that treats dimensionality selection as a joint criterion over representation capacity, gradient interpretability, and stability across nearby values of K K . We illustrate the method on a corpus of short posts about artificial intelligence written by Prolific participants who also completed Admiration and Rivalry narcissism scales. The sweep yields a stable, interpretable Admiration-related gradient contrasting optimistic, collaborative framings of AI with distrustful and derisive discourse, while no robust alignment emerges for Rivalry. We also show that a counterfactual using a high-PCA dimension solution heuristic produces diffuse, weakly structured clusters instead, reinforcing the value of the sweep-based choice of K K . The case study shows how the PCA sweep constrains researcher degrees of freedom while preserving SSD’s interpretive aims, supporting transparent and psychologically meaningful analyses of connotative meaning. 1 Introduction Supervised Semantic Differential (SSD) is a recent mixed-method that analyzes shifts in meaning of concepts and texts with regards to psychological individual differences of their authors ( 14 ) . The method is inspired by longstanding psychological Semantic Differential methods that measure connotative meaning of concepts via Likert-scales with polar concept opposites (e.g. warm/cold, strong/weak; 12 ) and by modern distributional semantics, where word embeddings model relational structure in language use ( 8 ; 7 ; 5 ) . Given texts annotated with an outcome variable, such as a trait score or attitude scale, SSD creates individual representations of specific concepts, also called Personal Concept Vectors (PCVs) by aggregating words that collocate with a lexicon of interest relating to a concept under analysis. SSD then fits a linear model in a reduced embedding space with regards to the outcome variable and uses the regression coefficients as a semantic direction or gradient, whose positive and negative poles are explored through clustering of nearby words and further elucidated by retrieving original text snippets which numerical representations are the most aligned to each cluster centroid. The statistical results of the regression are analyzed to estimate whether a linear shift in individual semantic representation explains a significant portion of the variance in the outcome variable. In the original SSD method, the PCVs are compressed with principal component analysis (PCA) before regression in order to make it possible to apply it to small corpora, and reduce the numer of redudant dimensions ( 6 ) . However, as of now there is no systematic method of choosing the number of components (K) to be extracted. While original results were relatively stable across similar component ranges, leaving the choice of dimensionality to the researcher introduces an avoidable researcher degree of freedom, thereby increasing the risk of overfitting, reducing the transparency of the analysis pipeline, and potentially biasing substantive interpretations of the resulting semantic gradients ( 15 ) . This work proposes a simple computational heuristic that can be used to pick K in a principled way. We introduce a PCA sweep procedure that treats K selection as a joint optimization problem over topic representation quality, interpretability, and the stability of the results. The sweep evaluates a sequence of K values, fits SSD at each K and tracks both cluster coherence (used as an interpretability criterion; 9 ), their allignment with the gradient, as well as a result stability measure based on cosine differences between consecutive gradients. We showcase the applicability of SSD with a PCA sweep by applying it to a dataset of posts written by Prolific study participants who have also answered a questionnaire assessing their Admiration (ADM) and Rivalry (RIV) Narcissistic traits ( 3 ) . The case study serves primarily as an illustration of the extended method; the main contribution is methodological. 2 Background: Supervised Semantic Differential Supervised Semantic Differential (SSD) assumes a collection of documents d_{i} paired with continuous outcomes y_{i} , such as trait or attitude scores. In its canonical formulation ( 14 ) , SSD focuses on a specific concept of interest by means of a lexicon: words that denote or closely relate to the target concept (e.g., abortion , immigration ). These lexicon terms are used to identify and aggregate the local semantic contexts in which the concept appears, yielding a PCV for each author, reflecting how the concept is represented in their texts. Each document is mapped to a dense vector \mathbf{x}_{i}\in\mathbb{R}^{D} using a fixed embedding model and a composition scheme (e.g., SIF-weighted averages with removal of the top principal component to reduce anisotropy; 1 ; 11 ). To reduce redundancy and to make the regression applicable in small datasets, SSD applies PCA to the vectors and projects them to a lower-dimensional representation \tilde{\mathbf{x}}_{i}\in\mathbb{R}^{K} . A linear model is then estimated: y_{i}=\alpha+\boldsymbol{\beta}^{\top}\tilde{\mathbf{x}}_{i}+\epsilon_{i}. The regression coefficients are normalized to unit length to obtain a semantic gradient \hat{\boldsymbol{\beta}} , which is back-projected to the original embedding space. SSD explains the positive and negative poles of this gradient by retrieving nearest neighbors, clustering them separately on each side, and retrieving original text snippets aligned with cluster centroids. These clusters provide qualitative evidence of how the meaning of the focal concept shifts as the outcome increases. The choice of PCA dimensionality K affects both regression and interpretation: too few components may collapse distinct semantic regularities, whereas too many may introduce unstable or noisy directions. A principled way to choose K is therefore essential. 3 PCA Sweep for SSD We introduce a PCA sweep procedure that operationalizes the choice of K as a structured model-selection problem. The key principle is that K is not optimized for predictive accuracy, but for three properties central to SSD as an interpretive method: (i) the quality of the semantic representation after dimensionality reduction, (ii) the interpretability of the derived semantic gradient, and (iii) the stability of that gradient across nearby values of K . Although these diagnostics are computed after the regression is fitted, they are not functions of variance explained by the regression or predictive fit, and therefore do not reward models that align more strongly with the outcome variable. Instead, they privilege solutions whose semantic structure is coherent and stable across K . Given a user-specified range of candidate dimensionalities (e.g., K\in\{20,22,\dots,120\} ), the sweep runs SSD repeatedly with identical preprocessing, weighting, and clustering settings, and records diagnostic quantities for each K . Representation statistics. For every configuration, we compute the cumulative proportion of variance explained by the first K principal components. This provides a monotonic measure of representation capacity: larger K values encode more information from the PCVs but also increase the risk of capturing idiosyncratic or noisy variation. Because this quantity grows mechanically with K , it is used only as a baseline for detrending rather than as a selection criterion. Interpretability diagnostics. After fitting the regression

Read the complete text