Measuring Self-Rating Bias in LLM-Generated Survey Data: A Semantic Similarity Framework for Independent Scale Mapping

Eduardo Vera Pichardo · 2026

Read Measuring Self-Rating Bias in LLM-Generated Survey Data: A Semantic Similarity Framework for Independent Scale Mapping by Eduardo Vera Pichardo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13862.txt.

Description

Eduardo Vera Pichardo, writing alone, presents a 2026 arXiv paper on a circular flaw in machine-made survey data. When one model family both writes survey answers and scores them, the scores may flatter the model. The paper calibrates a fix called Semantic Similarity Rating, which scores text by its closeness to preset anchor statements.

In pilots and cross-checks across eight domains, plain behavioral anchors beat jargon by 29 points, with 65 to 67% exact matches and 91% within one point. A pre-registered test of 345 cases found model self-rating compressed error variance fourfold, a bias that held across models. The method trades a little accuracy for independence, the author argues.

Catalog record

CREATOREduardo Vera Pichardo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-13862

Browse the full library · About the Tower of Babel