Measuring Self-Rating Bias in LLM-Generated Survey Data: A Semantic Similarity Framework for Independent Scale Mapping
Eduardo Vera Pichardo · 2026
Read Measuring Self-Rating Bias in LLM-Generated Survey Data: A Semantic Similarity Framework for Independent Scale Mapping by Eduardo Vera Pichardo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-13862.txt.
Description
Eduardo Vera Pichardo, writing alone, presents a 2026 arXiv paper on a circular flaw in machine-made survey data. When one model family both writes survey answers and scores them, the scores may flatter the model. The paper calibrates a fix called Semantic Similarity Rating, which scores text by its closeness to preset anchor statements.
In pilots and cross-checks across eight domains, plain behavioral anchors beat jargon by 29 points, with 65 to 67% exact matches and 91% within one point. A pre-registered test of 345 cases found model self-rating compressed error variance fourfold, a bias that held across models. The method trades a little accuracy for independence, the author argues.
Catalog record
| CREATOR | Eduardo Vera Pichardo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 83 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-13862