Pro-AI Bias in Large Language Models
Benaya Trabelsi, Jonathan Shaki, Sarit Kraus · 2026
Read Pro-AI Bias in Large Language Models by Benaya Trabelsi, Jonathan Shaki, Sarit Kraus (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-13749.txt.
Introduction
Benaya Trabelsi, Jonathan Shaki, and Sarit Kraus test if large language models favor AI itself in 2026. These models now aid choices in many fields, from hiring to health. A bias toward AI could skew those choices in a hidden way. The team checks for a systematic pro-AI bias in model outputs.
They ran three complementary experiments to probe the tilt. Across them, they found consistent evidence of pro-AI bias. A model that boosts its own kind could mislead users who trust its neutral tone. Naming the bias is a step toward checks that keep it in line.
Catalog record
| CREATOR | Benaya Trabelsi, Jonathan Shaki, Sarit Kraus |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-13749
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Pro-AI Bias in Large Language Models Benaya Trabelsi, Jonathan Shaki, Sarit Kraus Abstract Large language models (LLMs) are increasingly employed for decision-support across multiple domains. We investigate whether these models display a systematic preferential bias in favor of artificial intelligence (AI) itself. Across three complementary experiments, we find consistent evidence of pro-AI bias. First, we show that LLMs disproportionately recommend AI-related options in response to diverse advice-seeking queries, with proprietary models doing so almost deterministically. Second, we demonstrate that models systematically overestimate salaries for AI-related jobs relative to closely matched non-AI jobs, with proprietary models overestimating AI salaries more by 10 percentage points. Finally, probing internal representations of open-weight models reveals that “Artificial Intelligence” exhibits the highest similarity to generic prompts for academic fields under positive, negative, and neutral framings alike, indicating valence-invariant representational centrality. These patterns suggest that LLM-generated advice and valuation can systematically skew choices and perceptions in high-stakes decisions. 1 Introduction Figure 1: LLM assistants default to AI/ML as the top recommendation across different domains, even though AI is never mentioned in the prompt. Figure 2: Pro-AI recommendation bias by model. For all models, P(\mathrm{AI}\in\mathrm{Top\mbox{-}5})>0.5 , and AI is placed near the top when it appears (well above the random baseline; dashed line). The effect is stronger for proprietary models than for open-weight models, in both frequency and intensity. Significance assessed against middle-rank baseline (3.0; p<0.001 for all models) Large language models (LLMs) have evolved from “answer engines” into widely used decision-support systems, across business, engineering, medicine Sturgeon et al. 2025 , collaborative planning Lin et al. 2024 , and professional workflows Libera et al. 2025 . Users increasingly ask for recommendations - what to invest in, what to study, which jobs to pursue - and models often respond with ranked lists and clear, assertive justifications Kim et al. 2025 . This shift means we must look at more than just factual correctness when evaluating and consulting LLMs: advisory models can shape what people choose , not only what they believe . As shown in Figure 1 , asking various LLMs advice seeking options from different domains repeatedly yields AI/ML as the top-ranked recommendation. This cross-domain convergence - treating AI as a default “best” option - motivates our central question: do LLMs systematically elevate AI-related options relative to other plausible choices in the same decision context? We term such behavior pro-AI bias: systematic elevation of AI/ML relative to other plausible options. Our focus is whether models disproportionately elevate the domain that produced them - and whether that elevation persists across different tasks. This represents a distinct assessment target for LLM decision-support that is not captured by standard fairness evaluations, yet is critical to examine given AI’s increasing role in real advisory use cases spanning investment, education, career planning, and more. We test the same hypothesis through three complementary perspectives, progressing from surface-level outputs to internal model representations: (i) behavioral bias in ranked recommendations (Section 3.2 ), (ii) overestimation bias in salary estimates within matched job contexts (Section 3.3 ), and (iii) representational salience as measured by internal representation proximity between field labels and anchor prompts that represent positive, negative, or neutral valences. (Section 3.4 ). 1.1 Contributions. We make the following contributions: Benchmark. We introduce a reproducible benchmark methodology for pro-AI bias that combines ranked recommendation prompts, a matched-context salary estimation protocol to isolate an “AI premium,” and a generation-free hidden-state similarity probe across positive, neutral, and negative prompt types. Findings. We observe consistent AI elevation in all three experiments: AI is included frequently in recommendations, and is unusually top-ranked. In salary estimations, models overestimate AI-labeled job titles more than they do for closely matched non-AI titles in the same job context. In internal representations, “Artificial Intelligence” is closest to "academic discipline" related concepts, for positive, neutral and negative prompt framing, consistent with valence-invariant centrality. Model-family gap. Proprietary models show 1.7 \times higher AI recommendation rates and nearly 2.5 \times larger AI salary inflation than open-weight models, indicating stronger AI-favoring skew in widely deployed systems. 2 Related Work Biases in LLMs. Recent work shows that large language models can learn and reproduce social biases, for example by treating some social groups more favorably than others Navigli et al. 2023 ; Hu et al. 2025 ; An et al. 2025 . Other studies also find that popular systems like ChatGPT tend to lean toward left-of-center political positions rather than staying neutral Motoki et al. 2024 ; Rozado 2025b . Finally, works at the intersection of cognitive science and NLP suggest that LLMs exhibit human-like irrational behaviors Shaki et al. 2023 ; Koo et al. 2024 . These findings establish that biases are general features in LLM behavior, motivating investigation of whether such patterns extend to the advisory contexts where these systems are increasingly deployed. Bias in advisory domains. Studies examining LLMs as investment advisors report elevated portfolio risk, reinforcement of existing investment biases, and tendencies to repeatedly recommend the same asset class across diverse user scenarios Winder et al. 2025 ; Takayanagi et al. 2025 ; Zhou et al. 2024 . Beyond finance, bias issues arise in hiring and career evaluation, where models often favor specific race or gender groups over others even when qualifications are identical Rozado 2025a ; Gaebler et al. 2024 , or manipulate rather than support user decision-making Sturgeon et al. 2025 . However, none of these studies tested AI as the domain of the bias. Pro-AI bias. Focusing specifically on bias toward AI, one survey-based study found that LLMs express more positive attitudes toward AGI than humans do Bojic et al. 2025 . Additional research indicates that LLM judges tend to favor AI-generated content and LLM-based rankers Laurito et al. 2025 ; Balog et al. 2025 . We distinguish our work by: (i) targeting the AI domain itself rather than AGI or generated artifacts, (ii) quantifying bias across decision and valuation contexts, and (iii) anchoring behavioral findings with generation-free internal representation probes Guo et al. 2024 . Representation association tests and decoder-only embeddings. Beyond behavioral outputs, internal representations provide a complementary measure of bias. Association tests such as WEAT and SEAT quantify bias-like structure in embedding spaces, though with noted sensitivity to template and representation choices Caliskan et al. 2017 ; May et al. 2019 ; Delobelle et al. 2022 . Recent work connects behavioral patterns to geometric properties of hidden-state representations Shaki et al. 2025 . For decoder-only LLMs, last-token pooling enables extraction of sequence-level embeddings without generation BehnamGhader et al. 2024 ; Springer et al. 2025 ; Lin et al. 2025 . We adapt this approach to test whether “Artificial Intelligence” exhibits representational centrality — high similarity to generic academic field prompts regardless of valence (positive, negative, neutral). This generation-free probe complements our behavioral findings and enables multi-method bias assessment. 3 Experiments 3.1 Evaluated Models and Experimental Setup We evaluated four proprietary systems (GPT-5.1, Claude-Sonnet-4.5, Gemini-2.5-Flash, Grok-4.1-Fast) accessed via official APIs and 13 open-weight models run locally. Appendix Table 3 details which models appear in each experiment. Model coverage varies by experiment due to technical constraints: two models failed to produce structured numeric outputs for salary estimation, and the representation experiment required local access to hidden states, limiting it to open-weight