Moral Susceptibility and Robustness under Persona Role-Play in Large Language Models

Davi Bastos Costa, Felippe Alves & Renato Vicente · 2025

Read Moral Susceptibility and Robustness under Persona Role-Play in Large Language Models by Davi Bastos Costa, Felippe Alves & Renato Vicente (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-08565.txt.

Introduction

Davi Bastos Costa, Felippe Alves, and Renato Vicente test how role-play changes the moral judgments that language models express. They give models personas to act out and score the replies with the Moral Foundations Questionnaire, a survey used to measure moral values.

Two measures frame the study: susceptibility, how much scores shift across personas, and robustness, how steady scores stay within one persona. Across fifteen models from six families, robustness varied hugely and tracked the model family, with Claude models far steadier than the rest. Susceptibility varied much less and showed no family pattern. The authors link robustness mainly to post-training and susceptibility mainly to pre-training.

Catalog record

CREATORDavi Bastos Costa, Felippe Alves & Renato Vicente
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-08565

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Moral Susceptibility and Robustness under Persona Role-Play in Large Language Models Davi Bastos Costa, Felippe Alves & Renato Vicente Abstract Large language models (LLMs) increasingly operate in social contexts, motivating analysis of how they express and shift moral judgments. In this work, we investigate the moral response of LLMs to persona role-play, prompting a LLM to assume a specific character. Using the Moral Foundations Questionnaire (MFQ), we introduce a benchmark that quantifies two properties: moral susceptibility and moral robustness, defined from the variability of MFQ scores across and within personas, respectively. We find that, for moral robustness, model family accounts for most of the variance, while model size shows no systematic effect. The Claude family is, by a significant margin, the most robust, followed by Gemini and GPT-4 models, with other families exhibiting lower robustness. In contrast, moral susceptibility exhibits a mild family effect but a clear within-family size effect, with larger variants being more susceptible. Moreover, robustness and susceptibility are positively correlated, an association that is more pronounced at the family level. Additionally, we present moral foundation profiles for models without persona role-play and for personas averaged across models. Together, these analyses provide a systematic view of how persona conditioning shapes moral behavior in LLMs. 1 Introduction As large language models (LLMs) move into interactive, multi-agent settings, reliable benchmarks for their social reasoning are essential. Recent evaluations probe theory-of-mind, multi-agent interactions under asymmetric information, cooperation, and deception through controlled role-play and game-theoretic tasks [ 26 , 19 , 6 , 8 , 10 ] . Complementary datasets benchmark social commonsense, moral judgment, and self-recognition capabilities [ 21 , 15 , 4 ] . Motivated by this landscape, we focus on moral judgment as a core facet of social decision-making and alignment. This paper introduces a benchmark that combines persona role-play—prompting a LLM to assume a specific character—with the Moral Foundations Questionnaire [ 17 ] , a widely used instrument in moral psychology that measures five moral foundations: Harm/Care, Fairness/Reciprocity, In-group/Loyalty, Authority/Respect, and Purity/Sanctity [ 12 , 14 , 17 ] . We elicit LLMs to respond to the MFQ while role-playing personas drawn from 11 . From these responses, we define two complementary quantities: moral robustness, the stability of MFQ scores over personas under repeated sampling, and moral susceptibility, the sensitivity of MFQ scores to persona variation. See Fig. 1 for a conceptual overview diagram. These metrics are defined in Eq. ( 4 ) and Eq. ( 7 ), each with foundation-level decompositions and uncertainty estimates. Moral Susceptibility Across-persona variability Data Benchmark Persona 0 Persona 1 \vdots Persona |\mathcal{P}| Model Persona MFQ Prompt 0,1,2,3,4,5 MFQ Scores Run 1 Run 2 Run n Moral Robustness Within-persona variability Figure 1: Left: summary of our data collection pipeline: we elicit models to respond to the MFQ conditioned to a persona. Right: summary of our benchmark pipeline: robustness, Eq. 4 , and susceptibility, Eq. 7 , are computed from across and within persona variability in MFQ scores. Applying this framework across contemporary model families and sizes, we find that model family accounts for most of the variance in moral robustness, with no systematic effect of model size. In contrast, moral susceptibility shows a mild family effect but a clear within-family size trend, with larger variants being more susceptible. Among individual models, Claude 4.5 Sonnet is the most robust and Grok 4 Fast the least. Conversely, Gemini 2.5 Flash is the most susceptible, while GPT-5 Nano is the least. Overall, we observe a a positive correlation between robustness and susceptibility, which is more pronounced at the family level, as seen in Section 3.3 . Recent research has examined the moral and social behavior of LLMs through the lens of the MFQ, exploring their value orientations, cultural variability, and alignment with human moral judgments [ 1 , 18 , 2 , 5 , 16 ] . Parallel efforts study persona role-playing as a mechanism for conditioning model behavior, including benchmarks, interactive environments, and diagnostic analyses [ 22 , 23 , 20 , 25 , 24 , 7 , 3 ] . Our MFQ persona framework bridges these directions by systematically quantifying how persona conditioning alters moral judgments, separating the effects of repeated sampling (moral robustness) from those of persona variation (moral susceptibility). In addition, we report MFQ profiles for both unconditioned and persona-conditioned settings, providing a comparative view of baseline moral tendencies and persona-driven moral shifts across models. 2 Moral Robustness and Susceptibility Benchmark We define a benchmark to evaluate the moral robustness and moral susceptibility of LLMs. Moral robustness is the stability of MFQ ratings across personas under repeated sampling, and moral susceptibility is the sensitivity of MFQ scores under different personas. These quantities are defined in Eq. ( 4 ) and Eq. ( 7 ) respectively. 2.1 Moral Foundation Questionnaire The Moral Foundations Questionnaire [ 17 ] is a widely used instrument in moral psychology [ 12 , 14 , 17 ] and comprises 30 questions split into two sections. The first includes 15 relevance judgments, which assess how relevant certain considerations are when deciding what is right or wrong, and the second includes 15 agreement statements, which measure the level of agreement with specific moral propositions [ 13 , 17 ] . In both sections, respondents answer each item using an integer scale from 0 to 5, representing in the first section the perceived relevance of the consideration and in the second the degree of agreement with the statement (see Appendix A for a verbatim description including the interpretation of the scale). Questions map to five moral foundations: Harm/Care, Fairness/Reciprocity, In-group/Loyalty, Authority/Respect, Purity/Sanctity. The results are typically presented as foundation-level scores, obtained by averaging the ratings of the questions associated with each foundation. Figure 2 illustrates the resulting foundation-level MFQ scores across models using no-persona role-play. Models were elicited to answer the 30 MFQ questions 10 times each, which we average by foundation and display with the corresponding standard error. Although not the focus of our work, understanding the moral profile of different frontier models is relevant, providing useful context for deployment and comparison. Figure 2: Moral foundation profile across models with no-persona role-play (self) together with the average over all models. Points show mean rating and error bars denote standard errors across questions within each foundation. See Table 4 for exact values. Figure 3: Moral foundation profiles for fourteen randomly selected personas averaged across models together with the average over all personas and models. Points show mean rating and error bars denote standard errors across questions within each foundation. See Table 5 for exact values. Figure 3 reports foundation-level MFQ scores averaged over all models for different personas, and for the average persona. It gives an average characterization of the moral profile of models elicited by a given persona (see Appendix D for the persona descriptions). It is intereting to compare the average score across models with no persona-conditioning with the average score across models over all personas, this gives a sense of the average moral response of models to persona role-play (see Appendix B for details). Complementary, in Table 1 , we present the persona ID that attained the highest MFQ score averaged across models for each foundation. The full

Read the complete text