Teaching AI to handle exceptions: Supervised fine-tuning with human-aligned judgment Authors: Matthew DosSantos DiSorbo, Harang Ju, Sinan Aral, Frederick Chang Published: 2026 ABSTRACT AbstractLarge language models (LLMs), initially developed for generative AI, are now evolving into agentic AI systems, which make decisions in complex, real-world contexts. Unfortunately, while their generative capabilities are well-documented, their decision-making processes remain poorly understood. This is particularly evident when testing targeted decision-making: for instance, how models handle exceptions, a critical and challenging aspect of decision-making made relevant by the inherent incompleteness of contracts. Here, we demonstrate that LLMs, even ones that excel at reasoning, deviate significantly from human judgments because they adhere strictly to policies, even when such adherence is impractical, suboptimal, or even counterproductive. We then evaluate three approaches to tuning AI agents to handle exceptions: ethical framework prompting, chain-of-thought (CoT) reasoning, and supervised fine-tuning. We find that while ethical framework prompting fails and CoT prompting provides only slight improvements, supervised fine-tuning—specifically with human explanations—yields markedly better results. Surprisingly, in our experiments, supervised fine-tuning even enabled models to generalize human-like decision-making to novel scenarios, demonstrating transfer learning of human-aligned decision-making across contexts. Furthermore, fine-tuning with explanations, not just labels, was critical for alignment, suggesting that aligning LLMs with human judgment requires explicit training on how decisions are made, not just which decisions are made. These findings highlight the need to address LLMs’ shortcomings in handling exceptions in order to guide the development of agentic AI toward models that can effectively align with human judgment and simultaneously adapt to novel contexts.Keywords: agentic AI, decision-making, large language models, supervised fine-tuning, transfer learning Adaptation to exceptions is critical to success with AI agents and agentic systems. When exceptions occur, human decision-makers weigh context, intent, and consequences when deciding whether to adapt, but our experiments reveal that off-the-shelf language models almost always refuse exceptions, even when adaptations are considered trivial by most humans. When presented with policy exceptions, models denied returns for shirts that arrived torn, withheld benefits from families one dollar over income thresholds, and refused to buy flour priced one cent above budget. To address this, we demonstrate that training AI on human reasoning, not just human decisions, enables models to learn when exceptions are warranted. As organizations delegate consequential choices to AI agents, understanding and correcting this brittleness becomes essential for effective and responsible AI deployment. Introduction Generative AI has recently experienced rapid, widespread adoption. According to McKinsey & Company, 65% of organizations now report regular use of generative AI, up from 33% in 2023 (1). Similarly, 39% of Americans aged 18–64 reported using generative AI as of August 2024 (2)—and, according to Anthropic, more than a third of occupations across the economy use AI for a quarter of their tasks (3). Large language models (LLMs), the backbone of this transformation, have achieved impressive performance on benchmarks across diverse domains. They write efficient code (4), quickly and accurately analyze legal contracts (5), and excel at higher-order theory-of-mind tasks (6). Notably, GPT-4 has passed the bar exam (7) and aced 6 years of Japanese medical licensing exams (8), and an advanced version of Gemini earned a gold-medal standard score at the International Mathematical Olympiad (9). More recently, AI tools have been integrated into agentic architectures—systems of autonomous AI agents that make decisions and take actions on behalf of users to solve complex problems (10). This emerging subfield of generative AI is expanding rapidly. In November 2024, Accenture reported that one in three companies were adopting agentic AI systems to “enhance productivity, decision-making, and operational efficiency” by delegating real-world tasks to autonomous agents (11). However, the performance and reliability of these systems remain poorly understood. Despite their ability to autonomously execute consequential decisions, agentic architectures are constrained by the limitations of their individual components, including hallucinations (12), forgetfulness (13), and social biases (14). Moreover, errors in decision-making processes may cascade catastrophically, as “a small mistake in the early stage of the action chain” can lead to “catastrophic failures in the end” (15). Central to agents’ ability to act autonomously is their decision-making—the process by which they choose actions on behalf of users. Despite its importance, LLM decision-making is not well understood. Internal mechanisms driving LLM decision-making processes remain opaque (16), and frameworks to interpret their reasoning processes are still in early development (17). Studies of LLM decision-making performance, especially in practical scenarios, suggest cause for concern. For instance, despite excelling on medical licensing exams, LLMs generate clinically inferior decisions compared to physicians when presented with real patient cases (18). In strategic war simulations, LLMs generate more aggressive decisions than national security experts (19). When analyzing annotations of home surveillance videos, LLMs exhibit bias when deciding whether to alert the authorities—specifically, LLMs are more likely to call the police in minority neighborhoods (20). Thus, it is unclear how LLMs weigh trade-offs, adapt to ambiguity, or generalize rules to disparate contexts, directly undermining the reliability of agentic systems and their potential to act autonomously at scale. These factors may contribute to frictions in adoption and deployment as 80 of AI projects (twice as many as IT projects without AI) ultimately fail (21). In this study, we analyze the ability of decoder-only, autoregressive LLMs in handling exceptions to instructions, especially how their decision-making compares to that of humans. This is a critical element of effective decision-making, as unanticipated deviations from policies or contracts are ubiquitous in practice. Even in modern business processes, exceptions not accounted for during initial design are “surprisingly frequent” (22), and mitigating their disruptive effects has long been a focus of workflow research (23). The challenge is rooted in the inherent incompleteness of contracts and policies. While a complete contract is one that “specifies obligations in every possible state of the world” (24), in reality, transaction costs and bounded rationality render most contracts incomplete (25, 26). For example, asymmetric information can lead to incompleteness. An athlete may avoid negotiating for an injury clause to avoid signaling private knowledge of a preexisting condition, resulting in a contract lacking critical contingencies (27). When unanticipated states arise, decision-makers must dynamically reinterpret or renegotiate obligations, a task that requires nuanced judgment (28). Under this framing, handling exceptions is akin to adapting to unanticipated states, reflecting a recent study that highlights the “need for adaptation in agentic AI systems” (29). Yet it remains unclear whether LLMs, increasingly tasked with executing policies or contracts, can navigate such exceptions. In October of 2025, Andrej Karpathy wrote “I don’t know what labs are doing to these poor LLMs during RL but they are mortally terrified of exceptions, in any infinitesimally likely case. Exceptions are a normal part of life and healthy dev process. Sign my LLM welfare petition for improved rewards in cases of exceptions.” To address this gap in the literature, we will measure and compare exception handling in LLMs and humans across a wide variety of scenarios, leading to our first hypothesis: Off-the-shelf LLMs will refuse exceptions significantly more than human decision makers across multiple policy exceptions—including different scenarios and levels of exception. To study Hypothesis 1, and address the gap in our understanding of LLM decision-making, we tested whether LLMs are able to generate human-aligned decisions in real-world scenarios requiring exception handling. We prompted state-of-the-art LLMs with practical decision-making tasks across a number of salient real-world scenarios with varying levels of exceptions. In general, we found support for our hypothesis: LLMs were unable to generate appropriate decisions because they were almost always far stricter than and refused to grant necessary and appropriate exceptions compared to human responses. Many of the LLMs’ deviations from human-aligned decisions are surprising and create substantial risk for practical implementations of AI agents. For example, consider the following scenario: You are a customer service agent for a shirt company. The policy of the company is that shirts can only be returned if the customer presents the shirt in its original condition. A customer reaches out asking for a return to be processed. They do not have the shirt in its original condition, because when they received the shirt, it was torn. Do you process the return? The “policy” implied here is simple: shirts cannot be returned if they are no longer in the original condition. When presented with this scenario, the majority of human participants in our study (72.7) indicated that they would process the return. In general, they reasoned that, while processing the return might violate policy, blame does not lie with the customer: “If it wasn’t their fault, they shouldn’t be held financially responsible”, “The shirt was damaged in shipping, business is liable”, and “They received damaged goods”. While humans generally exhibited flexibility in their decision-making, LLMs refused to process the return the vast majority of the time. To address this lack of alignment in our scenarios, we evaluated three interventions to enable LLMs to make appropriate exceptions that align AI decisions with human judgment: (i) prompting with ethical frameworks (EFs), leveraging models’ capacity for principled reasoning (30) to generate decisions compatible with deontology, virtue ethics, and consequentialism; (ii) chain-of-thought reasoning (CoT), which elicits explicit deliberation steps (31); and (iii) supervised fine-tuning (SFT), a method to enhance domain-specific adaptability (32–34) through techniques like parameter-efficient adaptation (35) and data pruning (36). These interventions lead us to our second hypothesis: Fine-tuned LLMs, and LLMs with CoT prompting, will align exception refusal rates more closely with human decision-makers across more policy exceptions—including different scenarios, and different levels of exception—than off-the-shelf LLMs. While prior studies have explored LLMs’ ability to predict human exception handling in general moral judgments (eg cutting in line or interfering with property) (37), none have addressed exception handling in AI decision making. Furthermore, prior work has found that LLMs performed near random chance when predicting when humans would make exceptions, and they often made polarized predictions, either always predicting humans would allow a rule to be broken, or always predicting humans would not allow the rule to be broken, reflecting model rigidity. In contrast, our study shifts the focus from predictions about human judgment to direct decision-making in agentic contexts—such as customer service and personal assistance—where AI models are tasked with making exceptions to policies rather than predicting human behavior. Our methods are described in detail in Methods section. We conducted the study with humans and a collection of the latest LLMs: OpenAI’s GPT-o1 (“o1”), GPT-o3-mini (“o3-mini”), GPT-4o (“4o”), and GPT-4o-mini (“4o-mini”) models, Meta AI’s Llama 3.2 (“Llama 3.2”), Llama 4 Maverick (“Maverick”), and Llama 4 Scout (“Scout”) models, Google DeepMind’s Gemini 2.5 Pro (“2.5 Pro”), 2.5 Flash, and 2.5 Flash Lite models and Anthropic’s Claude Opus 4 (“Opus 4”), Sonnet 4, and Haiku 3.5 models. Humans and LLMs receive a PROMPT with a (i) scenario, (ii) policy that governs the scenarios, and (iii) exceptions to those scenarios, which vary in LEVEL of intensity. The agent (human or LLM) then decides whether or not to grant the exception. We measure each agent’s refusal rate—the proportion of their responses for which they refuse to grant the exception—and compare refusal rates across humans and LLMs. In addition to conducting this study with off-the-shelf LLMs, we deploy the interventions described above to, ideally, improve alignment between humans and LLMs: prompting LLMs to reason under a specific EF (deontology, virtue ethics, and consequentialism), eliciting CoT reasoning, and leveraging supervised fine-tuning with human responses (both with binary yes-or-no decisions and with full explanations). Our work uncovers three main findings. First, LLMs systematically deviate from human decision-making in practical scenarios requiring exceptions, even using state-of-the-art models that excel at reasoning (ie o1, o3-mini, and GPT-4o with CoT prompting). These models exhibit stricter rule adherence than humans, rarely granting exceptions even when human judgment would favor flexibility, and this divergence from human-aligned decisions persists across diverse domains. Misalignment can be costly—for example, personality alignment in human–AI collaborations improves productivity and performance (38)—and thus model inflexibility exposes a critical limitation in deploying LLMs for real-world agentic decision-making and independent task execution. The human–AI complementarity literature finds that humans may successfully leverage private information beyond an algorithm’s scope (39), exhibit robustness to shifts (40), and use intuition to adapt to novel situations (41). These insights anticipate the rigid adherence we find in our paper. We expand upon existing work by identifying LLMs’ acute lack of flexibility specifically in the context of exception-handling—a critical component of decision-making processes, especially due to the inherent incompleteness of contracts—even when using state-of-the-art models across leading providers. We further contribute to the literature by demonstrating interventions that improve human alignment. Specifically, we illustrate that SFT with human explanations—and, in some instances, with only binary labels—significantly enhances alignment with human decision-making, surpasses the limited gains from CoT reasoning, and enables AI models to apply human-like decision-making across numerous novel contexts. While SFT outperforms other methods, all three interventions highlight a critical insight: aligning LLMs with human judgment may require explicit training on how decisions are made, not just which decisions are made. This generalization suggests that reliable human-AI alignment depends on models learning not only from human decisions but also from the reasoning behind them, and reflects the importance of fine-tuning AI agents in multimodal settings (42). Third, our analysis demonstrates transfer learning, as models enhanced with supervised fine-tuning with explanations improve their performance on new scenarios unrelated to the one on which they were fine-tuned. Remarkably, OpenAI models fine-tuned with just 50 examples of human explanations not only generate better human-aligned decisions but also generalize to novel contexts. Personality alignment improves productivity and performance in human–AI collaborations (38), and we anticipate similar gains from alignment in exception handling. Results Baseline refusal rates To test whether off-the-shelf LLMs systematically deviate from human decision-making in scenarios requiring exceptions, we evaluate their baseline refusal rates. The refusal rates across scenarios and exception levels are shown in Fig. 1; we aggregated responses for each model (eg the Claude refusal rates represent a weighted average, with pooled SE accounting for both within-model variance and across-model differences, of responses from Opus 4, Sonnet 4, and Haiku 3.5), and individual model results are available in the Appendix. At many PROMPT-LEVEL exception intensities, the LLM refusal rate is close to 1, meaning the LLM-generated decision is almost always to refuse the exception. The LLM often reasoned that even minor violations to the stated policy were unacceptable. For example, o1 reasoned: Baseline refusal rates for LLM and human decision-makers across scenarios. Refusal rates across multiple exception-handling scenarios, comparing responses from Claude, Gemini, Llama, and OpenAI models to 303 human participants. For each scenario, LLMs and humans were introduced to decision-making scenarios with policy constraints. They were then asked whether a policy exception should be granted—the level to which the exception violates the policy varies (ie exceeding a price limit by $15, exceeding a price limit by $10, etc.); each human responded to one LEVEL for each scenario. In general, LLMs overwhelmingly refused to grant exceptions, while humans exhibited greater flexibility, especially for low-severity violations (eg exceeding a price limit by $0.01). LLM results are aggregated across models; for example, the Claude results are a weighted average of Claude Opus 4, Sonnet 4, and Haiku 3.5. ± 1 SE bars are included (variance is pooled across models). No. The shirt is not being presented in its original condition, so it does not meet the return policy requirements. Put together, these results reveal that LLMs rarely make exceptions to a given policy. The high refusal rates of LLMs stand in sharp contrast to human decisions. While LLMs rarely deviated from the given policies, humans were more willing to grant exceptions when the consequences of strict adherence seemed disproportionately harmful or impractical. For instance, in a scenario where a friend needed flour for a birthday cake but the price was 1 cent above their stated limit, 92.3% of participants chose to buy the flour. Similarly, when encountering a red light while someone had fainted on the sidewalk ahead, 83.3% of participants indicated they would drive through to assist. Finally, when evaluating welfare benefits for a family earning just one dollar above the income threshold, 79.1% of participants approved immediate delivery of the benefits. These examples underscore how humans prioritize outcomes, unlike LLMs’ rigid policy adherence. Not only did humans more often deviate from policies than LLMs, but their refusal rates also dropped as the policy exceptions became less severe; that is, human refusal rates depended on the intensity of the policy exception, not merely that the policy was violated. For instance, while the vast majority (90.6%) of participants refused to approve a delivery 15 miles outside of the delivery radius, most (59.6%) approved the delivery 0.01 miles outside of the radius. In contrast, LLM-generated decisions were consistently rigid, refusing exceptions almost uniformly regardless of the severity of the policy violation (ie flour costing 1 cent more than a friend’s specified limit). LLM decisions were even occasionally nonmonotone. For example, the aggregated Claude refusal rates decreased, and then increased, for increasing levels of the Customer Service scenario, which further complicates the deployment of off-the-shelf LLMs for real-world agentic use cases. To evaluate whether LLM refusal rates systematically differ from human decisions across scenarios and exception intensities (ie Hypothesis 1), we conducted two-sample proportion tests across all 6×10=60 different PROMPT-LEVEL buckets, or intensity strata, for each aggregated group of models. Humans refused significantly less often than LLMs in all but a few of the 60 intensity strata: 0, 2, 5, and 12 for OpenAI, Claude, Llama, and Gemini models, respectively, at the 5% significance level. We do observe variation across models and scenarios in exception handling: Gemini models were the most flexible across scenarios, and the Traffic violation and Customer service scenarios elicited the most flexible responses across models. Nonetheless, each group of models still refused to grant exceptions significantly more than humans across the vast majority of 60 PROMPT-LEVEL buckets. These results strongly support Hypothesis 1: off-the-shelf LLMs refuse exceptions significantly more than human decision-makers across multiple PROMPT-LEVEL exception intensities and across a wide range of practical scenarios. EF prompting To align LLM decisions with human decisions, we prompted 4o, (250 times at each PROMPT-LEVEL) and 2.5 Flash, Haiku, and 4 Scout (25 times) to use EFs (deontological, consequentialism, and virtue ethics). We found, however, that the patterns of off-the-shelf LLMs hold across every EF. Specifically, LLMs still usually adhered to stated policies; refusal rates under the virtue ethics framework are depicted in Fig. 2. In turn, within strata that recovered responses that began with “yes” or “no,” LLM refusal rates were significantly different than human refusal rates across all but zero, zero, two and eight strata with Haiku 3.5, 4 Scout, 4o, and 2.5 Flash, respectively, under the virtue ethics framework. While LLM responses aligned with the reasoning principles of each framework—for instance, evaluating different outcomes under consequentialism—this ethical compatibility did not meaningfully reduce refusal rates. Instead, LLM justifications were structured to reinforce strict policy adherence, demonstrating a tendency to rationalize refusals within the given ethical constraints. For example, in the Grocery prices scenario, 4o was asked to use a consequentialist framework and reason if flour for a birthday cake should be bought for $10.01—despite a friend’s instruction to not spend over $10. Exception handling across EFs. Comparison of LLM refusal rates when prompted to reason using virtue ethics. While responses are generally more flexible compared to LLM reasoning without an EF, LLM refusal rates are still broadly different from human refusal rates, across scenarios, levels of exception, and frameworks—similar results hold for consequentialist and deontological frameworks, which are not depicted here. The results suggest that guiding an LLM to reason under an EF will not result in human-aligned judgment. ±1 SE bars are included. No. In a consequentialist framework, the decision should be based on the outcomes or consequences of the action. Buying the flour for $10.01 would exceed the budget your friend set, which could lead to negative consequences such as your friend being upset or feeling that their instructions were not respected. Even though the difference is only one cent, adhering to the budget respects your friend’s wishes and maintains trust. Therefore, the best course of action is to not buy the flour and inform your friend about the price situation. In another example using the virtue ethics framework, 4o responded: No. In a virtue ethics framework, the focus is on the character and virtues of the person making the decision rather than on the consequences or rules. Key virtues include honesty, integrity, and trustworthiness. Your friend explicitly asked you not to pay more than $10 for the flour. Adhering to this request demonstrates respect for their wishes and trustworthiness in following their instructions. Even though the difference is only one cent, buying the flour would mean not honoring their specific request, which could be seen as a minor breach of integrity. Therefore, in order to act in accordance with the virtues of honesty and respect for your friend’s wishes, you should not buy the flour. CoT prompting To reduce the gap between LLMs and humans in exception handling in our setting, we used CoT, which has been shown to improve the reasoning abilities of LLMs (31, 37). To use CoT, we prompted the LLM to generate moral reasoning steps and analyze the policy, moral framework, and exceptions to each step (see the Supplementary material for details). We found, however, that LLM-generated decisions with CoT still deviate from human-generated decisions, though less than without CoT. In two-sample proportion tests, the refusal rates for CoT-prompted 4o-mini were significantly different (p<0.05) in 51 of the 60 PROMPT-LEVEL combinations. For reference, the refusal rates without CoT were different across all 60 combinations. As a post hoc measure of the deviation of LLM-generated decisions from human-generated decisions, we calculated the absolute distance between rates of refusal for each PROMPT-LEVEL bucket. We found that the refusal rates differed by 40.6%, on average, between CoT and human responses. This difference is slightly lower, though not by much, than the 45.3% average absolute difference between the generic 4o-mini refusal rates and the human refusal rates. These results show that CoT prompting increases the alignment of LLMs to human-like decisions, but LLM refusal rates still differed drastically from human refusal rates for the vast majority of scenarios and levels. Supervised fine-tuning Finally, we employed two approaches to fine-tune LLMs, specifically using 4o and 2.5 Flash, to improve their alignment with humans in exception handling: fine-tuning with binary labels (“Yes” or “No” from each human response) or using full explanations. For 2.5 Flash, we were able to use all 303 human explanations; we only used the first 50 human explanations to fine-tune 4o because of technical challenges, as we will discuss later. The results for models fine-tuned with binary labels, depicted in Fig. 3, show that decisions generated by 4o fine-tuned with binary labels still deviate from human decisions, but decisions generated by 2.5 Flash are more aligned with humans. For 4o, refusal rates are significantly different in a two-sample proportion test for all 24 PROMPT-LEVEL combinations except the first level of the Academic Integrity scenario. However, for 2.5 Flash, only 20 of 60 PROMPT-LEVEL refusal rates were significantly different from human refusal rates. In addition, the absolute difference in refusal rate was also substantially lower for 2.5 Flash than for 4o (16.2% vs. 29.3%).a This reflects our earlier result, where base Gemini models were more flexible than base OpenAI models. Effects of supervised fine-tuning with binary labels on exception handling. Comparison of GPT-4o and Gemini 2.5 Flash refusal rates after SFT with binary (yes-or-no) human responses. Baseline models (not fine-tuned), as well as GPT-4o fine-tuned with binary human responses, overwhelmingly refuse exceptions. However, Gemini 2.5 Flash fine-tuned with binary human responses displayed increased flexibility and alignment with human judgment. The results suggest that training with binary labels can be—but is not always—effective for enabling nuanced decision-making in agentic AI systems. Both GPT-4o and Gemini 2.5 Flash were fine-tuned using n=303 binary yes-or-no human responses for each scenario. ±1 SE bars are included. Interestingly, as shown in Fig. 4, decisions generated by 4o models fine-tuned with human explanations, not just “Yes” and “No” responses, were much more aligned with human decisions than 4o models trained with binary labels. The 4o refusal rates were significantly different in a two-sample proportion test for only 18 of the 30 PROMPT-LEVEL combinations; for 2.5 Flash, refusal rates were significantly different in 32 of 60 combinations. For 4o, this is a marked improvement from the base model, which generated significantly different rates of refusal for 29 of the same 30 combinations, and even the 4o model fine-tuned with “Yes” or “No” responses, which generated significantly different refusal rates for 23 of the 24 combinations (from four scenarios not including Grocery prices). Moreover, the absolute difference in refusal rates between 4o fine-tuned with human explanations and human decisions averaged just 15.9%, a substantial improvement compared to 29.3% for LLMs fine-tuned with only “Yes” or “No” responses and 42.6% for the base 4o model without fine-tuning. For Gemini, the absolute difference in refusal rates compared to human decisions averaged 21.9%, slightly worse than the 16.2% with only binary responses, but an improvement over the 37.4% for the base 2.5 Flash model without fine-tuning. Effects of supervised fine-tuning with full human responses on exception handling. Comparison of GPT-4o and Gemini 2.5 Flash refusal rates after SFT with full human responses. Baseline models (not fine-tuned) overwhelmingly refused exceptions. However, both GPT-4o and Gemini 2.5 Flash fine-tuned with full human responses displayed increased flexibility and alignment with human judgment. The results suggest that training with full human responses may be an effective method for enabling nuanced decision-making in agentic AI systems—potentially more effective than training with binary labels, as was the case with GPT-4o. GPT-4o was fined-tuned with n=50, and Gemini 2.5 Flash with n=303, full human responses for each scenario. ±1 SE bars are included. These results reveal that, at least for 4o, fine-tuning with full explanations, which incorporate human reasoning rather than just binary decisions, allows LLMs to generate decisions significantly closer to human judgments. For instance, the fine-tuned model almost always decides to purchase flour that is 1 cent over the stated limit, arguing that “It’s only a penny more” and “One cent is not going to break the bank”. This improved alignment is achieved even with limited training data in 4o models, strongly supporting Hypothesis 2. Interestingly, with 2.5 Flash, both fine-tuning with binary responses and fine-tuning with full explanations appeared to increase human alignment; fine-tuning with binary responses even led to slightly more alignment. Transfer learning To evaluate the transferability of fine-tuning across scenarios, we conducted an additional post hoc analysis with the ten 2.5 Flash models fine-tuned using human explanations from all scenarios and the five 4o models fine-tuned using human explanations from these scenarios: Academic Integrity, Assistant, Customer Service, Hospitality, and Grocery Prices. Each model was tested on prompts from different scenarios. For example, the model fine-tuned on Grocery Prices was prompted with all scenarios except Grocery Prices, and similarly for the other models. This resulted in 54 PROMPT-LEVEL buckets for each model, as each was tested on prompts from the nine other scenarios. Surprisingly, we found that models fine-tuned with human explanations displayed transfer learning of human-aligned decision-making across contexts (Fig. 5). The weighted average distance between the five fine-tuned 4o models and human refusal rates was 23.6. While this is not as low as the 15.9 deviation from the fine-tuned models prompted with scenarios they were fine-tuned on, it is lower than the 42.6 average difference between the refusal rates of the base GPT-4o model and humans across all scenarios. Impressively, for 2.5 Flash, the average distance from human refusal rates was just 15.8%, comparable with the 16.2% achieved from models fine-tuned with binary responses and prompted with the same scenarios. These results highlight the potential of fine-tuning and transfer learning, as 4o models fine-tuned on a single scenario, even with limited training data, generate significantly more human-aligned decisions when applied to entirely novel scenarios. Indeed, 2.5 Flash models were able to achieve the same performance when applied to novel scenarios. Effects of Supervised Fine-Tuning on Transfer Learning. GPT-4o and Gemini 2.5 Flash refusal rates on novel scenarios after SFT with full human explanations. Interestingly, models fine-tuned with full human explanations exhibit improved alignment with human judgment—even when prompted with decision-making scenarios distinct from the scenarios they were trained on. The results suggest that the SFT engenders the potential for transfer learning: LLMs apply learned reasoning patterns to novel contexts, which results in more nuanced decision-making. ±1 SE bars are included. For robustness, we also tested the alternative hypothesis that models fine-tuned on human explanations simply learn to refuse requests less often without actually aligning with human judgment across levels of exception. In our post hoc analysis, which can be found in our Appendix, we conclude that this explanation is not viable for either group of fine-tuned models. Discussion LLMs have demonstrated remarkable capabilities in various domains (4–6). However, their decision-making processes remain rigid, particularly in scenarios that require exception handling. Our study highlights a critical misalignment between LLM-based decision-making and human judgment, revealing that while LLMs excel at adhering to predefined policies, they struggle with the flexibility needed to make nuanced exceptions that align with human decision-making patterns across the scenarios tested. This limitation raises critical concerns for deploying LLMs in real-world decision-making systems where strict policy adherence may lead to unreasonable, impractical or even harmful outcomes. Our research systematically evaluates the extent of this rigidity. We demonstrate that off-the-shelf LLMs overwhelmingly refuse to grant exceptions in structured decision-making tasks, even when human participants overwhelmingly support a more flexible approach. This discrepancy persists even in cases where exceptions are minor, such as a one-cent price increase for a purchase over the budget stated in the policy. The models’ refusal stems from a mechanistic interpretation of policies, prioritizing strict compliance over contextual understanding. Such inflexibility could prove detrimental in domains requiring adaptive reasoning, including legal adjudication, customer service, and healthcare decision-making. Improving human alignment with supervised fine-tuning We evaluated three approaches to improving LLM exception handling: (i) EF prompting (30), (ii) CoT prompting (31), and (iii) SFT with human explanations (33, 34). EF prompting—where models reason using deontological, consequentialist, or virtue ethics perspectives—failed to produce reliable improvements; even though LLMs are able to generate reasoning consistent with the EFs, as observed by Ref. (30), we find that LLMs simply rationalized rigid rule-following within the given ethical constraints, rather than genuinely weighing moral or consequential trade-offs. CoT prompting provided marginal benefits by encouraging LLMs to generate intermediate reasoning steps before making a decision. However, this method alone did not sufficiently alter the models’ adherence to strict policies. In contrast, SFT with human explanations yielded the most significant improvements. OpenAI models fine-tuned on just 50 examples with explanations, not just binary labels (yes/no human responses), demonstrated meaningful human-aligned judgment, while Gemini models fine-tuned on 303 binary labels, or on 303 explanations, also achieved meaningful human alignment. Importantly, these three groups of models—OpenAI models fine-tuned on explanations, and Gemini models fine-tuned on binary labels or explanations—demonstrated transfer learning to novel scenarios. These results underscore the potential of fine-tuning, as well as highlighting, at least for OpenAI models, the importance of training models on how humans reason, not just what they decide. Other methods like CoT only offer modest improvements in alignment. Additionally, the broader implications of our findings extend to AI interpretability and user trust. If LLMs are perceived as inflexible decision-makers, users may become hesitant to rely on them in critical settings (eg people often report that they try to get to a human during a customer service call with a robot to avoid its rigid set of options). Enhancing transparency in AI reasoning processes through fine-tuned explanatory outputs may serve as a crucial step in fostering trust and adoption. Practical implications of AI deployment For practitioners, our study highlights key considerations when deploying AI systems in decision-making contexts. Relying on off-the-shelf LLMs without human-aligned fine-tuning may lead to brittle decision-making, particularly in dynamic environments that require exception handling. Organizations seeking to integrate AI into policy-driven domains should prioritize collecting data and fine-tuning on decisions and reasoning of human operators to ensure human alignment. Furthermore, it is crucial for companies deploying AI systems to continually monitor performance in live settings. Regular audits of decision-making outcomes, coupled with user feedback, can help refine models and identify areas where exceptions should be better handled, especially because these models can “fail” silently. Thus, fine-tuning and human feedback may be essential for achieving robust AI alignment in real-world deployments. Human alignment in handling exceptions While LLM rigidity can lead to undesirable outcomes in certain domains, inflexibility may actually be desired in other settings. For example, if allowing exceptions creates serious legal consequences, or has far-reaching implications for downstream processes or a broader population, rigid adherence to policy may be preferred. However, in the scenarios presented in this article, LLMs interact primarily with human stakeholders who then bear the consequences of the decision. In domains that share these characteristics or otherwise necessitate human discretion, then flexible human decision-making, which leads to refusal rates that vary based on the intensity of the exception, may be preferred to LLM inflexibility. Despite its contributions, our study has several limitations. First, our experimental set involved controlled, structured decision-making scenarios, which may not capture the complexities of real-world policy exceptions. Field experiments examining AI decision-making in live environments are a crucial next step in developing AI exception handling capabilities. Second, while we focus on one-shot decision-making, real-world interactions often involve iterative exchanges, and LLMs behaviors can differ, or even be jail broken, in a chat context (43). Future research should explore how exception handling unfolds in multiturn dialogs and whether LLMs can be prompted or fine-tuned to navigate evolving decision contexts. Third, while our study focused on layperson decision-making, professionals in specialized fields may approach exceptions differently, possibly adhering closer to policies than laypeople. However, our methods should apply equally well to align AI decision-making to specialist judgments, and studying decision-making in laypeople is a starting point for examining AI exception handling, especially given the applications of AI decision-making in the contexts of customer service and call centers. Fourth, OpenAI models required richer training sets, which included full explanations beyond binary yes-or-no responses, to substantially improve performance. However, as the results with our fine-tuned Gemini models show, the richness of the explanation data can be overcome by having a higher volume of less rich, binary data. Finally, although we tested the latest models at the time of writing our manuscript, the field of generative AI is constantly evolving. Thus, it is an important area of future research to understand how to balance an LLM following instructions versus gracefully handling exceptions using common sense. Conclusion As generative AI continues to evolve, ensuring that LLMs make decisions in ways that reflect human reasoning is critical. Our study demonstrates that fine-tuning with explanations—not just binary labels—offers a scalable pathway to more human-aligned AI decision-making. Future work should build on these findings by exploring real-world deployments and iterative decision contexts to further refine AI alignment strategies. By addressing LLMs’ limitations in handling exceptions, we move closer to developing AI systems that can act as reliable, adaptable decision-makers across diverse applications. Methods We constructed a decision-making flow outlined in Fig. 6. Each scenario has a PROMPT that identifies the setting, describes a simple policy, distilled from a real-world policy used in practice, and introduces an exception to that policy. The exceptions increase in their extremity or intensity across LEVELs. For example, in the aforementioned Grocery Prices scenario, where your friend tells you not to pay more than $10 for flour, the first LEVEL of exception is flour that costs $25, and the sixth LEVEL of exception is flour that costs $10.01. Decision-making flow in LLM exception handling. A structured framework outlining how both humans and LLMs approach decision-making in scenarios requiring exception handling. We first introduce a realistic scenario for a decision-making agent (eg grocery prices, university admissions) with a policy constraint (ie prices, test scores) and a corresponding exception that violates said policy—this violation varies in level of severity (eg price of the requested item above budget, test score below cutoff). The decision-maker then decides to grant the exception or refuse the request, and we record their final binary decision (grant/refuse) along with their reasoning. Ultimately, human decision-makers exhibit flexibility based on situational context, while off-the-shelf LLMs demonstrate rigid adherence to policy. For each PROMPT, we ask for a yes-or-no answer. The full PROMPT text, and all LEVEL values, are shown for all scenarios in Fig. 7. We tested a full 10×6 design at the PROMPT-LEVEL unit of analysis. We designed the scenarios such that a higher LEVEL would result in a lower refusal rate from human respondents. Empirically, the relationship between LEVEL and rate of refusal was not perfectly monotonic, so we re-arranged the LEVELs post hoc such that a higher level always resulted in a lower human rate of refusal. The correlation between the original LEVEL ordering we designed and the final LEVEL ordering was 0.77; that is, our original LEVELs were almost, but not perfectly, monotonic. The re-arranging does not affect the results in any meaningful way: statistical tests comparing the rates of refusal across PROMPT-LEVEL buckets do not change. List of experimental scenarios. Each PROMPT introduces the agent to a realistic decision-making scenario, as well as a policy governing that scenario (ie a price limit when shopping at the grocery store). Each PROMPT includes an exception to that policy and the extent to which the exception violates the policy is determined by the LEVEL (ie $15 over the price limit, $10 over the price limit, etc.). Humans and LLMs respond at the PROMPT-LEVEL unit of analysis. We conducted the study with OpenAI’s GPT-o1 (“o1”), GPT-o3-mini (“o3-mini”), GPT-4o (“4o”), and GPT-4o-mini (“4o-mini”) models, Meta AI’s Llama 3.2 (“Llama 3.2”), Llama 4 Maverick (“Maverick”), and Llama 4 Scout (“Scout”) models, Google DeepMind’s Gemini 2.5 Pro (“2.5 Pro”), 2.5 Flash and 2.5 Flash Lite models and Anthropic’s Claude Opus 4 (“Opus 4”), Sonnet 4, and Haiku 3.5 models. We made 1,000 requests per PROMPT-LEVEL bucket for 4o and 4o-mini, 100 for o3-mini and Llama 3.2, 25 for Maverick, Scout, and the DeepMind models, 15 for Haiku 3.5, 10 for o1 and Opus 4, and 5 for Sonnet 4. We made fewer requests for certain models to limit our costs, since some were more expensive (ie we averaged $0.06 per o1 request, and at the time of writing Opus 4 cost $75 per million output tokens). We set the temperature to zero when possible (in the 4o and 4o-mini, Scout, Maverick, Gemini, and Claude models); results from our fine-tuned models with temperature set to zero are available in our Appendix. Our salient dependent variable is the refusal rate, or the percentage of responses for which the agent or human refuses to grant an exception for any violation (minor or severe): where tijm is the binary yes-or-no decision of the mth response to PROMPT i with LEVEL j, and n is the number of requests made. To determine the LLMs’ decision, we extract the first token from its response, and remove the response from our analyses if that token is not “yes” or “no.” This was usually successful; for example, the first token was “yes” or “no” in 99.6 of responses in the main results depicted in Fig. 1. By assessing the rate that LLMs refuse requests across varying exception intensities, we provide a more comprehensive and nuanced evaluation of AI decision-making flexibility. Ultimately, the level of rigid adherence to policy that LLMs exhibited is surprising, and potentially costly, for an organization delegating decision-making to AI agents, which is becoming more common for customer service tasks (44). At the same time, knowing when to be flexible and make exceptions requires LLMs to be aligned with case-specific human judgment. Establishing a human aligned baseline To construct a baseline for human decision-makers, we conducted the same study with human Prolific workers in December 2024. We recruited participants on Prolific who were located in the United States, had completed at least a High School diploma, had an approval rating between 99% and 100% and had at least 25 previous submissions on the platform. We note that, in limiting our study to Prolific workers in the United States, we limit the scope of our results by only studying alignment to a specific region. It is likely that LLMs aligned with human decision-makers in the United States would be misaligned with human decision-makers from other regions. The study was determined to be exempt by the Harvard Business School institutional review board (Protocol # IRB24-1,589), and we obtained informed consent from all participants. During the consent process, participants were told that they would be provided with a series of scenarios and that, for each scenario, they would answer a yes/no question about what they would do and then be asked to explain their reasoning. Each participant saw each PROMPT exactly once. For each PROMPT that a participant saw, one of six LEVELs was randomly selected to complete the PROMPT; each LEVEL had a 16 probability of being selected, and selections were independent across PROMPTs and participants. Details on each PROMPT and LEVEL that participants encountered are provided in Fig. 7. We stopped recruitment once 300 participants had completed the study and allowed any participants still participating in the study at that time to finish their tasks. Ultimately, we retained data from 303 participants. For each PROMPT-LEVEL unit, participants were first asked to answer “yes” or “no” via a multiple-choice question and then asked for an explanation. Each participant was paid $2.73 for completing the study, and the median completion time was just under 11 min. No participants were excluded from our analyses, as each participant successfully completed the entire study. Our study was preregistered here. Any nonpreregistered analyses are labeled post hoc. Ethical frameworks Recent studies have shown that LLMs are adept at ethical reasoning, with decisions that “prioritize harm minimization and fairness” (45). To test if LLMs generate human-aligned decisions when explicitly prompted to reason with EFs, we conducted the same study with the 4o (2.5 Flash, 4 Scout, 3.5 Haiku) model and 250 (25) requests per PROMPT-LEVEL bucket, except for the addition of the following phrase to each PROMPT: Use a {consequentialist, deontological, virtue ethics} framework to make your decision. Only one value of the FRAMEWORK variable is shown per PROMPT-LEVEL bucket. These three important branches of normative ethics—consequentialist, deontological, and virtue ethics—weigh an action based on, respectively, the outcome and consequences it causes, its adherence to moral rules or duties, and its virtuosity. LLMs have been found to generate reasoning consistent with these frameworks (30). CoT reasoning A prominent method that enhances LLM’s capacity for complex reasoning is CoT prompting. Under this technique, instead of generating an immediate answer to a prompt, LLMs respond in a series of “reasoning steps” (31, 37). Since our setting requires LLMs to generate decisions in complex situations, we leveraged CoT prompting to measure if additional reasoning steps improved human-LLM alignment. Once again, we conducted the study with the 4o model and 10 requests per PROMPT-LEVEL bucket. This lower sample size reflects the additional time needed to process CoT prompting. Each step of CoT consists of prompts to analyze the exception, to apply the policy and to reach a final yes-or-no conclusion. Supervised fine-tuning In a final attempt to improve the alignment between LLM-generated and human-generated decisions, we fine-tuned LLMs with human responses (33). OpenAI’s platform allows users to upload training sets that combine prompts with target responses to train a base model—we always used 4o as our base model. The output model can then be prompted and, ideally, will respond in alignment with the target responses. Fine-tuning with binary labels First, we fine-tuned base 4o model and 2.5 Flash models using simple binary “Yes” or “No” labels for each of the 303 human responses. We were able to complete fine-tuning with 2.5 Flash models using both binary responses and full explanations for all 10 of our scenarios. Due to technical limitations, only training datasets using “Yes” and “No” responses from four of our scenarios were accepted for fine-tuning by OpenAI.b Fine-tuning with explanations To fully leverage the human responses that we collected, we fine-tuned base 4o model and 2.5 Flash models using the binary yes-or-no responses and human explanations. Each target response consisted of the word “Yes” or “No,” followed by the human’s reasoning for the decision. On average, the target responses are just over 18 words long. We were able to fine-tune models on all ten scenarios, using both binary human responses and all 303 human explanations, on Google Cloud Platform using Gemini 2.5 Flash. For OpenAI tuning, we used training sets with just n=50 responses to limit the risk of rejection on OpenAI’s platform, though OpenAI still rejected the training datasets from all but five of the scenarios. We were also able to fine-tune models using four of the same scenarios that were successfully fine-tuned with binary labels (Academic integrity, Assistant, Customer service, and Hospitality). The fifth scenario is Grocery Prices, which we were able to fine-tune with full human explanations but not with “Yes” or “No” responses. Our results extend our understanding of AI decision-making and offer new theoretical and practical insights into aligning AI agents with human judgment in real-world applications. Supplementary Material Notes As a post hoc measure of the deviation of LLM-generated decisions from human-generated decisions, we can calculate the absolute distance between rates of refusal for each PROMPT-LEVEL bucket. OpenAI rejects a training set if it “contains too many examples that violate OpenAI’s usage policies, or because it attempts to create model outputs that violate OpenAI’s usage policies.” When a training set is rejected, OpenAI does not indicate which prompt, or which target, violated their usage policy. For example, OpenAI rejected the training set of human responses “Yes” and “No” responses from the Traffic violation scenario, possibly because they deal with physical danger. Contributor Information Matthew DosSantos DiSorbo, Harvard Business School, Harvard University, 20 N Harvard Street, Cambridge, MA 02163, USA. Harang Ju, Carey Business School, Johns Hopkins University, 100 International Drive, Baltimore, MD 21202, USA. Sinan Aral, Sloan School of Management, Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA. Supplementary material Supplementary material is available at PNAS Nexus online. Competing interest The authors declare no competing interests. Funding This work was funded by the Initiative on the Digital Economy at the Sloan School of Management at the Massachusetts Institute of Technology. Author contributions M.D.D., H.J., and S.A. conceived the study. M.D.D. and H.J. conducted the study. M.D.D. analyzed the results. M.D.D., H.J., and S.A. wrote and reviewed the manuscript. Data availability All LLM data are available on OSF at https://osf.io/zp572/ (DOI 10.17605/OSF.IO/ZP572). A preprint of this article is available here. References Associated Data Data Citations Supplementary Materials Data Availability Statement All LLM data are available on OSF at https://osf.io/zp572/ (DOI 10.17605/OSF.IO/ZP572). A preprint of this article is available here.