CoRE: Condition-based Reasoning for Identifying Outcome Variance in Complex Events
Sai Vallurupalli, Francis Ferraro · 2025
Read CoRE: Condition-based Reasoning for Identifying Outcome Variance in Complex Events by Sai Vallurupalli, Francis Ferraro (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01253.txt.
Introduction
News claims often hinge on hidden conditions: a result holds only if some background fact is true. Sai Vallurupalli and Francis Ferraro build a test of that reasoning in a 2025 paper. They merge two data sets of goals and states to make tasks where a model must name the conditions that change an outcome.
The team ran open and closed language models of many sizes on the tasks. Named conditions helped most when the full story was missing, since they could stand in for lost context. Models varied widely in how well they could propose and spot such conditions. The largest model tested was also the most careful when the setup left room for doubt.
Catalog record
| CREATOR | Sai Vallurupalli, Francis Ferraro |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-01253
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
CoRE: Condition-based Reasoning for Identifying Outcome Variance in Complex Events Sai Vallurupalli, Francis Ferraro Abstract Knowing which latent conditions lead to a particular outcome is useful for critically examining claims made about complex event outcomes. Identifying implied conditions and examining their influence on an outcome is challenging. We handle this by combining and augmenting annotations from two existing datasets consisting of goals and states, and explore the influence of conditions through our research questions and Condition-based Reasoning tasks. We examine open and closed LLMs of varying sizes and intent-alignment on our reasoning tasks and find that conditions are useful when not all context is available. Models differ widely in their ability to generate and identify outcome-variant conditions which affects their performance on outcome validation when conditions are used to replace missing context. Larger models like GPT-4o, are more cautious in such less constrained situations. \ESO@isMEMOIR 1 Introduction Knowing which conditions influence a goal’s outcome is useful for understanding and planning goal-directed actions seen in complex events Csibra and Gergely 2007 . Consider the following 5-sentence short story, also shown in fig. 1 : Sam can’t sleep at night. Sam is afraid of the monsters under the bed. Dad tells Sam there is no such thing as monsters but it doesn’t help. So, dad give Sam a blanket and tells Sam that it’s a magic blanket. Sam believes the blanket protects against monsters. While one can reasonably infer that Sam’s goal (to sleep at night overcoming fears about monsters) was achieved in this story , how stable is this outcome? What states, or conditions, relating to Sam or the situation help support the outcome, and if those were changed, would Sam’s goal still be achieved? Some of these conditions, like Sam trusting his/her dad or Sam being a small child, are relevant to the goal, with a high likelihood of influencing the outcome, while other conditions that might be true are irrelevant to the outcome—like Sam being male. Among relevant conditions, Sam trusting his/her dad directly influences the outcome and is outcome-variant because if Sam did not trust his/her dad the outcome is less likely to be true. Alternatively, Sam being a small child does not directly influence the outcome; it is outcome-invariant as the contrastive condition is unlikely to change the outcome. In this paper, we examine the interplay between state conditions and goal outcomes, and in particular, we demonstrate how to use, and generate outcome-variant counterfactual conditions to effectively reason about whether a story outcome is true (or false) across altered narratives. Figure 1: Conditions and their relationship to an outcome inspired by a story from PASTA/SAGA Ghosh et al. 2023 ; Vallurupalli et al. 2024 . (a) We examine whether different descriptions, an ongoing Vs a completed action, change the truth value of an outcome. (b) We examine whether models are able to think counterfactually and generate/identify outcome-variant conditions. (c) We examine whether models are able to combine a condition with varying amounts of context to validate outcomes. Figure 2: Our research questions explore outcome validation through our condition-based reasoning tasks. We leverage PASTA Ghosh et al. 2023 and SAGA Vallurupalli et al. 2024 datasets to generate outcomes of SAGA and News Stories and validate these using both our generated and PASTA’s crowd-annotated conditions. This type of reasoning is challenging because: 1. Conditions that to relate to entity properties and states are not always explicitly stated in a narrative but are implicitly understood through forming a coherent mental representation Ghosh et al. 2023 and acquiring this implicit knowledge is not easy. 2. Outcomes can be vastly different even for slight variations in context Vallurupalli et al. 2024 pointing to a need for identifying nuanced properties and states that have an influence on the outcome. 3. Knowing a condition’s influence on the outcome is implicitly understood through constructing counterfactual mental representations Byrne 2016 ; Mercier et al. 2017 requiring robust means for acquiring counterfactual conditions and reasoning with them. 4. Large Language Models (LLMs) ( Brown et al. 2020 ; Ouyang et al. 2022 , inter alia ) , while powerful, do not necessarily perform well on tasks requiring counterfactual reasoning Fang et al. 2025 ; Lin 2004 ; Ghosh et al. 2023 ; Qin et al. 2019 . 5. Linguistic differences influence action understanding Pruś et al. 2024 ; Zhou et al. 2023 ; Salomon et al. 2013 ; Hart and Albarracín 2011 which can hinder an LLM in understanding condition and outcome descriptions. Inspired by Pruś et al. 2024 ’s findings that pragmatic knowledge influences how linguistic cues are interpreted, we examine how models use linguistic cues when understanding story outcomes. We explore how implicit conditions influence outcomes and whether they can be used in place of missing context for validating story outcomes. We leverage two previously released datasets PASTA Ghosh et al. 2023 and SAGA Vallurupalli et al. 2024 consisting of participants’ goal and state annotations (shown in fig. 2 ) and augment them for exploring conditions and outcomes. We address the following research questions and formulate our Condition-based Reasoning tasks to explore the capabilities of intent-aligned LLMs on these tasks. We highlight the usefulness of conditions in determining story outcomes across a variety of alternative narratives, from short five sentence stories to real life newswire articles. RQ1: Do Linguistic Cues Matter? We explore models’ ability to handle different types of outcome descriptions. For the example in fig. 1 , would models consider “Sam was sleeping well” to mean the same as “Sam slept well?” While these two descriptions do not mean the same according to Dowty’s imperfective paradox Dowty 1977 , Pruś et al. 2024 show that meaning agreement is more complex. We examine how models fare on such linguistic differences and find that the state-of-the-art models differ in performance on the imperfective paradox, but understand both ongoing and completed actions, with a slight preference for completed actions. See section 5.1 for more details. RQ2: Can models think about “what-if…?” We examine whether models can reason counterfactually, through generating and identifying outcome-variant contrastive condition pairs. We found that with one exception (Mistral-7B-Instruct-v0.3), all considered models are better at identifying whether a condition is outcome-related (which does not involve counterfactual thinking), but struggle at identifying outcome variance which requires counterfactual thinking. See section 5.2 for details. RQ3: When are conditions useful? We examine if models are able to use conditions when context is unavailable or incomplete. We find that while models are able to combine conditions with available context to validate outcomes, some larger models are more cautious when the context is less constrained. See section 5.3 for more details. Overall, we find that performance on outcome validation improves by 2-6% (macro F1 score) when outcome-variant (as opposed to outcome-invariant ) conditions are used with the story context. Since identifying or generating outcome-variant conditions is not easy, performance drops by 3-6% for generated conditions when compared to annotated conditions. For news stories, performance drops by up to 2-18%, when compared to the PASTA story generated conditions, likely attributable to a combination of the domain change and models’ going beyond the provided context and using their knowledge of past news events from their pretraining. See section 6