DecisionFlow: Advancing Large Language Model as Principled Decision Maker

Xiusi Chen, Shanyong Wang, Cheng Qian, Hongru Wang, Peixuan Han, Heng Ji · 2025

Read DecisionFlow: Advancing Large Language Model as Principled Decision Maker by Xiusi Chen, Shanyong Wang, Cheng Qian, Hongru Wang, Peixuan Han, Heng Ji (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-21397.txt.

Introduction

Chen and co-authors propose in 2025 DecisionFlow, a framework that guides language models to decide through structured reasoning. In health and finance, choices need clear, explainable grounds, not just right answers. Current models often give a decision first and bolt on reasons after.

Their method builds a grounded space of actions, attributes, and constraints from the prompt. A hidden utility function then weighs trade offs in the open, so reasons and choice stay tightly linked. Tests on two high stakes benchmarks gave accuracy gains up to thirty percent over strong prompting. Outcome alignment also improved, a step toward accountable decision support.

Catalog record

CREATORXiusi Chen, Shanyong Wang, Cheng Qian, Hongru Wang, Peixuan Han, Heng Ji
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE71 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-21397

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

DecisionFlow: Advancing Large Language Model as Principled Decision Maker Xiusi Chen, Shanyong Wang, Cheng Qian, Hongru Wang, Peixuan Han, Heng Ji Abstract In high-stakes domains such as healthcare and finance, effective decision-making demands not just accurate outcomes but transparent and explainable reasoning. However, current language models often lack the structured deliberation needed for such tasks, instead generating decisions and justifications in a disconnected, post-hoc manner. To address this, we propose DecisionFlow , a novel decision modeling framework that guides models to reason over structured representations of actions, attributes, and constraints. Rather than predicting answers directly from prompts, DecisionFlow builds a semantically grounded decision space and infers a latent utility function to evaluate trade-offs in a transparent, utility-driven manner. This process produces decisions tightly coupled with interpretable rationales reflecting the model’s reasoning. Empirical results on two high-stakes benchmarks show that DecisionFlow not only achieves up to 30% accuracy gains over strong prompting baselines but also enhances alignment in outcomes. Our work is a critical step toward integrating symbolic reasoning with LLMs, enabling more accountable, explainable, and reliable LLM decision support systems. We release the data and code at https://github.com/xiusic/DecisionFlow . 1 Introduction Understanding and navigating high-stakes situations is a defining challenge for both human and artificial intelligence ( Duan et al. 2019 ) . In critical domains such as healthcare, disaster management, and economic policy, decisions are rarely made by intuition alone ( Béchard et al. 2024 ; Cuthbertson and Penney 2023 ) . Instead, effective decision-making requires a deliberate and structured process: identifying relevant factors, modeling their interactions, and systematically evaluating the consequences of possible actions. This structured reasoning ensures that decisions are not only well-informed but also transparent and accountable. Such a process is formalized under the concept of Decision Modeling , which provides a principled framework for navigating complex choices. Definition of Decision Modeling Decision Modeling is the process of formulating an abstract representation of a decision scenario by identifying key variables, their attributes, relevant constraints, and possible courses of action, in order to evaluate trade-offs and arrive at the most rational and explainable outcome. While this approach is fundamental to human expertise, current Large Language Models (LLMs) often struggle with coherent and transparent decision-making as they lack an explicit representation of the decision space ( Yang et al. 2023 ) . Rather than engaging in structured reasoning, they generate answers directly from prompts, leading to fragmented logic, weak causal connections, and inconsistent use of relevant information ( Freedman et al. 2024 ) . Specifically, explanations are often constructed after the fact by retrieving semantically similar text, rather than emerging from the reasoning process itself. This undermines their reliability in domains where decisions must be justified and defensible ( Randl et al. 2024 ) . When humans make complex decisions, they reason over structured abstractions: considering multiple aspects, weighing pros and cons, and connecting the dots. Similarly, structured decision modeling should embed this deliberative process into models, ensuring that decisions and explanations emerge from the same principled, traceable reasoning framework. To address these limitations, we introduce DecisionFlow , a step-by-step decision modeling framework that transforms natural language scenarios into structured, utility-based reasoning processes. Rather than producing answers directly from prompts, DecisionFlow begins by constructing a semantically grounded representation of the decision space. It identifies the set of candidate actions, extracts context-relevant attributes, and incorporates any explicit constraints such as ethical rules or resource limitations. This structured representation forms the basis for an interpretable decision context. Building on it, DecisionFlow infers a latent utility function over the attribute space, enabling the model to score each action while respecting the given constraints. By reasoning over this structured space, DecisionFlow moves beyond superficial text generation, allowing the model to deliberate over trade-offs and align its decisions with task-specific objectives such as fairness, risk-aversion, or efficiency. Specifically, DecisionFlow bridges natural language understanding with symbolic utility modeling inside the LLM pipeline, without relying on external supervision or hand-crafted templates. Each decision is accompanied by rationale that traces back to the structured inputs, highlighting which attributes mattered most, which constraints ruled out alternatives, and how the selected option compares to its competitors. This not only increases transparency and robustness but also enables inspection, audit, and downstream use of the decision logic. By formulating decision modeling as a structured inference task, DecisionFlow equips LLMs with a deliberative process that mirrors human reasoning: incremental, interpretable, and context-aware. We evaluate DecisionFlow on two high-stakes decision modeling benchmarks—Medical Triage Alignment (MTA) ( Hu et al. 2024a ) and DeLLMa ( Liu et al. 2024 ) —covering domains such as clinical triage, agriculture, and finance. Across all settings, DecisionFlow consistently outperforms strong baselines, including Chain-of-Thought and Self-Consistency prompting, achieving up to 30% gains and setting new state-of-the-art results. Beyond accuracy, DecisionFlow also addresses imbalances in model reasoning by mitigating bias and promoting alignment between model outputs and target preferences in MTA. Human evaluations show high component-level reliability, especially in information extraction and symbolic reasoning, while identifying scoring and constraint-filtering as key areas for improvement. Ablation studies further highlight that accurate information gathering is crucial for enabling effective downstream reasoning, and that both reasoning depth and model scale contribute to performance. These results underscore DecisionFlow’s strength as a robust and interpretable framework for LLM-based decision modeling. To summarize, our paper’s contributions are threefold: • We introduce the concept of Decision Modeling as a critical lens through which to evaluate and enhance reasoning transparency and rationality in LLMs, particularly under high-stakes scenarios. • We design DecisionFlow, enabling language models to transform natural language scenarios into explicit modeling of actions, attributes, and constraints, and to reason over them via symbolic decision-making. • We empirically validate DecisionFlow’s effectiveness and generalizability across multiple domains, demonstrating substantial improvements in both decision quality and explainability. Finally, our research pioneers a new pathway for integrating symbolic reasoning structures with LLMs, bridging the gap between statistical learning and principled decision science. We anticipate that this line of work will catalyze further advances in building AI systems capable of transparent, accountable, and high-stakes decision support in critical real-world applications. 2 Related Work LLMs for Decision Making. Decision-making is a fundamental aspect of human life, and recent advances in LLMs have opened new opportunities for assisting humans to make more informed and better decisions Eigner and Händler 2024 . However, prior research has predominantly focused on low-stakes domains where the cost of suboptimal decisions is relatively low ( Wang et al. 2023 ; Nie et al. 2024 ; Shinn et al. 2023 ; Yang et al. 2024b ) . While recent efforts have begun to explore high-stakes domains, such as medicine ( Benary et al. 2023 ; Hu et al. 2024a ) , autonomous driving ( Mao et al. 2024 ) and financial investment ( Liu et al. 2024 ) , the decision-making process of LLMs remains largely opaque, limiting their trustworthiness and practical deployment in critical scenarios. In this work, we depart from prior domain-specific or task-limited approaches by proposing a more general and structured decision-making framework, enabling more robust and human-aligned decision-making, particularly for high-stakes applications. Symbolic Reasoning in LLMs. Recent work has explored the integration of symbolic reasoning mechanisms into large language models (LLMs) to improve interpretability,

Read the complete text