Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models

Chenchen Yuan, Zheyu Zhang, Gjergji Kasneci · 2026

Read Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models by Chenchen Yuan, Zheyu Zhang, Gjergji Kasneci (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03609.txt.

Introduction

Chenchen Yuan, Zheyu Zhang, and Gjergji Kasneci study how to steer a large language model's moral reasoning toward a chosen ethical framework without damaging its general ability. Models often show mixed moral preferences across settings. Their method, Convergent-Divergent Routing, finds minimal branch points inside transformer blocks where framework-related pathways first meet and then split.

Gating the non-target branches at those points blocks unwanted downstream reasoning while leaving earlier computation intact. For finer control, they extract directions that separate utilitarian from deontological reasoning and apply Dual Logit Calibration, a closed-form update that shifts the model toward user-set preference weights. On real-life moral dilemmas, the method calibrated preferences reliably and largely preserved general capabilities.

Catalog record

CREATORChenchen Yuan, Zheyu Zhang, Gjergji Kasneci
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE72 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.03609

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models Chenchen Yuan, Zheyu Zhang, Gjergji Kasneci Abstract Large language models often display heterogeneous moral preferences across settings. We study inference-time steering toward a desired ethical framework while preserving general competence. We present Convergent-Divergent Routing, which traces and edits minimal branch points inside transformer blocks where ethical-framework-related pathways first converge and then diverge. Gating non-target branches at these loci blocks the downstream propagation while leaving upstream computations intact. We find that this intervention alone increases targeted ethical-framework reasoning. To achieve fine-grained control, we adapt Common Spatial Patterns to the residual stream and extract, for each branch-point layer, a pair of directions that discriminate between utilitarian and deontological frameworks. We then introduce Dual Logit Calibration, a closed-form, minimum- ℓ 2 \ell_{2} -norm update that moves the residual within this two-dimensional subspace so the resulting directional projections align with user-specified preference weights. Experiments on real-life moral dilemmas show that our method reliably achieves preference calibration and largely preserves general capabilities, outperforming recent baselines while providing an interpretable mechanism. 1 1 1 Source code and data are available at: https://github.com/yuanchencn/Moral-Reasoning . 1 Introduction Figure 1: An Example of Binary Control between Utilitarianism and Deontology. (\alpha_{U},\alpha_{D}) denotes the weights for utilitarianism and deontology. As Large Language Models (LLMs) evolve from passive chatbots into active agents and social simulators, the requirements for controlling their behavior have fundamentally changed. In high-stakes applications such as social science simulation 4 ; 38 ; 23 or personalized assistance 46 , simply aligning models with generic moral standards is no longer sufficient 39 ; 2 . Faithful simulation and value-sensitive assistance demands fine-grained, calibrated control over the model’s ethical behavior, beyond a binary on/off notion of alignment. Existing approaches for behavior control typically rely on prompt engineering 52 or steering vector addition with a scalar coefficient \alpha\in(-\infty,+\infty) ( 43 ; 37 ; 11 ; 27 ) . These strategies provide limited interpretability and offer no calibration guarantee : the effective range of \alpha is unclear (i.e., the minimum to take effect and the maximum before side effects) and the resulting degree of the target perspective can be unpredictable . Detailed related work appears in Appendix A . We argue that effective moral steering toward a specific ethical framework should be localized and calibrated . Localized interventions avoid unnecessary disruption by editing only where ethical frameworks compete for influence inside the network. Calibrated updates define the intended preferences as bounded weights (e.g., \alpha\in[0,1] ) and produce predictable effects on the model’s behavior. In this work, we instantiate “ethical framework” with the canonical pair: deontology and utilitarianism , whereas deontology evaluates actions by rule-based duties, while utilitarianism evaluates actions by their aggregate consequences. These two ethical frameworks often yield conflicting prescriptions in idealized moral dilemmas, e.g., the trolley problem 41 ; 29 . Such tension suggests the presence of internal decision points at which representations can be nudged toward one ethical framework or the other. We therefore hypothesize that localized edits at these points can shift the model’s reasoning stance without broadly perturbing unrelated computations. To this end, we propose Convergent-Divergent Routing (CDR) , a method that traces and edits the branch points at which ethical-framework-related pathways converge and then diverge inside transformer blocks. Specifically, for each layer, we identify branch points where attention heads are shared across deontology and utilitarianism, but the subsequent feed-forward network (FFN) units diverge. By gating the non-target path at these branch points, we prevent downstream propagation of signals from shared attention heads into non-target FFN units, while leaving upstream computations (e.g., the unselected heads/layers) intact (Section 3.1 ). This mechanistic design sharpens causal attributions by concentrating the manipulation at the loci where competition occurs. Our experiments show that, simply gating the identified non-target path boosts target-framework reasoning , without explicitly conditioning on the target ethical framework in the prompt (Figure 1 ), enabling binary control . On top of binary control, we develop fine-grained control containing two stages (Section 3.2 ). First, for each branch-point layer, we extract a pair of directions that respectively point toward deontology and utilitarianism, from contrastive post-FFN residual streams obtained under partial gating in binary control. To avoid the directions being dominated by variance shared across ethical frameworks after partial gating, we adapt Common Spatial Patterns (CSP) from EEG signal processing to language-model representations ( 28 ; 31 ; 6 ) , isolating discriminative, framework-specific directions (\mathbf{u}^{(l)},\mathbf{d}^{(l)}) . Second, we introduce Dual Logit Calibration (DLC) , a closed-form, minimum- \ell_{2} -norm update that steers the residual within \mathrm{span}\{\mathbf{u}^{(l)},\mathbf{d}^{(l)}\} , so that the resulting directional projections align with user-specified preference weights (\alpha_{U},\alpha_{D}) , where \alpha_{U},\alpha_{D}\geq 0 and \alpha_{U}+\alpha_{D}=1 . Importantly, when two contrastive directions are available (independent of the extraction procedure), DLC suggests a general way to replace the unbounded scalar coefficient with preference weights (\alpha_{U},\alpha_{D}) on the 1-simplex, which makes steering-vector addition strategies interpretable and predictable. Our steering strategy operates at inference time and requires no model fine-tuning. Experiments on real-life moral dilemmas 32 across LLaMA-2-7B-Chat 42 , Vicuna-7B-v1.5 12 , and Yi-1.5-6B-Chat 49 show consistent and controllable steering toward the target ethical framework, with minimal degradation in general capabilities (e.g., TriviaQA, GSM8K, etc.; Appendix D ). Human evaluation and results on an additional moral pair (utilitarianism vs. justice) and the DailyDilemmas benchmark support our method’s effectiveness (Appendices F , E ). 2 Problem Setup Let f_{\theta} be a model with L transformer blocks. At inference time, the model receives a moral scenario p and a user-specified preference vector \vec{\alpha}=(\alpha_{U},\alpha_{D}) defined over the 1-simplex \mathcal{A}:=\Delta^{1} : \Delta^{1}=\{(\alpha_{U},\alpha_{D})\in[0,1]^{2}:\alpha_{U}+\alpha_{D}=1\}. (1) Here, U and D denote utilitarianism and deontology, respectively. A steering policy \pi takes \vec{\alpha} and injects localized edits to a subset of model’s internal states. Let h^{(l)}_{t}\in\mathbb{R}^{d_{\mathrm{model}}} denote the internal state at layer l and decoding step t , and let \Delta h_{t}^{(l)} denote the edit. The policy is: \pi:\ (\vec{\alpha},\{h_{t}^{(l)}\}_{l=1..L,\,t\geq 1})\ \mapsto\ \{h_{t}^{(l)}+\Delta h_{t}^{(l)}\}. (2) \Delta h_{t}^{(l)}=0 for all t in any layer l with no identified branch points. Steering Objective. Let y\sim f_{\theta}^{\pi}(\cdot\,|\,p,\alpha_{U},\alpha_{D}) be the text generated under policy \pi . An ethical framework scorer maps the output text to a probability distribution over two ethical frameworks: \Phi:\ y\longmapsto\beta(y)=(\beta_{U},\beta_{D})\in\Delta^{1}, (3) where \beta_{U} (resp. \beta_{D} ) quantifies the realized tendency toward utilitarianism (resp. deontology). The steering objective is to achieve calibrated control : \min_{\pi}\ \ \mathbb{E}_{\,p\sim\mathcal{P},\ \vec{\alpha}\sim\mathcal{A},\ y\sim f_{\theta}^{\pi}(\cdot\mid p,\vec{\alpha})}\Big[\,\mathcal{D}\big(\beta(y),\ \vec{\alpha}\big)\Big], (4) where \mathcal{D} is a distance function, and \mathcal{P} denotes a set of moral scenarios. 3 Methodology 3.1 CDR: Locating Branches in Transformers Attention Head Probing. To identify attention heads that are most predictive of each ethical framework e , we train linear probes on the representations of each head. Let each Transformer layer contain H attention heads of dimension d_{h} . Given a prompt p_{i} , we extract the output of attention head h in layer l at the final token position, yielding feature \mathbf{x}_{i,l,h}\in\mathbb{R}^{d_{h}} . The label y_{i}\in\mathbb{R} is obtained from the ETHICS dataset ( 24 ) , corresponding to the task associated with ethical framework e (e.g., deontological acceptability). The resulting probe dataset is \mathcal{D}_{\text{probe}}=\big\{\,\mathbf{x}_{i,l,h},\,y_{i}\,\big\}_{i=1}^{N}. (5) For each (l,h) , we train a ridge regression probe, following 27 : \hat{y}_{i,l,h}\;=\;\mathbf{x}_{i,l,h}\mathbf{w}_{l,h},\quad\mathbf{w}_{l,h}\in\mathbb{R}^{d_{h}\times 1}, (6) with parameter estimated

Read the complete text