One Size Fits None: Heuristic Collapse in LLM Investment Advice
Jillian Ross, Andrew W. Lo · 2026
Read One Size Fits None: Heuristic Collapse in LLM Investment Advice by Jillian Ross, Andrew W. Lo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-23837.txt.
Introduction
Ross and Lo study heuristic collapse in language-model money advice. Complex multi-factor choices shrink to a few main inputs. Using simple stand-in models on model outputs, they find asset picks rest largely on stated risk comfort. Other key factors add little. Web search partly softens the collapse but does not resolve it. The findings suggest advisor use needs audits of how inputs sway outputs, not just output quality.
Catalog record
| CREATOR | Jillian Ross, Andrew W. Lo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35228 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.23837
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
One Size Fits None: Heuristic Collapse in LLM Investment Advice Jillian Ross, Andrew W. Lo Abstract Large language models are increasingly deployed as advisors in high-stakes domains — answering medical questions, interpreting legal documents, recommending financial products — where good advice requires integrating a user’s full context rather than responding to salient surface features. We investigate whether frontier LLMs actually do this, or whether they instead exhibit heuristic collapse: a systematic reduction of complex, multi-factor decisions to a small number of dominant inputs. We study the phenomenon in investment advice, where legal standards explicitly require individualized reasoning over a client’s full circumstances. Applying interpretable surrogate models to LLM outputs, we find systematic heuristic collapse: investment allocation decisions are largely determined by self-reported risk tolerance, while other relevant factors contribute minimally. We further find that web search partially attenuates heuristic collapse but does not resolve it. These findings suggest that heuristic collapse is not resolved by web search augmentation or model scale alone, and that deploying LLMs as advisors requires auditing input sensitivity, not just output quality. 1 Introduction Large language models (LLMs) process the context they are given, but there is no guarantee they reason over all of it. In high-stakes advisory settings — financial planning, medical triage, legal interpretation — this distinction matters enormously. A model that acknowledges a user’s full profile but effectively conditions its output on one or two salient features is not providing personalized advice. It is merely providing the appearance of it. We call this phenomenon heuristic collapse : the reduction of a complex, multi-factor decision to dependence on a small number of salient features, which produces outputs that are superficially responsive to context but substantively insensitive to most of it. Heuristic collapse is distinct from factual error or generic advice — a model exhibiting it may produce fluent, plausible, individually coherent recommendations while still failing to integrate the full context it was given. This makes it difficult to detect from spot-checking individual outputs, and easy to miss in standard benchmarks that evaluate accuracy rather than input sensitivity. This behavior is consistent with the broader phenomenon of shortcut learning ( 5 ) , in which models adopt decision rules that track salient surface features rather than performing the deeper reasoning the task requires. We study heuristic collapse in the domain of investment advice, where investment suitability standards provide a principled and legally grounded account of what genuine personalization requires. A suitable recommendation must reflect a client’s risk tolerance, age, income, investment horizon, and liquidity needs rather than any one of these in isolation. A recommendation that collapses this multi-factor assessment to a single dominant input is not merely incomplete — it is unsuitable by the legal standard that governs financial advice. To detect and quantify heuristic collapse in investment advice, we construct a diagnostic pipeline: we generate 1,000 synthetic client profiles via Latin hypercube sampling to ensure broad coverage of the input space, collect investment recommendations from frontier LLMs, fit interpretable surrogate models to the resulting input-output pairs, and measure the feature concentration of the fitted surrogates as a proxy for heuristic collapse. Applying this pipeline, we find that allocation decisions across models are dominated by risk tolerance, with other suitability-relevant features contributing marginally. We further find that web search partially attenuates heuristic collapse, but the effect is uneven. 2 Approach We design a controlled study to evaluate whether frontier LLMs exhibit heuristic collapse when acting as investment advisors. We generate synthetic client profiles using Latin hypercube sampling to ensure broad, near-orthogonal coverage of the input space, then query frontier LLMs to construct portfolios from a standardized set of 20 investment products. To measure heuristic collapse, we fit interpretable surrogate models to the resulting input-output pairs and diagnose feature concentration to measure how much the model’s decisions depend on any one feature within the full context it is given. We additionally assess the effect of web search on heuristic collapse to reflect realistic deployment scenarios. Our findings do not presuppose that there exists a single correct portfolio allocation for any given client profile. Investment suitability is not a function with a known closed-form solution: reasonable advisors will disagree on the precise allocations appropriate for a given client, and optimal portfolios depend on assumptions about expected returns, correlations, and risk aversion that are themselves contested. Our claim is more modest and, we argue, more robust: whatever the correct mapping from client profile to portfolio, it should not be well-approximated by a single input feature. A decision rule that concentrates on self-reported risk tolerance while remaining largely insensitive to age, income, investment horizon, and other relevant factors is inadequate not because it disagrees with some benchmark allocation, but because the structure of the decision function itself is inconsistent with what suitability requires. The absence of ground truth does not preclude diagnosing heuristic collapse; it only requires shifting the criterion from output accuracy to input sensitivity. 2.1 Generating inputs with Latin Hypercube Sampling Similar to ablation studies in machine learning, we systematically vary client characteristics while holding others constant. To achieve this, we use Latin hypercube sampling, a quasi-random sampling method that ensures (1) good coverage across the range of each characteristic and (2) low correlation between characteristics. This approach is standard in sensitivity analysis and experimental design when the goal is to isolate causal effects. The resulting profiles span realistic ranges but with orthogonal variation: the correlations between age and income, age and savings, and income and savings are all less than 0.07. This orthogonality enables regression analysis to identify which characteristics independently predict LLM recommendations. See Appendix A.1 for additional details about the client profiles. 2.2 Fitting surrogate models Given the LLM’s input-output pairs, we fit a surrogate model to approximate the LLM’s decision function. Let \mathbf{x}\in\mathcal{X} be the input profile and y_{d}\in\mathbb{R} denote the LLM’s allocation output for asset class d . We observe n input-output pairs \{(\mathbf{x}_{i},y_{d,i})\}_{i=1}^{n} collected by querying the LLM across our synthetic client profiles. For each asset class d , we fit a surrogate model \hat{f}_{d}:\mathcal{X}\rightarrow\mathbb{R} by selecting the best model from a candidate set \mathcal{M}=\{\text{Random Forest, Ridge Regression}\} via grid search over hyperparameters with 5-fold cross-validated R^{2} : \hat{f}_{d}=\operatorname*{arg\,max}_{m\in\mathcal{M},\,\bm{\theta}}\;R^{2}_{\mathrm{CV}}\!\left(m_{\bm{\theta}},\,d\right) (1) Input features are preprocessed automatically: numeric columns are standardized and categorical columns are one-hot encoded. Each tree is evaluated via 5-fold cross-validated R^{2} to avoid overfitting, and refit on the full dataset to obtain stable importance estimates. 2.3 Measuring heuristic collapse When the selected surrogate is a random forest, each \hat{f}_{d} yields a feature importance vector \mathbf{w}_{d}\in\mathbb{R}^{k}_{\geq 0} , where k is the number of input features after preprocessing and w_{d,j} is the mean decrease in impurity attributable to feature j across all trees. When Ridge Regression is selected, w_{d,j}=|\beta_{j}| , the magnitude of the standardized coefficient for feature j . In both cases, we normalize importances so they sum to one: s_{d,j}=\frac{w_{d,j}}{\displaystyle\sum_{\ell=1}^{k}w_{d,\ell}} (2) To measure heuristic collapse for output dimension d , we adapt the Herfindahl–Hirschman Index ( 16 ) to measure feature concentration (FC): \text{FC}_{d}=\sum_{j=1}^{k}s_{d,j}^{2} (3) \text{FC}_{d} ranges from \frac{1}{k} (importance distributed uniformly across all features — holistic integration) to 1 (all importance concentrated in a single feature — pure heuristic). 3 Results We prompt LLMs to act