InvestPhilBench: A Multi-Layer Dynamic Benchmark for Evaluating Large Language Model Procedural Reasoning in Expert Investment Philosophy

Mingguang Chen · 2026

Read InvestPhilBench: A Multi-Layer Dynamic Benchmark for Evaluating Large Language Model Procedural Reasoning in Expert Investment Philosophy by Mingguang Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25984.txt.

Introduction

Mingguang Chen builds a benchmark that tests whether language models can think like expert investors. Models increasingly help with investment research. Yet no test checks whether they can rebuild a named investor's decision framework and apply it faithfully. InvestPhilBench fills that gap. Its tasks span eight tiers of difficulty. The lowest tier asks for principle identification. The highest asks for extending a framework to new ground.

Materials come from primary sources only. The release holds 118 verified principle cards and 25 framework cards. It also holds 243 questions, split between a development set and a held-out test set. Scoring runs through an automated pipeline with five metrics. A failure-mode protocol flags six computable error patterns. A small four-model study shows a sharp split between provider tiers. The composite score saturates at the frontier, while a per-gate accuracy metric still separates strong models.

Catalog record

CREATORMingguang Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE178 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.25984

Browse the full library · About the Tower of Babel