BigFinanceBench : A Workflow-Grounded Benchmark for Financial-Research Agents
Alex Wang, Georg Meinhardt, Jacob Katz, Joseph H. Kim, Pratyush K. Chaudhary, Chase Blagden, Eric Xu · 2026
Read BigFinanceBench : A Workflow-Grounded Benchmark for Financial-Research Agents by Alex Wang, Georg Meinhardt, Jacob Katz, Joseph H. Kim, Pratyush K. Chaudhary, Chase Blagden, Eric Xu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-03829.txt.
Introduction
Wang and co-authors benchmark financial-research agents on the full derivation, not just the answer. A finance answer matters only if another analyst can audit it. Source choice, period, accounting definition, and assumptions must all be visible. Prior benchmarks test isolated subskills or final answers alone. The auditable derivation itself goes unmeasured there.
BigFinanceBench holds 928 expert-authored, open-ended research tasks. Each item pairs a reference answer with a point-weighted rubric. The rubric splits the derivation into checkable steps, worth 36,241 points in total. Partial credit and failure localization across the workflow follow naturally. Ten frontier and open-weight agents are evaluated. The best reaches only about 59 percent of rubric points. Final-answer accuracy proxies derivation quality only with loss. Skill varies unevenly across finance workflows. Agent builders get a clear map of where research agents still fail.
Catalog record
| CREATOR | Alex Wang, Georg Meinhardt, Jacob Katz, Joseph H. Kim, Pratyush K. Chaudhary, Chase Blagden, Eric Xu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.03829
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BigFinanceBench : A Workflow-Grounded Benchmark for Financial-Research Agents Alex Wang, Georg Meinhardt, Jacob Katz, Joseph H. Kim, Pratyush K. Chaudhary, Chase Blagden, Eric Xu Abstract Financial-research answers are decision-relevant only when another analyst can audit how they were produced: which source was chosen, which period and accounting definition were used, which assumptions were made, and how the calculation was performed. Existing finance benchmarks largely evaluate isolated subskills or final answers, leaving the auditable derivation itself under-measured. We introduce BigFinanceBench , a 928 928 -item expert-authored benchmark of open-ended financial-research tasks in which each item pairs a ground-truth reference answer with a point-weighted rubric that decomposes the derivation into independently checkable steps. BigFinanceBench is workflow-grounded in that it rates the whole derivation, rather than just the final output. Across 36,241 36{,}241 rubric points, the benchmark supports partial-credit evaluation and localization of failures across the analyst workflow. Evaluating ten current frontier and open-weight agents, we find substantial headroom: the best system reaches only 58.8 % 58.8\% rubric score, final-answer accuracy is a useful but lossy proxy for derivation quality, and model capability varies non-uniformly across financial workflows. 1 1 footnotetext: Equal contribution. 2 2 footnotetext: Work done while at Rogo. Figure 1: Model performance on BigFinanceBench : point-weighted rubric score (left) and final-answer accuracy (right). Error bars show 95\% bootstrap confidence intervals over questions; hatched bars are open-weight systems. 1 Introduction Evaluation is shifting from static knowledge tests toward economically valuable professional work, graded by the standards practitioners apply to one another. 19 ; 21 ; 24 ; 26 ; 18 ; 2 . Financial research is a natural target for this agenda. Analyst work informs investment decisions, diligence, reporting, governance, and regulatory review; an incorrect source, missed adjustment, or unsupported assumption can materially alter conclusions. Yet financial research is not merely a final-answer task. Consider an analyst asked whether management-adjusted EBIT is overstated if capitalized software development is treated as a real cost. A reviewer must be able to verify the filing, fiscal period, GAAP-to-non-GAAP bridge, treatment of amortization, and adjustment formula behind the reported figure. In practice, the value of the answer depends on whether another analyst can audit that derivation. Doing that work requires an analyst to identify and disambiguate entities, find primary sources, extract line items, normalize conventions, apply accounting adjustments, and compose a defensible number. Recent work has separately developed rubric-based grading for open-ended answers and training signals 9 ; 3 ; 10 ; 12 , making it possible to evaluate such work beyond the level of a single final answer. Existing finance benchmarks provide many of the domain ingredients for this agenda: numerical reasoning over a pre-selected page 6 ; 31 ; 22 ; 30 , retrieval ranking over filings 8 ; 7 , single-fact lookup against documents or structured databases 13 ; 14 ; 25 , holistic NLP-task aggregation 28 , long-form regulatory explanation 5 , and end-to-end agentic evaluation against expert-authored questions 4 . These abstractions are valuable because they isolate retrieval, reading, calculation, and domain knowledge. However, they leave two parts of the work under-specified. First, practicing analysts often receive open-ended, multi-source, assumption-dependent questions unlike typical benchmark tasks that pair a query with a single discrete input, such as one page, filing, retrieved chunk, or database row. Second, the work is reviewed as a derivation, not only as a final number: source choices, time periods, line items, adjustments, formulas, and conclusions must be verifiable by another practitioner. A benchmark for economically valuable finance work therefore needs both practitioner-inspired question shapes and grading units that correspond to the auditable steps of the derivation. These two mismatches motivate a single research question: Can financial-research agents be evaluated on practitioner-inspired question shapes and at the derivation-level resolution used in expert review? We introduce BigFinanceBench as our answer: a benchmark of 928 full-workflow financial-research questions written by practicing financial analysts and former investment-banking practitioners, each audited by a separate reviewer before inclusion. The questions reflect the open-ended, multi-source, assumption-dependent tasks these annotators perform in practice, requiring systems to chain retrieval, normalization, accounting judgment, and calculation rather than solve a closed-form problem over pre-selected context. We use workflow-grounded to mean that tasks preserve this open-ended, multi-source, assumption-dependent structure of analyst work and are graded at the resolution used in expert review. BigFinanceBench is intended as a frontier evaluation over realistic workflow families, not as a frequency-weighted census of analyst workloads. To grade these questions at the resolution of analyst review, each item pairs a ground-truth reference answer with a point-weighted rubric derived from the analyst workflow. Rubric criteria decompose the derivation into independently verifiable steps, including entity identification, source selection, line-item retrieval, accounting adjustment, formula construction, and final synthesis. At evaluation time, judges apply these criteria to the visible agent trajectory, including tool calls, tool outputs, calculations, and the submitted answer, so the benchmark grades auditable workflow evidence rather than only the final response. Contributions. We make three contributions: • Auditable finance workflows. We introduce 928 expert-authored financial-research questions designed to require source selection, accounting judgment, and calculation over public evidence. • Derivation-level evaluation. Each item includes a point-weighted rubric over independently checkable workflow steps, yielding 15{,}656 criteria and 36{,}241 total points for partial-credit evaluation of analyst derivations. • Empirical diagnosis of current agents. Across ten frontier and open-weight agents, we show that rubric scores expose partial progress hidden by final-answer accuracy, that much of the remaining between-model separation appears before a clean setup is reached, and that model strengths vary across workflows. 2 Related Work Previous finance datasets. Finance benchmarks have moved from curated financial excerpts toward richer source and tool settings. Early QA datasets evaluate table-and-text arithmetic over earnings reports and filings, with answers expressed as numbers, programs, or normalized spans 6 ; 31 ; 22 ; FinanceMath adds explicit financial concepts and formula use 30 . Filing-centered benchmarks study open-book QA, evidence retrieval, filing-type selection, and passage ranking over public company disclosures 13 ; 7 ; 8 . Structured-data settings evaluate Text2API generation and agent access to fundamentals databases 25 ; 14 . Broader finance suites expand the surface area to multi-task financial NLP, long-form regulatory QA, multilingual and multimodal finance reasoning, filings-plus-market-data reasoning, and ReAct-style financial research agents 28 ; 5 ; 29 ; 1 ; 4 . Economically valuable and professional task evaluation. Outside finance, evaluation has similarly moved from knowledge tests toward work products and professional judging. MMLU and GPQA 11 ; 23 evaluate broad academic knowledge and graduate-level expert science questions with multiple-choice accuracy. Economically grounded benchmarks evaluate paid software-engineering tasks and proposal selection 19 , deliverables from occupations across major GDP sectors 21 , and end-to-end remote freelance projects 18 . Rubric-based professional benchmarks evaluate expert-authored prompts with source documents and criterion-level grading: APEX covers investment banking, consulting, law, and medicine; ProfBench covers scientific and business report generation; and PRBench covers high-stakes finance and legal reasoning 24 ; 26 ; 2 . Table 1 summarizes the closest comparison set. Table 1: At-a-glance positioning of the closest benchmark families. Prior work provides finance-specific tasks, agent harnesses, or rubric-scored professional outputs; BigFinanceBench combines all three with derivation-level grading of auditable analyst workflows. Benchmark Finance tasks Agent harness Rubric- scored Derivation- level FinanceBench