Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis
1 st Zhiyuan Cheng, 2 nd Longying Lai, 3 rd Yue Liu, 4 th Kai Cheng, 5 th Xiaoxi Qi · 2026
Read Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis by 1 st Zhiyuan Cheng, 2 nd Longying Lai, 3 rd Yue Liu, 4 th Kai Cheng, 5 th Xiaoxi Qi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-16877.txt.
Introduction
Cheng and co-authors build a retrieval augmented system that answers questions on long financial reports. Analysts struggle with 10-K filings that can exceed 100 pages. The system mixes full text and semantic search, then adds an optional reranking stage with a cross encoder model. Tests used the FinDER benchmark, with 1,500 queries in five groups. With reranking, correctness for high scoring answers rose to 49.0 percent, up from 33.5 percent without it. Fully wrong answers fell to 22.5 percent, down from 35.3 percent in baseline runs.
Catalog record
| CREATOR | 1 st Zhiyuan Cheng, 2 nd Longying Lai, 3 rd Yue Liu, 4 th Kai Cheng, 5 th Xiaoxi Qi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26481 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.16877
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis 1 st Zhiyuan Cheng, 2 nd Longying Lai, 3 rd Yue Liu, 4 th Kai Cheng, 5 th Xiaoxi Qi Abstract Financial analysts face significant challenges extracting information from lengthy 10-K reports, which often exceed 100 pages. This paper presents a Retrieval-Augmented Generation (RAG) system designed to answer questions about S&P 500 financial reports and evaluates the impact of neural reranking on system performance. Our pipeline employs hybrid search combining full-text and semantic retrieval, followed by an optional reranking stage using a cross-encoder model. We conduct systematic evaluation using the FinDER benchmark dataset, comprising 1,500 queries across five experimental groups. Results demonstrate that reranking significantly improves answer quality, achieving 49.0 percent correctness for scores of 8 or above compared to 33.5 percent without reranking, representing a 15.5 percentage point improvement. Additionally, the error rate for completely incorrect answers decreases from 35.3 percent to 22.5 percent. Our findings emphasize the critical role of reranking in financial RAG systems and demonstrate performance improvements over baseline methods through modern language models and refined retrieval strategies. I Introduction I-A Background and Motivation Financial 10-K reports represent comprehensive annual filings required by the U.S. Securities and Exchange Commission (SEC) for publicly traded companies. These documents typically span 100-300 pages and contain detailed information about a company’s financial performance, operations, risk factors, and management discussions. Financial analysts, investors, and researchers routinely consult these reports to make informed investment decisions, assess company performance, and understand business strategies. However, the sheer volume and complexity of these documents pose significant challenges for manual information extraction. Analysts must navigate dense prose, financial tables, and regulatory disclosures to locate specific information relevant to their queries. This process is time-consuming, error-prone, and does not scale efficiently across multiple companies or time periods. Retrieval-Augmented Generation (RAG) [ 1 ] offers a promising solution to this challenge by combining information retrieval with large language model (LLM) generation capabilities. Unlike pure generative approaches that rely solely on parametric knowledge, RAG systems retrieve relevant document passages to ground their responses, reducing hallucinations and improving factual accuracy. This approach is particularly valuable in the financial domain, where precision is paramount and errors can lead to significant financial consequences. I-B Problem Statement Despite the promise of RAG systems, achieving high accuracy in financial question-answering remains challenging. The quality of retrieved information directly impacts the final answer quality—irrelevant or marginally relevant passages can mislead the generation model, resulting in incorrect or incomplete responses. Traditional retrieval methods, including keyword-based search and semantic similarity search, often struggle with the nuanced language and domain-specific terminology prevalent in financial documents. This paper addresses a critical research question: How much does neural reranking contribute to RAG performance in the financial domain? While reranking has shown promise in general-domain information retrieval, its impact on financial document question-answering, particularly with modern LLMs, remains underexplored. I-C Contributions This work makes the following contributions: 1. Complete RAG Pipeline Implementation: We present an end-to-end RAG system for financial report question-answering, processing all 500 S&P 500 companies’ 10-K reports with a hybrid retrieval approach combining full-text search and semantic similarity. 2. Systematic Benchmark Evaluation: We conduct rigorous evaluation using 1,500 queries from the FinDER dataset [ 2 ] , organized into five independent groups of 300 queries each to ensure statistical robustness. 3. Empirical Reranking Analysis: We provide empirical evidence that neural reranking significantly improves performance, achieving a 15.5 percentage point improvement in correctness rate (49.0% vs. 33.5%) and reducing completely incorrect answers by 12.8 percentage points. 4. Performance Improvement Demonstration: Our system achieves approximately 49% correctness on the FinDER benchmark, representing substantial improvement over baseline performance reported in prior work ( \sim 33% correctness), attributable to modern LLMs (GPT-4.1), neural reranking, and hybrid retrieval strategies. I-D Paper Organization The remainder of this paper is organized as follows: Section II reviews related work on RAG systems, financial document analysis, and retrieval enhancement techniques. Section III describes our system architecture in detail. Section IV presents the experimental setup, including dataset description and evaluation methodology. Section V discusses results and provides analysis. Section VI concludes with future research directions. II Related Work II-A Retrieval-Augmented Generation Retrieval-Augmented Generation, introduced by Lewis et al. [ 1 ] , combines parametric memory (pre-trained language models) with non-parametric memory (external document collections) to improve factual accuracy in knowledge-intensive NLP tasks. The RAG paradigm has since been applied to various domains, including open-domain question answering [ 1 ] , dialogue systems, and domain-specific applications. The core RAG architecture consists of two components: a retriever that identifies relevant documents given a query, and a generator that produces natural language responses conditioned on both the query and retrieved documents. This approach addresses limitations of pure generative models, which struggle to access precise knowledge, cannot provide provenance for their responses, and cannot easily update their world knowledge [ 1 ] . II-B Financial Document Analysis Financial document analysis has received increasing attention from the NLP community. The FinDER dataset [ 2 ] , introduced by LinqAlpha, provides 5,703 query-evidence-answer triplets derived from real-world financial inquiries across 10-K filings. Unlike existing financial QA datasets, FinDER reflects realistic analyst workflows with ambiguous, concise queries featuring domain-specific abbreviations and jargon. The dataset’s characteristics present unique challenges: 84.5% qualitative questions, 15.5% quantitative questions (with half requiring multi-step reasoning), and relevant evidence often scattered across multiple document sections [ 2 ] . Multi-hop reasoning in RAG systems has been explored through graph-based and tree-based retrieval approaches [ 12 ] . Prior work reports that advanced models like GPT-4-Turbo achieve only 9% accuracy on closed-book financial questions, highlighting the necessity of retrieval-augmented approaches [ 2 ] . II-C Retrieval Enhancement Techniques Modern retrieval systems employ various techniques to improve result quality: Hybrid Search: Combining traditional keyword-based methods (BM25, full-text search) with dense semantic retrieval addresses complementary strengths—keyword search excels at exact matching while semantic search captures conceptual similarity [ 3 ] . Reciprocal Rank Fusion (RRF): Cormack et al. [ 4 ] introduced RRF as an unsupervised method for combining multiple ranked lists. Using the formula \text{RRFscore}(d)=\sum\frac{1}{k+r(d)} , where k=60 and r(d) is the rank of document d , RRF consistently outperforms individual systems and other meta-ranking methods without requiring training data [ 4 ] . Neural Reranking: Cross-encoder models re-score retrieved candidates by jointly encoding query-document pairs, capturing fine-grained relevance signals missed by first-stage retrievers [ 5 ] . Recent models like Jina Reranker v2 [ 6 ] employ flash attention mechanisms and support multilingual contexts, achieving state-of-the-art performance on benchmark datasets. Context pruning techniques [ 13 ] further refine retrieved context by selecting the most relevant passages, complementing reranking approaches. II-D Gap Analysis While reranking has demonstrated effectiveness in general information retrieval, its impact on financial RAG systems remains underexplored, particularly with modern large language models. Most existing studies focus on individual components (retrieval or generation) rather than evaluating the end-to-end pipeline. Additionally, systematic evaluation with realistic financial queries and proper statistical controls is limited. Our work addresses these gaps through controlled experimentation on a standardized benchmark. III System Architecture Our RAG pipeline consists of two main stages: document processing (offline) and query processing (online). Fig. 1 illustrates the document processing flow, and