FinSphere, a Real-Time Stock Analysis Agent Powered by Instruction-Tuned LLMs and Domain Tools
Shijie Han, Jingshu Zhang, Yiqing Shen, Kaiyuan Yan, Hongguang Li · 2025
Read FinSphere, a Real-Time Stock Analysis Agent Powered by Instruction-Tuned LLMs and Domain Tools by Shijie Han, Jingshu Zhang, Yiqing Shen, Kaiyuan Yan, Hongguang Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-12399.txt.
Introduction
Shijie Han and co-authors wrote a 2025 paper on FinSphere, an agent that writes real-time stock analyses. Stock models built on large language models have two weak spots. No objective score grades how good a stock analysis is. And the analyses lack the depth of professional work.
The team contributes three things. First, a scoring framework that grades stock analyses against set criteria. Next, a dataset built by industry professionals to teach the models the craft. Last, the agent itself, which combines live price feeds, quantitative tools, and a language model tuned on the craft dataset. In tests, the agent beat both general and finance-tuned language models. It also beat earlier agent systems, even ones given live data and worked examples. The team says the combination improves both the quality and the usefulness of the analyses.
Catalog record
| CREATOR | Shijie Han, Jingshu Zhang, Yiqing Shen, Kaiyuan Yan, Hongguang Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-12399
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
FinSphere, a Real-Time Stock Analysis Agent Powered by Instruction-Tuned LLMs and Domain Tools Shijie Han, Jingshu Zhang, Yiqing Shen, Kaiyuan Yan, Hongguang Li Abstract Current financial large language models (FinLLMs) struggle with two critical limitations: the absence of objective evaluation metrics to assess the quality of stock analysis reports and a lack of depth in stock analysis, which impedes their ability to generate professional-grade insights. To address these challenges, this paper introduces FinSphere, a stock analysis agent, along with three major contributions: (1) AnalyScore, a systematic evaluation framework for assessing stock analysis quality, (2) Stocksis, a dataset curated by industry experts to enhance LLMs’ stock analysis capabilities, and (3) FinSphere, an AI agent that can generate high-quality stock analysis reports in response to user queries. Experiments demonstrate that FinSphere achieves superior performance compared to both general and domain-specific LLMs, as well as existing agent-based systems, even when they are enhanced with real-time data access and few-shot guidance. The integrated framework, which combines real-time data feeds, quantitative tools, and an instruction-tuned LLM, yields substantial improvements in both analytical quality and practical applicability for real-world stock analysis. 1 Introduction Large Language Models (LLMs) have demonstrated remarkable capabilities in comprehending and processing natural language, extending their influence across various domains, including finance ( Li et al., 2023b ) . By leveraging their language comprehension capabilities, these models have exhibited exceptional performance in various financial applications, including sentiment analysis ( Liu et al., 2024b ; Zhang et al., 2023a ) and information extraction from unstructured financial texts ( Li et al., 2024 ; Huang et al., 2023 ) . The advent of finance-specific LLMs such as FinBERT ( Yang et al., 2020 ; Liu et al., 2021 ) , BloombergGPT ( Wu et al., 2023 ) , and PIXIU ( Xie et al., 2023 ) has further enhanced the capacity to process financial data effectively. These advancements have laid the foundation for developing more sophisticated financial analysis tools and shifted how investors interact with market data ( Krause, 2023 ; Nie et al., 2024 ) . These AI-powered systems have broadened access to professional financial insights, allowing retail investors to benefit from advanced analysis once reserved for institutions. As LLM technology continues to evolve, there is a growing expectation for these models to handle more complex financial tasks, particularly in stock analysis and real-time financial question-answering ( Yang et al., 2023b ; Zhao et al., 2024 ) . This has led to the development of tool-augmented agents that integrate LLMs’ natural language understanding with specialized financial tools, significantly enhancing automated financial analysis and interactive question-answering capabilities ( Ding et al., 2024 ; Zhang et al., 2024 ) . However, LLM-based financial QA systems still face substantial challenges in effectively interpreting and utilizing the outputs of these tools to generate high-quality analytical responses. Two primary obstacles include the absence of systematic evaluation frameworks to assess their performance in stock analysis, as well as the lack of specialized datasets for fine-tuning LLMs’ analytical reasoning capabilities. Moreover, existing research is constrained by LLMs’ heavy reliance on historical data, such as GPT-4o’s dependence on its pre-trained knowledge for generating responses ( Ni et al., 2024 ; Bhat and Jain, 2024 ) . This limitation in accessing and processing real-time financial data and domain-specific information restricts their ability to fully capture the dynamic and evolving nature of financial markets, posing a critical challenge for real-time financial question-answering systems. We address these limitations through three contributions: • AnalyScore : A comprehensive evaluation framework designed to systematically assess the accuracy, relevance, and analytical depth of LLM-driven stock question-answering. • Stocksis : A specialized dataset curated by industry experts to enhance LLMs’ financial question-answering and stock analysis capabilities. • FinSphere : A real-time stock question-answering agent capable of generating high-quality stock analysis reports in response to user queries. Prompt with Background Information (input) Expert-Edited Analysis (label) System Instruction. You are a proficient investment assistant, adept at stock analysis tasks. You demonstrate a robust aptitude for comprehending user-provided [Question] and meticulously organizing them into accurate and standardized analysis reports. You utilize the [Background Information] provided by the corresponding quantitative tools to respond to users in a systematic and standardized manner. Background Information (from quantitative tools). • Capital Flow Analysis: TF Securities is currently under moderate control by market leaders, indicating a positive institutional outlook. There is a strong willingness for margin financing inflow. Institutional quarterly shareholding has increased, suggesting institutions are optimistic about the company’s development, warranting attention to medium-term opportunities. • [Other specialized analysis from quantitative tools] Question. Please analyze TF Securities. From a capital flow perspective, the 5-day cumulative DDX of -14.865 indicates significant selling pressure from major market participants and strong bearish sentiment. Despite moderate institutional control and determined margin financing inflows showing relatively active main capital, short-term capital flows warrant caution. [Key Summaries of supporting conclusions drawn from other Specialized Analysis] In conclusion, TF Securities shows recent unusual market activity, largely driven by heightened trading within the securities sector. Technical analysis reveals short-term opportunities with a bullish engulfing pattern, though weak fundamentals warrant caution for medium to long-term positions. Investors should monitor market trends and company developments, as the stock benefits from sector momentum despite underlying fundamental concerns. Table 1: An abbreviated example of Stocksis. The complete content is detailed in Table 5 . Guide LLM to provide a comprehensive analysis based on specialized analyses returned from quantitative tools. The average cost of comprehensive analysis written by experts is $10 per query, with detailed cost breakdowns available in Appendix F . Our experiments demonstrate that FinSphere, by integrating real-time financial databases, specialized quantitative tools, and an instruction-tuned LLM optimized for financial question-answering, significantly outperforms both general-purpose and domain-specific LLMs, as well as existing agent-based systems. This superior performance holds even when baseline models are augmented with real-time background information and few-shot prompting, validating the effectiveness of our integrated approach to real-time financial question-answering and stock market analysis. 2 AnalyScore and Stocksis Stock market analysis is becoming increasingly complex, necessitating the integration of diverse data sources and sophisticated analytical approaches. While LLMs show promise in financial analysis, two key gaps remain: the lack of standardized evaluation frameworks for AI-generated stock analyses and the scarcity of high-quality training data. To address these, we introduce AnalyScore , a systematic evaluation framework, and Stocksis , a comprehensive dataset designed to enhance LLMs’ stock analysis capabilities. 2.1 AnalyScore: A Comprehensive Evaluation Framework for Stock Analysis Reports To address the lack of systematic evaluation standards in stock analysis, we introduce AnalyScore , a domain-specific framework co-developed with industry experts. It combines established financial assessment criteria with insights from LLM-generated content evaluation. AnalyScore employs a structured scoring system across four dimensions, with a total score of 100 points: • Conclusion (20 pts): Clarity, relevance, and personalization of investment recommendations. • Content (45 pts): Depth, coherence, and professionalism of analytical reasoning. • Expression (15 pts): Organization, fluency, and linguistic clarity. • Data (20 pts): Breadth, accuracy, and effective use of supporting quantitative data. This multi-dimensional evaluation ensures comprehensive coverage of both qualitative judgment and quantitative rigor. At present, AnalyScore is used exclusively by human experts to assess model-generated analyses. In future work, we aim to integrate AnalyScore