Can AI Help with Your Personal Finances?

Oudom Hean, Utsha Saha, Binita Saha{ oudom.hean, utsha.saha, binita.saha } @ndsu.edu North Dakota State University, Fargo, ND, USA · 2024

Read Can AI Help with Your Personal Finances? by Oudom Hean, Utsha Saha, Binita Saha{ oudom.hean, utsha.saha, binita.saha } @ndsu.edu North Dakota State University, Fargo, ND, USA (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-19784.txt.

Introduction

Oudom Hean, Utsha Saha, and Binita Saha present a 2024 study asking whether large language models can give sound personal finance advice. The team tested leading systems, including ChatGPT, Gemini, Claude, and Llama, on US topics such as mortgages, taxes, loans, and investments. The models were graded on the accuracy of their answers to practical money questions.

On average the models scored about 70 percent accuracy. Performance swung widely by topic, and complex questions exposed clear limits. Newer model versions did better than older ones, which the authors read as a sign of growing usefulness. They see promise for both household users and professional advisors. The paper also cautions that today's models still stumble often enough that blind trust in their money advice would be unwise.

Catalog record

CREATOROudom Hean, Utsha Saha, Binita Saha{ oudom.hean, utsha.saha, binita.saha } @ndsu.edu North Dakota State University, Fargo, ND, USA
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-19784

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Can AI Help with Your Personal Finances? Oudom Hean, Utsha Saha, Binita Saha{ oudom.hean, utsha.saha, binita.saha } @ndsu.edu North Dakota State University, Fargo, ND, USA Abstract Abstract In recent years, Large Language Models (LLMs) have emerged as a transformative development in artificial intelligence (AI), drawing significant attention from industry and academia. Trained on vast datasets, these sophisticated AI systems exhibit impressive natural language processing and content generation capabilities. This paper explores the potential of LLMs to address key challenges in personal finance, focusing on the United States. We evaluate several leading LLMs, including OpenAI’s ChatGPT, Google’s Gemini, Anthropic’s Claude, and Meta’s Llama, to assess their effectiveness in providing accurate financial advice on topics such as mortgages, taxes, loans, and investments. Our findings show that while these models achieve an average accuracy rate of approximately 70%, they also display notable limitations in certain areas. Specifically, LLMs struggle to provide accurate responses for complex financial queries, with performance varying significantly across different topics. Despite these limitations, the analysis reveals notable improvements in newer versions of these models, highlighting their growing utility for individuals and financial advisors. As these AI systems continue to evolve, their potential for advancing AI-driven applications in personal finance becomes increasingly promising. Keywords: Large Language Models; Artificial Intelligence; Generative Pretrained Transformers; Personal Finances; Financial Literacy JEL codes: D14; G11; G53 1 Introduction Large Language Models (LLMs), a significant development in artificial intelligence (AI), have emerged as a transformative technology, attracting substantial interest from both industry and academia. These advanced AI systems, trained on extensive datasets, have exhibited impressive capabilities across diverse domains, such as natural language processing, question answering, and content generation ( Brown et al. 2020a ; Devlin et al. 2018a ) . As LLMs continue to evolve and advance, their potential applications, particularly in personal finance, are becoming increasingly relevant and practical, extending well beyond academic research. In this paper, we thoroughly investigate the potential of AI, specifically LLMs, in addressing a spectrum of personal finance issues 1 1 1 In this study, we use the terms AI and LLMs interchangeably. Technically, LLMs are a subset of AI, focusing on processing and generating human language. . We systematically evaluate the responses of LLMs to a range of personal finance topics—such as mortgages, taxes, loans, and investments—within the United States context. We examine several prominent LLMs, including OpenAI’s ChatGPT, Google’s Gemini, Anthropic’s Claude, and Meta/Facebook’s Llama, to discern their comparative efficacy in delivering accurate and consistent financial advice. Our findings indicate that AI models, on average, correctly answer approximately 70 percent of posed questions. Notably, AI performance has shown consistent improvement over time. Older versions of these models answered only about 50-60 percent of questions correctly, while the latest models achieve accuracy rates up to 80 percent. ChatGPT and Claude emerged as the top performers among the various models tested, with Llama being the least accurate. Specifically, the latest versions of ChatGPT, such as ChatGPT 4o, and Claude 3.5 Sonnet, achieved accuracy rates exceeding 74 percent, whereas Llama3 70b maintained a lower accuracy rate of about 65 percent. Furthermore, our results demonstrate that these LLMs provide consistent answers when prompted with the same questions multiple times, alleviating concerns about inconsistent responses. These advancements in AI model performance have significant implications for future applications, particularly in the fields of natural language processing and question-answering systems within the finance industry and education sector. Personal finance is a critical aspect of everyone’s life, yet it is often overlooked due to its complexity and the lack of accessible, personalized guidance ( Lusardi & Mitchell 2014a ) . Many people struggle with budgeting, investing, taxes, and financial planning, leading to suboptimal financial decisions and outcomes ( Jappelli & Padula 2013a ) . Traditionally, individuals have turned to financial advisors for help, but this can be expensive and only within reach for a few. Financial advisors usually charge fees, with robo-advisors at the more affordable end of this range and traditional in-person advisors at the higher end ( Fisch et al. 2019a ) . These fees can mount up swiftly, making professional financial guidance a costly and out-of-reach service for many. Our study reveals that while AI currently faces limitations in assisting with personal finance, particularly with complex questions and variability in performance across topics, its potential for future applications is promising as technology advances. As AI models become more sophisticated, they could play an essential role in helping individuals manage their finances more effectively. One could use AI to provide tailored advice on various personal finance topics, such as budgeting, investments, loans, and tax planning. Moreover, AI has the potential to become an invaluable tool for financial advisors by improving their capability to analyze complex economic data, generate insights, and offer personalized recommendations to clients. The continued advancement of AI in this domain could lead to greater financial literacy and improved financial decision-making for both individuals and professionals. Our paper contributes to the growing body of literature analyzing the application of AI in finance and economics. While most studies focus on a limited set of AI models, they offer valuable insights into specific capabilities and limitations. For instance, Niszczota & Abbas 2023a examine GPT-3.5 and GPT-4 models, evaluating their financial literacy to determine their potential as financial advisors for the public. Using a standardized financial literacy test on core topics like compound interest, tax-advantaged assets, and risk diversification, they find that GPT-4 achieves a near-perfect score of 99%, a marked improvement over GPT-3.5’s 65-66%. This advancement in GPT-4 indicates significant progress in understanding fundamental financial concepts and its potential accuracy in financial advisement. Further extending the capabilities of LLMs, Kim et al. 2024a demonstrate that GPT-4 can outperform human analysts in predicting company earnings from financial statements, particularly in challenging scenarios. This study underscores the model’s capacity for analyzing complex financial information and positioning LLMs as competitive with, and in some cases superior to, human expertise in financial prediction tasks. Similarly, Dowling & Lucey 2023a and Korinek 2023a explore the broader potential of ChatGPT in financial and economic research, emphasizing LLMs’ expanding utility in these domains. Despite these advancements, other studies caution against over-reliance on LLMs for financial advisory roles. Lakkaraju et al. 2023a investigate the reliability and fairness of LLMs in financial advisement by comparing ChatGPT and Bard with SafeFinance, a rule-based chatbot. While LLMs provide plausible responses, they often make substantial errors in retrieving financial information and show inconsistent accuracy across user demographics. In contrast, the more limited SafeFinance offers safer, traceable advice, suggesting that current LLMs may fall short in consistently reliable advisement, especially in high-stakes financial contexts. Other research has focused on AI’s role in dynamic financial markets. Yu et al. 2023a introduce FINMEM, an LLM-based autonomous trading agent designed to enhance decision-making under volatile conditions. FINMEM integrates a profiling module that adjusts risk preferences, a layered memory system that processes time-sensitive financial data, and a decision-making module that synthesizes insights for trading actions. Their findings show that FINMEM can adapt to fluctuating market environments, demonstrating resilience and superior trading performance in complex scenarios. Similarly, Ding et al. 2023a develop a Local-Global model for predicting outcomes in the Chinese A-share market by combining LLM-processed financial news with stock-specific features.

Read the complete text