Sentiment-Aware Stock Price Prediction with Transformer and LLM-Generated Formulaic Alpha

Qizhao Chen, Hiroaki Kawashima · 2025

Read Sentiment-Aware Stock Price Prediction with Transformer and LLM-Generated Formulaic Alpha by Qizhao Chen, Hiroaki Kawashima (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-04975.txt.

Introduction

Qizhao Chen and Hiroaki Kawashima build a stock forecast system in 2025 that pairs market mood with machine made trading formulas. Traders have long crafted such formulas by hand, hunting for signals in price data. That craft is slow, and the signals decay as markets shift, so fresh ones are always needed.

Their system lets a language model draft a spread of formulas from past prices, standard indicators, and mood scores drawn from news and posts for the target stock and its peers. A transformer model then weighs those signals to call the next price move. Machine drafting could give a steady supply of fresh signals at low cost. The pair reports how the joined system fares against standard forecast methods.

Catalog record

CREATORQizhao Chen, Hiroaki Kawashima
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-04975

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Sentiment-Aware Stock Price Prediction with Transformer and LLM-Generated Formulaic Alpha Qizhao Chen, Hiroaki Kawashima Abstract Traditionally, traders and quantitative analysts address alpha decay by manually crafting formulaic alphas, mathematical expressions that identify patterns or signals in financial data, through domain expertise and trial-and-error. This process is often time-consuming and difficult to scale. With recent advances in large language models (LLMs), it is now possible to automate the generation of such alphas by leveraging the reasoning capabilities of LLMs. This paper introduces a novel framework that integrates a prompt-based LLM with a Transformer model for stock price prediction. The LLM first generates diverse and adaptive alphas using structured inputs such as historical stock features (Close, Open, High, Low, Volume), technical indicators, sentiment scores of both target and related companies. These alphas, instead of being used directly for trading, are treated as high-level features that capture complex dependencies within the financial data. To evaluate the effectiveness of these LLM-generated formulaic alphas, the alpha features are then fed into prediction models such as Transformer, LSTM, TCN, SVR, and Random Forest to forecast future stock prices. Experimental results demonstrate that the LLM-generated alphas significantly improve predictive accuracy. Moreover, the accompanying natural language reasoning provided by the LLM enhances the interpretability and transparency of the predictions, supporting more informed financial decision-making. K eywords LLM \cdot Stock Price Prediction \cdot Sentiment Analysis \cdot Transformer \cdot Deep Learning 1 Introduction Financial markets are complex systems where prices are influenced by company fundamentals, technical patterns, sentiment, and external events. Predicting stock prices remains a major challenge in both finance and machine learning. Traditional models like regression and tree-based methods often struggle to capture the highly non-linear relationships present in financial time series. Deep learning models, especially Transformers, have recently shown strong results in sequence modeling tasks. Their ability to learn long-range dependencies makes them suitable for predicting price movements [ 1 , 2 , 3 ] . However, using only raw financial features often leads to noisy and unstable predictions. To improve model performance, many researchers and practitioners create special features called "alpha factors" or "alphas." These are formulas designed to capture patterns that can predict future returns. Equation 1 is an example of a formulaic alpha. \text{Alpha}_{t}=\left(\frac{P_{t}-\text{SMA}_{20,t}}{\text{SMA}_{20,t}}\right)\cdot\log(1+\text{Sentiment}_{t}) (1) This alpha factor combines a technical indicator with sentiment information. The term \frac{P_{t}-\text{SMA}_{20,t}}{\text{SMA}_{20,t}} captures the percentage deviation of the current price P_{t} from its simple moving average of 20 days, indicating short-term overbought or oversold conditions. The logarithmic sentiment term \log(1+\text{Sentiment}_{t}) adjusts this signal based on the sentiment of the market for the day, where higher sentiment values amplify the signal and negative or neutral sentiment dampens it. By integrating both technical momentum and investor mood, this alpha aims to forecast short-term returns more effectively than using technical or sentiment data alone. In practice, alpha factors do not remain effective forever. Over time, as more traders discover and exploit an alpha, it tends to lose its predictive power. This phenomenon is known as alpha decay [ 4 ] . Alpha decay is a major problem in quantitative finance because it forces investors to constantly search for new and original alphas. Relying on some old alphas can quickly lead to underperformance as markets evolve and adapt. Traditionally, designing new alphas has been a manual process, driven by human intuition and experience. Analysts and quants study historical patterns, test hypotheses, and refine formulas through trial and error. However, this approach is slow, expensive, and limited by human creativity. In recent years, Large Language Models (LLMs) like GPT have shown strong abilities in reasoning, summarization, and idea generation. This opens up the possibility of using LLMs to automate the search for new alphas by generating formulaic trading signals based on structured financial data [ 5 ] . Prompt-based LLMs offer several advantages in this context. First, they can quickly generate a large number of candidate formulas following clear instructions (prompts). Second, the generated alphas are often interpretable because they are written in a human-readable form, unlike the hidden features learned by deep neural networks. Third, prompt-based LLMs are flexible. By adjusting the prompt design, users can guide the LLM to focus on different aspects of the market, such as momentum, mean reversion, or sentiment. This flexibility allows for fast exploration of many trading ideas without heavy manual effort. Lastly, since LLMs can be fine-tuned or adapted over time, they can continuously assist in generating fresh alphas to fight alpha decay. In this paper, we propose a new framework that uses prompt-based LLMs to automatically generate formulaic alphas from stock features, technical indicators, and sentiment scores. These generated alphas serve as additional inputs to a vanilla Transformer model that predicts next-day stock prices. By introducing new machine-generated alphas, it is expected that alpha decay will slow down and stronger predictive signals will be maintained over time. Following our previous work [ 6 ] , we also include the sentiments of related companies as one of the input features. Related companies are defined as those that have connections with the target companies, such as business partners, suppliers, or competitors. The reason is that the news sentiments of these companies can also affect the stock movement of the target company. In our earlier study, we applied Named Entity Recognition (NER) to extract company names from news articles about target firms. The underlying assumption is that companies frequently co-occurring in the same articles as the target company are likely to have some form of business relationship or relevance. In summary, this paper addresses the following two research questions (RQs): • RQ1: Can sentiment information from related companies be effectively incorporated into alpha formula generated by LLM? • RQ2: Does sentiment-aware LLM-generated alpha signals improve stock price prediction accuracy using deep learning techniques such as Transformer? The remainder of this paper is structured as follows. Firstly, related work is listed (Section 2). The methodology is then described (Section 3). In Section 4, the experimental results are presented. Section 5 presents further discussion of the findings. Finally, Section 6 concludes the paper. 2 Related Work 2.1 Stock Price Prediction with Machine Learning Models Predicting stock prices has attracted a lot of research attention. Traditional methods include statistical models such as ARIMA [ 7 ] and GARCH [ 8 ] . These models assume certain patterns in the data, such as stationarity or constant volatility, but real financial data often violates these assumptions. As a result, these models struggle when applied to complex market environments. With the rise of machine learning, models such as Linear Regression [ 9 ] , Support Vector Machines [ 10 ] , Random Forests [ 11 ] , and Gradient Boosting [ 12 ] have been widely used for stock prediction. These models can capture non-linear patterns better than traditional statistical approaches. However, they still require careful feature engineering, and their ability to model sequential data is limited. Deep learning methods have shown strong promise for stock prediction. Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks were among the first deep learning models used for financial time series [ 13 , 14 , 15 ] . They are designed to capture temporal dependencies in data. While LSTMs improve over RNNs by addressing

Read the complete text