StockBot 2.0: Vanilla LSTMs Outperform Transformer-based Forecasting for Stock Prices Journal: arXiv
Shaswat Mohanty · 2026
Read StockBot 2.0: Vanilla LSTMs Outperform Transformer-based Forecasting for Stock Prices Journal: arXiv by Shaswat Mohanty (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-00197.txt.
Introduction
Shaswat Mohanty, writing alone, reports that a plain old network beats fancy new ones at forecasting stock prices. StockBot 2.0 grows out of his earlier recurrent system. It tests attention, convolution, and recurrent models side by side. All models share one fair experimental setup. Default settings keep the contest honest and simple.
Extensive tests give a clear verdict. A carefully built vanilla LSTM wins on accuracy. Its buy and sell calls also stay more stable. Transformer style models offer more flexibility on paper. That freedom does not pay off here. Data are cut into single day steps, so samples stay scarce. In that regime, recurrent bias fits the task better. Mohanty stresses a wider lesson for finance. Architecture fit can matter more than novelty. Data efficiency is a real edge in noisy markets.
Catalog record
| CREATOR | Shaswat Mohanty |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 25 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-00197
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
StockBot 2.0: Vanilla LSTMs Outperform Transformer-based Forecasting for Stock Prices Journal: arXiv Shaswat Mohanty Abstract Accurate forecasting of financial markets remains a long-standing challenge due to complex temporal and often latent dependencies, non-linear dynamics, and high volatility. Building on our earlier recurrent neural network framework, we present an enhanced StockBot architecture that systematically evaluates modern attention-based, convolutional, and recurrent time-series forecasting models within a unified experimental setting. While attention-based and transformer-inspired models offer increased modeling flexibility, extensive empirical evaluation reveals that a carefully constructed vanilla LSTM consistently achieves superior predictive accuracy and more stable buy/sell decision-making when trained under a common set of default hyperparameters. These results highlight the robustness and data efficiency of recurrent sequence models for financial time-series forecasting, particularly in the absence of extensive hyperparameter tuning or the availability of sufficient data when discretized to single-day intervals. Additionally, these results underscore the importance of architectural inductive bias in data-limited market prediction tasks. 1 Introduction Predicting stock price movements remains a central challenge in quantitative finance due to the inherent non-linearity, noise, and structural volatility of financial time-series data. Traditional statistical and econometric models often rely on linear assumptions and struggle to capture the dynamic dependencies that govern market behavior Murphy 1999 , Tsai and Hsiao 2010 . Deep learning methods, particularly recurrent neural networks (RNNs) and their variants such as long short-term memory (LSTM) networks, have demonstrated improved performance in modeling temporal dependencies and nonlinear correlations Di Persio et al. 2016 , Fischer and Krauss 2018 , Selvin et al. 2017 . Mohanty et al. 2022 has presented the benefits of stacking multiple LSTMs through their ablative studies, demonstrating superior predictive performance of stock prices, especially within the energy sector. A large corpus of such research exists in the literature and has shown promise in predicting stock price movements, but they fall short in capturing long-term estimations due to hysteresis Chen 2023 as well as high volatility index (VIX) fluctuations in the short-term Finnegan et al. 2024 . Subsequent advances in sequence modeling have introduced alternative architectures such as transformers Vaswani et al. 2017 and temporal convolutional networks (TCNs) Lea et al. 2017 , which enable more efficient and expressive representations of long-range dependencies. Transformers, through their self-attention mechanism, can flexibly model contextual relevance across time, while TCNs leverage dilated causal convolutions to achieve comparable receptive fields with fewer parameters. These architectures have shown strong generalization across diverse time-series applications, motivating their adaptation for financial forecasting Liu et al. 2019 , Zhang et al. 2022 , Wang et al. 2022 . In this work, we extend our previous StockBot framework by incorporating a diverse family of temporal forecasting models, including recurrent, transformer-inspired attention-based, convolutional, and hybrid attention–recurrent architectures, within a unified evaluation setting. This design enables a controlled comparison of model expressiveness, generalization behavior, and downstream trading interpretability across multiple market sectors. By evaluating all architectures under identical training protocols and default hyperparameter settings, our analysis emphasizes the role of architectural inductive bias in shaping predictive performance and decision stability, as assessed through a simple buy/sell trading strategy. The remainder of this paper is organized as follows. Section 2 describes the dataset construction, normalization procedures, and model architectures. Section 3 presents comparative results across model families, along with an evaluation of StockBot’s trading performance. Finally, Section 4 summarizes key findings and outlines directions for future work. 2 Data and Methods 2.1 Data Preparation The dataset and preprocessing steps follow the procedure established in our previous work Mohanty et al. 2022 . Historical stock price data are obtained from Yahoo! Finance Yah , focusing on equities listed on the NYSE and NASDAQ. The adjusted closing price is used as the primary variable for prediction. A sliding window approach constructs supervised learning sequences. For a given past \_ history of k days, the model predicts the stock price for the subsequent forward \_ look days. While most experiments use forward \_ look =1 , the framework generalizes to multi-step prediction. The data span from 2010 - 2020 and are split into 80\% training and 20\% testing subsets, with temporal ordering preserved. All series are normalized using z-score scaling based on the training set statistics to prevent data leakage. This data selection and preprocessing are consistent with the earlier evaluation in [ Mohanty et al. 2022 ] . Past Observations \mathbf{x}_{t-k:t} Temporal Encoder LSTM, Transformer, TCN, TFT, Informer Latent Temporal Representation Prediction Head Forecast \hat{\mathbf{x}}_{t+1:t+h} Autoregressive Update Figure 1: Generic temporal forecasting framework shared by all models considered in this work. Past observations are encoded into a latent temporal representation, which is mapped to future predictions. The dashed loop indicates optional autoregressive forecasting for multi-step prediction. 2.2 Model Architectures We evaluate a family of temporal deep learning models under a unified framework, each implemented as a subclass of a base LSTM \_ Model . This enables consistent data handling, training, and evaluation across different model types. All models can perform two types of forecasting at the time of evaluation: (i) autoregressive forecasting by recursively feeding the most recent prediction as the next input. This allows evaluation of stability and drift behavior over multiple prediction horizons, and (ii) teacher-forcing forecasting, where ground-truth values are supplied at each step, providing an upper bound on predictive accuracy in the absence of compounding errors. 2.2.1 LSTM and Attention-Enhanced LSTM The baseline model uses a stacked long short-term memory (LSTM) network, which captures temporal dependencies through gated recurrent units. The forward pass of an LSTM layer at time step t is governed by Eqs. ( 1 )–( 6 ): \displaystyle f_{t} \displaystyle=\sigma_{g}(W_{f}x_{t}+U_{f}h_{t-1}+b_{f}), (1) \displaystyle i_{t} \displaystyle=\sigma_{g}(W_{i}x_{t}+U_{i}h_{t-1}+b_{i}), (2) \displaystyle o_{t} \displaystyle=\sigma_{g}(W_{o}x_{t}+U_{o}h_{t-1}+b_{o}), (3) \displaystyle\tilde{c}_{t} \displaystyle=\sigma_{c}(W_{c}x_{t}+U_{c}h_{t-1}+b_{c}), (4) \displaystyle c_{t} \displaystyle=f_{t}\odot c_{t-1}+i_{t}\odot\tilde{c}_{t}, (5) \displaystyle h_{t} \displaystyle=o_{t}\odot\sigma_{h}(c_{t}), (6) where x_{t} is the input vector to the LSTM unit; f_{t} , i_{t} , and o_{t} denote the forget, input, and output gate activations; c_{t} and h_{t} are the cell and hidden states, respectively. The functions \sigma_{g}(\cdot) and \sigma_{c}(\cdot)=\sigma_{h}(\cdot) correspond to the sigmoid and hyperbolic tangent activations. The matrices W , U , and biases b are learned during training. An attention-augmented variant introduces a learnable attention layer over LSTM outputs, enabling adaptive weighting of relevant time steps. Given the sequence of hidden states \{h_{1},h_{2},\dots,h_{T}\} produced by the stacked LSTM layers, a Bahdanau-style attention mechanism is applied to compute a context vector. The query vector is defined as the temporal average of the hidden states, q=\frac{1}{T}\sum_{t=1}^{T}h_{t}. (7) The attention score for each time step is computed as e_{t}=v^{\top}\tanh\!\left(W_{1}h_{t}+W_{2}q\right), (8) where W_{1} and W_{2} are learnable projection matrices and v is a learnable attention vector. The normalized attention weights are obtained via \alpha_{t}=\frac{\exp(e_{t})}{\sum_{k=1}^{T}\exp(e_{k})}. (9) The resulting context vector is a weighted sum of the hidden states, c=\sum_{t=1}^{T}\alpha_{t}h_{t}. (10) Finally, the context vector is mapped to the prediction horizon through a fully connected layer, \hat{y}=W_{o}c+b_{o}. (11) 2.2.2 Temporal Convolutional Network (TCN) The Temporal Convolutional Network (TCN) models sequential dependencies using causal one-dimensional convolutions, ensuring that predictions at time t depend only on inputs from time steps \leq t . Given an input sequence \mathbf{X}=\{x_{1},x_{2},\dots,x_{T}\} with x_{t}\in\mathbb{R}^{d_{\text{in}}} , the first causal convolutional layer computes \mathbf{H}^{(1)}_{t}=\phi\!\left(\sum_{k=0}^{K-1}W^{(1)}_{k}\,x_{t-k}+b^{(1)}\right), (12) where K is the kernel size, \phi(\cdot) denotes the ReLU activation, and W^{(1)}_{k} and b^{(1)} are the learnable convolutional