From Rattle to Roar: Optimizer Showdown for MambaStock on S&P 500
Maria Garmonina, Alena Chan · 2025
Read From Rattle to Roar: Optimizer Showdown for MambaStock on S&P 500 by Maria Garmonina, Alena Chan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-04707.txt.
Introduction
Alena Chan and Maria Garmonina compare training optimizers for a stock forecaster in a 2025 paper. Their model is a sequence network built for index return prediction on a major United States index. Gradient-smoothing and adaptive-rate optimizers gave the lowest test errors. A newer optimizer trained notably faster but with a bumpier loss curve.
To get both virtues, they introduce a new optimizer family that calms the fast method's oscillations while keeping its speed. The paper is a compact benchmark study, framed as a showdown, aimed at practitioners tuning sequence models for finance.
Catalog record
| CREATOR | Maria Garmonina, Alena Chan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-04707
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Rattle to Roar: Optimizer Showdown for MambaStock on S&P 500 Maria Garmonina, Alena Chan Abstract We evaluate the performance of several optimizers on the task of forecasting S&P 500 Index returns with the MambaStock model. Among the most widely used algorithms, gradient-smoothing and adaptive-rate optimizers (for example, Adam and RMSProp) yield the lowest test errors. In contrast, the Lion optimizer offers notably faster training. To combine these advantages, we introduce a novel family of optimizers, Roaree, that dampens the oscillatory loss behavior often seen with Lion while preserving its training speed. Source code can be found at https://github.com/maria-garmonina/snakes-on-trading-floor. 1 Introduction Interest in state-space models has surged as a competitive alternative to both simple statistical approaches and resource-intensive transformers. The recently introduced Mamba architecture has demonstrated strong performance on long time-series sequences gu2024mamba , making it particularly well-suited for financial data. In this work, we investigate the use of Mamba-inspired selective state-space models for forecasting financial time series. Specifically, we focus on predicting future returns of the S&P 500 Index with the MambaStock model shi2024mambastock . Our primary contribution is a thorough evaluation of several widely adopted optimization algorithms and the introduction of a novel family of optimizers tailored to this task – with the goal of improving both predictive accuracy and training efficiency on sequential financial data. 1.1 Problem Statement We explore how the choice of optimizer affects both training speed and forecasting accuracy in the MambaStock model. Training any machine learning model is an optimization problem, and a variety of algorithms exist for approximating the optimal parameters. We examine the trade-offs among predictive error, convergence rate, and overall training time – each metric is critical for practical trading applications. 1.2 Objectives and Scope This paper has two objectives: 1. Compare the performance of the MambaStock model under a range of optimizers. 2. Leverage the speed vs. MSE trade-off to design and validate a new optimizer family. 2 Literature Review 2.1 Related Work Traditional time-series models such as ARIMA assume linear relationships between past and future values and often fail to capture sudden market shocks, leading to inaccurate price predictions. Hybrid LSTM-CNN architectures have improved sensitivity to market shifts zhong2021data , but their computational demands can be prohibitive. Similarly, Kalman filters offer interpretable price trend filtering yet remain constrained by linearity assumptions. More recently, state-space models (SSMs) have gained traction for their ability to process very long sequences efficiently. Unlike transformers, which require O(n^{2}) attention operations and thus struggle with long-history inference (recalculating attention over extended price histories is impractical), SSMs scale linearly with input length and maintain constant memory usage. This makes them well-suited for real-time financial forecasting, especially in resource-constrained environments where training or deploying a full transformer might be prohibitive. Gu et al. gu2022s4 introduced the Structured State-Space Sequence (S4) model, which reparameterizes transition matrices to capture long-range dependencies with efficient computation. Building on S4, Gu & Dao gu2024mamba proposed Mamba, adding an optimized recurrent update block scan and a hardware optimization by parallelizing computation via convolution. Mamba’s ability to vary its SSM parameters based on the input enables it to focus on relevant information and forget the rest. Thus, Mamba often matches the accuracy of attention-based models while offering linear time complexity and fast inference. 2.2 Identification of Gaps in Existing Research For MambaStock shi2024mambastock , Shi adapted the Mamba architecture to the stock price prediction task. It uses historical series data to predict future price. While this model brought gains in accuracy, further optimizations of this model have not been studied much. While the Lion optimizer (Chen et al., 2023) chen2023lion has emerged as a memory-efficient alternative to Adam-style methods by leveraging a sign-based momentum, optimizers specifically designed for financial time series within Mamba architectures remain largely unexplored – most prior work has focused on transformer-based models. 3 Methodology 3.1 Data Collection and Preprocessing The dataset contains weekly S&P 500 Index observations from 2000 to 2019 with historical returns, 10 engineered technical - analysis signals, three fundamental valuation ratios, and two crowd - sentiment scores (Table 1 ) – collected by Zhong et al. zhong2021data . The prediction target is the forward one - week return ( return_t_plus_1 ), obtained by shifting the adjusted closing price one week ahead. Attribute Description Target return_t_plus_1 Forward one-week return: (\texttt{adj\_close}_{t+1}/\texttt{adj\_close}_{t})-1 History return_t Return for the current week Technical Indicators adx Average Directional Index – measures trend strength (higher = stronger trend) adxr Average Directional Movement Rating – a smoothed version of ADX trix Triple Exponential Average – a moving average of a moving average of a moving average, capturing momentum cci Commodity Channel Index – identifies overbought/oversold conditions macdh MACD Histogram – the difference between the MACD line and its signal line rsi_14 Relative Strength Index (14-day) – measures recent gains vs. losses (RSI > 70 overbought, < 30 oversold) kdjk K-line of the Stochastic Oscillator – another momentum indicator wr_14 Williams %R (14-day) – overbought/oversold oscillator atr Average True Range – measures volatility atr_percent ATR as a percentage of price – normalized volatility measure Valuation Ratios PbRatio Price-to-Book Ratio – price divided by book value per share PeRatio Price-to-Earnings Ratio – price divided by earnings per share PsRatio Price-to-Sales Ratio – price divided by revenue per share Sentiment Scores spsentiment Crowd sentiment (e.g., from Seeking Alpha, StockTwits) sentiment General daily sentiment score (positive/negative/neutral) Table 1: Summary of features in the weekly S&P 500 Index dataset We split the dataset into train, validation, and test parts: the test set contains the last 100 weeks of observations, the training set is 90% of the remaining data, with 10% being the validation data (used for tracking the validation loss over the training epochs). We keep all of our datasets causal – not letting the model peek into the future. 3.2 Model Selection We utilized the MambaStock model, which has proven itself for successful Chinese stock market predictions, with hidden size set to 64 and 2 layers. No architectural changes were made – the optimizer choice is the sole experimental factor. 3.3 Baseline Optimizers We benchmark eight widely - used methods: SGD, SGD with Momentum, Nesterov, RMSProp, Adagrad, Adam, AdamW, and Lion. 3.4 Roaree: a Smooth - Lion Family Due to the computational efficiency of the Lion optimizer, we based our Roaree algorithms on Lion. Lion’s original update \theta_{t}=\theta_{t-1}-\eta_{t}\ (\operatorname{sign}(c_{t})+\lambda\theta_{t-1})\ ,\ c_{t}=\beta_{2}m_{t-1}+(1-\beta_{2})g_{t} is non - differentiable . Roaree replaces \operatorname{sign}(\cdot) with a smooth surrogate s_{\kappa}(\cdot) controlled by a curvature hyper - parameter \kappa>0 (please refer to Algorithm 1): \displaystyle s\_{\kappa}^{\text{tanh}}(x) \displaystyle=\tanh(\kappa x) (1) \displaystyle s\_{\kappa}^{\text{atan}}(x) \displaystyle=\tfrac{2}{\pi}\arctan(\kappa x) (2) \displaystyle s\_{\kappa}^{\text{softsign}}(x) \displaystyle=\frac{\kappa x}{1+|\kappa x|} (3) \displaystyle s\_{\kappa}^{\text{sigmoid}}(x) \displaystyle=2\sigma(\kappa x)-1 (4) \displaystyle s\_{\kappa}^{\text{erf}}(x) \displaystyle=\operatorname{erf}(\kappa x) (5) \displaystyle s\_{\kappa}^{\text{norm}}(x) \displaystyle=\frac{x}{\sqrt{x^{2}+1}}\quad (6) Setting \kappa\!\to\!\infty recovers the hard sign, whereas a small \kappa yields a nearly linear step. We sweep \kappa\in\{10,\,100,\,1000\} . 3.5 Benchmarking Protocol and Evaluation Metrics All optimizers were trained for 64 epochs on MambaStock with identical random seeds for reproducibility. Standard optimizers were explored on the larger grid: \;\text{lr}\in\bigl\{10^{-5},\;5\!\times\!10^{-5},\;10^{-4},\;5\!\times\!10^{-4},10^{-3},\;5\!\times\!10^{-3},\;10^{-2},\;5\!\times\!10^{-2}\bigr\}, \text{wd}\in\bigl\{0,10^{-4},\;5\!\times\!10^{-4},\;10^{-3},\;5\!\times\!10^{-3},\;10^{-2},\;5\!\times\!10^{-2},\;10^{-1}\bigr\}. For the Roaree family we narrowed the grid on learning rate and weight decay in order to explore six surrogates and three curvature values: \;\text{lr}\in\bigl\{10^{-4},\;10^{-3},\;10^{-2}\bigr\},\qquad\;\text{wd}\in\bigl\{0,\;10^{-3},\;10^{-2},\;10^{-1}\bigr\}. For every configuration we log the following metrics: • Average Epoch Time – wall - clock seconds