Finding Optimal Trading History in Reinforcement Learning for Stock Market Trading
Sina Montazeri, Haseebullah Jumakhan, Amir Mirzaeinia · 2025
Read Finding Optimal Trading History in Reinforcement Learning for Stock Market Trading by Sina Montazeri, Haseebullah Jumakhan, Amir Mirzaeinia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12537.txt.
Introduction
How much past price data should a trading program see before it acts? Sina Montazeri, Haseebullah Jumakhan, and Amir Mirzaeinia treat that window length as a tunable choice in a 2025 study of learning-based stock trading.
The team trained trading agents that read price charts as images, growing their view from two weeks of history up to twelve. They ran the test two ways: with company data grouped together in the chart, and with data sorted by type instead. Short windows won when data stayed ungrouped. Once company data was grouped, longer windows gave better results. The test was repeated on two data sets covering the same thirty large firms, adding weight to the pattern.
Catalog record
| CREATOR | Sina Montazeri, Haseebullah Jumakhan, Amir Mirzaeinia |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-12537
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Finding Optimal Trading History in Reinforcement Learning for Stock Market Trading Sina Montazeri, Haseebullah Jumakhan, Amir Mirzaeinia Abstract This paper investigates the optimization of temporal windows in Financial Deep Reinforcement Learning (DRL) models using 2D Convolutional Neural Networks (CNNs). We introduce a novel approach to treating the temporal field as a hyperparameter and examine its impact on model performance across various datasets and feature arrangements. We introduce a new hyperparameter for the CNN policy, proposing that this temporal field can and should be treated as a hyperparameter for these models. We examine the significance of this temporal field by iteratively expanding the window of observations presented to the CNN policy during the deep reinforcement learning process. Our iterative process involves progressively increasing the observation period from two weeks to twelve weeks, allowing us to examine the effects of different temporal windows on the model’s performance. This window expansion is implemented in two settings. In one setting, we rearrange the features in the dataset to group them by company, allowing the model to have a full view of company data in its observation window and CNN kernel. In the second setting, we do not group the features by company, and features are arranged by category. Our study reveals that shorter temporal windows are most effective when no feature rearrangement to group per company is in effect. However, the model will utilize longer temporal windows and yield better performance once we introduce the feature rearrangement. To examine the consistency of our findings, we repeated our experiment on two datasets containing the same thirty companies from the Dow Jones Index but with different features in each dataset and consistently observed the above-mentioned patterns. The result is a trading model significantly outperforming global financial services firms such as the Global X Guru by the established Mirae Asset. 1 Introduction 1.1 Background and Motivation Integrating Deep Reinforcement Learning (DRL) in financial market analysis significantly evolved investment analysis with Deep Learning. DRL combines deep learning and reinforcement learning to offer a sophisticated framework for adapting strategies in the dynamic financial domain. It allows a deep learning model to effectively decipher complex patterns in historical market data often overlooked by traditional quantitative models. It is no secret that financial markets are inherently complex and influenced by economic trends and geopolitical events. Therefore, traditional financial modeling often struggles to adapt to these ever-changing conditions. However, with its direct learning from data and adaptive strategies, DRL presents a promising solution to these challenges. With its autonomous learning ability and continual adaptation to the financial environment, it leverages historical market data to identify complex relationships and patterns. 1.2 Overview of Our Previous Work In recent years, significant progress has been made in applying deep reinforcement learning (DRL) to stock trading strategies. For instance, Wang et al. proposed a parallel multi-module DRL algorithm that effectively captures both current market conditions and long-term dependencies using fully connected and LSTM layers [ 11 ] . Zhang et al. introduced an automated stock trading system based on the Proximal Policy Optimization algorithm, modeling trading as a Markov decision process [ 21 ] . Additionally, Huang et al. demonstrated the importance of integrating market sentiment data to enhance the performance of DRL models in trading [ 8 ] . Liu et al. developed an end-to-end trading strategy using a multi-view environment representation neural network, incorporating a Long Memory mechanism to improve decision-making [ 19 ] . Lastly, Li et al. focused on adaptive trading strategies using Gated Recurrent Units to capture time-series data effectively [ 20 ] . These studies collectively highlight the potential of DRL in creating robust and adaptive trading strategies. Liu et al. significantly advanced Deep Reinforcement Learning in Finance by developing platforms such as FinRL-Meta [ 10 ] . This platform is a comprehensive tool for training and evaluating data-driven reinforcement learning agents within several simulated financial markets, offering a robust benchmarking system for algorithm comparison and facilitating the simulation of complex market conditions. The FinRL platform enables researchers to refine and test the efficacy of various DRL strategies, and it has been pivotal in democratizing access to sophisticated financial simulation tools and propelling research in financial analysis. FinRL uses environments that offer broad simulation capabilities. These specialized environments, such as ABIDES-Gym [ 1 ] , provide the necessary infrastructure that allows FinRL to create discrete event simulations explicitly tailored for financial markets. ABIDES-Gym extends the OpenAI Gym interface to accommodate the complex dynamics of financial trading, allowing for a nuanced replication of market mechanisms and agent interactions. This level of detail will enable researchers and practitioners to explore the impact of individual agent behaviors and market responses, thus enhancing the understanding of market microstructure and agent-based modeling. The framework also streamlines the model training process on financial datasets, epitomizing the intersection of DRL and high-performance computing. It Leverages distributed computing resources to reduce training times significantly and optimizes computational workflows to enable the application of complex DRL models to extensive financial tasks. Their efforts have led to the creation of scalable and efficient financial models. Our previous work [ 14 ] demonstrated the efficiency and capability of CNNs when used as policies for deep reinforcement learning. We utilized the FinRL platform to conduct experiments on CNNs as a significantly improved policy to FinRL’s original proposition. We also showed [ 15 , 13 ] that rearranging the stock market features used in the FinRL platform to group them per company could benefit the mode’s performance. This study also utilizes the FinRL platform with its original dataset, containing features generated through traditional Technical Analysis used in Finance. It also uses the new dataset introduced in FinRL Meta, which contains statistically engineered features such as Simple Moving Average (SMA), momentum, and rate of change. Building upon these foundational studies, our research aims to bridge the gap between CNN architecture optimization and financial market analysis. By introducing a systematic approach to temporal window selection, we seek to enhance the adaptability and performance of DRL models in capturing complex market dynamics. 2 Objectives of the Current Study So far, we have presented the literature and the setting in which our study operates. The primary objective of our research is to explore the effects of changing the temporal window of a Convolutional Neural Network (CNN) used as a policy in a FinRL. By progressively expanding the observation period, beginning with a concise two-week window and incrementally enlarging it by two weeks in each iteration and culminating in twelve weeks, we aim to observe and analyze the performance of our model as its temporal window changes in the FinRL platform. This iterative window expansion is designed to examine the impact of different temporal scales on the model’s performance. This process enables a comprehensive analysis of how varying lengths of financial data affect the model’s predictive capabilities, offering insights and an opportunity to optimize the temporal granularity for financial market analysis. Our study also examines the arrangement of feature vectors within these expanding windows to better understand the model-market dynamics. Furthermore, we contrast the model’s performance across these different temporal windows to discern patterns in market behavior and model performance. In our study, short-term windows, particularly the initial