Autocorrelation effects in a stochastic-process model for decision making via time series
Tomoki Yamagami, Mikio Hasegawa, Takatomo Mihana, Ryoichi Horisaki, Atsushi Uchida · 2026
Read Autocorrelation effects in a stochastic-process model for decision making via time series by Tomoki Yamagami, Mikio Hasegawa, Takatomo Mihana, Ryoichi Horisaki, Atsushi Uchida (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-05559.txt.
Introduction
Yamagami and team analyze a stochastic-process model for the two-armed bandit problem, where a threshold and a Markov signal evolve jointly. Test results show a setting-based pattern, with negative autocorrelation best when rewards are rich and positive autocorrelation best when rewards are poor. Negative autocorrelation wins when the winning chances sum to more than one, and positive when the sum is less than one. Results do not depend on autocorrelation when the sum equals one. The findings point toward bandit solving for reinforcement learning in wireless comms.
Catalog record
| CREATOR | Tomoki Yamagami, Mikio Hasegawa, Takatomo Mihana, Ryoichi Horisaki, Atsushi Uchida |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60477 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.05559
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Autocorrelation effects in a stochastic-process model for decision making via time series Tomoki Yamagami, Mikio Hasegawa, Takatomo Mihana, Ryoichi Horisaki, Atsushi Uchida Abstract 1 Department of Information and Computer Sciences, Saitama University, 255 Shimo-Okubo, Sakura-ku, Saitama City, Saitama 338–8570, Japan. 2 Department of Electrical Engineering, Tokyo University of Science, 6–3–1, Niijuku, Katsushika-ku, Tokyo 125–8585, Japan. 3 Department of Information Physics and Computing, The University of Tokyo, 7–3–1 Hongo, Bunkyo-ku, Tokyo 113–8656, Japan. ∗ [email protected] † [email protected] Abstract Decision makers exploiting photonic chaotic dynamics obtained by semiconductor lasers provide an ultrafast approach to solving multi-armed bandit problems by using a temporal optical signal as the driving source for sequential decisions. In such systems, the sampling interval of the chaotic waveform shapes the temporal correlation of the resulting time series, and experiments have reported that decision accuracy depends strongly on this autocorrelation property. However, it remains unclear whether the benefit of autocorrelation can be explained by a minimal mathematical model. Here, we analyze a stochastic-process model of the time-series-based decision making using the tug-of-war principle for solving the two-armed bandit problem, where the threshold and a two-valued Markov signal evolve jointly. Numerical results reveal an environment-dependent structure: negative (positive) autocorrelation is optimal in reward-rich (reward-poor) environments. These findings show that negative autocorrelation of the time series is advantageous when the sum of the winning probabilities is more than 1 , whereas positive autocorrelation is useful when the sum of the winning probabilities is less than 1 . Moreover, the performance is independent of autocorrelation if the sum of the winning probabilities equals 1 , which is mathematically clarified. This study paves the way for improving the decision-making scheme for reinforcement learning applications in wireless communications and robotics. 1 Introduction The rapid progress of artificial intelligence and machine learning have stimulated research into optics- and photonics-based computing systems [ 13 , 18 , 33 ] . One promising direction is photonic decision making, in which physical properties of light are exploited to solve multi-armed bandit (MAB) problems [ 31 ] . The MAB problem represents one of the simplest settings of reinforcement learning. Reinforcement learning is a framework for solving tasks through interaction between a decision maker (an agent) and the object of its action (an environment). After the agent takes an action, the environment probabilistically returns a reward as feedback, and the goal of reinforcement learning is to maximize the cumulative reward over a sequence of actions. In general, the agent does not know in advance which behavior is more effective for obtaining higher rewards; thus, the agent must explore different actions and learn how to optimize its behavior. Moreover, the best choice found by the agent may not always remain optimal because the scenario can change depending on external conditions and the agent’s action history, which is modeled as states. In contrast, the MAB problem assumes a simple setting with respect to the state: the optimal choice does not change, although it is hidden from the agent. Specifically, the MAB problem considers repeated selections among multiple slot machines (arms) in the environment. Each selection of an arm corresponds to an action in the reinforcement-learning framework, and the selected arm yields a reward with an arm-dependent probability, which we call the winning probability of the arm. The agent aims to maximize the cumulative reward through a series of arm selections. A central challenge in the MAB problem is the exploration–exploitation trade-off [ 21 , 7 ] . As described above, the agent needs to try each possible selection multiple times to learn which arm should be chosen to optimize the total reward. On the other hand, the number of selections is limited in general, meaning that excessive exploration can waste opportunities to choose the arm that is expected to provide the best outcome. Therefore, balancing exploration and exploitation is crucial in the MAB problem. To achieve this balance, various algorithms have been developed, such as the \varepsilon -greedy [ 37 ] , softmax [ 37 , 7 ] , upper confidence bound [ 4 ] , and quantum walks [ 41 ] . Given this broad applicability and the need for efficient decision making under uncertainty, MAB problems have also attracted sustained attention as a practical framework. Originally formulated in the context of sequential allocation in clinical trials [ 39 ] , MAB problems are now employed in applications such as online advertising [ 24 ] and wireless communications [ 20 ] . Photonic implementations based on semiconductor lasers, referred to as the laser-chaos-based decision maker, have attracted significant attention for solving the MAB problem [ 26 ] . This system utilizes chaotic time series generated by a semiconductor laser with optical feedback [ 27 , 36 ] to physically implement the decision-making process. Chaotic laser signals provide quasi-irregular oscillations, and chaotic laser dynamics have been studied for secure key distribution [ 5 , 3 , 32 , 43 ] , random number generation [ 40 , 2 , 15 ] , and reservoir computing [ 1 , 8 , 14 ] . In the context of decision making, the chaotic signal is fed to a decision-making scheme established on the tug-of-war (ToW) principle [ 16 , 17 ] . This algorithm, inspired by slime-mold behavior, dynamically adjusts a threshold to regulate arm-selection frequencies in two-armed bandit environments. In this scheme, the signal amplitude is binarized by comparison with the threshold, corresponding to which arm is selected, and ultrafast (GHz-order) decision-update rates have been demonstrated [ 26 ] . Building on this baseline, later studies have extended photonic bandit solvers toward larger-scale and more practical settings by harnessing additional physical degrees of freedom and control parameters [ 11 , 30 , 12 , 23 , 34 , 6 ] . This ultrafast adaptive property is also a crucial resource for the research fields that demand rapid decision making such as wireless communication [ 38 ] , real-time model selection [ 14 ] , and optimizing Ising machine [ 42 ] . A key empirical finding is that the autocorrelation properties of the chaotic signal have a strong influence on decision-making accuracy: sampling the signal at delay times that induce negative autocorrelation reliably improves performance compared with time series lacking such autocorrelation [ 26 ] . This improvement has also been confirmed using Fourier surrogate data that preserve autocorrelation while removing higher-order temporal dependencies [ 28 ] . Related motivations appear in other contexts as well, such as code-division multiple access (CDMA), where minimizing correlation is essential for improving performance [ 19 , 9 ] . To advance theoretical understanding of autocorrelation-driven performance, a recent work has introduced a stochastic-process model that reproduces the time evolution of decision making based on comparing time series and threshold, as in the laser-chaos-based decision maker [ 29 ] . In this model, the threshold is regarded as the position of a random walker whose transition probabilities depend on the sign of the time series; the signal values are governed by a two-valued Markov chain. The prior study [ 29 ] reports that the model demonstrates how negative autocorrelation can enhance performance under specific environmental conditions. However, the evidence is limited to only three cases regarding the