On the Effectiveness of Classical Regression Methods for Optimal Switching Problems

Martin Andersson, Benny Avelin, Marcus Olofsson · 2025

Read On the Effectiveness of Classical Regression Methods for Optimal Switching Problems by Martin Andersson, Benny Avelin, Marcus Olofsson (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-15436.txt.

Introduction

Martin Andersson, Benny Avelin, and Marcus Olofsson test classical regression methods in two thousand twenty-five on optimal switching problems, where a system must jump between states at the right times. Exact theory demands solving hard equation systems, so practitioners lean on simulation algorithms like Longstaff-Schwartz. The question is which regressor to plug in.

The team tried linear models, tree ensembles, nearest neighbors, and neural networks on four benchmark problems, including fifty-dimensional ones. Several simple methods stayed stable across problem types and beat the neural networks tested. Nearest-neighbor regression did consistently well with almost no tuning. The authors prove error bounds for it and show a dimension-reduction step lets it scale to high dimensions. Simple tools, tuned lightly, win here.

Catalog record

CREATORMartin Andersson, Benny Avelin, Marcus Olofsson
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE92 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-15436

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On the Effectiveness of Classical Regression Methods for Optimal Switching Problems Martin Andersson, Benny Avelin, Marcus Olofsson Abstract . Simple regression methods provide robust, near-optimal solutions for optimal switching problems in dimensions ranging from 1 to 50. While the theory requires solving intractable PDE systems, the Longstaff-Schwartz algorithm with classical approaches like k k -NN achieves excellent switching decisions without extensive hyperparameter tuning. Testing eight regression approaches on four benchmark problems, we find that simple methods maintain stable performance across diverse problem characteristics, even after extensive neural network optimization. The contaminated training targets inherent to backward induction-where each target contains both approximation bias and Monte Carlo noise-actually favor these robust approaches over more complex alternatives such as neural networks. Further, we establish concentration bounds for k k -NN regression under jump-diffusion dynamics and show that PCA enables k k -NN to scale to high dimensions. For practitioners: simple, minimally-tuned regression methods offer reliable performance for computationally demanding switching problems. 1. Introduction Managers of complex production systems face a challenging problem: how to decide when to switch between different operational modes when facing uncertain market conditions and significant switching costs? This question arises across industries, from energy production choosing between different fuel sources to manufacturing plants shifting between different product lines. The mathematical framework of Optimal Switching (OS) provides a principled approach to these decisions, yet a significant gap exists between theoretical solutions and practical implementation. These switching decisions carry substantial economic weight. In energy markets, the difference between optimal and suboptimal switching can represent a substantial loss in annual revenue. A poorly-timed switch can lock a facility into an inefficient mode just as conditions shift, while excessive switching erodes profits through transition costs. While the mathematical foundations of OS are well-established through solutions to systems of parabolic PDEs with interconnected obstacles, these theoretical solutions remain computationally intractable for realistic problems. These PDE systems are difficult to manage even numerically, and without explicit solutions we cannot construct switching strategies to support actual decision-making. Consider a manager facing a just 10-dimensional state space, (comprised of multiple correlated fuel costs, energy prices, weather conditions and forecasts). Traditional finite differences with just 100 grid points in each dimension would require 10^{20} points to solve for, exceeding any current computational abilities. A classical approach to similar problems, particularly in the context of American options, involves simulation and regression methods, see the seminal papers [ LS01 ] and [ TVR99 ] . These involve simulating data and then using dynamic programming principles, recursively defining the problem, and iteratively applying regression methods. These methods shift the computational burden from solving high-dimensional PDEs to a statistical estimation problem: given simulated paths of the underlying stochastic factors, can we learn the continuation values needed for optimal decisions? This approach faces its own dimensional challenges but offers more flexibility in addressing them. While this approach has been studied extensively in option pricing, its effectiveness in the more complex OS setting with multiple sequential decisions rather than a single exercise time is underexplored, both theoretically and practically. Given the complexity of OS problems, one might expect that sophisticated machine learning methods, particularly deep neural networks, would be necessary for optimal performance. Our findings challenge this intuition. Through a combination of new theoretical analysis and comprehensive experiments, we demonstrate that classical methods, in particular k -nearest neighbor regression, can match or exceed the performance of complex alternatives while maintaining stable decision boundaries, requiring minimal hyperparameter tuning, and offering computational efficiency. Our contributions in this paper are the following: • Theoretical guarantees : We establish error bounds on regression with k -NN regression in the Longstaff-Schwartz algorithm under both diffusion and jump-diffusion dynamics, covering subgaussian and subexponential tails ( Section 5 ) • Comprehensive empirical comparison : We comprehensively compare a range of parametric and non-parametric regression models, ranging from linear models to neural networks, on a suite of management problems from the OS literature, including high-dimensional ( \sim 50 ) ones ( Section 6 ). • Simple methods excel : We find that very simple models, like k -NN, can achieve almost optimal management decisions, including in high dimensions, when paired with straight-forward dimensionality reduction such as Principal Component Analysis (PCA) ( Section 7 ) 2. Literature overview The mathematical theory of OS is well studied and contributions have been made from a long list of researchers. Without any aspiration to give a complete list, contributions can be found in, e.g., [ BGSS22 , BJK10 , DHP09 , EAF17 , EAH09 , HM13 , HT10 , Kha16 , LNO14a , LNO14b , LO22 , Mar16 , Per18 , Per20 , Per21 ] . On a more fundamental level, the concepts of backward stochastic differential equations and viscosity solutions for PDEs are of critical importance for the development of OS theory, and we refer to [ CIL92 , PP90 , PP92 ] for introductions to these concepts. As the standard setting has been understood, many alternative formulations of the OS have been considered, e.g., game settings [ HMM19 ] , delayed decisions and memory [ Per18 , Per20 ] , partial information [ LNO15 , Olo18 ] , state constraints [ Kha16 ] , and numerical methods based on simulations [ ACLP14 , LOO21 ] . When it comes to applications, the literature is far less developed. This contrasts to what could be expected given the importance of closely related topics in financial mathematics; see, e.g., [ EKPQ97 ] , and [ PS06 ] for an overview. Typically, the applications considered are low-dimensional and rather unrealistic from a practitioner point of view; see, e.g., [ BØ94 , BS85 ] . A literature survey indicates that, besides [ ACLP14 , CL08 , CL10 , LOO21 ] concrete applications of optimal switching in at least semi-realistic settings are hard to come by. A major challenge in the quest for explicit value functions of OS problems is the quickly increasing dimensionality of the problem when applied to more realistic settings. This dimensionality becomes an issue as most numerical methods suffer heavily from the “curse of dimensionality”. Starting with [ Til93 ] , a classical approach to overcoming this difficulty in option pricing is to ignore the connection to PDEs and instead exploit the stochastic formulation of the optimization problem as an expected value of some functional of a stochastic process; the value function can then be found using Monte Carlo methods. Several attempts have been made to improve this idea using regression, some of the most well-known attempts being [ BG97 , Car96 , HK04 , LS01 , Rog02 , TVR99 ] . There have been a few efforts to apply more general regression methods to the problem; see [ CL08 , ACLP14 , BCN23 ] , but the literature is scarce. Regarding machine learning and neural networks (NNs) for stochastic control problems, attempts were made already a decade ago to study the optimal stopping problem using a NN with a single hidden layer, see [ KKT10 ] . Recently, much attention has been directed to this problem from researchers in various fields, including mathematics, finance, data science, and physics. In particular, the closely related case of optimal stopping has been extensively studied during the last

Read the complete text