Quantifying the Risk–Return Tradeoff in Forecasting

Philippe Goulet Coulombe · 2026

Read Quantifying the Risk–Return Tradeoff in Forecasting by Philippe Goulet Coulombe (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-09712.txt.

Introduction

Goulet Coulombe argued that average forecast accuracy tells only half the story about a forecaster. The paper treats forecast losses against a benchmark as a return series, then grades that series with finance-style risk measures: Sharpe, Sortino, and Omega ratios plus drawdown gauges. A new Edge Ratio captures how often a model delivers uniquely useful predictions ahead of the forecasting frontier.

Applied to United States macroeconomic forecasting, the framework compared econometric models, machine-learning models, the TabPFN foundation model, and the Survey of Professional Forecasters. Beating professionals on average accuracy proved feasible, but beating them on risk-adjusted terms proved much harder, since they rarely fail catastrophically. Selected machine-learning methods still showed attractive risk profiles for specific targets.

Catalog record

CREATORPhilippe Goulet Coulombe
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE101 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.09712

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Quantifying the Risk–Return Tradeoff in Forecasting Philippe Goulet Coulombe Abstract Average forecast accuracy is not the same as forecast reliability. I treat forecast loss differentials relative to a benchmark as a return series. These return series are then evaluated using risk-adjusted performance measures from finance, including the Sharpe ratio, Sortino ratio, Omega ratio, and drawdown-based metrics. I also introduce the Edge Ratio capturing a model’s propensity to deliver uniquely informative predictions relative to the forecasting frontier. I apply this framework to U.S. macroeconomic forecasting, comparing econometric benchmarks, machine learning models, a foundation model (TabPFN), and the Survey of Professional Forecasters. While it is often feasible to beat professional forecasters in terms of average accuracy, it is much harder to beat them on a risk-adjusted basis . They rarely exhibit catastrophic failures and often achieve high Edge Ratios, plausibly reflecting the value of contextual judgment. Nonetheless, selected machine learning methods deliver attractive risk profiles for specific targets. The framework naturally extends to meta-analyses across targets, horizons, and samples, illustrated with a density forecast evaluation and the M4 competition. 1 Introduction Macroeconomic forecasters typically compare models using loss-based criteria such as root mean squared error (RMSE), mean absolute error (MAE), log scores, and equal predictive ability tests—most prominently the Diebold–Mariano test. These tools address two central questions: does a model reduce forecast errors relative to a benchmark, and is this improvement statistically significant? The literature on forecast evaluation is vast and sophisticated, but its unifying focus remains average performance gains and testing whether they differ from zero. Central banks, finance ministries, and asset managers invest heavily in forecasting infrastructure: human capital, computing resources, and model development pipelines. The return on this investment is not a lower RMSE number but a more reliable path of forecast losses —consistently reduced uncertainty about the economic outlook. These users have naturally asymmetric preferences: they tolerate, even appreciate, episodes where a model dramatically outperforms expectations, but they cannot afford episodes of severe underperformance. A machine learning model that beats the benchmark by 15% on average yet falters badly during the 2008 financial crisis or the 2022 inflation surge offers a less attractive risk-return profile than its average performance suggests. It is not realistic to require a new model to dominate a benchmark at every realization. Yet this unavoidable fact introduces risk. Two models may deliver similar RMSE ratios and both beat a benchmark with statistical significance while exhibiting markedly different patterns of gains and losses over time, resulting in different risk profiles. This distinction is difficult to discern from the results tables that typically accompany forecasting studies. Forecasting Models as a Risky Investment. This paper imports risk-adjusted performance measures from finance to address this gap. In portfolio management, returns are never judged by their mean alone. The Sharpe ratio ( Sharpe, 1966 ) is the ratio of average return to return volatility, measuring improvement per unit of risk. The Sortino ratio ( Sortino and van der Meer, 1991 ) refines this by focusing on the semi-standard deviation—the volatility of negative returns only—which penalizes episodes where the model underperforms the benchmark while remaining agnostic to upside variability. The Omega ratio ( Keating and Shadwick, 2002 ) captures the full distribution of gains versus losses, comparing total upside to total downside without imposing distributional assumptions; this is particularly appealing in the forecasting context, where the distribution of loss differentials can be highly non-Gaussian. Finally, maximum drawdown tracks the worst cumulative underperformance from peak to trough, identifying prolonged episodes of sustained forecast failure Magdon-Ismail and Atiya, 2004 ; Chekhlov et al., 2005 . These metrics have direct analogues for forecast evaluation. Defining the “return” at time t as the loss differential r_{t}=L_{t}^{\text{benchmark}}-L_{t}^{\text{model}} , one can compute Forecast Sharpe, Forecast Sortino, and Forecast Omega ratios that summarize not just how much a model improves upon the benchmark, but how reliably it does so. As I show in the results, some models delivering very similar MAE/RMSE improvements, and sometimes similar levels of statistical significance, feature strikingly different risk-adjusted ratios. That is, while there could be ambiguity with respect to standard metrics, the risk-adjusted ones provide a sharper answer as to which model is likely operationally preferable. Importantly, the framework is not completely estranged from traditional tests of statistical significance. In fact, the simplest ratio—the Sharpe ratio—coincides with the Diebold-Mariano t -statistic (up to a scaling constant) when forecast errors are serially uncorrelated. Moreover, I show in Section 2.3 that the Diebold-Mariano test can itself be interpreted as a utility-based statistic: the long-run variance in its denominator penalizes not only for the variance of loss differentials but also for their autocorrelation, implying that persistent deviations in performance are treated as more costly than transient failures. From an institutional standpoint, this asymmetric treatment of clustered outcomes has intuitive appeal: a prolonged streak of underperformance is more likely to materialize into sub-optimal policy decisions. The Edge Ratio. I also introduce the Edge Ratio, a novel metric that measures whether a model delivers unique predictive advantage relative to all available alternatives. Unlike benchmark-relative metrics, the Edge Ratio tracks how often a model reaches the forecasting frontier and how decisively it does so, while penalizing the magnitude of regret when it fails to be the best. Conceptually, it is an Omega ratio with a moving benchmark: at each point in time, the benchmark is the best-performing alternative model rather than a fixed reference. A model with high Edge Ratio provides value that cannot be replicated by other methods in the forecaster’s toolkit without incurring a significant amount of realized regret with respect to competitors. Meta-Analyses. Forecasting studies routinely evaluate new methods across many targets, horizons, geographies, and out-of-sample periods. Results are typically summarized by averages of, e.g., RMSEs, but substantial heterogeneity often lurks beneath. From a model development perspective, knowing which method wins on average is useful, but so is knowing which methods are robust—and which ones excel on a few targets while failing catastrophically on others. This matters especially because published studies tend to emphasize favorable cases; the question is whether these gains come at the cost of severe underperformance elsewhere. The same risk-adjusted metrics apply here: the Sortino ratio, for instance, asks how large the average improvement is relative to the magnitude of the worst failures across the design space. For practitioners moving from research to implementation, this is precisely the relevant question: if the setup differs slightly from the original study, how much variability in performance should one expect? Since model devolvement is itself a risky investment, such metrics can bring useful information to the table. Three Applications. I develop this framework and apply it to three settings. ML/AI versus SPF. The first application evaluates risk-adjusted forecast performance over time. The exercise follows a standard recursive out-of-sample design from 2007 to the present, split into pre- and post-COVID eras. The model pool is inspired by Goulet Coulombe et al., 2025 and includes standard econometric benchmarks (autoregressive, factor-augmented), regularized linear methods (Ridge, Kernel Ridge), and established ML approaches (Random Forests, gradient boosting, neural networks). Three recent contenders are of particular interest: (i) LGB+, a hybrid boosting algorithm that alternates tree ensembles with linear corrections Goulet Coulombe, 2026 ; (ii) the Survey of Professional Forecasters (SPF),

Read the complete text