The Uncertainty of Machine Learning Predictions in Asset Pricing Yuan Liao, Xinjie Ma, Andreas Neuhierl, Linda Schilling Abstract Machine learning in asset pricing typically predicts expected returns as point estimates, ignoring uncertainty. We develop new methods to construct forecast confidence intervals for expected returns obtained from neural networks. We show that neural network forecasts of expected returns share the same asymptotic distribution as classic nonparametric methods, enabling a closed-form expression for their standard errors. We also propose a computationally feasible bootstrap to obtain the asymptotic distribution. We incorporate these forecast confidence intervals into an uncertainty-averse investment framework. This provides an economic rationale for shrinkage implementations of portfolio selection. Empirically, our methods improve out-of-sample performance. We thank Damir Filipovic, Semyon Malamud and seminar audiences at École Polytechnique Fédéral de Lausanne, Washington University in St. Louis, Wolfe Research. 1 Introduction Pension funds, wealth advisors, and hedge funds worldwide employ predictive methods to forecast asset returns and construct optimal portfolios that aim to maximize client returns. Recently, machine learning (ML) models have gained prominence in predicting asset returns, selecting portfolios, and estimating stochastic discount factors, with significant success in these areas. ML techniques, by capturing complex and nonlinear relationships in financial data, are particularly well-suited for enhancing portfolio management decisions. For example, within the mean-variance portfolio framework, ML methods are increasingly used to estimate expected returns and (co)variances, often leading to more effective portfolio allocations. The literature consistently demonstrates the effectiveness of machine learning in these and other applications (e.g., Gu, Kelly, and Xiu (2020) ; Bianchi, Büchner, and Tamoni (2021) ; Cong, Tang, Wang, and Zhang (2021) ; Kelly, Malamud, and Zhou (2021) ; Patton and Weller (2022) ; Didisheim, Ke, Kelly, and Malamud (2023) ; Filipovic and Schneider (2024) ). Despite the success of machine learning in asset pricing, existing literature typically treats ML predictions as point estimates and conducts asset pricing analyses as if they were true values, overlooking the associated uncertainty. This is surprising, given that uncertainty about input parameters is widely acknowledged as critical in portfolio selection (e.g., DeMiguel, Garlappi, and Uppal (2009) ), and Garlappi, Uppal, and Wang (2007) show that incorporating forecast uncertainty in mean-variance portfolio allocation leads to distinct economic insights. However, quantifying prediction uncertainty in ML forecasts, particularly with neural networks, remains a complex challenge, limiting their broader application in asset pricing. This paper addresses this gap by rigorously quantifying uncertainty in ML predictions and incorporating it into portfolio selection, specifically by constructing forecast confidence intervals (FCIs) for return predictions from neural networks. We provide two methods for constructing forecast confidence intervals: one based on closed-form approximations and the other on the bootstrap. A key theoretical contribution of this paper is our proof that ML-based forecast methods exhibit an asymptotic distribution independent of the specific ML model used to generate the forecast. For example, the asymptotic distribution of neural network forecasts is not specific to neural networks. This implies that, under appropriate technical conditions, simpler machine learning methods—such as Fourier series, particularly those with closed-form estimators—can be utilized to construct confidence intervals. These intervals can be applied to algorithmically more complex machine learning estimators, whose confidence intervals would otherwise be intractable. Building on this insight, we derive an analytic formula for the ML forecast standard error, which is straightforward to calculate and paves the way for constructing the FCI. For the second method, we propose a novel k -step bootstrap approach to simulating the asymptotic distribution of the ML forecast. This method overcomes the substantial computational burden associated with conventional bootstrap procedures, which require repeatedly fully training multiple neural networks from scratch. In contrast, the k -step bootstrap, originally developed by Davidson and MacKinnon (1999) and Andrews (2002) , starts with the previously trained neural network and retrains it for additional k steps on a bootstrap resample. Through the k -step method, a pre-trained neural network can achieve a high level of training accuracy as quantified by the validity of the FCIs’ coverages, even if k is relatively small. We verify that both methods give correct asymptotic coverage probabilities for ML forecasts of expected returns, while effectively addressing cross-sectional dependence. Empirically, the two methods produce qualitatively similar results. We recommend that researchers compare the two as robustness check. In some applications, it may also be prudent to follow a conservative approach and use the larger of the two standard errors. Furthermore, through extensive simulations, we demonstrate that alternative bootstrap approaches, such as bootstrapping across assets or jointly across assets and time series, fail to produce a valid forecast confidence interval for machine learning forecasts. In the second part of the paper, we apply our methods for quantifying estimation uncertainty to two standard problems in portfolio selection. We adopt the view of an uncertainty-averse (UA) investor. In the first application, we build on the framework introduced by Garlappi et al. (2007) : we treat the expected return in the classic mean-variance optimization problem as an unknown parameter varying within a confidence interval. Then, we find the optimal portfolio weights for the worst-case scenario of the mean-variance utility, varying the mean within the given interval. Extending the characterization by Garlappi et al. (2007) to the machine learning context, we apply the proposed neural network FCI to forecast the expected return of the UA-portfolio. We show that the solution can be formulated as an \ell_{1} -penalized regression problem. Our results establish that portfolio weights exhibit a “non-participation” region for risky assets, meaning an uncertainty-averse investor may choose not to invest in a risky asset if uncertainty about the asset’s expected value exceeds a certain level. In addition, this region expands as the investor’s uncertainty about the expected value increases. In contrast, the non-participation region is not present in the standard mean-variance approach which treats the expected returns as known. This insight offers a clear rationale for employing shrinkage approaches in portfolio selection, as developed by Ao, Li, and Zheng (2019) and Kozak, Nagel, and Santosh (2020) . Empirically, the behavior of UA-portfolio of individual stocks aligns closely with our theoretical characterization, and generates higher Sharpe ratios than benchmarks that disregard forecast uncertainty. In our second application, we focus on selecting assets with statistically significantly positive expected returns and constructing long-only portfolios. This exercise is particularly relevant for mutual funds, which do not employ short positions and thus face exactly this challenge of building long-only portfolios. The forecast confidence interval facilitates the selection of individual securities with significantly positive expected returns while controlling the false discovery rate, thereby mitigating the multiple testing problem as documented by Barras, Scaillet, and Wermers (2010) and Harvey and Liu (2020) . Empirically, our long-only investment strategy, which accounts for forecast uncertainty, yields higher out-of-sample average returns and similar standard deviations compared to benchmark methods that select assets without considering forecast uncertainty. In addition, it yields economically significant alphas when regressing the returns on various risk factors. Our result holds in a setting where the machine learning model is not over-parameterized. Recently, Kelly et al. (2021) and Didisheim et al. (2023) derive results about the out-of-sample properties of portfolios in the context of overparameterized models where the number of parameters far exceeds the sample size. They show the interesting virtue of complexity in asset pricing models. Developing forecast confidence intervals in the virtue of complexity regime is an important question which we leave for future research. Related Literature The literature on machine learning in asset pricing has grown rapidly in recent years, such as Kelly, Pruitt, and Su (2019) ; Kozak, Nagel, and Santosh (2020) ; Freyberger, Neuhierl, and Weber (2020) ; Chen, Pelger, and Zhu (2020) ; Baba-Yara, Boyer, and Davis (2022) and Li, Rossi, Yan, and Zheng (2024) . Recently, several papers in asset pricing have made progress in the theoretical analysis of machine learning predictions. Fan, Ke, Liao, and Neuhierl (2022) and Jagannathan, Liao, and Neuhierl (2023) propose the so-called “period-by-period” ML and developed the FCI around the forecast indices, which relies on estimating latent risk factors. Period-by-period learning requires training the machine learning model each period. In this paper, we instead focus exclusively on the popular “pooled machine learning” approach, which is the most common in forecasting applications and does not involve estimating factors. 1 1 1 Pooled machine learning pools the data over the cross-sectional and time series dimension in estimation. Allena (2021) is one of the first papers in asset pricing that formally develops confidence intervals for risk premia predicted using machine learning. He proposes a Bayesian ML approach that flexibly draws from posterior distributions of the predicted risk premia and demonstrates that it successfully yields a confidence-based strategy with insightful economic interpretations. The Bayesian approach leverages the fact that posterior distributions, deduced from properly specified priors, can provide valid confidence intervals. In this paper, we take a different approach. In particular, we do not pursue Bayesian estimation. One of the two methods we develop for the forecast confidence interval is based on a derived analytic formula for the ML standard error. Meanwhile, we move to expand the insights obtained by Allena (2021) and Garlappi et al. (2007) . In the context of portfolio selection under uncertainty aversion, we show that machine learning confidence intervals lead to qualitatively different investment behavior relative to the standard mean-variance model. Another strand of literature in asset pricing studies model uncertainty (e.g., Avramov (2002) ; Anderson and Cheng (2016) ; Bianchi et al. (2024) ). In this line of research, no stance is taken on the “correct model,” as the goal is often to achieve robust portfolio allocations and predictions via Bayesian model averaging. Researchers therefore usually specify a probability distribution over the different models but do not derive the forecast uncertainty within a given model. In theoretical econometrics, research on machine learning uncertainty is still at a very early stage. There are, however, few papers that rigorously develop the forecast standard error in the i.i.d. setting. In a seminal paper, Chen and White (1999) set the stage for a rigorous analysis of the distributional properties of neural network predictions. In the field of econometric program evaluation, there is a popular method known as “doubly robust ML inference”, which aims to develop asymptotic confidence intervals for some structural parameters in econometric models (e.g., Chernozhukov, Chetverikov, Demirer, Duflo, Hansen, Newey, and Robins (2018) ). These methods develop sophisticated procedures that require so-called orthogonal moment conditions and cross-fitting, which are quite different from the typical implementation of ML models in asset pricing. In addition, both approaches rely on the assumption of i.i.d. (or weakly dependent) data, which is invalid in asset pricing due to the strong cross-sectional dependence driven by common risk factors. This dependence is at the heart of the theoretical challenge in asset pricing. This paper proposes new approaches that explicitly account for this dependence and develops forecast confidence intervals with frequentist guarantees. 2 Machine Learning Forecast Confidence Intervals 2.1 The Model Consider the excess return of a portfolio: z_{t+1}=\sum_{i=1}^{N}w_{i}y_{i,t+1}, where y_{i,t+1},i=1,...,N denote the excess return for base asset i from t to t+1 , relative to the risk-free rate. The portfolio weights w_{i} are assumed to be known at time t , but they may vary over time. This contains the special case of an individual asset, i.e., z_{t+1}=y_{1,t+1} , or a broad market index. At period T , the objective is to forecast the expected excess return of the portfolio: z_{T+1|T}=\mathbb{E}(z_{T+1}|\mathcal{F}_{T}) where \mathcal{F}_{T} denotes the information set up to time T . To do so, researchers observe a matrix of asset-specific characteristics (features), x_{t-1}=(x_{1,t-1},...,x_{N,t-1}), where x_{i,t-1} is an d -dimensional vector of characteristics for asset i at time t-1 , such as momentum, volatility, financial liabilities. In this setting, machine learning regression, e.g. Gu et al. (2020) ; Bianchi et al. (2021) have become a very successful and popular methodology to obtain point predictions. ML regressions build on the nonparametric model, y_{i,t}=g(x_{i,t-1})+e_{i,t} with an unknown function g(\cdot):\mathbb{R}^{d}\to\mathbb{R} , where e_{i,t} is the error term. The unknown function is learned by pooling all observed data (cross-sectionally and over time) and solving a least squares problem: \widehat{g}(\cdot)=\arg\min_{g\in\mathcal{G}}\sum_{i=1}^{N}\sum_{t=1}^{T}(y_{i,t}-g(x_{i,t-1}))^{2}, (2.1) where the optimal solution is searched for in a function space \mathcal{G} that typically corresponds to a specific machine learning method. Once \widehat{g}(\cdot) has been computed, the expected excess return is predicted by plugging in the most recent characteristic x_{i,T} and constructing a portfolio: \widehat{z}_{T+1|T}:=\sum_{i=1}^{N}w_{i}\widehat{g}(x_{i,T}). (2.2) This approach is widely used in both academic research and industry applications, making it the primary forecasting method analyzed in this paper. The choice of the function space \mathcal{G} corresponds to the specific ML method to forecast excess returns. For example, neural networks, random forests, boosted regression trees, or random feature regressions give rise to different \mathcal{G} ’s. For the purpose of deriving confidence intervals, we focus on two types of machine learning methods, \mathcal{G}=\text{either }\mathcal{G}_{\DNN}\text{ or }\mathcal{G}_{B}. Here \mathcal{G}_{\DNN} corresponds to the use of deep neural network (DNN) functions, which is a collection of all possible neural network functions with a predetermined architecture — specifying the width and depth of the layers and the activation functions for each neuron. Then by minimizing ( 2.1 ), we find the optimal neuron biases and weights so that the neural network function \widehat{g}(\cdot) optimally fits the in-sample data. Alternatively, the more classic nonparametric regression gives rise to an alternative specification \mathcal{G}=\mathcal{G}_{B} , which uses a set of basis functions: \Phi(x)=(\phi_{1}(x),...,\phi_{J}(x)) such as the Fourier basis. Then, ( 2.1 ) searches for the optimal function within a simpler space: \mathcal{G}_{B}=\{\Phi(x)^{\prime}\theta:\theta\in\mathbb{R}^{J}\}, and the function \widehat{g}(\cdot) is estimated by finding the best combination of the basis functions. The crucial distinction between \mathcal{G}_{\DNN} and \mathcal{G}_{B} is that the estimators in \mathcal{G}_{B} admit a closed-form representation, whereas \mathcal{G}_{\DNN} does not. However, \mathcal{G}_{\DNN} is more appealing in predicting expected returns in asset pricing leveraging the advantages of sophisticated machine learning methods, as documented in Gu et al. (2020) and Bianchi et al. (2021) . The objective of this paper is to construct forecast confidence intervals for predictions made with neural networks \mathcal{G}_{\DNN} , but it will soon become clear that \mathcal{G}_{B} also plays an important role in our construction. Throughout, we refer to \mathcal{G}_{B} as the “closed-form ML”, such as Fourier series regressions, B-Splines – which are more classic nonparametric regression models. Despite the great popularity and empirical success of machine learning predictions, little work has been devoted to understanding the structure and sources of predictability. However, understanding the structure of the prediction, \widehat{z}_{T+1|T} , is crucially important to quantifying the prediction uncertainty. Recently, Fan et al. (2022) provide a thorough analysis by bridging between the machine learning model and factor models. They suppose that excess returns follow a conditional factor model (also see Gagliardini, Ossola, and Scaillet (2016) ; Zaffaroni (2019) ): y_{i,t}=\alpha_{i,t-1}+\beta_{i,t-1}^{\prime}f_{t}+u_{i,t} where \alpha_{i,t-1} and \beta_{i,t-1} are respectively the “alpha” and “beta” of the asset, f_{t} is the set of (possibly latent) risk factors, and u_{i,t} is the idiosyncratic return. In addition, suppose characteristics are informative about factor loadings (betas) and mispricing (alpha), i.e., there are functions g_{\alpha} and g_{\beta} so that we can rewrite alpha and beta as: 2 2 2 This formulation is formalizing the notion that asset characteristics are informative about risk exposures which is also documented in Rosenberg and McKibben (1973) ; Jagannathan and Wang (1996) ; Connor et al. (2012) ; Gagliardini et al. (2016) ; Kelly et al. (2019) . \alpha_{i,t-1}=g_{\alpha}(x_{i,t-1}),\quad\beta_{i,t-1}=g_{\beta}(x_{i,t-1}). We can thus rewrite the asset pricing model as y_{i,t}=g_{\alpha}(x_{i,t-1})+g_{\beta}(x_{i,t-1})^{\prime}\mathbb{E}f_{t}+g_{\beta}(x_{i,t-1})^{\prime}v_{t}+u_{i,t}, (2.3) where v_{t}:=f_{t}-\mathbb{E}f_{t} . Both v_{t} and u_{i,t} are mean-zero processes contributing to the error term: e_{i,t}:=g_{\beta}(x_{i,t-1})^{\prime}v_{t}+u_{i,t}, (2.4) then the first term is the exposure to factor shocks, while the second term is the idiosyncratic shock. Under the assumption of constant prices of risk, i.e. \lambda_{f}:=\mathbb{E}f_{t} does not change over time, we can define g(x):=g_{\alpha}(x)+g_{\beta}(x)^{\prime}\lambda_{f}. (2.5) Then indeed, ( 2.3 ) can be formulated as the ML model y_{i,t}=g(x_{i,t-1})+e_{i,t} , with g(\cdot) and e_{i,t} defined in ( 2.4 ) and ( 2.5 ) respectively. Therefore, by applying ML regressions within the context of this model, Fan et al. (2022) show that \widehat{g}(x) is estimating g_{\alpha}(x)+g_{\beta}(x)^{\prime}\lambda_{f} . More formally, they show that the ML function has the following (probability) limit: \widehat{z}_{T+1|T}\to^{P}z_{T+1|T}=\sum_{i=1}^{N}w_{i}[g_{\alpha}(x_{i,T})+g_{\beta}(x_{i,T})^{\prime}\lambda_{f}]. This shows that the predictive ability of machine learning regressions arises from capturing mispricing and risk premia, i.e. both \sum_{i=1}^{N}w_{i}\alpha_{i,T} and \sum_{i=1}^{N}w_{i}\beta_{i,T}^{\prime}\lambda_{f} – are key components of expected returns. Understanding the properties of the predicted expected portfolio return, \widehat{z}_{T+1|T} , is an essential first step towards understanding machine learning predictability. Studying the uncertainty of ML forecasts is yet a challenging problem. Most statistical results are developed for i.i.d. or weakly dependent errors, however in asset pricing this is a tenuous assumption as explained by Allena (2021) : we should expect to see strong cross-sectional dependence. The source of the cross-sectional dependence can be illustrated through the lens of the (characteristic-based) factor model. Recall the standard ML model: y_{i,t}=g(x_{i,t-1})+e_{i,t}. (2.6) The errors are strongly cross-sectionally dependent. Take two assets, i and j , then we can characterize the covariance of their errors by using equation ( 2.4 ): \Cov(e_{i,t},e_{j,t})=\mathbb{E}\left[g_{\beta}(x_{i,t-1})^{\prime}\Cov(f_{t})g_{\beta}(x_{j,t-1})\right]\neq 0. (2.7) We expect to see a strong correlation because the assets are exposed to the same sources of systematic risk. Hence, the regression model ( 2.6 ) is not the usual ML model with i.i.d. errors. This is why new methods are needed to explicitly consider the strong cross-sectional dependence structure. 3 3 3 Allena (2021) specified a novel prior to account for the strong dependences in the Bayesian framework. A possible alternative approach is to treat factors as “interactive fixed effects” as in Bai (2009) and explicitly estimate them. However, the method in Bai (2009) or Freyberger (2018) does not cover the case of sophisticated machine learning methods, nor is it the “standard” implementation in the applied forecasting literature. We will therefore not pursue this approach. It is essential to stress that accounting for the dependence in the errors is not purely an econometric challenge. From ( 2.7 ), it is clear that we vastly underestimate the risk associated with a prediction if we incorrectly assume i.i.d. errors. The Monte Carlo evidence in Section 2.4 will illustrate that ignoring the cross-sectional dependences will vastly understate the prediction uncertainty. 2.2 The Intuition of the ML Forecast Error We now discuss the intuition of our approach to computing the ML standard error. Recall that the expected excess return, \widehat{z}_{T+1|T} , is predicted using either neural networks (corresponding to \mathcal{G}_{\DNN} ) or a closed-form ML method (corresponding to \mathcal{G}_{B} ). The intuition of our constructed FCI is based on the following theorem, which is our first main result, showing that \widehat{z}_{T+1|T} has the same asymptotic distribution regardless of which specific ML method is used. Theorem 1 . Suppose \sum_{i}|w_{i}|<\infty and Assumption 1 , Assumption 2 and Assumption 3 in the appendix hold. There exists a function \zeta^{*}(\cdot) so that \displaystyle\widehat{z}_{T+1|T}-z_{T+1|T} \displaystyle= \displaystyle\frac{1}{T}\sum_{t=1}^{T}\mathcal{B}_{t-1}(f_{t}-\mathbb{E}f_{t})+o_{P}(T^{-1/2}),\quad\text{where} (2.8) \displaystyle\mathcal{B}_{t-1} \displaystyle:= \displaystyle\frac{1}{N}\sum_{i=1}^{N}\zeta^{*}(x_{i,t-1})\beta_{i,t-1}^{\prime}. (2.9) The function \zeta^{*} only depends on the joint distribution of (x_{i,t-1}) , but does not depend on whether \mathcal{G}_{\DNN} or \mathcal{G}_{B} are used for constructing \widehat{z}_{T+1|T} . In other words, the same asymptotic expansion holds for \widehat{z}_{T+1|T} if the closed-form ML space \mathcal{G}_{B} is used in place of \mathcal{G}_{\DNN} . In addition, \sqrt{T}V^{-1/2}(\widehat{z}_{T+1|T}-z_{T+1|T})\to^{d}\mathcal{N}(0,1) (2.10) where V=\frac{1}{T}\sum_{t=1}^{T}\Var(\mathcal{B}_{t-1}f_{t}) does not depend on whether \mathcal{G}_{\DNN} or \mathcal{G}_{B} are used to predict. Hence \widehat{z}_{T+1|T} has the same asymptotic distribution for both neural networks and closed-form- ML. Proof. The proof is given in Section B.3.1 . ∎ The theorem has two important implications. First and foremost, fundamentally, the asymptotic distribution of ML forecasts does not depend on the specific machine learning model \mathcal{G} in ( 2.1 ). In the asymptotic expansion, \mathcal{B}_{t-1} depends on a function \zeta^{*}(\cdot) . Although the closed-form expression for the function \zeta^{*}(\cdot) is very difficult to derive for neural networks, it is entirely determined by the quantities of the asset pricing model ( 2.3 ), but not by the specific choice of the ML method, which can be sophisticated machine learning prediction \mathcal{G}_{\DNN} (neural networks) or closed-form prediction \mathcal{G}_{B} (e.g., Fourier series regression). The main econometric intuition is that the predictor is obtained by minimizing a regular loss function – it is the loss function rather than the choice of \mathcal{G} that ultimately determines the asymptotic distribution. 4 4 4 Note that this result also allows for small shrinkage in penalized regressions. However, if models are strongly penalized, as with the Lasso, this induces a shrinkage bias, leading to different asymptotic phenomena Zhang and Zhang (2014) . This insight forms the foundation of our approach to constructing confidence intervals for the expected return predicted by neural networks. As explained in more detail in Section 2.3 , one of the two proposed methods approximates the standard error of the neural network forecast using an analytic standard error from closed-form ML. The latter is straightforward to derive due to its closed-form nature. The second implication of the theorem is that the prediction error \mathcal{B}_{t-1}(f_{t}-\mathbb{E}f_{t}) is only driven by the common factor shocks rather than the idiosyncratic errors. Thus, the rate of convergence is O_{P}(1/\sqrt{T}) , which is much slower than O_{P}(1/\sqrt{NT}) for the usual panel data models. This aligns with the well-established asset pricing intuition that the factor risk premium can only be learned well over time, rather than cross-sectionally, due to the strong cross-sectional dependence. Therefore, the dominant source of uncertainty comes from the time series rather than the cross-sectional variation. Theorem 1 requires three technical assumptions, which are stated in the appendix. Assumption 1 is a standard condition about the dependence of the data and the tail behavior. Assumptions 2 and 3 concern the complexity of the machine learning space. Essentially, it says that the complexity must be controlled and the models cannot be overparametrized. This is in contrast to the setting of Kelly et al. (2021) and Didisheim et al. (2023) , who derive appealing features of overparametrized models in portfolio construction. Developing distribution theory in such a setting is extremely challenging, but interesting and will be left for future work in theoretical econometrics. 2.3 Constructing ML Forecast Confidence Interval Throughout the rest of the paper, we denote \widehat{z}_{T+1|T} as the expected return predicted by neural networks, i.e. \mathcal{G}_{\DNN} . These models are among the most successful in asset pricing, as reviewed before. The objective is to construct its forecast confidence interval. Each of the two implications outlined in the previous subsection motivates a distinct method for this purpose. In the sequel, we introduce these two methods. We then later demonstrate the usefulness of these methods in asset pricing in Section 3 . 2.3.1 Method I: Closed-form ML Approximation We first introduce a closed-form ML method, building on our main result in Theorem 1 . The asymptotic distribution ( 2.10 ) derived in Theorem 1 demonstrates that the asymptotic variance of \widehat{z}_{T+1|T} is the same regardless of whether \mathcal{G}=\mathcal{G}_{\DNN} or \mathcal{G}_{B} . This intuition allows us to approximate the forecast standard error using that of the closed-form ML method, which is much easier to derive. For the closed-form ML we set \mathcal{G}=\mathcal{G}_{B} in ( 2.1 ) and use Fourier series to make predictions. In this case, let \Phi(x) be the vector of Fourier bases, and the regression model is: \displaystyle\widehat{g}_{B}(\cdot) \displaystyle= \displaystyle\Phi(\cdot)^{\prime}\widehat{\theta},\quad\text{ where }\widehat{\theta}:=\arg\min_{\theta}\sum_{i=1}^{N}\sum_{t=1}^{T}(y_{i,t}-\Phi(x_{i,t-1})^{\prime}\theta)^{2}. Estimators using the Fourier series can be obtained in closed form because of their OLS-type analytic solution: \widehat{g}_{B}(\cdot)=\Phi(\cdot)^{\prime}(\Psi^{\prime}\Psi)^{-1}\sum_{i,t}\Phi(x_{i,t-1})y_{i,t}, where \Psi is the NT\times J matrix stacking all \Phi(x_{i,t-1}) . Therefore, we can easily derive its asymptotic standard error: \text{SE}(\widehat{z}_{T+1}):=\sqrt{\sum_{t=1}^{T}H^{\prime}\Phi_{t-1}^{\prime}\beta_{t-1}\Cov(f_{t})\beta_{t-1}^{\prime}\Phi_{t-1}H}, (2.11) where W=(w_{1},...,w_{N}) are the portfolio weights, \Phi_{t-1}=(\Phi(x_{1,t-1}),...,\Phi(x_{N,t-1})) , and H^{\prime}=W^{\prime}\Phi_{T}(\Psi^{\prime}\Psi)^{-1}. This is straightforward to estimate by: \widehat{\text{SE}}(\widehat{z}_{T+1|T}):=\sqrt{\sum_{t=1}^{T}H^{\prime}\Phi_{t-1}^{\prime}\widehat{e}_{t}\widehat{e}_{t}^{\prime}\Phi_{t-1}H}, (2.12) where \widehat{e}_{t} is the vector of residuals, \widehat{e}_{i,t}:=y_{i,t}-\widehat{g}(x_{i,t-1}) . It is important to note that we employ closed-form ML, \widehat{g}_{B}(\cdot) , only to compute the forecast standard error. The forecast itself, \widehat{g}(\cdot) as in ( 2.1 ), is still generated by a sophisticated neural network. This leverages our result that the two ML predictions have the same asymptotic standard error. It is therefore tempting to ask why we should employ more sophisticated neural networks in the first place. The answer is that the benefits of using neural networks or related methods do not arise from a smaller standard error, but from fewer constraints in handling highly nonlinear functions and the capability of approximating larger classes of asset pricing functions. To be specific, Theorem 1 shows that for both DNN and closed-form ML predicted \widehat{z}_{T+1|T} , \widehat{z}_{T+1|T}-z_{T+1|T}=\frac{1}{T}\sum_{t=1}^{T}\mathcal{B}_{t-1}(f_{t}-\mathbb{E}f_{t})+o_{P}(T^{-1/2}). The remainder error term o_{P}(T^{-1/2}) captures what is known as “approximation bias”, which arises from approximating the unknown expected return function g(x) using either machine learning method. While both methods exhibit a diminishing approximation bias, the rate of decay differs. Neural networks, due to the adaptivity to the intrinsic dimension of the input features, 5 5 5 See Schmidt-Hieber (2020) ; Kohler and Langer (2021) and Fan et al. (2022) for detailed discussions on theoretical advantages of neural networks. can flexibly approximate a broad class of nonlinear functions with a rapidly diminishing approximation bias. In contrast, the closed-form ML (e.g., Fourier series) is capable of approximating a much narrower class of functions, and its approximation bias decays at a slower rate as the number of input features increases, due to the well-known curse of dimensionality. This makes closed-form ML unsuitable for direct use in return prediction. However, its larger bias does not affect its asymptotic variance, which still provides a good approximation to the variance of the predicted returns from the neural network. The following theorem is our second main result, which formally justifies the validity of \widehat{\SE}(\widehat{z}_{T+1|T}) as the standard error when \widehat{z}_{T+1|T} is constructed using neural networks. Theorem 2 (Closed-form ML approximation) . Suppose Assumption 1 , Assumption 2 and Assumption 3 in the appendix hold, then \widehat{\SE}(\widehat{z}_{T+1|T})^{-1}(\widehat{z}_{T+1|T}-z_{T+1|T})\to^{d}\mathcal{N}(0,1), where \widehat{z}_{T+1} is obtained from neural networks and \widehat{\SE}(\widehat{z}_{T+1|T}) is obtained via Fourier series as in ( 2.12 ). Then or a given significance level 1-\alpha\in(0,1) , P(|\widehat{z}_{T+1|T}-z_{T+1|T}|<\widehat{\SE}(\widehat{z}_{T+1|T})\times\epsilon_{\alpha})\to 1-\alpha, where \epsilon_{\alpha} is the critical value of the standard normal distribution corresponding to size \alpha . Proof. The proof is given in Section B.3.2 . ∎ 2.3.2 Method II: The k -step Bootstrap We propose an alternative method for constructing FCIs, based on the time-series bootstrap. The bootstrap computes the critical value by repeatedly resampling from the original data set and using the quantile of the neural network predictors recomputed from the resampled data. Unlike the method of analytic standard error, the bootstrap does not involve closed-form ML methods, and therefore requires weaker conditions. However, the performance and validity of bootstrap depend critically on how the bootstrap data are generated. In the asset pricing context, the bootstrap data should properly capture the primary sources of uncertainty to predicted expected returns. Theorem 1 shows that the forecast uncertainty is mainly driven by time series variation. Thus, we can cluster in time by applying the wild bootstrap to mimic the sampling distribution of \widehat{z}_{T+1|T} . 6 6 6 In the case of serial correlation, the block bootstrap ( Künsch (1989) ) or the stationary bootstrap ( Politis and Romano (1994) ) can be applied directly in our setting. Specifically, let \eta_{t}^{*} denote an i.i.d. sequence of standard normal random variables and let \widehat{g}(\cdot) denote the learned function using neural networks. Define the bootstrap residuals as: e_{i,t}^{*}=(y_{i,t}-\widehat{g}(x_{i,t-1}))\eta_{t}^{*}. (2.13) We then apply neural networks to the resampled excess return y_{i,t}^{*}=\widehat{g}(x_{i,t-1})+e_{i,t}^{*} and x_{i,t-1} to repredict the expected return, and repeat this step many times. The forecast critical value is given by the bootstrap quantile of the repeated predictions. A potential limitation of the bootstrap procedure is the enormous computational burden. For example, estimating with 100 bootstrap iterations requires training 100 separate neural networks, one for each bootstrap sample. Fully training these neural networks is computationally very costly, limiting the applicability of bootstrap-based inference for larger machine learning models. To address this issue, we propose a k -step bootstrap method for neural network inference, which significantly reduces computational burden. The k -step bootstrap was initially proposed and studied by Davidson and MacKinnon (1999) and Andrews (2002) in the context of making inference for nonlinear models. The idea is that, instead of fully training the neural network for each bootstrap sample, we only train it iteratively for k epochs, with a relatively small k such as 10 or 20. This approach takes advantage of the observation that the fully trained function from the original data, \widehat{g}(\cdot) , provides an excellent starting point for training in the bootstrap data. Thus, for each bootstrap re-sample, we initialize with \widehat{g}_{0}^{*}(\cdot)=\widehat{g}(\cdot) and then train the network for k epochs. The full algorithm is given as follows: k -step Bootstrap Algorithm. Step 1. Generate \eta_{t}^{*}\sim\mathcal{N}(0,1) independently; generate \displaystyle e_{i,t}^{*} \displaystyle= \displaystyle(y_{i,t}-\widehat{g}(x_{i,t-1}))\eta_{t}^{*} \displaystyle y_{i,t}^{*} \displaystyle= \displaystyle\widehat{g}(x_{i,t-1})+e_{i,t}^{*}. Step 2. Train the neural network on the bootstrap resampled data starting from the original \widehat{g}(\cdot) , and train for k epochs. Obtain \widehat{g}^{*}(\cdot) . Step 3. Repeat Steps 1-2, B times to get \widehat{g}^{*^{1}}(\cdot),...,\widehat{g}^{*^{B}}(\cdot) . Let q^{*}_{\alpha} be the 1-\alpha quantile of \left|\sum_{i}w_{i}\widehat{g}^{*,b}(x_{i,T})-\widehat{z}_{T+1|T}\right|,\quad b=1,...,B. The bootstrap 1-\alpha level FCI for z_{T+1|T} is [\widehat{z}_{T+1|T}-q_{\alpha}^{*},\quad\widehat{z}_{T+1|T}+q_{\alpha}^{*}]. It is critical to note that we generate \eta_{t}^{*} in Step 1, which only varies over time but not across individual assets. Hence, assets share the same \eta_{t}^{*} at each period. This is because expression ( 2.8 ) clearly shows that it is the time series variation that determines the sampling distribution of the ML model. In contrast, if the bootstrap sample were wrongly generated, such as independently from cross-sectional residuals by either generating \eta^{*}_{it} (resampled idenpendently across time and firms, this corresponds to the standard bootstrap implementation) or \eta_{i}^{*} (resampled indepdently across firms, but fixed over time) instead of \eta_{t}^{*} , it will not correctly capture the strong cross-sectional dependence, and will dramatically understate the forecast uncertainty. We illustrate this in simulation in Section 2.4 . Theorem 3 formally justifies the proposed bootstrap confidence interval for predicted expected returns. Theorem 3 (Bootstrap) . Suppose Assumption 1 and Assumption 2 in the appendix hold. Then for any \alpha\in(0,1) , P(|\widehat{z}_{T+1|T}-z_{T+1|T}|0 is the coefficient of risk aversion. In this problem, \widehat{z}_{T+1|T} is taken as given without accounting for its associated estimation uncertainty. In contrast, the uncertainty-averse formulation explicitly considers the estimation uncertainty. It takes a “max-min” form. It can be interpreted as finding the best portfolio in the worst case for the expected return, i.e. UA-MV problem \displaystyle\max_{\mbox{\boldmath$\omega$}}\min_{\mu\in\text{FCI}}\mbox{\boldmath$\omega$}^{\prime}\mu-\frac{\gamma}{2}\mbox{\boldmath$\omega$}^{\prime}\Sigma_{T}\mbox{\boldmath$\omega$}, (3.2) where FCI is the forecast confidence interval for the expected return. The forecast confidence interval takes the following form: \text{FCI}=[\widehat{z}_{1,T+1|T}-q_{1,\alpha},\widehat{z}_{1,T+1|T}+q_{1,\alpha}]\times\cdots\times[\widehat{z}_{R,T+1|T}-q_{R,\alpha},\widehat{z}_{R,T+1|T}+q_{R,\alpha}], where q_{i,\alpha} is the critical value for \widehat{z}_{i,T+1|T}-z_{i,T+1} under significance level \alpha obtained using either the analytic forecast standard error or bootstrap. For instance, using the analytic standard error, we can take q_{i,\alpha}=\widehat{\SE}(\widehat{z}_{i,T+1|T})\times\epsilon_{\alpha}, where \epsilon_{\alpha} corresponds to the level uncertainty aversion. If the bootstrap is used to obtain a forecast confidence interval, we can use q_{i,\alpha}=q^{*}_{i,\alpha}, which is the 1-\alpha quantile of the bootstrap distribution of \widehat{z}_{i,T+1|T} . In applications, particularly when multiple assets are considered, controlling for the Type I error rate for multiple testing is also desirable. A simple adjustment is the Bonferroni correction, i.e. setting \alpha=0.05/R . While the intuition of problem ( 3.2 ) is clear, it may appear hard to solve because of the inner maximization. Garlappi et al. (2007) characterize each element of the solution as a function of other elements. Our contribution in this framework, is in the following theorem, which shows that the UA-MV problem can be reformulated as a \ell_{1} - penalized optimization problem, known as Lasso. Theorem 4 . The multivariate UA-MV problem, ( 3.2 ), is equivalent to the following adaptive Lasso formulation: \min_{\mbox{\boldmath$\omega$}=(\omega_{1},...,\omega_{R})}\frac{\gamma}{2}\mbox{\boldmath$\omega$}^{\prime}\Sigma_{T}\mbox{\boldmath$\omega$}-\sum_{i=1}^{R}\omega_{i}\widehat{z}_{T+1|T,i}+\sum_{i=1}^{R}q_{\alpha,i}|\omega_{i}|. (3.3) Proof. The proof is given in Section B.3.4 . ∎ This result offers an economic justification for using penalized shrinkage portfolio selection, as developed by Ao et al. (2019) and Kozak et al. (2020) . Furthermore, it justifies that the penalization parameter should be explicitly chosen as q_{\alpha,i} , the quantile of the ML forecast error. The following remarks provide additional guidance for implementation in different cases. Remark 1 . If a risk-free asset is not available, \widehat{z}_{T+1|T,i} denotes the raw return. Then the above problem becomes a constrained adaptive Lasso problem, i.e. \min_{\mbox{\boldmath$\omega$}=(\omega_{1},...,\omega_{R})}\frac{\gamma}{2}\mbox{\boldmath$\omega$}^{\prime}\Sigma_{T}\mbox{\boldmath$\omega$}-\sum_{i=1}^{R}\omega_{i}\widehat{z}_{T+1|T,i}+\sum_{i=1}^{R}q_{\alpha,i}|\omega_{i}|,\;\;\text{subject to}\;\;\sum_{i=1}^{R}\omega_{i}=1 Remark 2 . The problem can be solved with standard software packages. To see this, write the Cholesky decomposition of \Sigma_{T} as \Sigma_{T}=LL^{\prime} where L is a R\times R lower triangular matrix and denote Q_{\alpha}=\text{diag}(q_{\alpha,1},...,q_{\alpha,R}) be a diagonal matrix with elements q_{\alpha,i} . Then it is easy to derive that ( 3.2 ) is equivalent to the following adaptive Lasso problem: \min_{\mbox{\boldmath$\omega$}}\frac{1}{2}\|Y^{*}-X^{*}\mbox{\boldmath$\omega$}\|_{2}^{2}+\|Q_{\alpha}\mbox{\boldmath$\omega$}\|_{1},\quad\text{with}\quad X^{*}=\sqrt{\gamma}L^{\prime},\;\;Y^{*}=\frac{1}{\sqrt{\gamma}}L^{-1}\widehat{z}_{T+1|T}. (3.4) If a risk-free asset is not available, the problem is again augmented with the constraint \sum_{i=1}^{R}\omega_{i}=1 . It is well-known that the Lasso solution may be sparse and thus produce portfolio allocations, for which some (or many) \omega_{i} ’s are zero. The \ell_{1} -penalized formulation facilitates the economic interpretation of the behavior of uncertainty-averse investors compared to that of mean-variance investors. To gain the intuition, in below we study a simple case of a single risky asset: \max_{\omega}\min_{\mu}\omega\mu-\frac{\gamma}{2}\omega^{2}\sigma^{2}\quad\text{subject to}\quad|\mu-\widehat{z}_{T+1|T}|\leq q_{\alpha}, (3.5) where q_{\alpha} denotes the (1-\alpha) -quantile for the predicted expected return. Corollary 5 provides a closed-form solution and allows for a comparison with the mean-variance solution in which the effect of estimation uncertainty can be seen directly. In the following, we denote (x)_{+}=\max\{x,0\} and \sgn(x) as the sign of x . Corollary 5 . The uncertainty-constrained MV problem ( 3.5 ) is equivalent to the following: \min_{\omega}\frac{1}{2}\left(\omega-\omega^{\MV}\right)^{2}+\lambda_{\alpha}|\omega| and the optimal solution is \omega^{\UA}=\sgn(\omega^{\MV})(|\omega^{\MV}|-\lambda_{\alpha})_{+} where \omega^{\MV}=\frac{\widehat{z}_{T+1|T}}{\gamma\sigma^{2}},\quad\lambda_{\alpha}=\frac{q_{\alpha}}{\gamma\sigma^{2}}. Proof. The proof is given in Section B.3.4 . ∎ In the above theorem, \omega^{\MV} is the classic mean-variance portfolio without uncertainty constraints, i.e. the solution to \max_{\omega}\omega\widehat{z}_{T+1|T}-\frac{\gamma}{2}\omega^{2}\sigma^{2}. The corollary directly illustrates why the UA-MV problem yields possible non-participation. For a fixed coefficient of risk aversion ( \gamma ) and variance ( \sigma^{2} ), an investor will choose not to invest in the risky asset if the estimation uncertainty ( q_{\alpha} ) is too large. More importantly, this result provides a direct economic interpretation of the UA-MV problem. The solution captures investor behavior in which she tests the following null hypothesis: H_{0}:\quad z_{T+1|T}=0. On the one hand, when she does not reject the null hypothesis, |\widehat{z}_{T+1|T}|\leq q_{\alpha} , the investor finds the predicted excess return not significantly different from zero as the level of uncertainty is too large. In this case, she will hold no position in the risky asset and \omega^{\UA}=0 . Her optimal portfolio is then solely holding the risk-free asset. At first glance, this may seem like excessively conservative behavior. However, Bessembinder (2018) performs an extensive empirical investigation of this hypothesis and finds that many stocks do not outperform treasuries ex-post in a significant way. On the other hand, when the investor predicts that the expected return of the risky asset is significantly different from the risk-free rate, which happens if |\widehat{z}_{T+1|T}|>q_{\alpha} , she then starts investing in the risky asset. The decision of whether to short or long the risky asset (the sign of \omega^{\UA} ) is determined by the sign of \widehat{z}_{T+1|T} . However, even in this case, the investor will still invest more cautiously in the risky asset by shrinking her investment towards the risk-free rate. For instance, suppose the investor finds that \widehat{z}_{T+1|T}>q_{\alpha} , then her allocation in the risky asset is \omega^{\UA}=\omega^{\MV}-\frac{q_{\alpha}}{\gamma\sigma^{2}}>0. Instead of adopting the classic mean-variance portfolio, she reduces her allocation to the risky asset, and the amount of reduction, q_{\alpha}/(\gamma\sigma^{2}) , reflects her tolerance towards uncertainty, which is closely linked to risk aversion. We illustrate the optimal solution to the UA-MV problem in Figure 2 . The upper panel of Figure 2 plots the optional min-max weight, \omega^{\UA} , against the classic MV portfolio. Here we fix the level of forecast standard error (FSE) and thus q_{\alpha} . As we can see, \omega^{\UA} is zero for small magnitudes of |\omega^{\MV}| , i.e., the investor does not allocate towards this asset. The allocation toward the risky assets starts to increase but with constant shrinkage relative to the MVE as the latter deviates from zero. The lower panel of Figure 2 plots \omega^{\UA} as a function of the forecast standard error for a fixed \omega^{\MV} . As the forecast standard error increases, her position in the risky asset decays linearly and eventually becomes zero. In the case of two risky assets and no risk-free asset, we also provide an explicit solution and discuss its properties in A.3 . Figure 2 : Mean-Variance vs. Uncertainty Averse (max-min) Portfolio Allocation This figure shows the uncertainty averse (max-min) solution for the optimal portfolio weight in comparison with the standard mean-variance portfolio weight. In the upper panel, we hold the level of the forecast standard error (FSE) fixed and vary the mean-variance weight ( \omega^{\MV} ). In the lower panel, we fix the mean-variance weight at 0.2 and vary the forecast standard error (FSE). 3.2 Asset Selection In the second application, we apply the FCI to security selection. The literature often considers long-short portfolios sorted on characteristics or return predictions typically ignoring short-sale costs. In a recent study Muravyev et al. (2024) obtain implied short-sale costs from the equity options markets and find that many anomalies are no longer profitable after accounting for short-selling fees. Moreover, several major market participants such as mutual funds are either prohibited from short-selling or do not engage in shorting for other reasons. Chen et al. (2013) document that fewer than 10% of mutual funds engage in short-selling and in general do not have large short positions. More generally, and in particular in the light of the findings of Bessembinder (2018) , it is natural to ask whether expected returns are indeed significantly positive. In an intriguing analysis, he documents that the US equity premium is indeed highly concentrated and can be attributed to about 5% of stocks. While it is unlikely that our forecast confidence intervals (or any other method) can extract all ex-post winners, the use of machine learning FCI, as we describe below, highlights a good way of finding securities with positive expected returns. In the following, we consider a long-only mutual fund manager. She needs to build a portfolio with K stocks. Rather than merely allocating her funds toward the K stocks with the highest predicted returns, she aims to allocate toward securities for which she has stronger conviction, i.e. those that have significantly positive predictions. So for each security under consideration, she conducts a hypothesis test H_{0}^{i}:z_{i,T+1|T}=0, versus the one-sided alternative H_{a}^{i}:z_{i,T+1|T}>0 for each i=1,...,R, i.e., she aims to detect which stocks have significantly and positively predicted expected returns. Given that there may be a large number of securities under consideration, this is a multiple hypotheses testing problem. Alternatively, one might consider testing if an expected excess return is significantly larger than the transaction costs associated with trading the security. We leave this extension for future research. 7 7 7 Jensen et al. (2024) highlight the importance of transaction costs in the context of machine learning portfolios. We compute the t-statistics of the neural network predicted excess return for each asset: t_{i}=\frac{\widehat{z}_{i,T+1|T}}{\widehat{\SE}(\widehat{z}_{i,T+1|T})}. Let p_{i} denote the corresponding p-value. We then determine a cutoff value c_{0}>0 so that H_{0}^{i} is rejected if p_{i}