Temporal Wasserstein Imputation: Versatile Missing Data Imputation for Time Series
Shuo-Chieh Huang Department of Statistics, Rutgers University Tengyuan Liang Booth School of Business, University of Chicago and Ruey S. Tsay Booth School of Business, University of Chicago · 2024
Read Temporal Wasserstein Imputation: Versatile Missing Data Imputation for Time Series by Shuo-Chieh Huang Department of Statistics, Rutgers University Tengyuan Liang Booth School of Business, University of Chicago and Ruey S. Tsay Booth School of Business, University of Chicago (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-02811.txt.
Description
Abstract Missing data can significantly hamper standard time series analysis, yet in practice they are frequently encountered. In this paper, we introduce temporal Wasserstein imputation, a novel method for imputing missing data in time series. Unlike existing techniques, our approach is fully nonparametric, circumventing the need for model specification prior to imputation, making it suitable for potential nonlinear dynamics.
Catalog record
| CREATOR | Shuo-Chieh Huang Department of Statistics, Rutgers University Tengyuan Liang Booth School of Business, University of Chicago and Ruey S. Tsay Booth School of Business, University of Chicago |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-02811
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Temporal Wasserstein Imputation: Versatile Missing Data Imputation for Time Series Shuo-Chieh Huang Department of Statistics, Rutgers University Tengyuan Liang Booth School of Business, University of Chicago and Ruey S. Tsay Booth School of Business, University of Chicago Abstract Missing data can significantly hamper standard time series analysis, yet in practice they are frequently encountered. In this paper, we introduce temporal Wasserstein imputation, a novel method for imputing missing data in time series. Unlike existing techniques, our approach is fully nonparametric, circumventing the need for model specification prior to imputation, making it suitable for potential nonlinear dynamics. Its principled algorithmic implementation can seamlessly handle univariate or multivariate time series with any non-systematic missing pattern. In addition, the plausible range and side information of the missing entries (such as box constraints) can easily be incorporated. As a key advantage, our method mitigates the distributional bias typical of many existing approaches, ensuring more reliable downstream statistical analysis using the imputed series. Leveraging the benign landscape of the optimization formulation, we establish the convergence of an alternating minimization algorithm to critical points. We also provide conditions under which the marginal distributions of the underlying time series can be identified. Numerical experiments, including extensive simulations covering linear and nonlinear time series models and a real-world groundwater dataset laden with missing values, corroborate the practical usefulness of the proposed method. Keywords: Optimal transport, Nonlinear time series, Threshold AR model, Compositional time series, Nonstationary time series 1 Introduction In many scientific fields, data collection is often subject to various disruptions such as idiosyncratic or system-level instrument failures or changes in data collection policies, leading to missing entries in the data ( Little and Rubin, 2019 ) . In time series analysis, missing data pose serious challenges to the application of many statistical tools since it is typically presumed that observations are available at equally spaced time points. Moreover, since the dynamic relationship is of crucial importance in time series analysis, one cannot proceed by throwing away the missing data, even if the missing data appear completely at random (i.e., the missing pattern is independent of the series). To tackle this issue, one approach is to impute the missing entries, after which the vast array of time series methods becomes readily applicable for statistical inference in the downstream. Naturally, the estimation of the optimal interpolator , which is the expectation of the missing data conditional on the observed data, has been extensively studied ( Harvey and Pierse, 1984 ; Pourahmadi, 1989 ; Peña and Maravall, 1991 ; Gómez et al., 1999 ; Alonso and Sipols, 2008 ; Peña and Tsay, 2021 ; McElroy and Politis, 2020 ; McElroy and Politis, 2022 ; McElroy, 2022 ) . Since the optimal interpolator would depend on the probabilistic structure of the underlying process, most methods involve EM-type iterations that alternate between model identification and missing data imputation (see, for example, Chpt. 4 of Peña and Tsay, 2021 ). Besides the optimal interpolator, ad hoc deterministic imputation methods, such as smoothing with linear interpolation, B-splines and wavelets ( Hastie et al., 2009 , see, for instance,) , are also used in practice for their simplicity. However, the optimal interpolator can cause distributional biases that are detrimental to downstream statistical analysis. To illustrate, consider an AR(1) process, x_{t}=\phi x_{t-1}+\epsilon_{t} , where 0<|\phi|<1 and \{\epsilon_{t}\} is a sequence of i.i.d. standard normal random variables. The optimal interpolator can be expressed in terms of the inverse autocorrelation functions ( Peña and Maravall, 1991 ; McElroy and Politis, 2022 ; see also Chpt. 6 of Peña et al., 2011 ). For instance, suppose that data at time indices T and T+1 are missing. Then the optimal interpolators are given by \displaystyle\hat{x}_{T}=\frac{\phi(1+\phi^{2})}{1+\phi^{2}+\phi^{4}}x_{T-1}+\frac{\phi^{2}}{1+\phi^{2}+\phi^{4}}x_{T+2},\quad\hat{x}_{T+1}=\frac{\phi^{2}}{1+\phi^{2}+\phi^{4}}x_{T-1}+\frac{\phi(1+\phi^{2})}{1+\phi^{2}+\phi^{4}}x_{T+2}. Using the fact that \mathbb{E}(x_{t}x_{t-h})=\phi^{h}/(1-\phi^{2}) , for h\geq 0 , it is not difficult to show that \displaystyle\mathbb{E}(\hat{x}_{T}^{2})=\mathbb{E}(\hat{x}_{T+1}^{2})=\frac{\phi^{2}+3\phi^{4}+3\phi^{6}+2\phi^{8}}{(1+\phi^{2}+\phi^{4})^{2}}\mathbb{E}(x_{t}^{2})<\mathbb{E}(x_{t}^{2}),\mbox{ and} \displaystyle|\mathbb{E}(\hat{x}_{T}\hat{x}_{T+1})|=\frac{2\phi^{2}+3\phi^{4}+3\phi^{6}+\phi^{8}}{(1+\phi^{2}+\phi^{4})^{2}}|\mathbb{E}(x_{t}x_{t+1})|<|\mathbb{E}(x_{t}x_{t+1})|. Hence, the variance and the autocovariances of the imputed data are different from those of the original AR(1) process. In general, the joint distribution of the optimally interpolated time series is often quite different from that of the original process, leading to questionable downstream statistical analysis, especially when the number of missing data is non-negligible. Similar biases are also observed for some deterministic imputation methods ( Carrizosa et al., 2013 ) . In addition to the distributional bias, the above example also reveals further predicaments of the optimal interpolator. First, in order to compute the optimal interpolator, one often needs to choose a model class before imputation. However, such a decision relies heavily on domain expertise and may be ill-informed when there are many missing values. While it is possible, for certain time series models, to estimate the model parameters without imputation, it still engages in model specification before imputation. Second, it may be difficult to compute the optimal interpolator if some nonlinear model is employed. With different missing patterns and different chosen models, the computational procedure may change drastically. Finally, incorporating side information about the missing data into this methodology appears less straightforward. Side information about the missing data restricts the set of admissible imputations. For instance, a compositional time series is known to lie in [0,1] and sums to unity at each time point. Besides the optimal interpolator, many deterministic interpolators such as B-spline smoothing are also inclined to over-(or under-)shoot, leading to imputations that are outside plausible ranges. In this paper, we present the temporal Wasserstein imputation (TWI), a novel approach for time series imputation which circumvents the aforementioned limitations. As a nonparametric method, TWI is highly versatile and can be applied to a wide range of time series data. In particular, as a key advantage of TWI, it requires no model class specification prior to imputation, allowing it to perform well even when the underlying time series is nonlinear, such as the threshold autoregressive processes (TAR, Tong, 1983 ). Formulated as an optimization problem, TWI can be implemented efficiently through an alternating minimization algorithm. This principled algorithmic approach allows for the seamless integration of side information by restricting the admissible imputations, and its extension to multivariate time series is also straightforward. The optimization and statistical underpinnings of TWI are closely related to the optimal transport problem ( Villani, 2008 ; Villani, 2021 ; Cuturi, 2013 ) , which is briefly reviewed in Section 2 . Section 3 maps out the optimization formulation and introduces the computational scheme in detail. The core idea of TWI is that by minimizing the Wasserstein distance (over all admissible imputations) between the marginal distributions before and after a specified time point, the marginal distributions implied by the imputation shall resemble those of the original series, thereby lowering the distributional bias discussed above. In Section 4 , we investigate the optimization and statistical properties of TWI. Our analysis shows that the optimization landscape of TWI is biconvex and the alternating algorithm is guaranteed to converge to a critical point. However, from the statistical viewpoint, missing data imputation suffers from an identification issue: Many distributions (of the stochastic process) may be equally plausible in describing the corrupted series. This raises the question of whether and under what conditions TWI can identify the underlying marginal distributions. In Section 4.2 , we show, via a two-state Markov chain example, that