Variable Selection for Comparing High-dimensional Time-Series Data

Kensuke Mitsuzawa, Margherita Grossi, Stefano Bortoli, Motonobu Kanagawa · 2024

Read Variable Selection for Comparing High-dimensional Time-Series Data by Kensuke Mitsuzawa, Margherita Grossi, Stefano Bortoli, Motonobu Kanagawa (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06870.txt.

Introduction

Kensuke Mitsuzawa, Margherita Grossi, Stefano Bortoli, and Motonobu Kanagawa pinpoint where two wide data streams diverge in a 2024 paper. When one stream comes from a costly simulator, naming the exact variables and time spans that mismatch real data guides fixes.

The method chops time into slices. On each slice it runs a two sample test and picks the variables that split the pair most. Fake data trials map where the approach works and where it strains. Real uses check a neural emulator against a particle fluid simulator and probe a traffic simulator's knobs.

Catalog record

CREATORKensuke Mitsuzawa, Margherita Grossi, Stefano Bortoli, Motonobu Kanagawa
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-06870

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Variable Selection for Comparing High-dimensional Time-Series Data Kensuke Mitsuzawa, Margherita Grossi, Stefano Bortoli, Motonobu Kanagawa Abstract Given a pair of multivariate time-series data of the same length and dimensions, an approach is proposed to select variables and time intervals where the two series are significantly different. In applications where one time series is an output from a computationally expensive simulator, the approach may be used for validating the simulator against real data, for comparing the outputs of two simulators, and for validating a machine learning-based emulator against the simulator. With the proposed approach, the entire time interval is split into multiple subintervals, and on each subinterval, the two sample sets are compared to select variables that distinguish their distributions and a two-sample test is performed. The validity and limitations of the proposed approach are investigated in synthetic data experiments. Its usefulness is demonstrated in an application with a particle-based fluid simulator, where a deep neural network model is compared against the simulator, and in an application with a microscopic traffic simulator, where the effects of changing the simulator’s parameters on traffic flows are analysed. Impact Statement When engineering experiments on the system of interest are impossible or costly, computer experiments are performed on a simulator as a substitute for the intended experiments. Such situations can be widely seen in engineering for manufacturing, transportation and urban design, and disaster prevention. To trust the results of computer experiments, however, the simulator must be systematically validated beforehand. Similarly, when a machine learning model is trained to emulate a simulator to accelerate test-time execution, the emulator must be validated against the simulator before the emulator is deployed. In these cases, a systematic validation is challenging because the simulator’s output is usually a high-dimensional time series. The proposed approach is expected to be helpful in these situations for performing a systematic validation. 1 Introduction Consider the following problem: Given two multivariate time-series data of the same length and dimensions, identify time intervals and variables where they are significantly different . This problem will be referred to as variable selection for two multivariate time series . More specifically, given two time series {\bm{x}}_{1},\dots,{\bm{x}}_{T}\in\mathbb{R}^{D} and {\bm{y}}_{1},\dots,{\bm{y}}_{T}\in\mathbb{R}^{D} , where T is the number of time points and D is the number of variables, the problem is to identify the subintervals of the entire interval [1,T]:=\{1,\dots,T\} and variables d=1,\dots,D where the two time series are significantly different. This problem is motivated from the following situations involving simulators [ 13 , e.g.,] . • Model Validation. One multivariate time series may be output from a simulator of a spatio-temporal phenomenon (e.g., fluid, traffic, etc.), and the other may be real observed data. In this case, one can understand when and where the simulator fails to model reality by solving the above problem. Such insights may be useful in deciding whether the simulator is acceptable for an intended application (e.g., forecasting) and may provide hints on improving the simulator. • Model Comparison. One multivariate time series may be output from a simulator under one scenario (e.g., a traffic simulator during rush hours), and the other may be output from the same simulator under a slightly modified scenario (e.g., some roads are blocked.). Solving the above problem, one can understand when and where this modification influences. • Emulator Validation. One multivariate time series may be output from a simulator, and the other may be from a machine learning model (e.g., deep neural networks) that emulates the simulator. Solving the above problem enables identifying when and where the emulator fails to imitate the ground-truth simulator. In this way, the above problem appears in many scientific and engineering fields where simulators are developed to model spatio-temporal phenomena of interest, such as climate science, fluid mechanics, and digital twin [ 22 , 17 ] . There, the number D of variables can be high (e.g., hundreds to thousands), as it is usually the number of spatial locations or sensors on which observations are simulated. Manually analysing such data to detect the differing variables is time-consuming and cannot be systematic. Thus, an automatic method for variable selection is desired. One possible approach might be formulating the problem as a two-sample variable selection problem (see Sections 1.1 and 3 ), but this requires multiple realisations of each of the two multivariate time series X=({\bm{x}}_{1},\dots,{\bm{x}}_{T})\in\mathbb{R}^{D\times T} and Y=({\bm{y}}_{1},\dots,{\bm{y}}_{T})\in\mathbb{R}^{D\times T} . Here, we identify each multivariate time series as a D\times T matrix. More precisely, if we are given independently and identically distributed (i.i.d.) samples X_{1},\dots,X_{n}\in\mathbb{R}^{D\times T} of X and i.i.d. samples Y_{1},\dots,Y_{m}\in\mathbb{R}^{D\times T} of Y , where n and m are sample sizes, a two-sample variable selection method can select variable-time pairs (d,t) on which the underlying distributions of X and Y differ. However, this approach cannot be directly used in the above problem where only a single pair of multivariate-time series X and Y are given. For example, suppose that X is output from a computationally expensive simulator, which takes one day to finish the simulation. In this case, simulating n samples X_{1},\dots,X_{n} requires n days, so n cannot be large. However, an accurate two-sample testing (and variable selection) would require at least roughly n=50 samples, and this requires 50 days of simulations, which may be unfeasible in practice. Such a computationally expensive simulator is common, since generally a simulator’s cost increases arbitrarily as its resolution becomes higher. In such a situation, assuming many samples of X and Y is not realistic. The present work proposes an approach to variable selection for two multivariate time series. The proposed approach, named “Time-Slicing Variable Selection," divides the entire time interval into multiple subintervals, and performs two-sample variable selection on each subinterval to compare the two time series (see Figure 1 ). This is a meta algorithm in the sense that any existing two-sample variable selection method may be combined with the proposed framework. As a specific example, the proposed approach using a kernel-based two-sample variable selection method [ 18 ] is demonstrated to be effective for the problem, compared with baselines. This is demonstrated in synthetic data experiments and two applications. One application involves fluid simulations, where a simulator and its emulator based on a neural net are compared (emulator validation). The other application focuses on traffic simulations, where one scenario and its modified scenario are compared (model comparison). Through these experiments, we show that the proposed approach can perform interpretable variable selection only from a single pair of two time series data. The paper proceeds as follows. After briefly reviewing related studies in Section 1.1 , we describe the proposed approach in Section 2 and explain two-sample variable selection methods used in our experiments in Section 3 . Section 4 reports synthetic data experiments, Section 5 on fluid simulations, and Section 6 on traffic simulations. 1.1 Related Work To our knowledge, the problem of variable selection for two multivariate time series is new. Thus, we review existing papers that study problems related to (but not exactly the same as) our problem. Comparison of Multivariate Time Series. Tapinos and Mendes [26] propose a semi-metric to compare two multivariate time series with different dimensions. To compare two simulation outputs, Cess and Finley [5] propose to train a neural network for

Read the complete text