TSRating: Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment

Shunyu Wu, Dan Li, Haozheng Ye, Zhuomin Chen, Jiahui Zhou, Jian Lou, Zibin Zheng, See-Kiong Ng · 2025

Read TSRating: Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment by Shunyu Wu, Dan Li, Haozheng Ye, Zhuomin Chen, Jiahui Zhou, Jian Lou, Zibin Zheng, See-Kiong Ng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01290.txt.

Introduction

Shunyu Wu and co-authors present a framework for grading the quality of time series data gathered from many different fields, in this 2025 paper. Models are only as good as their training data, and time series, streams of measurements over time, vary wildly between medicine, finance, industry, and weather. Existing quality scoring methods work within one domain and stumble across domains with different shapes and rhythms.

Their system, TSRating, borrows the broad knowledge of large language models. Prompts first draw out the models' quality judgments on pairs of samples. A compact rating model is then trained to imitate those judgments, so future data can be scored quickly without calling the big model each time. A meta-learning scheme teaches the rater to adapt across domains. The goal is one trustworthy quality score for messy, mixed collections of time series.

Catalog record

CREATORShunyu Wu, Dan Li, Haozheng Ye, Zhuomin Chen, Jiahui Zhou, Jian Lou, Zibin Zheng, See-Kiong Ng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-01290

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TSRating: Rating Quality of Diverse Time Series Data by Meta-learning from LLM Judgment Shunyu Wu, Dan Li, Haozheng Ye, Zhuomin Chen, Jiahui Zhou, Jian Lou, Zibin Zheng, See-Kiong Ng Abstract High-quality time series (TS) data are essential for ensuring TS model performance, rendering research on rating TS data quality indispensable. Existing methods have shown promising rating accuracy within individual domains, primarily by extending data quality rating techniques such as influence functions and Shapley values to account for temporal characteristics. However, they neglect the fact that real-world TS data can span vastly different domains and exhibit distinct properties, hampering the accurate and efficient rating of diverse TS data. In this paper, we propose TSRating, a novel and unified framework for rating the quality of time series data crawled from diverse domains. TSRating is built on the assumption that LLMs inherit ample knowledge, acquired during their extensive pretraining, enabling them to comprehend and discern quality differences in diverse TS data. We verify this assumption by devising a series of prompts to elicit quality comparisons from LLMs for pairs of TS samples. We then fit a dedicated rating model, termed TSRater, to convert the LLMs’ judgments into efficient quality predictions via TSRater’s inference on future TS samples. To ensure cross - domain adaptability, we develop a meta - learning scheme to train TSRater on quality comparisons collected from nine distinct domains. To improve training efficiency, we employ signSGD for inner-loop updates, thus circumventing the demanding computation of hypergradients. Extensive experimental results on eleven benchmark datasets across three time series tasks, each using both conventional TS models and TS foundation models, demonstrate that TSRating outperforms baselines in terms of estimation accuracy, efficiency, and domain adaptability. The code for TSRating is available at https://github.com/clsr1008/TSRating . 1 Introduction Time series data arise ubiquitously across a broad range of domains, including healthcare [ 52 ] , finance [ 8 ] , industrial manufacturing [ 17 ] , and traffic monitoring [ 41 ] , which could exhibit vast distinctions in data characteristics from one domain to another. Recent advances in large language model (LLM) techniques have unleashed new potential for leveraging these time series data to achieve promising performance in myriad time series tasks such as forecasting [ 65 , 57 , 47 ] , imputation [ 16 , 69 , 20 ] , and classification [ 18 , 9 , 73 , 13 ] . There are roughly two caterogies of approaches, one finetunes or prompts off-the-shelf LLMs with format-converted time series data [ 44 , 25 ] , and the other trains time series foundation models from scratch on vast time series datasets by adapting from LLM-inspired model architectures and training strategies [ 4 , 23 , 60 , 48 , 75 , 21 ] . In both approaches, the quality of time series data is critical to model performance, as low-quality data can negate the benefits of the bespoke technical advancements, even potentially leading to detrimental performance. In practice, time series datasets are frequently marred by pervasive quality issues, including missing values, sensor-failure-induced corruption, and irregular sampling [ 16 , 69 , 43 , 78 ] . Consequently, it has become a pressing need to accurately and efficiently assess the quality of time series data crawled from diverse domains. Existing attempts in this direction point out the necessity of developing tailored quality estimation methods for time series data to account for their unique temporal characteristics. For instance, TimeInf [ 79 ] adapts the influence function [ 27 ] to time series data, drawing inspiration from works like [ 38 ] (Infinitesimal robustness for autoregressive processes) and [ 51 ] (Influence functionals for time series). Similarly, TimeShap [ 5 ] extends the Shapley value [ 64 ] , a cooperative game-theoretic attribution method, to time series data for quality assessment. However, current methods have largely neglected the fact that real-world time series data originate from diverse domains, rendering their effectiveness limited to a single domain. Furthermore, both influence function and Shapley value-based techniques face challenges in striking a balance between estimation fidelity and computational efficiency. Specifically, influence functions require computationally intensive Hessian and gradient calculations, while Shapley values incur exponential computational costs. Recently, data quality estimation through the LLMs’ judgments has emerged as a promising approach in the text field, offering accurate quality assessments by guiding LLMs with carefully designed prompts [ 74 , 26 ] . The underlying rationale is that LLMs are proficient in understanding quality criteria for natural language, such as writing style, required expertise, factual accuracy, and educational value. As a result, LLMs can be successfully leveraged to assess the quality of text data. Propelled by this success in the text data, it is tempting to explore whether LLMs can be exploited to determine the quality of diverse time series data. This approach has the potential to leverage LLMs’ inherent knowledge of time series data from diverse domains that are acquired during their large-scale pretraining, thereby circumventing the prohibitive computational costs of calculating the influence function or Shapley value in a domain-by-domain manner. However, it remains under-explored whether LLMs can fully comprehend the key characteristics that underpin the quality rating for diverse time series data, and how to effectively steer them to distinguish between high- and low-quality time series data based on desired quality criteria. Our Work. In this paper, we introduce TSRating, a novel and unified framework for rating the quality of time series data across diverse domains. First, we confirm that LLMs can indeed comprehend time series quality and discern differences in data quality along four criteria widely recognized as fundamental for characterizing time series data: trend, frequency, amplitude, and pattern. Next, we train a quality - prediction model, TSRater, which can efficiently rate the quality of large volumes of future time series data, incurring only minimal amortized computational cost once trained. To enhance adaptability across diverse domains, TSRater is trained via meta - learning scheme on nine distinct domains, leveraging a large-scale and publicly available dataset released very recently by Time - MoE [ 65 ] . We further alleviate training - time computation by using signSGD for inner - loop updates, thereby avoiding the computational burden of hypergradient calculations. Experimental results across eleven benchmark datasets and three representative time series tasks show that TSRating consistently surpasses state-of-the-art data selection baselines, including methods based on Shapley values and influence functions. We also perform data pruning experiments demonstrating that TSRating effectively pinpoints critical high-quality samples, whose removal significantly harms model performance. Additionally, ablation studies confirm the essential contribution of LLM-based criteria. Notably, a case study on time series foundation models further demonstrates TSRating’s practical value: fine-tuning on high-quality subsets selected by TSRating significantly enhances generalization performance. These findings corroborate TSRating as an effective solution for time series quality assessment. Our Appendix and code can be found in Supplementary Material . 2 Related Works General Data Quality Estimation. A range of data attribution methods has been proposed to assess the contribution of individual training samples to model behavior. Influence functions [ 37 , 12 ] , originating from robust statistics, estimate the marginal impact of a sample on model predictions or parameters by computing the product of gradients and the

Read the complete text