QuAnTS: Question Answering on Time Series

\name Felix Divo \email [email protected], \name Maurice Kraus \email [email protected], \name Anh Q. Nguyen \email [email protected], \name Hao Xue \email [email protected], \name Imran Razzak \email [email protected], \name Flora D. Salim \email flora.sa · 2025

Read QuAnTS: Question Answering on Time Series by \name Felix Divo \email [email protected], \name Maurice Kraus \email [email protected], \name Anh Q. Nguyen \email [email protected], \name Hao Xue \email [email protected], \name Imran Razzak \email [email protected], \name Flora D. Salim \email flora.sa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05124.txt.

Introduction

Felix Divo and co-authors present QuAnTS, a data set for answering questions about time series, a field that has drawn far less work than image or text questions. Text can open dense numeric records to users and aid choices.

QuAnTS asks many kinds of questions about human motion, given as tracked skeleton paths over time. Models track the moving joints and reply in words. The set is built to be hard, so scores show real skill at linking numbers to language. It gives builders a shared test for time-aware question systems.

Catalog record

CREATOR\name Felix Divo \email [email protected], \name Maurice Kraus \email [email protected], \name Anh Q. Nguyen \email [email protected], \name Hao Xue \email [email protected], \name Imran Razzak \email [email protected], \name Flora D. Salim \email flora.sa
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE74 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-05124

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

QuAnTS: Question Answering on Time Series \name Felix Divo \email [email protected], \name Maurice Kraus \email [email protected], \name Anh Q. Nguyen \email [email protected], \name Hao Xue \email [email protected], \name Imran Razzak \email [email protected], \name Flora D. Salim \email [email protected], \name Kristian Kersting \email [email protected], \name Devendra Singh Dhami \email [email protected] Abstract Text offers intuitive access to information. This can, in particular, complement the density of numerical time series, thereby allowing improved interactions with time series models to enhance accessibility and decision-making. While the creation of question-answering datasets and models has recently seen remarkable growth, most research focuses on question answering (QA) on vision and text, with time series receiving minute attention. To bridge this gap, we propose a challenging novel time series QA (TSQA) dataset, QuAnTS, for Qu estion An swering on T ime S eries data. Specifically, we pose a wide variety of questions and answers about human motion in the form of tracked skeleton trajectories. We verify that the large-scale QuAnTS dataset is well-formed and comprehensive through extensive experiments. Thoroughly evaluating existing and newly proposed baselines then lays the groundwork for a deeper exploration of TSQA using QuAnTS. Additionally, we provide human performances as a key reference for gauging the practical usability of such models. We hope to encourage future research on interacting with time series models through text, enabling better decision-making and more transparent systems. Keywords: time series, multimodal question answering, dataset 1 Introduction Language is a highly intuitive medium of communication ( Fedorenko et al. 2024 ) . Humans learn to listen and verbally express themselves early on in life, and most kids learn to read and write soon after. For example, we can easily describe perceived objects, their relations, chains of events, theories, uncertainties, and much more in this unifying modality ( Walker and Gopnik 2014 ; Jiang et al. 2023 ) . However, in the age of automation, we often encounter other kinds of data, including temporal data consisting of numerical values recorded over time. Time series are complex and precise pieces of data that are highly relevant in many applications. In contrast to the natural use of text, however, end users are often not immediately able to understand and interact with them ( Eggleton 1982 ) and might require specific training and tooling ( Harold et al. 2015 ; Zhao et al. 2023 ) . Therefore, employing text as a flexible bridge to more intuitively access time series is an exciting opportunity ( Imani et al. 2019 ; Slack et al. 2023 ) . Figure 1 shows how this even allows for back-and-forth discussions contextualized by data. Figure 1 : Text offers a highly intuitive medium of communication to interact with otherwise opaque multivariate time series. Practical machine learning applications require solving a wide variety of tasks on time series, including the classics of forecasting, classification, and anomaly detection. While tremendous progress has been made in building models for each individual use, most are only effective at one, and changes necessitate expensive remodeling and retraining. In the era of large language models (LLMs), we can solve many tasks by embedding both our data and the assignment description as text ( Brown et al. 2020 ; Lewis et al. 2020 ; Fan et al. 2024 ) . This often does not require specific retraining, as providing some examples by in-context learning is often sufficient. It would be highly desirable to tweak outputs of time series models in a similarly flexible way, as some initial works already started exploring ( Xie et al. 2025 ) . However, interacting with time series via text is only just emerging as a field of study. This combination is further motivated by the common need to enrich data with further context and task descriptions, which is eased by the possibility of encoding such information in text ( Han et al. 2023 ) . One example is multimodal forecasting, where textual data enriches numerical time series to provide crucial context ( Jia et al. 2024 ; Williams et al. 2025 ) . Interactive feedback promises even better results by allowing a human-in-the-loop to offer richer information ( Imani et al. 2019 ; Wu et al. 2022 ; Kraus et al. 2025b ) . Generally, time series are used to describe and subsequently analyze weather dynamics ( Hewage et al. 2020 ) , plant growth ( Yasrab et al. 2021 ) , electrocardiograms ( Sahoo et al. 2020 ) , electrical brain activity ( Andrzejak et al. 2001 ) , human activities ( Lara and Labrador 2013 ) , manufacturing processes ( Farahani et al. 2023 ) , traffic volumes ( Jiang and Luo 2022 ) , financial markets ( Nazareth and Ramana Reddy 2023 ; Divo et al. 2025 ) , and much more. In all those fields, combining time series with text could help domain experts better leverage data to drive more informed decisions. For instance, one could supplement traffic volume predictions with information about local festivities, or human activity recognition with details on a person’s disabilities. In other tasks, the textual domain not only serves as context but is an integral part of it, such as answering questions about time series or their textual summarization ( Novák 2016 ) . Yet, we find that very little work has been performed in that direction. We attribute this mainly to the lack of sufficient high-quality data. This limits both the training data to learn from and the opportunities to evaluate new systems. Therefore, we present QuAnTS : A challenging multimodal dataset for question answering on numerical time series data. We use human motion in the form of tracked skeleton positions as the type of signal to reason about. This has several key advantages: (1) It is natural to visualize the numerical values as human figures, making them interpretable for humans while still challenging for machines. (2) This field of application is central to building intelligent devices for, e.g., applications in health and sports. (3) There is a very long line of research on human motion data, highlighting its significance and providing tools and insights we can leverage in this work, such as the synthetic motion generation and visualization tools. To move toward rich interactions with complex time series signals via textual conversations, we contribute the following: (i) We investigate the design space of time series QA datasets to compile a comprehensive and challenging set of question and answer types. (ii) We propose a procedure for generating the diverse and large-scale QuAnTS dataset. It is readily extensible and freely available alongside the dataset itself. 1 1 1 The code for generating QuAnTS can be found at https://github.com/mauricekraus/quants-generate . The dataset DOI is 10.57967/hf/6663 , see https://huggingface.co/datasets/dasyd/quants . (iii) Extensive dataset validation and benchmarking of existing and novel baseline models provide a fertile ground for further research into this very young field. (iv) We provide human performance to contextualize current and future machine capabilities. Outline. We continue by reviewing related work in Section 2 as a foundation for the subsequent description of QuAnTS’s construction in Section 3 . Extensive empirical findings on solving QuAnTS are then presented in Section 4 . We conclude the work with an outlook in Section 5 and provide

Read the complete text