A Modular Multitask Reasoning Framework Integrating Spatio-temporal Models and LLMs
Kethmi Hirushini Hettige, Jiahao Ji, Cheng Long, Shili Xiang, Gao Cong, Jingyuan Wang · 2025
Read A Modular Multitask Reasoning Framework Integrating Spatio-temporal Models and LLMs by Kethmi Hirushini Hettige, Jiahao Ji, Cheng Long, Shili Xiang, Gao Cong, Jingyuan Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-20073.txt.
Introduction
Kethmi Hirushini Hettige and co-authors present STReason, a framework that joins large language models with space-time data models to answer complex queries, in a 2025 paper. Space-time mining guides choices in traffic, weather, and many other fields. Yet most models handle one narrow task and cannot reason at length or explain their answers.
STReason breaks a question asked in plain words into a modular program, then runs that program on its analytic models. Because every claim rests on computed output, the explanations avoid the made-up facts that direct prompting often yields. It needs no task-specific retraining. The team also built a benchmark and metrics for long-form space-time reasoning.
Catalog record
| CREATOR | Kethmi Hirushini Hettige, Jiahao Ji, Cheng Long, Shili Xiang, Gao Cong, Jingyuan Wang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-20073
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Modular Multitask Reasoning Framework Integrating Spatio-temporal Models and LLMs Kethmi Hirushini Hettige, Jiahao Ji, Cheng Long, Shili Xiang, Gao Cong, Jingyuan Wang Abstract Spatio-temporal data mining plays a pivotal role in informed decision making across diverse domains. However, existing models are often restricted to narrow tasks, lacking the capacity for multi-task inference and complex long-form reasoning that require generation of in-depth, explanatory outputs. These limitations restrict their applicability to real-world, multi-faceted decision scenarios. In this work, we introduce STReason, a novel framework that integrates the reasoning strengths of large language models (LLMs) with the analytical capabilities of spatio-temporal models for multi-task inference and execution. Without requiring task-specific finetuning, STReason leverages in-context learning to decompose complex natural language queries into modular, interpretable programs, which are then systematically executed to generate both solutions and detailed rationales. To facilitate rigorous evaluation, we construct a new benchmark dataset and propose a unified evaluation framework with metrics specifically designed for long-form spatio-temporal reasoning. Experimental results show that STReason significantly outperforms advanced LLM baselines across all metrics, particularly excelling in complex, reasoning-intensive spatio-temporal scenarios. Human evaluations further validate STReason’s credibility and practical utility, demonstrating its potential to reduce expert workload and broaden the applicability to real-world spatio-temporal tasks. We believe STReason provides a promising direction for developing more capable and generalizable spatio-temporal reasoning systems. 1 Introduction In the realm of data science, spatio-temporal data, characterized by both spatial and temporal dimensions, plays a critical role in a wide array of fields such as environmental monitoring [ 1 , 2 ] , urban planning and traffic management [ 3 , 4 ] , and public health [ 5 ] . Over the years, research in spatio-temporal data mining has progressed from conventional statistical and machine learning approaches [ 6 ] to advanced deep learning frameworks [ 7 , 8 , 9 ] . In recent years, the development of Foundation Models (FMs) has sparked a surge in research aimed at improving spatio-temporal modeling through Large Language Models (LLMs) [ 3 , 10 , 11 ] . By leveraging the strengths of LLMs in generalization, cross-modal reasoning, and long-sequence modeling, several LLM-based spatio-temporal models have been developed for various applications [ 12 , 13 , 14 , 15 ] with notable performance improvements in zero-shot and few-shot scenarios. Despite significant advancements, current LLM-based spatio-temporal models exhibit critical limitations. First , while these models are commonly applied for numerical tasks such as forecasting, their use in inferential problem-solving, such as reasoning or decision-making, remains underexplored [ 3 , 13 , 16 , 17 ] . For example, a forecasting system might violate real-world constraints, and erroneously predict traffic speeds that exceed safety thresholds, thus limiting interpretability and reliability of their outputs. Second , although foundation models generalize well, their performance compared to specialized smaller spatio-temporal models, is still debatable [ 18 , 19 ] . This raises the need to reassess the trade-offs between scalability, efficiency, and task optimization by proposing a hybrid approach that combines the strengths of foundation models and expert spatio-temporal models. Third , most current models are restricted to fixed spatio-temporal input formats; typically tensors with pre-defined dimensions (e.g. [batch, time, location, feature]) and struggle with processing natural language queries, highlighting a gap in their utility as general-purpose AI systems [ 13 , 15 , 20 , 21 ] . Building on these limitations, few recent studies have explored models for spatio-temporal reasoning, though they predominantly focus on highly task-specific inference tasks. Majority of these approaches convert spatio-temporal data into textual descriptions, for processing by LLMs [ 22 , 23 , 24 ] . This translation often leads to significant information loss, shallow reasoning, and inability to capture complex dependencies inherent in spatio-temporal phenomena. To tackle these limitations, certain program-based approaches have been introduced. For example, UrbanLLM [ 25 ] finetunes LLMs for urban planning by breaking down queries into sub-tasks handled by spatio-temporal AI models. However, its effectiveness is limited by the specific urban contexts it was trained on, reducing generalizability, and its reliance on pre-trained models may hinder detailed geospatial understanding. Similarly, TS-Reasoner [ 26 ] decomposes complex time-series tasks, yet it remains confined to niche time-series applications within climate and energy data, highlighting a gap in versatility and broader applicability. Moreover, the outputs of the aforementioned models are often brief and lack the depth required for long-form answers, limiting their practical utility. Long-form question answering [ 27 ] where comprehensive, explanatory, and interpretable outputs are generated from complex inputs still remains underexplored in spatio-temporal settings. Figure 1 : Comparison between Human Expert and STReason Model workflows for answering a complex spatio-temporal query. Addressing these limitations requires a paradigm shift to a task and domain-agnostic framework that can adapt to complex spatio-temporal problems and generate outputs with rich context, depth, and clarity making results more interpretable and actionable. Consider answering a query on analyzing historical air quality data for anomalies, integrating temporal and spatial reasoning by considering corresponding weather patterns and nearby locations as shown in Figure 1 . Typically, a human expert (Left) begins by understanding the intent of the query, followed by executing a series of analytical steps and finally providing a comprehensive answer that synthesizes all gathered insights with appropriate reasoning. While existing LLM-based systems can handle these individual steps in isolation, they fall short of effectively executing complex, language driven spatio-temporal queries end-to-end. They struggle to fully comprehend inputs, integrate multi-dimensional knowledge, and produce reasoned, actionable outputs. Bridging this gap requires advanced LLM agents that function like full-stack data scientists, supporting the entire pipeline from data ingestion and analysis to interpretation and decision-making. In response to these challenges, we introduce STReason, a novel framework that combines the reasoning and comprehension strengths of LLMs with the analytical power of state-of-the-art spatio-temporal models for multi-task inference and execution. Without requiring task-specific fine-tuning, STReason leverages LLMs to decompose complex tasks, articulated in natural language, into structured programs ("ST Program") using predefined in-context query–program example pairs and a Function Pool; a curated dictionary of available modules and their specifications that guides the LLM in aligning sub-tasks with the appropriate executable functions (see Figure 1 (Right)). The generated ST Programs are then executed by specialized, end-to-end trained models or tailored analytical programs to produce coherent, long-form answers that go beyond simple numeric predictions to include structured reasoning, explanatory narratives and meaningful interpretations inspired by long-form question answering [ 27 ] . A demonstration of STReason can be viewed at: https://anon.to/T5lL94 . STReason revolutionizes spatio-temporal data analysis by addressing the limitations of conventional LLMs or LLM agents with limited spatio-temporal understanding [ 28 , 29 ] . Unlike task-specific models [ 25 , 3 ] , STReason requires no pre-training, enhancing flexibility and generalization across diverse domains. Highlighting its versatility, we apply STReason to three primary tasks in the traffic and air quality domains. These tasks include (i) Spatio-temporal Analysis, (ii) Spatio-temporal Anomaly Detection, and (iii) Spatio-temporal Prediction and Reasoning. The modular nature [ 30 ] facilitates easy customization, promoting adaptation to new tasks without significant retraining. Moreover, STReason’s structured execution allows for validation of logic and inspection of intermediate outputs crucial for high accountability domains like environmental monitoring. To enable rigorous evaluation, we