Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks

Venkatesh Sivaraman, Anika Vaishampayan, Xiaotong Li, Brian R Buck, Ziyong Ma, Richard D Boyce, Adam Perer · 2025

Read Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks by Venkatesh Sivaraman, Anika Vaishampayan, Xiaotong Li, Brian R Buck, Ziyong Ma, Richard D Boyce, Adam Perer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-10526.txt.

Introduction

Venkatesh Sivaraman and co-authors presented Tempo in 2025, a system that helps mixed teams set up prediction tasks over time. Models that forecast health or service outcomes often miss what decision makers expect. Many of those misses trace back to how the task was set: how, when, and for whom forecasts are made.

Tempo uses a simple but exact query language for time-based tasks. Data staff can draft a task setup fast and show the choices behind it. Domain experts can check results within subgroups to see if the model acts as expected. In three case studies, teams used Tempo to drop weak setups early and find better paths to test.

Catalog record

CREATORVenkatesh Sivaraman, Anika Vaishampayan, Xiaotong Li, Brian R Buck, Ziyong Ma, Richard D Boyce, Adam Perer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-10526

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tempo: Helping Data Scientists and Domain Experts Collaboratively Specify Predictive Modeling Tasks Venkatesh Sivaraman, Anika Vaishampayan, Xiaotong Li, Brian R Buck, Ziyong Ma, Richard D Boyce, Adam Perer Abstract . Temporal predictive models have the potential to improve decisions in health care, public services, and other domains, yet they often fail to effectively support decision-makers. Prior literature shows that many misalignments between model behavior and decision-makers’ expectations stem from issues of model specification, namely how, when, and for whom predictions are made. However, model specifications for predictive tasks are highly technical and difficult for non-data-scientist stakeholders to interpret and critique. To address this challenge we developed Tempo, an interactive system that helps data scientists and domain experts collaboratively iterate on model specifications. Using Tempo’s simple yet precise temporal query language, data scientists can quickly prototype specifications with greater transparency about pre-processing choices. Moreover, domain experts can assess performance within data subgroups to validate that models behave as expected. Through three case studies, we demonstrate how Tempo helps multidisciplinary teams quickly prune infeasible specifications and identify more promising directions to explore. Conference: CHI Conference on Human Factors in Computing Systems; April 26-May 1, 2025; Yokohama, Japan CHI Conference on Human Factors in Computing Systems (CHI ’25), April 26-May 1, 2025, Yokohama, Japan DOI: 10.1145/3706598.3713664 ISBN: 979-8-4007-1394-1/25/04 CCS: Human-centered computing Visualization systems and tools CCS: Human-centered computing Collaborative and social computing systems and tools CCS: Computing methodologies Model development and analysis CCS: Information systems Decision support systems Figure 1. Tempo helps data scientists define and evaluate specifications of predictive modeling tasks by bringing non-technical domain experts into the loop early in development. Here, we worked with pharmacy experts to predict hospital readmission for patients receiving home health care (see Sec. 5.3 ). Data scientists on the team developed predictive model specifications using Tempo’s query language, enabling rapid experimentation with different patient cohorts and target variables. Pharmacy experts could then critique the models’ design and suggest improvements by examining subgroups most associated with readmission. † † cc-license: by 1. Introduction Predictive AI models aim to enhance human decision-making by inferring individuals’ future outcomes, relating their past observations against vast amounts of historical data. In expert communities, ranging from medical and public health professionals to public services agencies to business analysts, predictive capabilities are increasingly expected to streamline and standardize decision processes. However, many decision support tools (DSTs) built on predictive models fail to effectively support decision-makers in practice. While in some cases these failures can be attributed to the nature of the AI’s deployment, such as incompatibility with existing workflows or poor usability ( Dziorny et al., 2022 ; Khairat et al., 2018 ; Wang et al., 2021 ) , negative perceptions of predictive models often stem from more fundamental concerns about their behavior. In particular, issues often arise when existing decision processes conflict with the model specification , or the precise task that the predictive model is trained to perform. Model specifications determine what , how , and for whom predictions will be made ( Sherman et al., 2018 ; Tal, 2023 ) , all of which can impact how decision-makers perceive the resulting models to behave. In high-stakes fields such as medical treatment ( Sherman et al., 2018 ; Sivaraman et al., 2023 ) , child welfare ( Kawakami et al., 2022a ) , and housing allocation ( Kuo et al., 2023 ; Cheng et al., 2024 ) , misaligned model specifications can lead experts to avoid using the models out of concern that they will introduce or perpetuate systemic decision-making errors. Therefore, helping data scientists and domain experts collaboratively define the right predictive modeling task is key to developing DSTs that decision-makers find relevant and useful. To demonstrate how these obstacles might play out in a practical scenario (inspired by our previous research collaborations), let us consider a fictional data scientist named Ava who, along with non-technical clinical collaborator Ben, is building a model to predict whether a patient will be readmitted to a hospital in the future. Existing literature and the landscape of available tools ( Santos et al., 2019 ; Amershi et al., 2015 ; Zgraggen et al., 2015 ) suggest that even without the need to incorporate domain experts’ feedback, specifying a model for this task could be a challenging technical problem. In contrast to other types of models, building algorithms to predict future outcomes depends on making sense of temporal event data , which can be sparse and irregularly spaced and therefore demand extensive wrangling and pre-processing ( Gellad et al., 2023 ; Allegheny County et al., 2019 ) . For example, Ava might need to aggregate past diagnoses at varying time intervals and carry forward previous values to produce a clinically meaningful value at each hospital admission, which itself is an irregularly-spaced observation. However, existing tools for data scientists largely focus on the more general, technical problems of “data cleaning” ( Epperson et al., 2023 ; Kandel et al., 2011 ) and model selection ( Krause et al., 2014 ; Amershi et al., 2015 ) without considering the temporal nature of the data. As a result, a representative workflow for Ava’s task might consist of building an elaborate feature extraction pipeline in SQL, performing modeling using Python in a Jupyter Notebook, then exporting model summary statistics to a slide deck to share with Ben. If during their discussion of the model’s behavior Ben identified a specification issue in the model, Ava would likely have to rewrite sizable portions of the code, then execute the rest of the analysis over again. Human-AI interaction literature reflects the ad-hoc nature of model specification and the difficulties of bringing non-technical domain experts into the loop. Most studies with decision-makers often take place only after a model has been fully developed and/or deployed ( Kawakami et al., 2022a ; Kuo et al., 2023 ; Zytek et al., 2021 ; Beede et al., 2020 ; Grgić-Hlača et al., 2018 ; Sivaraman et al., 2023 ) . This delays opportunities for experts to identify critical issues until after a team has largely committed to a specification, such that changes to the specification represent a significant step back in progress. For example, a model that alerted clinicians to patients with sepsis was found to be largely redundant with clinical judgment during deployment ( Guidi et al., 2015 ) , leading to a subsequent overhaul of the system focusing on longer-term prediction ( Ginestra et al., 2019 ) . Ethnographic work with data science and domain expert teams has highlighted the importance of collaboration in finding these pitfalls, but the processes depicted in these studies are unstructured and time-consuming because of the technical challenges described above ( van den Broek et al., 2021 ; Passi and Barocas, 2019 ) . Accelerating this iterative process may require tools to not just help domain experts visually inspect models after the fact ( Amershi et al., 2015 ; Cabrera et al., 2023 ) , but also to support the more technical process of converting a modeling idea into a working prototype. Therefore, this paper seeks to address the following research question: How can we enable data scientists and domain experts to collaboratively specify and prototype models so

Read the complete text