TimeTox: An LLM-Based Pipeline for Automated Extraction of Time Toxicity from Clinical Trial Protocols
Saketh Vinjamuri, Marielle Fis Loperena, Marie C. Spezia, Ramez Kouzy · 2026
Read TimeTox: An LLM-Based Pipeline for Automated Extraction of Time Toxicity from Clinical Trial Protocols by Saketh Vinjamuri, Marielle Fis Loperena, Marie C. Spezia, Ramez Kouzy (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21335.txt.
Introduction
Vinjamuri and co-authors build TimeTox, a pipeline that extracts time toxicity from clinical trial protocols. Time toxicity counts the health care contact days that trial participation demands from a patient. The pipeline uses Gemini models in three stages. First it extracts summaries from protocol PDFs. Then it counts contact days at six timepoints for each treatment arm. Last, it builds consensus across runs. On six hundred forty four real oncology protocols, a single pass design proves more reproducible than a two stage design.
Catalog record
| CREATOR | Saketh Vinjamuri, Marielle Fis Loperena, Marie C. Spezia, Ramez Kouzy |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39293 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.21335
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
TimeTox: An LLM-Based Pipeline for Automated Extraction of Time Toxicity from Clinical Trial Protocols Saketh Vinjamuri, Marielle Fis Loperena, Marie C. Spezia, Ramez Kouzy Abstract Background: Time toxicity, the cumulative healthcare contact days from clinical trial participation, is an important but labor-intensive metric to extract from protocol documents. We developed TimeTox, an LLM-based pipeline for automated extraction of time toxicity from Schedule of Assessments tables. Methods: TimeTox uses Google’s Gemini models in three stages: summary extraction from full-length protocol PDFs, time toxicity quantification at six cumulative timepoints for each treatment arm, and multi-run consensus via position-based arm matching. We validated against 20 synthetic schedules (240 comparisons) and assessed reproducibility on 644 real-world oncology protocols. Two architectures were compared: single-pass (vanilla) and two-stage (structure-then-count). Results: The two-stage pipeline achieved 100% clinically acceptable accuracy ( ± \pm 3 days) on synthetic data (MAE 0.81 days) versus 41.5% for vanilla (MAE 9.0 days). However, on real-world protocols, the vanilla pipeline showed superior reproducibility: 95.3% clinically acceptable accuracy (IQR ≤ \leq 3 days) across 3 runs on 644 protocols, with 82.0% perfect stability (IQR = 0). The production pipeline extracted time toxicity for 1,288 treatment arms across multiple disease sites. Conclusions: Extraction stability on real-world data, rather than accuracy on synthetic benchmarks, is the decisive factor for production LLM deployment. Code is available at https://github.com/sakethbuild/TimeTox . 1 Fairview Hospital, Cleveland Clinic Foundation, Cleveland, OH, USA 2 The George Washington University School of Medicine and Health Sciences, Washington, DC, USA 3 University of Missouri-Columbia School of Medicine, Columbia, MO, USA 4 The University of Texas MD Anderson Cancer Center, Houston, TX, USA Keywords: time toxicity, large language models, clinical trial protocols, healthcare contact days, schedule of assessments, automated extraction 1 Introduction Clinical research generates a wealth of data that remains largely untapped, hidden within complex document structures and dense PDF files. Clinical trial protocols, often exceeding 100 pages, contain detailed specifications of study procedures, visit schedules, and assessment requirements. Within these documents, there exist additional layers of variables that are inherently difficult to identify, subjective in their interpretation, and dispersed across multiple sections without standardized formatting. These characteristics make systematic extraction and analysis at scale exceedingly difficult through manual approaches alone. One such variable is time toxicity, defined as the cumulative number of healthcare contact days experienced by patients as a result of clinical trial participation 1 , 2 . Time toxicity is primarily embodied in the Schedule of Assessments (SoA) table, a dense, multi-page document embedded within clinical trial protocols that specifies all required activities, their timing, and their applicability to each treatment arm. Critically, the SoA table does not follow a unified presentation format across protocols, making it difficult for regulators, trial coordinators, researchers, and patients themselves to understand the per-protocol time commitment of these trials. While we focus here on time, many other clinically relevant variables are similarly embedded within protocol documents and have been the subject of prior extraction efforts 3 , 4 , 5 . Manually extracting healthcare contact days from SoA tables requires mapping cycle-based schedules to calendar days, expanding grouped column headers (e.g., “Cycles 4 to 12”) into individual cycles, distinguishing arm-specific assessments from universal ones, and accumulating unique contact days across cumulative time windows while avoiding double-counting. This process is time-consuming, error-prone, and difficult to standardize across reviewers, as we demonstrate through our own attempts at manual extraction in Section 3 . Recent advances in large language models (LLMs) have demonstrated remarkable capabilities in processing complex structured documents, including medical texts and tabular data 6 , 7 . LLMs can parse visual table layouts, understand domain-specific terminology, and perform multi-step reasoning over structured content, skills directly applicable to SoA table interpretation. However, deploying LLMs for clinical data extraction introduces its own challenges: output variability across runs, sensitivity to prompt design, and the difficulty of validating extractions at scale when ground truth is scarce. We propose TimeTox, an end-to-end LLM-based pipeline that automates the extraction and calculation of a metric that has traditionally been considered highly subjective. Our contributions include: 1. A complete pipeline from raw protocol PDFs to structured time toxicity data, using Gemini models for both document preprocessing and data extraction. 2. A programmatically generated ground truth dataset of 20 synthetic SoA schedules with clinician-verified time toxicity values, enabling systematic validation. 3. A comparative evaluation of two extraction architectures, single-pass (vanilla) and two-stage (structure then count), revealing a critical trade-off between synthetic accuracy and real-world stability. 4. A position-based consensus mechanism that handles LLM-generated arm name instability across runs, enabling robust multi-run aggregation. 5. Application of the production pipeline to 644 real-world oncology trial protocols, demonstrating feasibility at scale. 2 Data Collection and Preprocessing 2.1 Protocol Acquisition We compiled a dataset of 649 clinical trial protocol documents from ClinicalTrials.gov, spanning multiple oncology disease sites including breast, thoracic, gastrointestinal, genitourinary, gynecologic, head and neck, skin, central nervous system, and hematologic malignancies. Protocols were selected based on the availability of full-text documents with Schedule of Assessments (SoA) tables. 2.2 Summary Extraction Full-length clinical trial protocols typically span 100 to 400 pages, of which only 5 to 15 pages contain the SoA table and related scheduling information. To reduce input size and improve extraction accuracy, we developed an automated summary extraction stage using Google’s Gemini 2.5 Flash model. A typical SoA table is presented in a dense grid format, with assessment categories organized as rows and visit time points arranged as columns. Row categories commonly include administrative procedures (informed consent, randomization), clinical assessments (physical examination, vital signs, performance status), laboratory assessments (complete blood count, metabolic panels, liver function tests), cardiac assessments (ECG, echocardiogram), safety monitoring (adverse events, concomitant medications), study treatment administration and accountability, patient reported outcomes (quality of life instruments), and imaging or tumor assessments. Columns represent discrete visit timepoints, including screening, individual cycle days (e.g., C1D1, C1D8, C2D1), grouped cycle ranges (e.g., C4 to C12), end of treatment, and follow-up visits. Cells are marked with indicators such as “X” for assessments required of all patients and “X A ” for assessments specific to a particular treatment arm. A legend section specifies arm-specific visit patterns and cycle structures, while footnotes provide additional details on assessment timing, visit windows, and procedural requirements. Figure 1: Representative SoA table from a complex synthetic breast oncology protocol (BRST-2025-01) showing two treatment arms with three visit days per cycle. For each protocol PDF, the summary extraction pipeline: 1. Uploads the full PDF to the Gemini API. 2. Prompts the model to identify pages containing SoA tables, visit schedules, assessment timing, or related scheduling information. The prompt instructs the model to be liberal in page identification to avoid missing relevant content. 3. Generates a consolidated summary PDF containing: (a) a concise trial summary (study design, treatment arms, duration), and (b) the identified SoA pages plus one surrounding page on each side for context. Figure 2: Step-by-step pipeline for processing protocol PDFs via the Gemini API to extract relevant schedules and generate a consolidated summary document. The summary extraction uses forced JSON output via response_mime_type="application/json" with a structured schema, temperature 0.0, and top-p 0.95. Processing time averages 2 to 3 minutes per protocol. Of the 649 protocols processed, summaries were successfully generated for 644, yielding