TraCeR : Transformer-Based Competing Risk Analysis with Longitudinal Covariates

Maxmillan Ries, Sohan Seth · 2025

Read TraCeR : Transformer-Based Competing Risk Analysis with Longitudinal Covariates by Maxmillan Ries, Sohan Seth (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-18129.txt.

Introduction

Maxmillan Ries and Sohan Seth present TraCeR, a survival analysis tool that learns from a patient's full history. Survival models guess how long until an event such as death or relapse. Most past tools took one snapshot of a patient, missing how measures change visit by visit, and few were checked for honest odds.

TraCeR reads the whole sequence of measurements with a transformer, the same family behind modern chatbots, and needs no set rules about how risk must behave. It handles patients who leave a study early and cases where several events compete. On real data sets it beat the best past methods by clear margins, and the team also graded how well its odds matched real rates.

Catalog record

CREATORMaxmillan Ries, Sohan Seth
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-18129

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

TraCeR : Transformer-Based Competing Risk Analysis with Longitudinal Covariates Maxmillan Ries, Sohan Seth Abstract Survival analysis is a critical tool for modeling time-to-event data. Recent deep learning-based models have reduced various modeling assumptions including proportional hazard and linearity. However, a persistent challenge remains in incorporating longitudinal covariates, with prior work largely focusing on cross-sectional features, and in assessing calibration of these models, with research primarily focusing on discrimination during evaluation. We introduce TraCeR , a transformer-based survival analysis framework for incorporating longitudinal covariates. Based on a factorized self-attention architecture, TraCeR estimates the hazard function from a sequence of measurements, naturally capturing temporal covariate interactions without assumptions about the underlying data-generating process. The framework is inherently designed to handle censored data and competing events. Experiments on multiple real-world datasets demonstrate that TraCeR achieves substantial and statistically significant performance improvements over state-of-the-art methods. Furthermore, our evaluation extends beyond discrimination metrics and assesses model calibration, addressing a key oversight in literature. Introduction Time-to-event analysis, or survival analysis, is a statistical framework for modeling the duration until an event occurs ( 27 ) . It aims to understand the relationship between the timing of an event (e.g., equipment failure or disease onset) and a set of predictive covariates (e.g., telemetry or lab results). The focus of this analysis is the hazard function , which quantifies the instantaneous risk of an event at a specific time, given that the event has not yet occurred. This framework is a powerful tool for decision making in fields ranging from medicine, where it guides treatment plans, to finance, where it can model credit card default ( 17 ) . However, applying survival analysis to real-world data introduces two significant challenges that can lead to biased models if ignored. First, data is often censored : the true event time is masked because a subject is lost to follow-up or the observation period ends, i.e., we only know that the subject was event-free up to a certain point in time. Second, subjects are often exposed to competing risks , where the occurrence of one event (e.g., patient death) precludes the occurrence of another (e.g., onset of cancer). The presence of competing risks introduces a critical distinction from censoring. While a censored subject is assumed to remain “at risk” for the event of interest, a subject who experiences a competing event is not, and therefore, not addressing for competing risk might lead to overestimation of risk ( 28 ) . This distinction has implications for the application of classical survival models. For instance, in the widely used Cox Proportional Hazards (CoxPH) model, competing risks are treated as censored observations ( 24 ) . This simplification is statistically inconsistent with the non-informative censoring assumption. Furthermore, classical models are frequently constrained by other fundamental assumptions, e.g., the CoxPH model operates under a strict “proportional hazard” assumption, requiring the ratio of hazard rates between subjects to be constant over time ( 6 ) . Moreover, such models are not inherently designed to utilize longitudinal data (e.g., routine lab tests) and often require experts to derive a cross-sectional summary vector. Model Static Cov. Long. Cov. Comp. Events Censor. Miss. Cov. Non-P.H. 1 Discr. 2 Cali. 2 Output Core Arch. CoxPH ✓ ✗ ▲ ✓ ✗ ✗ ✓ ✓ Log-risk Linear Model DeepSurv ✓ ✗ ✗ ✓ ✗ ✗ ✓ ✗ Log-risk MLP DeepHit ✓ ✗ ✓ ✓ ✗ ✓ ✓ ✗ PMF MLP DynamicDeepHit ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ PMF RNN MMT ∗ ( 29 ) ✓ ✓ ✗ ✓ ✗ ✓ ✓ ✗ Hazard Transformer N-MTLR ∗ ( 6 ) ✓ ✗ ✗ ✓ ✗ ✓ ✓ ⚫ Hazard MLP SurvFormer ∗ ( 19 ) ✓ ✗ ✗ ✓ ✗ ✓ ✓ ✗ Hazard Transformer SurvTRACE ✓ ✗ ✓ ✓ ✗ ✓ ✓ ✗ Hazard Transformer SurvTimeSurvival ✓ ✓ ✗ ✓ ✗ ✓ ✓ ⚫ Hazard Transformer TBDSA ∗ ( 10 ) ✓ ✗ ✗ ✓ ✗ ✓ ✓ ✓ Hazard Transformer TraCeR ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Hazard Transformer Table 1: Summary of prior work. ▲ : With extensions. ⚫ : Partially. 1 : Proportional Hazard. ∗ Not included in comparison. 1 Discr/Cali reported To overcome these limitations, recent research employed recurrent neural architectures (RNNs), such as Gated Recurrent Units (GRUs) and Long-Short Term Memory (LSTM) networks ( 26 ) . These data-driven models allow the relaxation of strong parametric assumptions and directly allow the inclusion of longitudinal measurements, allowing the temporal relationship between subject history and risk to be captured. While a significant improvement over classical models, the sequential and summarization nature of RNNs struggle with capturing long-term dependencies ( 35 ) . More recently, the discovery of the Transformer architecture has presented a compelling solution, demonstrating superior performance in capturing long-range dependencies and complex interactions to RNNs. Models such as SurvTRACE first demonstrated the capability of self-attention to learn complex relationships in cross-sectional data ( 34 ) . Subsequent work, such as SurvTimeSurvival, extended this work to longitudinal settings by employing a transformer to summarize temporal information, and subsequently feeding the representation into SurvTRACE ( 16 ) . Contribution This paper presents a Transformer-based architecture, TraCeR , for survival analysis with longitudinal covariates. We extend previous work applying the Transformer architecture to (predominantly) cross-sectional data, to a longitudinal setting. To robustly handle missing data that is ubiquitous in clinical applications, TraCeR first employs a covariate missingness gate that uses a time-decay function to modulate feature importance based on the time elapsed since the last observation ( 2 ) . The processed inputs are then fed into a factorized attention mechanism that operates across both the temporal and covariate axes ( 22 ) . This allows the model to explicitly capture the evolving influence of covariates over time. Rather than using only the final hidden state, TraCeR then generates a single context vector through a learned summarization of the entire sequence of temporal outputs. This vector, which represents the subject’s history, is finally passed to cause-specific subnetworks to estimate the discrete-time hazards for each potential event. To demonstrate the effectiveness of our method, we compare TraCeR ’s performance against competing prior works on four challenging longitudinal datasets derived from the eICU ( 23 ) , MIMIC-IV ( 11 ) , PBC2 ( 20 ) , and MIMIC-III ( 12 ) . These cohorts feature competing risks and, in the eICU dataset, a high degree of censoring. We evaluate performance on two key aspects, discrimination and calibration . Our results show that TraCeR achieves state-of-the-art performance on both fronts. Critically, we also discuss how these two aspects of performance are often conflated in prior work. Our deeper analysis reveals that TraCeR provides not only superior discrimination but also significantly better-calibrated risk estimates, a crucial factor for clinical trust and utility. Related Work The most widely used statistical model in medical research and clinical survival analysis is the Kaplan-Meier estimation. While fundamental for estimating survival functions from a population, it does not incorporate subject-specific covariates into its estimation ( 13 ) . To address this, the semi-parametric CoxPH model has become a cornerstone of clinical trials and observational studies by allowing the inclusion of a static set of

Read the complete text