Temporal Representation Learning of Phenotype Trajectories for pCR Prediction in Breast Cancer
Ivana Janíčková, Yen Y. Tan, Thomas H. Helbich, Konstantin Miloserdov, Zsuzsanna Bago-Horvath, Ulrike Heber, Georg Langs · 2025
Read Temporal Representation Learning of Phenotype Trajectories for pCR Prediction in Breast Cancer by Ivana Janíčková, Yen Y. Tan, Thomas H. Helbich, Konstantin Miloserdov, Zsuzsanna Bago-Horvath, Ulrike Heber, Georg Langs (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-14872.txt.
Introduction
Ivana Janíčková leads a team of seven authors predicting breast cancer treatment response from imaging over time in a 2025 medical AI paper. The target is pathological complete response, where no cancer remains after chemotherapy given before surgery. Responses vary widely between patients, so early prediction could guide therapy choices.
Their model learns from magnetic resonance imaging scans taken during treatment. Changes across scans form trajectories in a learned latent space, and a classifier reads those trajectories to predict response. A multi-task design also encourages smooth trajectories over time and handles the wide variety among non-responders. On the public ISPY-2 dataset, accuracy rose steadily as more imaging time points were added, reaching its best with all four scans. The work shows early imaging dynamics carry real predictive signal.
Catalog record
| CREATOR | Ivana Janíčková, Yen Y. Tan, Thomas H. Helbich, Konstantin Miloserdov, Zsuzsanna Bago-Horvath, Ulrike Heber, Georg Langs |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 21 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-14872
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Temporal Representation Learning of Phenotype Trajectories for pCR Prediction in Breast Cancer Ivana Janíčková, Yen Y. Tan, Thomas H. Helbich, Konstantin Miloserdov, Zsuzsanna Bago-Horvath, Ulrike Heber, Georg Langs Abstract Effective therapy decisions require models that predict the individual response to treatment. This is challenging since the progression of disease and response to treatment vary substantially across patients. Here, we propose to learn a representation of the early dynamics of treatment response from imaging data to predict pathological complete response (pCR) in breast cancer patients undergoing neoadjuvant chemotherapy (NACT). The longitudinal change in magnetic resonance imaging (MRI) data of the breast forms trajectories in the latent space, serving as basis for prediction of successful response. The multi-task model represents appearance, fosters temporal continuity and accounts for the comparably high heterogeneity in the non-responder cohort.In experiments on the publicly available ISPY-2 dataset, a linear classifier in the latent trajectory space achieves a balanced accuracy of 0.761 using only pre-treatment data ( T 0 T_{0} ), 0.811 using early response ( T 0 + T 1 T_{0}+T_{1} ), and 0.861 using four imaging time points ( T 0 → T 3 T_{0}\rightarrow T_{3} ). The full code can be found here: https://github.com/cirmuw/temporal-representation-learning 1 Introduction Pathological complete response (pCR) to neoadjuvant chemotherapy (NACT) of breast cancer is a key marker of success determined on the basis of tissue resected during surgery [ 16 ] . Predicting pCR by assessing early response dynamics can steer treatment decisions. Even after concluded NACT, it may inform important choices such as forgoing surgery in case of expected pCR, given sufficient prediction reliability. While single time point observations lack information on subtle, dynamic changes to treatment [ 10 , 19 ] , longitudinal imaging may capture changes associated with individual treatment efficacy or disease progression. Here, we propose a multi-task model to learn trajectory representations of imaging features observed during treatment. We show how a simple classifier can use this representation to predict future pCR with high accuracy. Our approach addresses the challenge of high inter-label similarity [ 13 ] and relatively substantial inter-individual variability not associated with treatment response. The dynamics of early response enables better prediction, while multi-task representation learning accounts for the response heterogeneity. Related work Prior efforts to predict pCR in breast cancer imaging have used radiomics-based [ 16 , 9 , 14 ] and deep learning-based approaches [ 23 , 10 , 11 , 19 , 1 , 3 , 4 ] . These methods largely focus on single [ 11 , 14 , 1 , 4 ] or two time point predictions [ 23 , 9 , 19 ] , limiting their ability to capture the full temporal dynamics of tumor progression. Although some studies incorporate multiple time points for pCR prediction [ 10 , 3 ] , only [ 10 ] explicitly models temporal relationships using an LSTM layer [ 8 ] . Using three imaging time points, the model achieved AUC = 0.706, Sensitivity = 0.483 and Specificity = 0.773 [ 10 ] . Learning temporal relationships by creating individualized treatment progression trajectories or leveraging the learned temporal relationships to improve single time point predictions remains unexplored. The publicly available ISPY-2 dataset consists of series of MRIs taken before and during NACT [ 15 , 18 ] . It provides an opportunity to address these limitations by modelling temporal dynamics more effectively. Existing methods typically classify patients as responders (positive) or non-responders (negative) [ 21 , 23 , 22 ] . For instance, [ 23 ] pre-trains a pCR prediction model by clustering pre- and post-NACT non-pCR images (assuming minimal change during NACT) while separating pre- and post-NACT pCR images (assuming greater change). However, this binary framework overlooks the heterogeneity among non-responders, including partial responders. The classification performance of this approach resulted in a test set AUC of 0.695 for the binary pCR label [ 23 ] . Contribution We propose a representation learning approach for image trajectories observed during treatment. We assume that, over time, responders change similarly, while non-responders are more heterogeneous since they include both partial responders and non-responders. The multi-task model learns to embed appearance change, while fostering temporal continuity by a dynamic-margin triplet loss adapted to nuanced temporal relationships. It aligns trajectories of responders to identify common temporal dynamics associated with successful treatment response, while accounting for the heterogeneity within the non-responder group. It avoids label-driven loss functions for non-responders and instead identifies response-specific patterns by aligning positive-outcome trajectories. A multi-task attention mechanism (MTAN) [ 17 ] enables the focus on feature changes associated with disease and treatment as opposed to comparably static inter individual variability of anatomy. Experiments on the multi-center ISPY-2 dataset [ 15 , 18 ] demonstrate that the learned representation enables high prediction accuracy with a linear classifier, even with a single or only few time points. The MRI-based classification performance surpasses the state of the art as reported in [ 23 , 10 ] . To our knowledge, this is the first pre-training method designed to (1) explicitly distinguish responder vs. non-responder progression patterns and (2) encode temporal relationships within ISPY-2. Unlike [ 10 ] our approach directly learns temporal dynamics for improved treatment progression modeling. Additionally, it leverages learned trajectories to refine single time point predictions, such as early pCR prediction at T_{0} , enhancing its ability to capture treatment response patterns. Figure 1: Method Overview Multi-task representation learning balances reconstruction performance L_{Rec} with temporal continuity of trajectories L_{Temp} , and alignment of changes in responders L_{Align} . A U-shaped denoising network extracts multi-scale features via its encoder. An MTAN module [ 17 ] steers attention across these tasks. The resulting trajectory representations are used for pCR prediction with a linear classifier. 2 Method For each patient i , we observe a time-series of images \mathbf{x}^{i}_{t} for t=0,\dots,T acquired before ( t=0 ), during, and after treatment ( t=T ), and a label y^{i}\in\{0,1\} , where 1 denotes a positive outcome (achieving pCR) and 0 a negative outcome (not achieving pCR). The multi-task training model for representing images integrates three loss functions: a triplet loss [ 20 ] with dynamic margin for temporal modeling L_{Temp} , a cosine similarity loss for responder alignment L_{Align} , and a reconstruction loss for robust image feature learning L_{Rec} . A set of two distinct augmented examples is generated for each training example: A_{1}(\mathbf{x}^{i}_{t}) and A_{2}(\mathbf{x}^{i}_{t}) . The final multiscale representations, derived from the encoder f , are then given by: \mathbf{z}^{i}_{t}=f(A_{1}(\mathbf{x}^{i}_{t})) , \mathbf{\bar{z}}^{i}_{t}=f(A_{2}(\mathbf{x}^{i}_{t})) as shown in Fig. 1 . Our model is a U-shaped encoder-decoder network, where the multi-scale representations are generated from the encoder. The representation tensors are normalized to a unit hypersphere and were used to predict the pCR in patients. 2.1 Constructing Multi-scale Visual Representations We train a U-shaped encoder-decoder network to obtain a multi-scale representation of images, while at the same time fostering temporal continuity and alignment of trajectories observed in responders (Fig. 1 ). Fine-grained features are extracted from the encoder’s multi-scale feature maps. These are then pooled, projected and concatenated into a multi-scale representation tensor \mathbf{z} . To ensure that the extracted features are anatomically relevant, we incorporate a reconstruction task: L_{Rec}=\sum_{i\in N}\sum_{t\in T}E(\mathbf{x}^{i}_{t},\mathbf{\hat{x}}^{i}_{t}) (1) Here, \mathbf{x}^{i}_{t}