Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation
Pafue Christy Nganjimi [email protected], Andrew Soltan [email protected], Danielle Belgrave [email protected], Lei Clifton [email protected], David Clifton [email protected], Anshul Thakur [email protected] · 2026
Read Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation by Pafue Christy Nganjimi [email protected], Andrew Soltan [email protected], Danielle Belgrave [email protected], Lei Clifton [email protected], David Clifton [email protected], Anshul Thakur [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21638.txt.
Introduction
Nganjimi and five colleagues examine dataset condensation. A small synthetic dataset tries to keep the training value of a much larger real one, a useful trick for governed fields like health care. Trajectory matching steers synthetic data from parameter changes seen in real training, but can only span a limited range. Their Bezier Trajectory Matching replaces jagged training paths with smooth quadratic curves. It matches or beats standard matching on five clinical datasets.
Catalog record
| CREATOR | Pafue Christy Nganjimi [email protected], Andrew Soltan [email protected], Danielle Belgrave [email protected], Lei Clifton [email protected], David Clifton [email protected], Anshul Thakur [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 108652 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.21638
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Geometric Characterisation and Structured Trajectory Surrogates for Clinical Dataset Condensation Thanks: An earlier version of this work appeared as a preprint. Pafue Christy Nganjimi [email protected], Andrew Soltan [email protected], Danielle Belgrave [email protected], Lei Clifton [email protected], David Clifton [email protected], Anshul Thakur [email protected] Abstract Dataset condensation constructs compact synthetic datasets that retain the training utility of large real-world datasets, enabling efficient model development and potentially supporting downstream research in governed domains such as healthcare. Trajectory matching (TM) is a widely used condensation approach that supervises synthetic data using changes in model parameters observed during training on real data, yet the structure of this supervision signal remains poorly understood. In this paper, we provide a geometric characterisation of trajectory matching, showing that a fixed synthetic dataset can only reproduce a limited span of such training-induced parameter changes. When the resulting supervision signal is spectrally broad, this creates a conditional representability bottleneck. Motivated by this mismatch, we propose Bézier Trajectory Matching (BTM), which replaces SGD trajectories with quadratic Bézier trajectory surrogates between initial and final model states. These surrogates are optimised to reduce average loss along the path while replacing broad SGD-derived supervision with a more structured, lower-rank signal that is better aligned with the optimisation constraints of a fixed synthetic dataset, and they substantially reduce trajectory storage. Experiments on five clinical datasets demonstrate that BTM consistently matches or improves upon standard trajectory matching, with the largest gains in low-prevalence and low-synthetic-budget settings. These results indicate that effective trajectory matching depends on structuring the supervision signal rather than reproducing stochastic optimisation paths. 1 INTRODUCTION Modern machine learning systems increasingly rely on large-scale datasets ( 13 ) , yet storing, sharing, and repeatedly training on such datasets remains computationally and operationally expensive. In clinical settings, this challenge is compounded by strict governance and access constraints on electronic health records (EHRs) ( 21 ; 16 ) , which limit data sharing, restrict multi-centre collaboration, and can make external validation across diverse real-world cohorts more difficult. Dataset condensation ( 23 ; 26 ) addresses this problem by constructing compact synthetic datasets that retain the training utility of the original data. In governed domains, such condensed datasets may support downstream methodological research and model development where direct data access is restricted. However, they do not by themselves provide formal privacy guarantees, and their safe deployment would require combining them with differential privacy. (a) Parameter-space trajectories (b) Training loss profiles (c) Trajectory-matching optimisation Figure 1 : Illustrative comparison between raw SGD teacher trajectories and Bézier trajectory surrogates used in BTM. (a) Between the same initial and final model states, raw SGD can follow jagged, directionally variable paths, whereas a Bézier surrogate (mode connected path) provides a smoother low-complexity connection. (b) The two forms of supervision also differ functionally. SGD training exhibits irregular epoch-to-epoch loss behaviour, while the Bézier surrogate defines a smooth loss profile between the same endpoints. (c) In trajectory matching, replacing raw SGD segments with structured surrogate supervision can yield a more stable optimisation objective and faster convergence. Trajectory Matching (TM) ( 2 ; 5 ; 7 ) is a widely used approach to dataset condensation. Rather than matching gradients or feature statistics at a single point, TM supervises synthetic data using changes in model parameters observed during training on real data. By transferring optimisation dynamics into synthetic datasets, TM provides a direct mechanism for encoding task-relevant structure. Despite strong empirical performance, however, progress in TM has largely relied on empirical heuristics—such as trajectory regularisation, selecting which parts of the training trajectory to match, smoothing ( 5 ; 7 ; 28 ) , and scalability improvements ( 3 ) —without a principled understanding of the supervision signal being matched. This leaves a basic question unresolved. Which aspects of real-data training trajectories can a fixed synthetic dataset actually reproduce? In TM, supervision is conveyed through changes in model parameters observed during training on real data, but the structure of this signal and its representational demands are not well understood. Without such an understanding, it remains unclear why trajectory matching performs well in some settings, degrades in others, or how its supervision should be structured to make condensation more effective. In this work, we answer this question through a geometric characterisation of trajectory matching. We show that, for any fixed synthetic dataset, the trajectory matching residual is lower bounded by how well the parameter changes induced by training on real data can be approximated within the gradient span reachable from that dataset. This leads to a conditional representability bottleneck. When the resulting supervision signal is spectrally broad relative to the reachable span, some components necessarily lie outside what the synthetic dataset can reproduce, and a non-zero matching error remains unavoidable. We then study this mismatch empirically for the supervision signal induced by standard SGD training. We find that the parameter changes used by TM often exhibit broad spectral structure, with non-negligible mass distributed across many weakly aligned directions. This effect appears across stochastic mini-batching, random initialisations, and distinct trained solutions, although we do not attempt to isolate the contribution of each source. The mismatch is especially pronounced in low-prevalence settings, where updates associated with the minority class are sparser and less consistently reinforced during training. As a result, standard TM may require a fixed synthetic dataset to reproduce a supervision signal that is broader than its reachable span, offering a plausible explanation for weaker performance in rare-event clinical prediction. To mitigate this limitation, we propose Bézier Trajectory Matching (BTM), which replaces discrete SGD trajectories with task-optimised quadratic Bézier surrogates connecting the initial and final model states. The approach is inspired by mode connectivity ( 6 ; 4 ; 20 ) , but our setting differs from the classical case because one endpoint is a random initialisation rather than a second converged solution. A quadratic Bézier curve provides a simple nonlinear path between the endpoints through a single learned control point, allowing the surrogate to capture curvature without storing a long sequence of SGD iterates. In practice, this yields smoother start-to-solution paths whose loss decreases more consistently from initialisation to solution. Compared with standard SGD trajectories, the resulting supervision signal is more structured and has lower effective rank, making it better aligned with the representational constraints of a fixed synthetic dataset while also substantially reducing trajectory storage. Figure 1 provides an intuitive comparison between raw SGD supervision and the structured Bézier surrogates used in BTM. The figure illustrates the smoother parameter-space geometry and loss profile of the surrogate paths, together with their effect on trajectory-matching optimisation. We evaluate BTM on five real-world clinical datasets spanning tabular and time-series modalities. Across these tasks, BTM matches or outperforms prior baselines, with the largest gains in low-prevalence and low-synthetic-budget settings, where identifying positive cases reliably is especially important. The main contributions of this paper are summarised as follows: • Geometric characterisation of trajectory matching: We formulate TM for a fixed synthetic dataset as a constrained subspace-approximation problem and show that its residual is lower bounded by the projection error of real-data parameter changes onto the dataset’s reachable gradient span. This reveals a conditional representability bottleneck when that span is limited. • Empirical analysis of standard trajectory supervision: We show that supervision induced by standard