MoCHA: Denoising Caption Supervision for Motion-Text Retrieval
Nikolai Warner, Cameron Ethan Taylor, Irfan Essa, Apaar Sadhwani · 2026
Read MoCHA: Denoising Caption Supervision for Motion-Text Retrieval by Nikolai Warner, Cameron Ethan Taylor, Irfan Essa, Apaar Sadhwani (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-23684.txt.
Introduction
Warner and three co-authors clean up motion captions so text and movement match better. One motion earns many captions, and each mixes true cues with a writer's style and guesses. Standard training treats one caption as the sole right answer, which blurs the shared space. MoCHA rewrites each caption down to what the motion itself can show. A big language model does the rewrite, or a small distilled model at run time. Scores jump on both standard sets, and transfer across datasets soars.
Catalog record
| CREATOR | Nikolai Warner, Cameron Ethan Taylor, Irfan Essa, Apaar Sadhwani |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58786 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.23684
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MoCHA: Denoising Caption Supervision for Motion-Text Retrieval Nikolai Warner, Cameron Ethan Taylor, Irfan Essa, Apaar Sadhwani Abstract Text-motion retrieval systems learn shared embedding spaces from motion-caption pairs via contrastive objectives. However, each caption is not a deterministic label but a sample from a distribution of valid descriptions: different annotators produce different text for the same motion, mixing motion-recoverable semantics (action type, body parts, directionality) with annotator-specific style and inferred context that cannot be determined from 3D joint coordinates alone. Standard contrastive training treats each caption as the single positive target, overlooking this distributional structure and inducing within-motion embedding variance that weakens alignment. We propose MoCHA, a text canonicalization framework that reduces this variance by projecting each caption onto its motion-recoverable content prior to encoding, producing tighter positive clusters and better-separated embeddings. Canonicalization is a general principle: even deterministic rule-based methods (e.g., stopword stripping) improve cross-dataset transfer, though learned canonicalizers provide substantially larger gains. We present two learned variants: an LLM-based approach (GPT-5.2) and a distilled FlanT5 model requiring no LLM at inference time. MoCHA operates as a preprocessing step compatible with any retrieval architecture. Applied to MoPa (MotionPatches), MoCHA sets a new state of the art on both HumanML3D (H) and KIT-ML (K): the LLM variant achieves 13.9% T2M R@1 on H (+3.1pp) and 24.3% on K (+10.3pp), while the LLM-free T5 variant achieves +2.5pp / +8.1pp. Canonicalization reduces within-motion text-embedding variance by 11–19% and improves cross-dataset transfer substantially—H → \to K by +94% and K → \to H by +52%—demonstrating that standardizing the language space yields more transferable motion-language representations. 1 Introduction “a person nervously walks forward , stops , turns around , and walks back .” ann. 1 “a person is walking around and they are scared .” ann. 2 “a person walks forward and back glancing frighteningly to his sides .” ann. 3 Same motion, three annotators: MoCHA: “ walk forward \to stop \to turn around \to walk back ” deterministic, motion-recoverable \blacksquare motion-recoverable semantics ( s ) \blacksquare nuisance factors ( a ): stylistic variation Figure 1 : Each caption is a different sample from a distribution of valid descriptions. Three annotators describe the same motion (top) with different captions, each mixing motion-recoverable semantics s ( blue ) with annotator-specific nuisance factors a ( red )—stylistic variation. Standard contrastive training treats each as the single correct target; MoCHA projects each onto s , producing a single deterministic positive. Motion-text retrieval—the task of matching natural language descriptions with 3D human motion sequences—is a foundational capability for motion generation [ 1 , 2 ] , action understanding, and human-computer interaction. Recent contrastive approaches [ 1 , 5 , 6 ] learn joint embedding spaces by treating each motion-caption pair as clean ground truth. We argue that this treatment is misspecified: caption supervision is structurally noisy, with each annotation mixing kinematically grounded content with annotator style, hallucinated intent, and non-kinematic additions that have little bearing on the joint coordinates. This noise structure has not been formally analyzed in the motion retrieval literature. Multi-caption datasets expose this as a many-to-one problem: the same motion of walking forward, stopping, and turning back receives “nervously walks forward, stops, turns around, and walks back,” “walking around and they are scared,” and “walks forward and back glancing frighteningly”—three independent draws from a distribution over valid descriptions (Fig. 1 ). The shared motion content is buried under annotator-specific style and speculation: “nervously,” “scared,” and “frighteningly” are hallucinated emotional states that have no signature in the joint coordinates. Yet standard contrastive objectives treat each draw as the ground-truth target. No existing retrieval method models this many-to-one structure; the positive key \mathbf{k}_{+} —the text embedding that the contrastive loss pulls toward the motion query—is implicitly assumed to be deterministic when it is in fact stochastic. The problem compounds across datasets. HumanML3D [ 2 ] and KIT-ML [ 3 ] describe overlapping motions but with different noise distributions: HumanML3D captions are verbose and intent-laden, while KIT-ML uses terse templates. A model trained on one noise distribution is miscalibrated for another—not because the motions differ, but because the corruption process differs. Cross-dataset failure is a direct consequence of fitting to dataset-specific supervision noise rather than to the shared latent motion semantics. We propose MoCHA (Motion Canonicalization for Human Action retrieval), a supervision denoising framework that projects captions onto their motion-recoverable content before they enter the contrastive objective. We formalize caption noise as a decomposition into motion-recoverable semantics s and nuisance factors a , which capture annotator-dependent variation such as linguistic style and inferred context. We then define a canonicalization operator C(t) that removes a from the input caption t . The operator is first implemented via an LLM (GPT-5.2), then distilled into a lightweight FlanT5-base model that requires no LLM at inference time. Because MoCHA operates entirely on the text supervision channel, it is compatible with any retrieval architecture. We introduce blend training , a dual-pass contrastive scheme that balances denoised and original caption views during training, avoiding over-canonicalization while retaining the denoised alignment signal. Our contributions are: 1. We define a text canonicalization operator C(t) that projects captions onto motion-recoverable content s , stripping annotator-specific nuisance a . Canonicalization is a general principle: even rule-based variants improve retrieval. C is implemented via GPT-5.2 and distilled into FlanT5 for LLM-free inference. 2. We propose MoCHA with blend training, which balances canonicalized and original captions during contrastive learning. MoCHA sets a new state of the art on HumanML3D (H), KIT-ML (K), and cross-dataset H \to K retrieval (+3.1pp/+10.3pp T2M R@1 on H/K). 3. We show that denoising supervision outperforms augmenting it: paraphrase augmentation widens p(t|s) and can degrade R@1, while canonicalization consistently improves all ranks by collapsing within-motion embedding variance by 11–19%. 4. We provide empirical evidence that caption supervision contains systematic non-kinematic noise: canonicalization produces tighter positive clusters, better text-motion alignment, and improved separation from negatives in the learned embedding space. 2 Related Work Motion–language learning. Text-conditioned motion generation has progressed from VAE-based models [ 12 , 23 ] through diffusion [ 18 , 19 ] , discrete-token [ 20 , 21 ] , and masked-modeling [ 22 ] approaches, with retrieval typically serving as an auxiliary evaluation. TMR [ 1 ] formalized text-to-motion retrieval as a standalone task via contrastive training, and MotionPatches (MoPa) [ 6 ] later showed that ViT-style patch tokenization with InfoNCE can outperform heavier generative formulations. CLIP [ 14 ] provides a widely reused text-embedding prior; MotionCLIP [ 13 ] and LaMP [ 15 ] adapt vision–language pretraining to the motion domain. Motion–language datasets. HumanML3D [ 2 ] ( {\sim} 15k motions, {\sim} 45k captions) and KIT-ML [ 3 ] ( {\sim} 3.9k motions, {\sim} 6.3k captions) are the primary benchmarks, sharing AMASS [ 10 ] motion capture but differing sharply in annotation style: HumanML3D uses crowd-sourced free-form sentences while KIT-ML uses semi-templated descriptions. BABEL [ 4 ] provides short action labels for {\sim} 43k temporal segments, offering broad coverage but minimal linguistic detail. These stylistic differences—verbosity, templating, granularity—create a domain gap that confounds cross-dataset evaluation even when the underlying motions overlap. Cross-dataset transfer. TMR++ [ 5 ] established cross-dataset protocols and showed that LLM paraphrases can partially reduce the annotation gap, but substantial