Time Series, Vision, and Language: Exploring the Limits of Alignment in Contrastive Representation Spaces

Pratham Yashwante, Rose Yu · 2026

Read Time Series, Vision, and Language: Exploring the Limits of Alignment in Contrastive Representation Spaces by Pratham Yashwante, Rose Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-19367.txt.

Description

The Platonic Representation Hypothesis claims that models trained on different modalities converge on a shared latent structure of the world. Pratham Yashwante and Rose Yu test whether time series join that convergence, a case little examined beside vision and language. Independently pretrained encoders for time series, vision, and language turn out nearly orthogonal in geometry without explicit coupling. No shared structure appears on its own.

Post-hoc alignment with contrastive projection heads over frozen encoders improves with model size, but asymmetrically. Time series line up more closely with visual representations than with text. Images can serve as intermediaries between time series and language. Richer textual descriptions help only up to a threshold, with denser captions adding nothing further. The findings guide multimodal systems involving time series.

Catalog record

CREATORPratham Yashwante, Rose Yu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE105 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-19367

Browse the full library · About the Tower of Babel