Multimodal Latent Reasoning via Predictive Embeddings

Ashutosh Adhikari & Mirella Lapata · 2026

Read Multimodal Latent Reasoning via Predictive Embeddings by Ashutosh Adhikari & Mirella Lapata (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-08065.txt.

Description

Adhikari and Lapata propose Pearl, a framework for multimodal latent reasoning via predictive codes. Tool-augmented visual language models improve perception by calling external tools like cropping and depth maps, but pay heavy run costs, need special training, and risk bad calls. Pearl learns from expert tool-use paths in latent space, following a JEPA-inspired design, so no explicit tool use is needed at run time. Unlike rebuild-based latent reasoning, which generates latent tokens in steps and suffers training mismatch, Pearl directly learns predictive codes from mixed paths.

Catalog record

CREATORAshutosh Adhikari & Mirella Lapata
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55856
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.08065

Browse the full library · About the Tower of Babel