Frame-Level Internal Tool Use for Temporal Grounding in Audio LMs

Joesph An, Phillip Keung, Jiaqi Wang, Orevaoghene Ahia, Noah A. Smith · 2026

Read Frame-Level Internal Tool Use for Temporal Grounding in Audio LMs by Joesph An, Phillip Keung, Jiaqi Wang, Orevaoghene Ahia, Noah A. Smith (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10230.txt.

Description

Audio language models turn sound into rich frame-level forms. Yet temporal localization usually emits timestamps as text tokens, discarding those forms for slow, non-parallel decoding that hallucinates outside the training distribution. A 2026 paper led by Joseph An, with four co-authors, proposes internal frame-level reuse. It trains audio models to reuse their own inner forms for finding directly, bypassing token generation entirely.

A light prediction head is trained with two frame-level goals: a binary frame classifier and a new inhomogeneous Poisson process loss modeling temporal event intensity. Across word finding, speaker diarization, and event finding, the approach runs over fifty times faster than token-based generation and holds up across audio lengths where token models collapse. Finding accuracy matches or beats fine-tuned token baselines.

Catalog record

CREATORJoesph An, Phillip Keung, Jiaqi Wang, Orevaoghene Ahia, Noah A. Smith
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE37 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-10230

Browse the full library · About the Tower of Babel