Frame-Level Internal Tool Use for Temporal Grounding in Audio LMs
Joesph An, Phillip Keung, Jiaqi Wang, Orevaoghene Ahia, Noah A. Smith · 2026
Read Frame-Level Internal Tool Use for Temporal Grounding in Audio LMs by Joesph An, Phillip Keung, Jiaqi Wang, Orevaoghene Ahia, Noah A. Smith (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-10230.txt.
Description
Audio language models turn sound into rich frame-level forms. Yet temporal localization usually emits timestamps as text tokens, discarding those forms for slow, non-parallel decoding that hallucinates outside the training distribution. A 2026 paper led by Joseph An, with four co-authors, proposes internal frame-level reuse. It trains audio models to reuse their own inner forms for finding directly, bypassing token generation entirely.
A light prediction head is trained with two frame-level goals: a binary frame classifier and a new inhomogeneous Poisson process loss modeling temporal event intensity. Across word finding, speaker diarization, and event finding, the approach runs over fifty times faster than token-based generation and holds up across audio lengths where token models collapse. Finding accuracy matches or beats fine-tuned token baselines.
Catalog record
| CREATOR | Joesph An, Phillip Keung, Jiaqi Wang, Orevaoghene Ahia, Noah A. Smith |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602-10230