NEXUS-O: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision
Che Liu ∗ \text{Che Liu}^{*}, Yingji Zhang ∗ \text{Yingji Zhang}^{*}, Dong Zhang* Weijie Zhang* Chenggong Gong*, Yu Lu, Shilin Zhou, Ziliang Gan, Ziao Wang, Haipang Wu Ji Liu, Andre Freitas, Qifan Wang, Zenglin Xu, Rongjunchen Zhang † \dagger, Yong Dai † ‡ \text{Yong Dai}^{\dagger}\ddagger · 2025
Read NEXUS-O: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision by Che Liu ∗ \text{Che Liu}^{*}, Yingji Zhang ∗ \text{Yingji Zhang}^{*}, Dong Zhang* Weijie Zhang* Chenggong Gong*, Yu Lu, Shilin Zhou, Ziliang Gan, Ziao Wang, Haipang Wu Ji Liu, Andre Freitas, Qifan Wang, Zenglin Xu, Rongjunchen Zhang † \dagger, Yong Dai † ‡ \text{Yong Dai}^{\dagger}\ddagger (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01879.txt.
Description
One model that can see, hear, and talk at once is an industry goal, and Che Liu and co-authors present their entry in a 2025 paper. Their pipeline lets builders mix and match encoders, a language core, and decoders like interchangeable parts. Instead of training vision from scratch, they bolt ears onto a strong existing vision-language model.
A synthetic speech factory supplies extra training audio drawn from everyday scenes. The resulting model beat its own vision backbone at image understanding, led a spoken question benchmark among its peers, and held its own at speech making. Adding ears, they found, even tightened how sight and language line up inside.
Catalog record
| CREATOR | Che Liu ∗ \text{Che Liu}^{*}, Yingji Zhang ∗ \text{Yingji Zhang}^{*}, Dong Zhang* Weijie Zhang* Chenggong Gong*, Yu Lu, Shilin Zhou, Ziliang Gan, Ziao Wang, Haipang Wu Ji Liu, Andre Freitas, Qifan Wang, Zenglin Xu, Rongjunchen Zhang † \dagger, Yong Dai † ‡ \text{Yong Dai}^{\dagger}\ddagger |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 39 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-01879