Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition

Jiamin, Ju, Yiteng, Tyler, Zhaojiang, Zhaojun, Peng, Prashant, Sangeeta, Ming, Florian · 2025

Read Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition by Jiamin, Ju, Yiteng, Tyler, Zhaojiang, Zhaojun, Peng, Prashant, Sangeeta, Ming, Florian (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-14973.txt.

Description

A team led by Jiamin Xie built a speech model that can tell where a voice comes from. Their twenty twenty-five paper uses the microphone array on smart glasses, which hears sound from many angles at once. The task is hard when several people talk at the same time.

Their model, directional-SpeechLlama, writes down speech, finds its source direction, and damps cross-talk from bystanders. Two tricks teach it direction sense. One trains it to list speakers in a fixed order tied to direction. The other builds contrast data that pairs the same words with distinct directions. Tests show strong scores on both transcription and source finding.

Catalog record

CREATORJiamin, Ju, Yiteng, Tyler, Zhaojiang, Zhaojun, Peng, Prashant, Sangeeta, Ming, Florian
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-14973

Browse the full library · About the Tower of Babel