BP-GPT: Auditory Neural Decoding Using fMRI-prompted LLM

Xiaoyu Chen 1, 2 Changde Du 1, 2 Che Liu 1, 3 Yizhe Wang 4 Huiguang He ∗ 1, 2 · 2025

Read BP-GPT: Auditory Neural Decoding Using fMRI-prompted LLM by Xiaoyu Chen 1, 2 Changde Du 1, 2 Che Liu 1, 3 Yizhe Wang 4 Huiguang He ∗ 1, 2 (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-15172.txt.

Description

Xiaoyu Chen and four co-authors decode heard language from brain scans by feeding the scan itself to a language model as a prompt. Reading meaning from functional MRI is a prized goal for brain-computer interfaces, yet earlier systems kept the language model out of the direct mapping. Their system, BP-GPT, turns a brain pattern into a prompt that steers GPT-2 to write the heard text. A matching text prompt helps align and steady the brain prompt during training.

On a public auditory decoding dataset, the method beat the prior best system for every subject tested. Scores rose by up to 4.61 points on METEOR and 2.43 on BERTScore. The gains suggest large language models can do more than polish outputs — they can carry the decoding itself when prompted well. The authors release their code for others to build on.

Catalog record

CREATORXiaoyu Chen 1, 2 Changde Du 1, 2 Che Liu 1, 3 Yizhe Wang 4 Huiguang He ∗ 1, 2
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-15172

Browse the full library · About the Tower of Babel