MindFlow: Harmonizing Cognitive Semantics and Acoustic Dynamics for Facial Animation Generation in Dyadic Conversations
Hejia Chen, Haoxian Zhang, Xu He, Xiaoqiang Liu, Pengfei Wan, Shoulong Zhang, Shuai Li · 2026
Read MindFlow: Harmonizing Cognitive Semantics and Acoustic Dynamics for Facial Animation Generation in Dyadic Conversations by Hejia Chen, Haoxian Zhang, Xu He, Xiaoqiang Liu, Pengfei Wan, Shoulong Zhang, Shuai Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-27779.txt.
Description
Animating a face for a two-person conversation is a balancing act. The face must reflect what is meant and felt. It must also move with the fine timing of speech. Hejia Chen and six colleagues propose MindFlow in 2026. MindFlow is a dual-pathway framework inspired by the ventral–dorsal model in neuroscience. One stream handles deep semantic reasoning. The other handles fine-grained motion control.
The ventral module replaces sentence-level modelling with a Chunk-State approach. Raw audio becomes an evolving chain of emotional states. Subtle tone cues and mid-sentence emotion shifts are captured that way. The dorsal module generates facial motion with a conditional flow matching network. High-frequency audio cues drive it, emotion states modulate it. A selective acoustic injector gates audio adaptively. That keeps talking and listening phases from interfering. Experiments show better semantic appropriateness and motion naturalness than state-of-the-art baselines.
Catalog record
| CREATOR | Hejia Chen, Haoxian Zhang, Xu He, Xiaoqiang Liu, Pengfei Wan, Shoulong Zhang, Shuai Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.27779