A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyond

Shreya Shukla, Jose Torres, Abhijit Mishra, Jacek Gwizdka, Shounak Roychowdhury School of Information, University of Texas at Austin {shreya.shukla, jtorres1221, abhijitmishra, jacekg, shounak.roychowdhury}@utexas.edu · 2025

Read A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyond by Shreya Shukla, Jose Torres, Abhijit Mishra, Jacek Gwizdka, Shounak Roychowdhury School of Information, University of Texas at Austin {shreya.shukla, jtorres1221, abhijitmishra, jacekg, shounak.roychowdhury}@utexas.edu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-12048.txt.

Introduction

Shreya Shukla and co-authors survey in 2025 work that turns brain wave recordings into images, text, and sound with generative AI. Note that the head byline lists more authors than the catalog field. Decoding brain activity into readable output is a central aim of brain-computer interfaces. They searched major databases for studies from 2017 to 2025 and extracted methods, data sets, and metrics.

Image models mostly use encoder-decoder designs with adversarial, variational, or diffusion methods. Text decoding leans on transformer language models. Audio methods map signals to spectrograms, then render sound. Small, varied data sets and weak cross-person generalization remain the field's main limits.

Catalog record

CREATORShreya Shukla, Jose Torres, Abhijit Mishra, Jacek Gwizdka, Shounak Roychowdhury School of Information, University of Texas at Austin {shreya.shukla, jtorres1221, abhijitmishra, jacekg, shounak.roychowdhury}@utexas.edu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-12048

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyond Shreya Shukla, Jose Torres, Abhijit Mishra, Jacek Gwizdka, Shounak Roychowdhury School of Information, University of Texas at Austin {shreya.shukla, jtorres1221, abhijitmishra, jacekg, shounak.roychowdhury}@utexas.edu Abstract Integration of Brain-Computer Interfaces (BCIs) and Generative Artificial Intelligence (GenAI) has opened new frontiers in brain signal decoding, enabling assistive communication, neural representation learning, and multimodal integration. BCIs, particularly those leveraging Electroencephalography (EEG), provide a non-invasive means of translating neural activity into meaningful outputs. Recent advances in deep learning, including Generative Adversarial Networks (GANs) and Transformer-based Large Language Models (LLMs), have significantly improved EEG-based generation of images, text, and speech. This paper provides a literature review of the state-of-the-art in EEG-based multimodal generation, focusing on (i) EEG-to-image generation through GANs, Variational Autoencoders (VAEs), and Diffusion Models, and (ii) EEG-to-text generation leveraging Transformer based language models and contrastive learning methods. Additionally, we discuss the emerging domain of EEG-to-speech synthesis , an evolving multimodal frontier. We highlight key datasets, use cases, challenges, and EEG feature encoding methods that underpin generative approaches. By providing a structured overview of EEG-based generative AI, this survey aims to equip researchers and practitioners with insights to advance neural decoding, enhance assistive technologies, and expand the frontiers of brain-computer interaction. 1 Introduction & Motivation The convergence of Brain-Computer Interfaces (BCIs) and Generative Artificial Intelligence (GenAI) is transforming human-computer interaction by enabling direct brain-to-device communication. These advancements have enabled applications in assistive communication for individuals with disabilities, cognitive neuroscience, mental health assessment, augmented reality (AR)/virtual reality (VR), and neural art generation. Electroencephalography (EEG), a widely used non-invasive neural recording technique, enables both passive and active Brain-Computer Interfaces (BCIs) and holds potential for applications in real-time adaptive human-computer interaction Zander et al. 2010 ; Wolpaw and Boulay 2010 . Recent advancements in deep learning and generative models have significantly improved the decoding of EEG signals, enabling the translation of neural activity into text, images, and speech. Specifically, Generative AI, including Generative Adversarial Networks (GANs) Goodfellow et al. 2014 and Transformers Vaswani et al. 2017 , has significantly advanced brain decoding, facilitating visual reconstruction, language generation, and speech synthesis Bai et al. 2023 ; Srivastava and Shinde 2020 ; Lee et al. 2023b . GANs improve cross-subject classification and EEG data augmentation Song et al. 2021 , while Transformer-based architectures and multimodal deep learning frameworks Liu et al. 2024a ; Wang and Ji 2022 enhance EEG-to-text translation and semantic decoding Ali et al. 2024 , pushing the boundaries of brain-signal interpretation. Figure 1: General Steps from EEG Data Gathering to Stimuli Reconstruction (Image, Text, or Sound) In light of recent breakthroughs in Generative AI , this survey provides a scope review of recent advancements in EEG-based generative AI, with a focus on two primary directions. The first explores how brain signals can be used to generate or reconstruct visual stimuli , utilizing models such as GANs and Diffusion Models to decode perceptual representations. The second investigates the application of deep learning for EEG-to-text translation , where recurrent neural network and Transformers Vaswani et al. 2017 based language models, and contrastive learning techniques play a crucial role in learning linguistic representation. The survey also examines emerging trends in speech decoding from EEG signals and multimodal integration considerations surrounding the use of generative AI for brain signal interpretation. Through this, we hope to provide a structured understanding of EEG-based generative AI to researchers and practitioners, offering insights to drive innovation in neural decoding, assistive technology, and brain-computer interaction. Before proceeding, we would like to highlight that Figure 1 provides a high-level overview of the EEG-to-stimuli generation pipeline, illustrating the key stages from neural data acquisition to the generation of text, images, or audio. Additionally, Table 1 summarizes key datasets that incorporate EEG and other modalities, serving as a valuable resource for researchers exploring multimodal neural decoding. Finally, Figure 2 centralizes a detailed overview of the techniques and approaches covered in this survey, contextualizing their purpose and applications in EEG-based generative modeling. 2 Related Work Several surveys have explored EEG-based brain-computer interfaces (BCIs) and deep learning techniques for neural decoding. Chen et al. 2022 provides a broad overview of EEG-based BCI applications, discussing signal processing methodologies and traditional machine learning techniques but lacks an in-depth analysis of generative models. Gong et al. 2021 and Weng et al. 2024 review deep learning techniques, particularly CNNs and RNNs, for EEG classification, emotion recognition, and cognitive state monitoring, yet they focus more on supervised learning approaches rather than self-supervised and generative modeling. More recently, Murad and Rahimi 2024 examined EEG-to-text decoding but primarily covered classification-based approaches without exploring the role of large language models (LLMs) or multimodal integration with vision-based EEG applications. Additionally, Sun and Mou 2023 discusses self-supervised learning (SSL) techniques for EEG but does not emphasize their applicability to generative tasks, such as EEG-to-image or EEG-to-text generation. Our survey specifically examines the intersection of EEG and generative AI, focusing on recent approches for EEG-to-text and EEG-to-image translation. While previous studies focus on EEG feature extraction and classification, our work highlights how self-supervised learning, contrastive learning, and multimodal alignment improve EEG-based generation. Dataset Stimuli Type Channels/Electrodes Stimuli Details Subjects Zuco 1.0 Hollenstein et al. 2018 Text 128 Sentences from Stanford Sentiment Treebank, Wikipedia corpus 12 Zuco 2.0 Hollenstein et al. 2019 Text 128 Expanded subjects with similar content as Zuco 1.0 18 Alice Bhattasali et al. 2020 Text 61 + 1 ground 2,129 words, 84 sentences from Alice in Wonderland 52 Envisioned Speech Kumar et al. 2018 Imagined Speech 14 20 text stimuli (digits, characters), 10 objects 23 Alljoined Xu et al. 2024 Image 64 10,000 images per participant from 80 MS-COCO categories 8 ImageNet EEG Spampinato et al. 2017 Image 128 40 ImageNet classes, 50 images/class, 2000 total 6 DM-RE2I Zeng et al. 2023b Image 32 200 ImageNet images across 26 subjects 26 Texture Perception Orima and Motoyoshi 2021 Image 19 166 grayscale natural texture images 15 THINGS-EEG Grootswagers et al. 2022 Image 64 22,248 images across 1854 object concepts 50 DCAE Zeng et al. 2023b Image 32 200 ImageNet images (cats, dogs, flowers, pandas) 26 ThoughtViz Tirupattur et al. 2018 Imagined Objects 14 EEG recorded while participants imagined digits, characters, and objects 23 OCED Kaneshiro et al. 2015 Image 128 12 images per 6 object categories 10 NMED-T Losorelli et al. 2017 Music 128 10 songs (4:30-5:00 mins) with tempos 56-150 BPM 20 NMED-H Kaneshiro et al. 2016 Music 125 4 versions of 4 songs, total 16 stimuli 48 KARA ONE Zhao and Rudzicz 2015 Text, Audio, Speech 64 Rest state, stimulus, imagined speech, speaking task 12 Japanese Speech EEG Mizuno et al. 2024 Audio 64 503 spoken sentences (male/female speaker) 1 Phrase/Word Speech EEG Park et al. 2024 Audio 64 Audio of 13 words/phrases, followed by speech replication 10 Table 1: EEG-Based Datasets from Surveyed Studies with Text, Image and Audio/Speech/Music Stimuli EEG-Text Studies and main use cases CNNs •Generate medical reports Biswal et al. 2019 • Translate user’s active intent to text represented by morse code Srivastava and Shinde 2020 • Folded ensemble technique to minimize the computational complexity and solve all the class imbalance issues to enhance the accuracy of text generation

Read the complete text