Speak the Art: A Direct Speech to Image Generation Framework
Mariam Saeed, Manar Amr, Farida Adel, Nada Hassan, Nour Walid, Eman Mohamed, Mohamed Hussein, Marwan Torki es-[mariamzaho4, ManarAmr2023, Faridaadel2023, Nada.Hassan2023, nourwalid70, Eman.abdo2023]@[email protected], [email protected] · 2026
Read Speak the Art: A Direct Speech to Image Generation Framework by Mariam Saeed, Manar Amr, Farida Adel, Nada Hassan, Nour Walid, Eman Mohamed, Mohamed Hussein, Marwan Torki es-[mariamzaho4, ManarAmr2023, Faridaadel2023, Nada.Hassan2023, nourwalid70, Eman.abdo2023]@[email protected], [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-00827.txt.
Description
Mariam Saeed and seven co-authors present, in 2026, a system that turns speech straight into pictures. Past systems first coded the speech, then fed the code to a GAN, a kind of image maker. GANs also train poorly, often collapsing to a few same-like outputs. The new system, named Speak the Art, swaps the GAN for a diffusion model, which builds images in steady steps.
Its speech coder learns under the eye of a big pre-trained model that links images and text. That watch forces the code to keep the sense of the words. Training is steadier, and the outputs vary more. The team also stretched the system to two languages, English and Arabic, as a proof of concept. On standard tests it beat the best past systems by a wide margin. The work narrows the gap between spoken and typed prompts for art tools.
Catalog record
| CREATOR | Mariam Saeed, Manar Amr, Farida Adel, Nada Hassan, Nour Walid, Eman Mohamed, Mohamed Hussein, Marwan Torki es-[mariamzaho4, ManarAmr2023, Faridaadel2023, Nada.Hassan2023, nourwalid70, Eman.abdo2023]@[email protected], [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 32 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-00827