ShadAR: LLM-driven shader generation to transform visual perception in Augmented Reality

Yanni Mei, Samuel Wendt, Florian Müller, Jan Gugenheimer · 2026

Read ShadAR: LLM-driven shader generation to transform visual perception in Augmented Reality by Yanni Mei, Samuel Wendt, Florian Müller, Jan Gugenheimer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17481.txt.

Description

Yanni Mei led a four-author team presenting ShadAR, an augmented reality app that transforms visual perception through language-model shader generation. Headsets can already mimic varied ways of seeing, such as the view of a person with colorblindness. But each effect must be fixed in advance by a developer, which limits choice. ShadAR drops that limit by building shaders on the fly with large language models.

A user states what they want to see in plain words. The model reads that request and writes matching shader code. The code is then compiled at once to change the headset view. The paper sets out this pipeline and shows how it can reshape sight for access and for play, without waiting for a developer to preset each effect.

Catalog record

CREATORYanni Mei, Samuel Wendt, Florian Müller, Jan Gugenheimer
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE7 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17481

Browse the full library · About the Tower of Babel