CodePercept: Code-Grounded Visual STEM Perception for MLLMs

Tongkun Guan, Zhibo Yang, Jianqiang Wan, Mingkun Yang, Zhengtao Guo, Zijian Hu, Ruilin Luo, Ruize Chen, Songtao Jiang, Peng Wang, Wei Shen, Junyang Lin, Xiaokang Yang · 2026

Read CodePercept: Code-Grounded Visual STEM Perception for MLLMs by Tongkun Guan, Zhibo Yang, Jianqiang Wan, Mingkun Yang, Zhengtao Guo, Zijian Hu, Ruilin Luo, Ruize Chen, Songtao Jiang, Peng Wang, Wei Shen, Junyang Lin, Xiaokang Yang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-10757.txt.

Description

Tongkun Guan and co-authors ask whether weak STEM visual reasoning in multimodal models comes from perception or from reasoning, by scaling each part separately. Scaling perception beat scaling reasoning every time, marking perception as the true bottleneck. Their answer is code as a perceptual medium, since executable code carries precise semantics matching structured STEM visuals. They built ICC-1M, a million image-caption-code triplets, using code as ground truth for captions. A new benchmark tests perception directly through reconstruction code, giving deterministic scores.

Catalog record

CREATORTongkun Guan, Zhibo Yang, Jianqiang Wan, Mingkun Yang, Zhengtao Guo, Zijian Hu, Ruilin Luo, Ruize Chen, Songtao Jiang, Peng Wang, Wei Shen, Junyang Lin, Xiaokang Yang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE109170
LICENSEhttp://creativecommons.org/licenses/by-sa/4.0/
RIGHTSCC BY SA
SOURCEarXiv

Source: https://arxiv.org/abs/2603.10757

Browse the full library · About the Tower of Babel