MobileDev-Bench: A Comprehensive Benchmark for Evaluating Language Models on Mobile Application Development

Moshood A. Fakorede, Krishna Upadhyay, A.B. Siddique, Umar Farooq · 2026

Read MobileDev-Bench: A Comprehensive Benchmark for Evaluating Language Models on Mobile Application Development by Moshood A. Fakorede, Krishna Upadhyay, A.B. Siddique, Umar Farooq (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24946.txt.

Description

Fakorede and co-authors introduce MobileDev-Bench, a benchmark for issue resolution in mobile app development. Existing software benchmarks use library-style repositories, leaving mobile work untested. Their set holds 407 real issue-resolution tasks from 19 production apps across Android, React Native, and Flutter. Each task pairs a verified developer issue with executable test patches for automatic validation. Fixes are complex, touching 12.9 files and 334.6 lines on average, and 41 percent span several artifact types. Four frontier models resolve only about three to six percent of tasks end to end.

Catalog record

CREATORMoshood A. Fakorede, Krishna Upadhyay, A.B. Siddique, Umar Farooq
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73901
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.24946

Browse the full library · About the Tower of Babel