MobileDev-Bench: A Comprehensive Benchmark for Evaluating Language Models on Mobile Application Development
Moshood A. Fakorede, Krishna Upadhyay, A.B. Siddique, Umar Farooq · 2026
Read MobileDev-Bench: A Comprehensive Benchmark for Evaluating Language Models on Mobile Application Development by Moshood A. Fakorede, Krishna Upadhyay, A.B. Siddique, Umar Farooq (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-24946.txt.
Description
Fakorede and co-authors introduce MobileDev-Bench, a benchmark for issue resolution in mobile app development. Existing software benchmarks use library-style repositories, leaving mobile work untested. Their set holds 407 real issue-resolution tasks from 19 production apps across Android, React Native, and Flutter. Each task pairs a verified developer issue with executable test patches for automatic validation. Fixes are complex, touching 12.9 files and 334.6 lines on average, and 41 percent span several artifact types. Four frontier models resolve only about three to six percent of tasks end to end.
Catalog record
| CREATOR | Moshood A. Fakorede, Krishna Upadhyay, A.B. Siddique, Umar Farooq |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73901 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.24946