MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research
James Burgess*, Jeffrey J Nirschl*, Laura Bravo-Sánchez, Alejandro Lozano, Sanket Rajan Gupte, Jesus G. Galaz-Montoya, Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, Sarina M. Hasan, Alexandra Johannesson, William D. Leineweber, Malvika G Nair, Ridhi Yarlagadda, Connor Zuraski, Wah Chiu, Sar · 2025
Read MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research by James Burgess*, Jeffrey J Nirschl*, Laura Bravo-Sánchez, Alejandro Lozano, Sanket Rajan Gupte, Jesus G. Galaz-Montoya, Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, Sarina M. Hasan, Alexandra Johannesson, William D. Leineweber, Malvika G Nair, Ridhi Yarlagadda, Connor Zuraski, Wah Chiu, Sar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-13399.txt.
Introduction
James Burgess and co-authors build a research-grade test of how well AI models reason over microscope images. Existing benchmarks stop at college level or check only basic seeing. Their MicroVQA set holds 1,042 multiple-choice questions written by biology experts across many microscopy types, probing three skills real research needs: reading expert images, proposing hypotheses, and designing follow-up experiments. A two-stage pipeline, ending with a refining agent, scrubbed the questions of language shortcuts.
The best current models scored only 53 percent. Smaller language engines inside the models cost little performance, suggesting the hard part is the visual reasoning, not the words. Extra training on scientific papers helped. When the authors read the models' reasoning, misreading the image was the commonest failure, ahead of missing knowledge and overgeneralising. They present the set as a sterner yardstick for scientific AI.
Catalog record
| CREATOR | James Burgess*, Jeffrey J Nirschl*, Laura Bravo-Sánchez, Alejandro Lozano, Sanket Rajan Gupte, Jesus G. Galaz-Montoya, Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, Sarina M. Hasan, Alexandra Johannesson, William D. Leineweber, Malvika G Nair, Ridhi Yarlagadda, Connor Zuraski, Wah Chiu, Sar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 113 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-13399
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MicroVQA: A Multimodal Reasoning Benchmark for Microscopy-Based Scientific Research James Burgess*, Jeffrey J Nirschl*, Laura Bravo-Sánchez, Alejandro Lozano, Sanket Rajan Gupte, Jesus G. Galaz-Montoya, Yuhui Zhang, Yuchang Su, Disha Bhowmik, Zachary Coman, Sarina M. Hasan, Alexandra Johannesson, William D. Leineweber, Malvika G Nair, Ridhi Yarlagadda, Connor Zuraski, Wah Chiu, Sarah Cohen, Jan N. Hansen, Manuel D Leonetti, Chad Liu, Emma Lundberg, Serena Yeung-Levy Abstract Scientific research demands sophisticated reasoning over multimodal data, a challenge especially prevalent in biology. Despite recent advances in multimodal large language models (MLLMs) for AI-assisted research, existing multimodal reasoning benchmarks only target up to college-level difficulty, while research-level benchmarks emphasize lower-level perception, falling short of the complex multimodal reasoning needed for scientific discovery. To bridge this gap, we introduce MicroVQA, a visual-question answering (VQA) benchmark designed to assess three reasoning capabilities vital in research workflows: expert image understanding, hypothesis generation, and experiment proposal. MicroVQA consists of 1,042 multiple-choice questions (MCQs) curated by biology experts across diverse microscopy modalities, ensuring VQA samples represent real scientific practice. In constructing the benchmark, we find that standard MCQ generation methods induce language shortcuts, motivating a new two-stage pipeline: an optimized LLM prompt structures question-answer pairs into MCQs; then, an agent-based ‘RefineBot’ updates them to remove shortcuts. Benchmarking on state-of-the-art MLLMs reveal a peak performance of 53%; models with smaller LLMs only slightly underperform top models, suggesting that language-based reasoning is less challenging than multimodal reasoning; and tuning with scientific articles enhances performance. Expert analysis of chain-of-thought responses shows that perception errors are the most frequent, followed by knowledge errors and then overgeneralization errors. These insights highlight the challenges in multimodal scientific reasoning, showing MicroVQA is a valuable resource advancing AI-driven biomedical research. MicroVQA is available here , project here . 1 Stanford University, 2 Tsinghua University, 3 University of North Carolina at Chapel Hill, 4 Princeton University, 5 KTH Royal Institute of Technology, 6 Chan Zuckerberg Biohub Network Figure 1 : A scientific experimentation workflow drives discovery: researchers analyze experiments, develop hypotheses, and design further experiments to test their ideas. We release MicroVQA, a visual question answering (VQA) benchmark to test these three tasks in the context of biological microscopy. Each of the 1,042 samples is created by a biology expert, and transformed into a multiple choice question (MCQ). † † footnotetext: Correspondence {jmhb,jnirschl,lmbravo}@stanford.edu ∗ These authors contributed equally 1 Introduction The transformative potential of AI in scientific discovery lies in its ability to interpret and reason over complex, multimodal data while integrating specialist domain knowledge. Recently, multimodal large language models (MLLMs) have demonstrated notable capabilities in data comprehension, recall, and multi-step reasoning across various modalities on exam-style questions [ 3 , 1 , 69 , 6 , 49 , 81 ] . This progress has generated enthusiasm for scientific AI applications, ranging from multimodal chat AI assistants [ 33 , 70 , 96 , 85 ] to AI agents capable of orchestrating complex experiments [ 26 , 12 , 55 , 16 ] . However, while MLLMs perform well on general exam tasks, they often lack the specialized, contextual reasoning needed for real scientific challenges, where tasks require advanced image-based reasoning, analysis, and hypothesis-driven experimentation. In recent years, efforts to apply scientific MLLMs across a range of domains have laid an essential groundwork. Advances have been achieved in image understanding [ 50 , 47 , 41 , 43 , 57 ] , literature analysis [ 66 , 77 , 47 , 41 ] , and problem-solving at school and early college levels [ 53 , 87 ] . In applied settings, AI has facilitated scientific tasks in biology [ 60 , 28 , 68 , 63 ] , chemistry [ 12 , 55 , 38 ] , software [ 73 , 63 ] , and diagnostic pathology and radiology [ 52 ] . However, there has been less progress on multimodal tasks requiring reasoning in research-level settings. The gap is due in part to a lack of multimodal benchmarks for scientific reasoning – unlike college-level tasks where existing exams can form benchmarks, research tasks require expert curation. Microscopy provides a compelling use case for developing such a benchmark. It requires more than pattern recognition or factual recall – it demands the ability to synthesize visual information with experimental context, formulate hypotheses, and propose experimental follow-ups. To evaluate these advanced capabilities, we introduce MicroVQA, a visual-question answering (VQA) benchmark tailored for multimodal scientific reasoning. Grounded in the real-world scientific practice of biological microscopy, MicroVQA incorporates 1,042 multiple-choice questions (MCQs) manually created by biological researchers across diverse disciplines, with each question taking more than 30 minutes to generate. Careful expert curation ensures the benchmark is relevant to real scientific applications. MicroVQA is designed to evaluate three key reasoning tasks crucial to scientific inquiry: (1) expert image understanding – the ability to recognize scientifically relevant features and integrate contextual information about sample preparation; (2) hypothesis generation – the formulation of scientifically plausible explanations from experimental data; and (3) experiment proposal – the ability to suggest further experiments that could validate or refute hypotheses. These capabilities form a common workflow in scientific experimentation. They demand both abductive reasoning – inferring the best explanation from multiple possibilities – and deductive reasoning – moving from general principles to specific conclusions [ 81 ] . We tested standard methods to map open scientific VQA samples to multiple-choice questions (MCQs), but found that they induce language shortcuts – the MCQs failed to truly test multimodal abilities. To ensure question quality and difficulty, we developed a two-stage pipeline for MCQ generation: first, an optimized LLM prompt structures QA pairs into well-formatted MCQs; then, a novel agent-based ‘RefineBot’ system increases question difficulty by rewriting questions without language shortcuts. This ensures that MicroVQA’s MCQs better test true scientific capabilities, rather than simple pattern recognition. We benchmarked frontier MLLMs on MicroVQA, finding that the best performers achieved a modest peak of 53%, which shows a significant gap between current models and expert-level scientific reasoning. The variation between different MLLMs was small; notably, smaller LLMs only slightly underperform their larger counterparts, suggesting that the key challenge for our MCQs is not language-only reasoning, but multimodal reasoning or knowledge. We also find that finetuning MLLMs on scientific publications improves MicroVQA performance. Finally, a group of biomedical researchers performed a detailed qualitative analysis into MLLM failure modes, uncovering three major failure modes: Expert perception is the most common, followed by knowledge errors, and then overgeneralization reasoning errors. This further supports that visual reasoning is a key challenge in MicroVQA. In summary, our contributions are as follows: • We introduce MicroVQA, a benchmark for multimodal reasoning in biological microscopy. We defined three key tasks for scientific research – expert image understanding, hypothesis generation, and experiment proposal. Then, expert researchers curated 1,042 diverse VQA samples. • We develop a novel two-stage MCQ generation pipeline to address the challenge of creating MCQs from long-format scientific questions without language shortcuts. • We provide quantitative and qualitative benchmarking of frontier MLLMs, highlighting areas for improvement. 2 Related work MLLM reasoning benchmarks While perception focuses on identifying an image’s visual features, visual reasoning often integrates prior knowledge to derive new conclusions using logical inference strategies like deduction, induction, abduction, and