Scientific Reasoning: Assessment of Multimodal Generative LLMs

Florian Dreyer, Ekaterina Kolos, Daria Matiash · 2025

Read Scientific Reasoning: Assessment of Multimodal Generative LLMs by Florian Dreyer, Ekaterina Kolos, Daria Matiash (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-01064.txt.

Description

Florian Dreyer, Ekaterina Kolos, and Daria Matiash grade AI models on science questions in a 2025 paper. Modern multimodal models can look at images and reason aloud. Science tests offer a fair yardstick for that skill. The team ran several leading models on the ScienceQA benchmark.

Gemini models scored highest when given little extra context. With richer context, their explanations matched human wording most closely. Tuning smaller models with adapter pieces gave no steady gains. Training models on Gemini's own outputs did worse than training on the original data. The short paper gives a snapshot of where multimodal reasoning stands. It also warns that copying a top model's answers is no shortcut.

Catalog record

CREATORFlorian Dreyer, Ekaterina Kolos, Daria Matiash
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-01064

Browse the full library · About the Tower of Babel