These Aren’t the Reviews You’re Looking For How Humans Review AI-Generated Pull Requests Conference: The 30th International Conference on Evaluation and Assessment in Software Engineering; Tue 9 - Fri 12 June 2026; Glasgow, United Kingdom

Kacper Duma, Patryk Wróblewski, Jagoda Bobińska, Julia Winiarska, Piotr Przymus · 2026

Read These Aren’t the Reviews You’re Looking For How Humans Review AI-Generated Pull Requests Conference: The 30th International Conference on Evaluation and Assessment in Software Engineering; Tue 9 - Fri 12 June 2026; Glasgow, United Kingdom by Kacper Duma, Patryk Wróblewski, Jagoda Bobińska, Julia Winiarska, Piotr Przymus (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-02273.txt.

Description

Kacper Duma and colleagues analyse how humans review pull requests written by AI on GitHub, using the AIDev data set. They compare AI-authored requests with human-authored ones inside the same repositories. The question matters because review is often read as a sign of human oversight in large studies of coding activity.

Most AI-generated requests received no review at all. When review happened, AI agents dominated it rather than humans. Human-authored requests were more likely to get human-only review and direct human feedback. Reviews of AI requests more often ran through automation, with humans steering agents instead of judging the code alone. The authors warn that review metrics can mislead as measures of human oversight in agentic workflows.

Catalog record

CREATORKacper Duma, Patryk Wróblewski, Jagoda Bobińska, Julia Winiarska, Piotr Przymus
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.02273

Browse the full library · About the Tower of Babel