Are We on the Same Page? Examining Developer Perception Alignment in Open Source Code Reviews

Yoseph Berhanu Alebachew, Minhyuk Ko, Chris Brown · 2025

Read Are We on the Same Page? Examining Developer Perception Alignment in Open Source Code Reviews by Yoseph Berhanu Alebachew, Minhyuk Ko, Chris Brown (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-18407.txt.

Introduction

Yoseph Berhanu Alebachew, Minhyuk Ko, and Chris Brown study whether contributors and maintainers see open source code review the same way, in a 2025 paper. Code review guards software quality and builds community, yet the two sides may judge the same review differently. The team mixed surveys, interviews, and repository analysis.

Surveys drew 289 answers, backed by 23 interviews and 81 project repositories. Both groups share core goals, but views diverge: maintainers stress fit with project goals, while contributors overvalue novelty. Familiarity bias falls hardest on underrepresented groups, discouraging them from taking part. The authors call for better documentation, better tools, and automation to cut delays and make open source review fairer.

Catalog record

CREATORYoseph Berhanu Alebachew, Minhyuk Ko, Chris Brown
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-18407

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Are We on the Same Page? Examining Developer Perception Alignment in Open Source Code Reviews Yoseph Berhanu Alebachew, Minhyuk Ko, Chris Brown Abstract . Code reviews are a critical aspect of open-source software (OSS) development, ensuring quality and fostering collaboration. This study examines perceptions, challenges, and biases in OSS code review processes, focusing on the perspectives of Contributors and Maintainers. Through surveys ( n = 289 n=289 ), interviews ( n = 23 n=23 ), and repository analysis ( n = 81 n=81 ), we identify key areas of alignment and disparity. While both groups share common objectives, differences emerge in priorities, e.g, with Maintainers emphasizing alignment with project goals while Contributors overestimated the value of novelty. Bias, particularly familiarity bias, disproportionately affects underrepresented groups, discouraging participation and limiting community growth. Misinterpretation of approach differences as bias further complicates reviews. Our findings underscore the need for improved documentation, better tools, and automated solutions to address delays and enhance inclusivity. This work provides actionable strategies to promote fairness and sustain the long-term innovation of OSS ecosystems. DOI: XXXXXXX.XXXXXXX Conference: The 29th International Conference on Evaluation and Assessment in Software Engineering; 17–20 June, 2025; Istanbul, Türkiye ISBN: 978-1-4503-XXXX-X/18/06 CCS: Software and its engineering Programming teams CCS: Software and its engineering Open source model 1. Introduction The code review process is a cornerstone of open-source software (OSS) development, ensuring that code changes meet quality standards, detecting defects early, and maintaining the integrity of the codebase ( 8 , 9 ) . Unlike closed-source projects, where hierarchical teams oversee code quality, OSS relies on a decentralized, community-driven model ( 70 , 71 ) . In OSS, developers (referred to as Contributors in this study) submit changes as pull requests, which are reviewed by a smaller group of developers with higher privileges, referred to here as Maintainers . These Maintainers evaluate the submission and decide its outcome (i.e., acceptance or rejection). The OSS model brings together developers with varying levels of expertise and experience, working collaboratively toward the shared goal of building a software. However, this diversity also introduces conflicts and biases in the code review process, which can undermine inclusivity and hinder the growth of OSS communities ( 15 ) . Several studies show the existence of bias in the code review process ( 13 , 16 , 27 , 38 , 61 , 77 ) . Studies show that biases in code reviews, such as favoring known Contributors over new or underrepresented ones, can discourage continued participation and hinder collaboration ( 25 , 37 ) . These biases not only result in inefficiencies, delays, and conflicts but also reduce the talent pool, limiting diversity and innovation in open-source projects ( 42 , 15 ) . Inclusive communities are more effective at solving complex problems and sustaining long-term growth. Addressing bias is therefore not just a fairness issue but a critical factor in fostering sustainable and innovative OSS ecosystems ( 43 ) . However, not all pushbacks and conflicts in code review are caused by bias. Prior work suggests the frequency and impact of bias is often overestimated in human judgment ( 63 ) . Tsay et. al ( 25 ) also report that misunderstandings in OSS environments are often rooted in differences in expectations rather than actual biases. To effectively address the challenges posed by biased code reviews, it is crucial to differentiate between noise, which represents misunderstandings or misinterpretations, and signal, which reflects instances of true bias affecting the review process. Building on the foundational work of Tversky and Kahneman, who described bias as a systematic deviation from objective standards, norms, or rationality in judgment or decision-making ( 80 ) , we aim to explore whether different stakeholders in OSS code reviews share a consensus on the objectives, key elements, and various aspects of the code review process. Understanding these differences is crucial, as misalignment in expectations or priorities can exacerbate misunderstandings, hinder collaboration, and potentially be understood as bias in the review process. Towards this goal, this study seeks to answer the following research questions (RQs): RQ1 : To what extent do maintainers and contributors in open-source projects agree on the objectives, key success factors, challenges, and potential improvements in the code review process? RQ2 : How does alignment (or lack thereof) between maintainers and contributors influence perceptions of bias in open-source projects’ code review? RQ3 : To what extent do developers believe the objectives, success factors, and challenges of the code review process reflected in project documentation, such as contribution guidelines? To answer these questions, we conducted online surveys ( n=289 ) and qualitative interviews ( n=23 ) with OSS developers experienced in contributing to and maintaining GitHub 1 1 1 https://github.com/ repositories to investigate OSS code review processes. Our study explores the dynamics between Contributors and Maintainers, addressing gaps in understanding bias, communication, and documentation. To the best of our knowledge, this is the first study to examine the alignment of expectations, opinions, challenges, and suggestions in OSS code reviews across roles. Our contributions include an analysis that identifies widespread misinterpretation of “approach differences” as bias, underscoring the need for clearer communication and documentation. Additionally, our study reveals differing priorities between Contributors and Maintainers, offering actionable strategies to enhance alignment. While Maintainers prioritize aligning contributions with project goals, Contributors often emphasize innovation, underscoring the importance of bridging this gap through effective communication. By addressing both technical and interpersonal aspects of code reviews, we propose a roadmap for more inclusive and sustainable OSS ecosystems. Empirical evidence also shows bias disproportionately affects newer Contributors and underrepresented groups, aligning with prior research ( 41 , 38 ) . Key contributions of this study include: • Evidence that while groups align on core aspects of code review, subtle but significant differences exist, warranting further research to improve alignment. • Empirical proof of misattributing approach differences as bias, calling for interventions to clarify perceptions and improve understanding. 2. Related Work Prior research has extensively explored the dynamics of code reviews and collaboration in software development. Santos and Nunes demonstrated the effectiveness of peer code reviews in distributed development for improving code quality, collaboration, and maintainability, while emphasizing the need to balance technical rigor with constructive feedback ( 81 ) . Similarly, Beller et al. highlighted the benefits of modern code reviews (MCRs) in enhancing code quality and fostering collaboration but noted their limitations in addressing architectural concerns, underscoring the need for complementary tools ( 87 ) . The dual role of OSS code reviews in ensuring quality and fostering trust was further examined by Alami et al., who emphasized transparent communication and mentorship, while identifying challenges such as cultural differences and asynchronous communication ( 88 ) . Eisty and Carver focused on peer code reviews in research software, identifying critical factors such as clear objectives and domain-specific feedback, while advocating for better documentation and tailored review practices to address inconsistencies and time constraints ( 82 ) . Research has also investigated the broader roles of OSS maintainers and contributors. Dias et al. identified key attributes of maintainers, including decision-making and inclusivity, while highlighting challenges like burnout and contributor management. Their recommendations to leverage automation, mentorship, and clear guidelines underscore the importance of alignment between

Read the complete text