From Papers to Progress: Rethinking Knowledge Accumulation in Software Engineering Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering: Future of Software Engineering ; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3793657.3793888 ISBN: 979-8-4007-2479-4/2026/04 CCS: Software and its engineering Software creation and management CCS: Software and its engineering Empirical software validation CCS: General and reference Empirical studies

Jason Cusati, Chris Brown · 2026

Read From Papers to Progress: Rethinking Knowledge Accumulation in Software Engineering Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering: Future of Software Engineering ; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3793657.3793888 ISBN: 979-8-4007-2479-4/2026/04 CCS: Software and its engineering Software creation and management CCS: Software and its engineering Empirical software validation CCS: General and reference Empirical studies by Jason Cusati, Chris Brown (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16208.txt.

Introduction

Cusati and Brown survey 280 experienced software engineering researchers about why the field struggles to accumulate knowledge despite booming output. Respondents point to a tension between rising productivity and weak tools for synthesising results, tracking claims, and building shared understanding over time. The authors diagnose four linked breakdowns, isolated papers, lost context, untracked claim changes, and incentives favouring novelty over consolidation, and frame these as barriers to long-term progress.

Catalog record

CREATORJason Cusati, Chris Brown
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23247
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16208

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Papers to Progress: Rethinking Knowledge Accumulation in Software Engineering Conference: 2026 IEEE/ACM 48th International Conference on Software Engineering: Future of Software Engineering ; April 12–18, 2026; Rio de Janeiro, Brazil 2026 IEEE/ACM 48th International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE), April 12–18, 2026, Rio de Janeiro, Brazil DOI: 10.1145/3793657.3793888 ISBN: 979-8-4007-2479-4/2026/04 CCS: Software and its engineering Software creation and management CCS: Software and its engineering Empirical software validation CCS: General and reference Empirical studies Jason Cusati, Chris Brown Abstract . Software engineering research has experienced rapid growth in both output and participation over the past decades. Yet concerns persist about the field’s ability to accumulate, integrate, and reuse knowledge in ways that support long-term progress. To better understand how the community itself perceives these challenges, we analyze responses from the ICSE 2026 Future of Software Engineering pre-survey, which captures perspectives from 280 globally distributed and highly experienced researchers. Our analysis reveals a tension between increasing research productivity and the limited mechanisms available for synthesizing results, tracking evolving claims, and supporting cumulative understanding over time. Building on these observations, we diagnose four interrelated structural breakdowns: papers function as isolated knowledge units with claims embedded in prose; context and provenance are lost as knowledge moves through the publication pipeline; claims evolve without systematic tracking; and incentive structures favor novelty over consolidation. We argue that addressing these barriers requires rethinking the fundamental properties of research artifacts. We articulate four technology-agnostic principles for future research artifacts: structured and interpretable representations of claims and evidence; inspectable and provenance-aware documentation of methodological decisions; long-lived and reusable substrates that evolve beyond publication; and governance mechanisms that align individual incentives with collective knowledge-building goals. We discuss implications for research practice, publication norms, and community infrastructure, positioning FOSE as a venue for experimenting with alternative artifact designs that support cumulative scientific progress. † † cc-license: by 1. Introduction Software engineering research is thriving by most measures. Conference submissions keep rising, publication venues have multiplied, and the community has gone global. Researchers are productive, techniques keep advancing, and new areas of inquiry emerge every year. But a persistent challenge lurks beneath all this activity: we struggle to build cumulative knowledge that actually connects, consolidates, and extends prior work. This is not a new concern, nor one unique to our field. Across science, more publications have not automatically meant deeper understanding ( 13 ; 10 ) . In software engineering specifically, fragmentation and the difficulty of synthesizing results have been raised repeatedly ( 8 ; 19 ; 20 ) . Yet the structural factors limiting cumulative progress remain underexplored. This paper offers a community-informed perspective on knowledge accumulation in software engineering. Drawing on 280 ICSE 2026 FOSE pre-survey responses from an experienced (57% with 10+ years), productive (44% with 11+ papers in 3 years), globally distributed community, we diagnose four structural breakdowns: papers as isolated units, lost context and provenance, untracked claim evolution, and incentives favoring novelty over consolidation. We then propose four principles for research artifacts: structured and interpretable, inspectable and provenance-aware, long-lived and reusable, and community-governed. We do not prescribe specific tools or claim to solve these challenges; instead, we synthesize community observations into a diagnosis and principles to guide future work. 2. Community Signals from the ICSE 2026 FOSE Pre-Survey We ground our diagnosis in the ICSE 2026 Future of Software Engineering pre-survey (n=280) ( 22 ) . Two researchers performed a lightweight qualitative analysis, examining comments for insights about knowledge-building in software engineering. The respondents span six geographic regions (Europe 50%, North America 25%, Asia 15%, others 10%) and represent a mature, productive community. This profile matters for our argument: if knowledge accumulation remains limited despite all this expertise and output, the barriers are structural , not individual. The community is not lacking effort. Respondents themselves recognize systemic challenges, noting fragmentation across resubmissions, difficulties building on prior work, and incentives that reward novelty over consolidation. These observations motivate our diagnosis. 3. Where Knowledge Accumulation Breaks Down If the community is experienced, productive, and global, yet still struggles to synthesize results and build on prior work, the barriers are structural. We identify four interrelated breakdowns in how research artifacts are produced, represented, and connected. Anonymous participant identifiers appear in parentheses. 3.1. Papers as Isolated Knowledge Units Research papers package claims, evidence, and context into narrative prose optimized for dissemination. This works well for presenting new ideas but creates barriers to cumulative building. One participant noted that “ 15 years ago [it was the norm to] present and discuss a research idea at a workshop then go back home, build and evaluate the thing, then publish at a conference. These days, reviewers repeatedly reject such submissions ‘because the work has already appeared at this and that workshop’ ” (195811076). Claims get buried in paragraphs alongside motivation and methods. Extracting and comparing claims across papers means reading entire documents and reconciling different terminology. Evidence linkages stay implicit; readers infer which results support which claims under what assumptions. Even when artifacts are shared, results freeze at publication time. Updates to datasets or baselines require new papers, while original claims sit unchanged. As another participant observed, “ despite having a software artifact track, not many of the software artifacts are being actively maintained ” (196030130). Each paper becomes an island connected only through citations and prose, requiring manual synthesis that does not scale. 3.2. Loss of Context and Provenance Research involves countless contextual decisions: which dataset, how to split data, which baselines, how to handle edge cases. Papers describe these choices but often omit the rationale. One participant noted frustration with “ software engineering researchers trying to do research with generative AI tools without specifying context first ” (196032564). As work gets cited and summarized, motivation and assumptions fade. Carefully qualified findings become unqualified facts in subsequent literature. Methodological decisions must be reverse-engineered, sometimes revealing that subtle choices significantly impacted results. Papers present polished final versions, hiding the path from hypothesis to result. When context and provenance are lost, later researchers either accept prior work at face value or invest substantial effort reconstructing the reasoning. Moreover, despite goals to enhance practice, research has “ minimal ” and “ almost no relevance ” to industry contexts (192101946). 3.3. Claims Evolve Without Tracking Scientific claims get refined, qualified, contradicted, and superseded as evidence accumulates, but the publication system provides few tracking mechanisms. The literature may contain conflicting claims (technique A outperforms B in one paper, the reverse in another) without systematic flagging. Survey respondents noted that “ most of the SE research being published is neither reproducible nor replicable ” (195853386) and “ we are reluctant to accept papers that replicate previous study…see medical sciences as counter example - they publish single cases as they allow to build body of konowledge in the long run ” (196067582). When researchers discover conflicts, they must investigate causes themselves, often finding subtle methodological differences explain the divergence. Refinements stay implicit: a follow-on paper may qualify a prior claim, but the original remains unchanged. Claim relationships (does one extend, contradict, or depend on another?) appear only in natural language like "building on [12]" or "in contrast to [34]." Without structured representations, determining what is

Read the complete text