Recipe for Discovery: A Framework for Systematic Open Source Project Identification

Juanita Gomez, Emily Lovell, Stephanie Lieggi, Alvaro A. Cardenas, James Davis · 2025

Read Recipe for Discovery: A Framework for Systematic Open Source Project Identification by Juanita Gomez, Emily Lovell, Stephanie Lieggi, Alvaro A. Cardenas, James Davis (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-18359.txt.

Introduction

Juanita Gomez and co-authors present a 2025 paper on finding all the open source software a university system produces. Academic coding is scattered across personal accounts and labs, so institutions cannot see their own footprint. The authors built a pipeline that sweeps GitHub, pulls project metadata, and guesses each project's home institution and type.

Across ten universities the pipeline found over 200,000 repositories. It also scored how each project handles licensing, documentation, and community care. Many strong tools stay invisible because they skip those basic practices. The framework turns the harvest into action items for open source program offices. It can track whether support programs actually raise the quality and lifespan of campus software.

Catalog record

CREATORJuanita Gomez, Emily Lovell, Stephanie Lieggi, Alvaro A. Cardenas, James Davis
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE39 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-18359

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Recipe for Discovery: A Framework for Systematic Open Source Project Identification Juanita Gomez, Emily Lovell, Stephanie Lieggi, Alvaro A. Cardenas, James Davis Abstract Open source software development, particularly within institutions such as universities and research laboratories, is often decentralized and difficult to track. Despite producing highly impactful tools in science, these efforts often go unrecognized due to a lack of visibility and institutional awareness. This paper addresses the challenge of discovering, classifying, and analyzing open source software projects developed across distributed institutional systems. We present a framework for systematically identifying institutional affiliated repositories, using the University of California (UC) system as a case study. Using GitHub’s REST API, we build a pipeline to discover relevant repositories and extract meaningful metadata. We then propose and evaluate multiple classification strategies, including both traditional machine learning models and large language models (LLMs), to distinguish affiliated projects from unrelated repositories and generate accurate insights into the academic open source landscape. Our results show that the framework is effective at scale, discovering over 52,000 repositories and predicting institutional affiliation with high accuracy. 1 Introduction Open source software plays a central role in modern software engineering as the foundation of critical infrastructure and scientific discovery. Despite its impact, understanding where and how open source is produced remains a challenge, particularly when it is developed by decentralized institutions such as universities and research laboratories. In these settings, development is often distributed across personal accounts, making it difficult to discover and attribute contributions to their source organizations. Institutions around the world are increasingly recognizing the need to discover and map their open source ecosystems. Efforts such as the Institutional Innovation Grapher at the University of Texas at Austin [ 27 ] , the OSPO Database Development Playbook at the University of Wisconsin [ 11 ] , the Stanford Open Source Registry [ 22 ] , the Janelia OSSI initiative on the Howard Hughes Medical Institute Research Campus [ 20 ] and France’s national research software catalog [ 17 ] demonstrate a growing interest in creating tools to discover, classify and showcase open source software within institutions. These initiatives aim to support researchers, improve digital infrastructure, and make institutional impact more visible. Unlike software developed within companies or centralized teams, academic open source research emerges usually from small university groups or even individual researchers. There is often no unified platform, naming convention, or metadata standard for these efforts. Websites like GitHub are commonly used by researchers to develop code collaboratively, but the vast scale of these systems makes discovery an arduous challenge. Several factors make the task of identifying institutionally affiliated repositories particularly difficult: 1. Contributor profiles do not always include standardized institutional information and, when they do, it can be inconsistent, incomplete, or ambiguous. 2. Not all repositories represent meaningful contributions. Forks, class assignments, and inactive personal experiments are hard to filter from mature and widely used projects. 3. Contributors may use multiple GitHub identities, switch institutions, or contribute in a personal capacity, making it difficult to trace contributions to a specific organization. 4. Repository searches often yield tens of thousands of results per institutional keyword or phrase (e.g. "UC Davis"). Manual filtering is impractical, and simple keyword filters are insufficient. In this paper, we discuss methods to systematically discover and classify open source repositories affiliated with academic institutions. Our approach combines GitHub’s REST API with a filtering pipeline that integrates machine learning and large language models (LLMs) to infer repository affiliation. This work is carried out in collaboration with the Open Source Program Office Network of the University of California (UC OSPO Network) [ 26 ] , a multicampus initiative launched in 2024 to support open source research, promote sustainability, and establish best practices in academic settings [ 19 ] . We apply our methods at scale to analyze repositories across the ten UC campuses, illustrating the effectiveness of our approach in a large, distributed university system. Our contributions include, • We present a method to discover open source repositories affiliated with academic institutions and discuss the main challenges in determining institutional affiliation. • We develop a framework for systematically classifying the discovered repositories by affiliation. • We compare our methods with baseline approaches previously used in similar contexts and show that our approach improves the accuracy of identifying affiliated repositories. • We apply our framework to a large public university system and provide insights into open source development practices across campuses. • We release the code for repository scraping and classification as an open-source tool, enabling replication and further research. The code is available at: https://github.com/UC-OSPO-Network/repofinder . 2 Related Work There is growing interest in discovering and classifying open source software repositories to better understand their purpose, usage, and practices. For discovery , some universities have developed tools such as the Institutional Innovation Grapher [ 27 ] , which identifies GitHub accounts that mention a university in their bios. However, this method has limitations, as many developers do not explicitly mention institutional affiliations. Similarly, the University of Wisconsin open source monitoring project [ 11 ] aims to discover open source contributions but does not automate affiliation detection. Schwartz et al. [ 24 ] present a more comprehensive effort by combining GitHub keyword searches with university website scraping and LLM-based filtering to identify research software engineering (RSE) repositories; however, affiliation is not directly studied. For classification , previous work has addressed several angles. One of the most common, is distinguishing software engineering projects from other non-research projects. Munaiah et al. [ 18 ] implemented a tool called Reaper to classify engineered software projects using a combination of score-based heuristics and random forest classifiers. Pickerill et al. [ 23 ] extended this idea with PHANTOM, a time-series clustering method based on Git logs. Schwartz et al. [ 24 ] also classified RSE projects using LLMs. Many efforts focus on categorizing repositories by topic and functionality [ 7 ] , [ 8 ] , [ 28 ] [ 29 ] , [ 12 ] . Sharma et al. [ 25 ] proposed automatic cataloging based on topic modeling of README files, while Zhang et al. [ 30 ] used keyword-driven hierarchical classification to assign repositories to topic categories. Aslam et al. [ 2 ] built a multilingual source code classifier using machine learning, and Linares-Vasquez et al. [ 15 ] classified software by application domain using (API) calls from third-party libraries. Embedding-based models have also been explored: Bharadwaj and Kadam [ 5 ] applied BERT-style models to classify GitHub issues, while Izadi et al. [ 13 ] used semantic knowledge graphs to improve topic recommendation. GitHub mining at scale has also been a focus of previous work. Kalliamvakou et al. [ 14 ] highlighted challenges in GitHub data quality and usage patterns. Ma et al. [ 16 ] introduced World of Code, an infrastructure for mining global VCS data and exploring interdependencies. Cosentino et al. [ 6 ] conducted a systematic mapping study of related GitHub research. Although these works significantly advance methods for repository discovery and classification, they generally focus on project characteristics, topic modeling, or development activity, without directly addressing the question of institutional ownership or affiliation. This point is emphasized by a recent systematic mapping study

Read the complete text