Nerfify : A Multi-Agent Framework for Turning NeRF Papers into Code
Seemandhar Jain, Keshav Gupta, Kunal Gupta, Manmohan Chandraker · 2026
Read Nerfify : A Multi-Agent Framework for Turning NeRF Papers into Code by Seemandhar Jain, Keshav Gupta, Kunal Gupta, Manmohan Chandraker (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-00805.txt.
Introduction
Jain and three colleagues turn NeRF research papers into runnable code with NERFIFY, a multi agent system making Nerfstudio plugins. Generic methods that turn papers into code and frontier models usually fail to produce runnable code. NERFIFY adds six domain innovations: grammar constrained synthesis within Nerfstudio rules, graph of thought generation in dependency order, and citation recovery from reference graphs. Visual feedback checks images and geometry, knowledge enhancement improves methods beyond reproduction, and benchmarking measures results. The system thus makes paper reproduction reliable for neural radiance field research.
Catalog record
| CREATOR | Seemandhar Jain, Keshav Gupta, Kunal Gupta, Manmohan Chandraker |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37833 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.00805
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Nerfify : A Multi-Agent Framework for Turning NeRF Papers into Code Seemandhar Jain, Keshav Gupta, Kunal Gupta, Manmohan Chandraker Abstract The proliferation of neural radiance field (NeRF) research requires significant efforts to reimplement papers before building upon them. We introduce Nerfify , a multi-agent framework that reliably converts NeRF research papers into trainable Nerfstudio plugins, in contrast to generic paper-to-code methods and frontier models like GPT-5 that usually fail to produce runnable code. Nerfify achieves domain-specific executability through six key innovations: (1) Context-free grammar (CFG): LLM synthesis is constrained by Nerfstudio formalized as a CFG, ensuring generated code satisfies architectural invariants. (2) Graph-of-Thought code synthesis: Specialized multi-file-agents generate repositories in topological dependency order, validating contracts and errors at each node. (3) Compositional citation recovery: Agents automatically retrieve and integrate components (samplers, encoders, proposal networks) from citation graphs of references. (4) Visual feedback: Artifacts are diagnosed through PSNR-minima ROI analysis, cross-view geometric validation, and VLM-guided patching to iteratively improve quality. (5) Knowledge enhancement: Beyond reproduction, methods can be improved with novel optimizations. (6) Benchmarking: An evaluation framework is designed for NeRF paper-to-code synthesis across 30 diverse papers. On papers without public implementations, Nerfify achieves visual quality matching expert human code (±0.5 dB PSNR, ±0.2 SSIM) while reducing implementation time from weeks to minutes. Nerfify demonstrates that a domain-aware design enables code translation for complex vision papers, potentiating accelerated and democratized reproducible research. Code, data and implementations will be publicly released. 1 Introduction Figure 1 : Overview. Manual NeRF implementation requires weeks of specialized effort (left). Existing paper-to-code systems fail to produce trainable code. Nerfify automates this process through grammar-constrained synthesis and compositional citation recovery, generating fully trainable Nerfstudio plugins in minutes (right). Since its publication in 2020, the Neural Radiance Fields (NeRF) paper 32 has spawned over 1,000 follow-ups. Yet, unavailability of code or standardized implementations for most of them 50 means each subsequent work requires a significant effort to reimplement existing NeRF papers. While the issue exists in broader machine learning research 41 , effective NeRF implementations present unique challenges. We consider the question of devising a large language model (LLM) agent to automatically produce trainable, performant and standardized NeRFs from their papers. NeRF implementations are uniquely challenging, with expertise required across volumetric rendering, computer vision and neural optimization. A single misplaced activation or incorrect ray-sphere intersection produces failures ranging from the catastrophic (NaN gradients) to the subtle (degenerate solutions). Debugging requires understanding whether failures stem from code bugs, scene geometry, hyperparameters, or the paper’s own ambiguities, with computationally demanding cycles (the original NeRF requires 100k-300k iterations taking 24-48 hours on high-end GPUs 32 ). Generic paper-to-code approaches do not suffice, given that the current best-performing system, O1, achieves only 26.6% accuracy on complex papers compared to 41.4% for human experts 44 . The challenge is exacerbated for NeRFs, where the strong domain coupling needed for its unique combination of rendering mathematics and neural architectures leads to execution failures and large performance gaps. As discussed in Section 2 and exhaustively shown in the Supplementary, recent advances like Paper2Code 42 and AutoP2C 25 – despite progress on general machine learning tasks – usually do not produce executable code or trainable NeRFs, with issues ranging from incorrect implementations (such as K-Planes with regular MLPs instead of planar factorization), or catastrophic failures (producing only dataset loading code). Another key challenge is that modern NeRF papers build upon chains of dependencies that generic systems cannot resolve, for example, see Figure 3 for K-Planes 10 . A single phrase such as “we adopt the distortion loss from 3 ” requires navigating to that paper, locating the correct equation, translating to executable code and implementing stop-gradient operations critical for stable training. We introduce Nerfify , a multi-agent framework that reliably converts NeRF research papers into code that trains, converges and matches the visual quality of expert implementations (Figure 1 ). Its success is based on five key technical innovations. 1) Use of Nerfstudio 47 architecture as a context-free grammar (CFG) that encodes domain-aware module compositions and interface contracts, leading to LLM synthesis of architecturally correct code by construction. 2) Graph-of-thought code generation to coordinate specialized agents that generate multi-file repositories in topological order, validating type signatures, tensor shapes and circular dependencies at each dependency before proceeding. 3) A compositional citation recovery that traverses reference graphs to retrieve implicit dependencies (such as proposal networks from Mip-NeRF 360 3 , hash encoders from Instant-NGP 33 ). 4) A critique agent that provides feedback by analyzing training runs to diagnose visual artifacts through PSNR-minima analysis and cross-view geometric validation, using vision-language models (VLMs) to guide targeted fixes. 5) Besides reproduction, an option to improve with optimizations where applicable. We extensively evaluate with Nerfify-Bench , a novel benchmark with 30 diverse papers stratified across code status and implementation complexity. Nerfify achieves full executability and rendering quality within 0.5 dB PSNR of expert implementations, while generic baselines fail to produce trainable code in 95% of cases. Nerfify will accelerate reproducible research, allowing researchers to reproduce complex NeRF papers in hours rather than weeks, rapidly prototype techniques and make accessible to the community papers that would otherwise remain purely theoretical. As the NeRF community continues its growth, Nerfify ensures that every paper’s contributions become available to all researchers, democratizing access to cutting-edge techniques and enabling the compositional research that drives the field forward. With our demonstration that depth and specialization in a multi-agent framework unlocks transformative capabilities for NeRFs, we believe future work will achieve similar transformations for other communities too. Code and data for Nerfify , Nerfify-Bench and generated implementations will be publicly released. 2 Related Work Generic Paper-to-Code Systems. Paper2Code 42 employs a three-stage pipeline (planning, analysis, generation), but its generic design lacks neural field architectures for NeRF implementation. AutoP2C 25 advances multimodal understanding with high executability on recent papers, but generates non trainable repositories with just placeholders for NeRFs. AutoReproduce 64 introduces paper lineage-extracting domain knowledge from citations. On PaperBench 37 , Claude 3.5 Sonnet achieves only 21% accuracy versus 41.4% for human researchers on ICML papers, motivating our domain-specific approach. RPG 26 employs test-driven development with repository planning graphs but cannot effectively parse papers or extract mathematical formulations. Paper2Agent 31 converts papers into conversational interfaces rather than trainable implementations. Recent works 65 ; 11 explore fine-grained verification and dynamic planning, but remain domain-agnostic. Multi-Agent Code Generation has emerged as a key field for complex code synthesis 17 ; 40 ; 56 ; 18 , with recent systems proposing repository-level code generation 51 ; 62 ; 7 ; 52 ; 57 ; 16 . However, general software systems lack the mathematical understanding and architectural constraints required for successful synthesis of research code. Domain-Specific Code Synthesis outperforms generic approaches by encoding specialized knowledge. Scene Language 63 uses CFG to structure visual program synthesis, CODEP 49 employs Pushdown Automaton and TSL+LLM 30 combines Temporal Stream Logic to act as CFG. Feedback and Planning Approaches like Graph of Thoughts (GoT) 4 generalize reasoning into directed graphs enabling aggregation, refinement loops, and backtracking, while Tree of Thoughts 58 and Tree-of-Code 35 explore over reasoning trees. Self-Debugging 6 , Reflexion 43 , Self-Refine 29 , Clover 45 and xKG 27 introduce refinement and consistency