Facilitating analysis of open neurophysiology data on the DANDI Archive using large language model tools Authors: Jeremy F Magland, Ryan Ly, Oliver Rübel, Benjamin Dichter Published: 2025 ABSTRACT AbstractThe DANDI Archive is a key resource for sharing open neurophysiology data, hosting over 400 datasets in the Neurodata Without Borders (NWB) format. While these datasets hold tremendous potential for reanalysis and discovery, many researchers face barriers to reuse, including unfamiliarity with access methods and difficulty identifying relevant content. Here we introduce an AI-powered, agentic chat assistant and a notebook generation pipeline. The chat assistant serves as an interactive tool for exploring DANDI datasets. It leverages large language models (LLMs) and integrates with agentic tools to guide users through data access, visualization, and preliminary analysis. The notebook generator analyzes dataset structure with minimal human input, executing inspection scripts and generating visualizations. It then produces an instructional Python notebook tailored to the dataset. We applied this system to 12 recent datasets. Review by neurophysiology data specialists found the generated notebooks to be generally accurate and well-structured, with most notebooks rated as “very helpful.” This work demonstrates how AI can support FAIR principles by leveraging data standards and lowering barriers to data reuse and engagement.Subject terms: Neuroscience, Education Introduction Since it was launched in 2020, the DANDI Archive1 has become a central resource for open data in neurophysiology. As of this writing, it holds over 400 neurophysiology “dandisets” (DANDI datasets) in the Neurodata Without Borders (NWB) standard1,2. Together, these datasets contain more than 350 TB of neurophysiology data from over 20 species, including intracellular and extracellular electrophysiology, calcium imaging, fiber photometry, and a range of behavioral measures, both naturalistic and task-based. These datasets are the basis for many influential studies in systems neuroscience, and provide insight into many contemporary topics in systems neuroscience, such as memory and navigation; auditory, visual, olfactory, and tactile sensory processing; motor control; and decision making. Analysis of these open datasets holds enormous potential for scientific discovery3–7 and tool development8–10, and a core goal of the DANDI project is to promote their use. However, there are barriers to broader engagement with data reuse: many scientists are more familiar with analyzing data that they acquired or working within established collaborations11. Analyzing public datasets requires learning new methods for data access and processing, trust in the quality and completeness of the data, and efficient tools to identify datasets relevant to specific research questions. While significant effort has gone into developing general tutorials and documentation for working with NWB data on DANDI (https://docs.dandiarchive.org; https://pynwb.readthedocs.io/; https://matnwb.readthedocs.io/), these resources are often either too broad for investigating a specific dandiset, or too narrowly focused on datasets that may not be relevant to the user’s current goals. The diversity of data on the archive, spanning a wide range of scientific questions, experimental designs, stimuli, and recording techniques, makes it particularly difficult to build tools and guidance that support effective dataset discovery, evaluation, and analysis. Some tools are already in place to help neuroscientists explore DANDI. Neurosift12 enables interactive exploration of neurophysiology data by streaming directly from the archive and providing modality-specific visualizations. However, it is built in the language of the web browser (TypeScript/JavaScript), which limits accessibility for most neuroscientists. Furthermore, its visualization plugins are generic and not tailored to individual dandisets and specific scientific questions. For more specific guidance, the DANDI team maintains an “example_notebooks” GitHub repository (https://github.com/dandi/example-notebooks), where users, often data contributors, share Jupyter notebooks demonstrating how to load, visualize, and sometimes reproduce figures from corresponding papers. Additional resources like the OpenScope DataBook13 offer high-quality tutorials for selected datasets from the Allen Institute OpenScope program. While valuable, these currently cover only a small fraction of the 400+ dandisets available. Recent advances in large language models (LLMs) offer a promising new approach to lowering the barriers to data reuse by automatically or semi-automatically generating dataset-specific introductory material to help scientists get started with analysis. In addition to coding capabilities14, popular LLMs have the required domain knowledge about neuroscience, in some cases surpassing world experts in the field15. These models also support natural language interaction, making it possible to build conversational tools that can guide users through the process of dataset evaluation and analysis. To realize this potential, we developed an AI-powered system consisting of an interactive, agentic chat interface and an automated notebook generator. Users can engage with the system in two complementary ways. They can interact directly with an AI agent through a chat interface, asking questions about a specific dandiset, requesting visualizations, or generating code tailored to that dataset. Alternatively, users can work with a Jupyter notebook produced by the system, which serves as a structured, executable guide for understanding and analyzing the dataset. This notebook is generated by an AI agent that first explores the dataset by inspecting NWB files, analyzing metadata, and producing visualizations. The exploration process supports human intervention at key steps, allowing users to correct errors or guide the analysis if needed. Together, the chat interface and notebooks provide both interactive support and reusable documentation to help researchers engage with public neurophysiology data. Results Chat interface The Dandiset Explorer, shown in Fig. 1, is an interactive chat interface where users can explore a particular dandiset using natural language. The chat agent is equipped with tools to query the DANDI API for dandiset-level metadata and to list the NWB files contained in a given dataset. Leveraging the standardized structure of NWB files, the agent can retrieve a structured summary of each file, including the types of neurodata present and associated metadata. It can also retrieve Python usage documentation for individual NWB files describing how to programmatically access these objects (see get_nwbfile_info in Methods). The agent executes Python scripts in a sandboxed environment, allowing it to stream data directly from remote NWB files and produce both textual and visual outputs in response to user questions. Because we do not want users to blindly trust the output of the assistant without understanding the basics of NWB, a message at the top of the chat directs users to read through the PyNWB documentation and introductory tutorials. Screenshots of an interactive chat conversation for Dandiset 001354. Notebook generation The notebook generation process (Fig. 2) begins with an initial prompt (Appendix A) provided to the chat interface, which explains that the system will conduct a structured exploration to collect the information needed for generating an introductory analysis notebook. This prompt outlines how the exploration should proceed and specifies the criteria for a high-quality notebook. The assistant then proceeds step by step, pausing after each action to wait for user input. If the user replies with “proceed,” the assistant continues autonomously; otherwise, the user can interject at any point with corrections or instructions. This interaction loop enables a flexible balance between automation and human guidance. During exploration, the assistant enumerates NWB files, inspects their structure, and summarizes the types of data present. The agent generates and executes code to create visualizations illustrating key content. If execution raises any exceptions, the resulting error messages are returned to the LLM, which revises the code accordingly. This process can repeat multiple times, providing the agent with multiple attempts to create functioning code. The system is designed to accommodate human oversight in resolving issues such as data inconsistencies or ambiguous metadata. The process continues until the assistant determines that it has gathered sufficient information for notebook generation. Two-stage process for notebook generation. Following the exploration phase, a separate LLM agent is tasked with generating a complete Jupyter notebook based on the accumulated interaction history (Appendix A, first prompt). This includes summaries, visual outputs, and code produced during exploration. The LLM is guided by structured instructions (Appendix A, second prompt) to produce the notebook (see Methods). Notebooks for twelve published dandisets To evaluate the system across a variety of real-world datasets, we applied it to 12 recently published dandisets spanning a range of species, modalities, and experimental designs: 00134916, 00135417, 00143318, 00056319, 00136120, 00136621, 00135922, 00137523, 00117424, 00069025, 00119526, and 00061727. Published dandisets are immutable, in contrast to draft versions, and therefore represent stable targets for notebook generation. Each was processed using the full pipeline: initiating structured exploration via the chat interface, generating a notebook, and executing the result with iterative error correction. Figure 3 shows screenshots for part of one of the notebooks, and Fig. 4 presents a montage of all of the generated images in the notebook. The full notebooks are available on Zenodo (https://zenodo.org/records/16033603). Screenshots from part of the generated notebook for dandiset 001349. Montage of the 45 images from the 12 AI-generated notebooks. Human interventions in the chat phase User intervention was required in 8 out of 12 dandisets to correct errors or guide the assistant (Supplementary Table 1). Common issues included misinterpretation of data structures, such as incorrect handling of timestamp fields; ineffective or misleading visualizations, including blank images, sparse raster plots, or misaligned regions of interest (ROIs); and performance problems, such as inefficient data access or code that ran too slowly. For example, for Dandiset 001433, the assistant misinterpreted a dataset of event times, leading to incorrect visualizations. For Dandiset 000617, the assistant repeatedly failed in its attempt to align ROIs with background images, and the user ultimately advised abandoning that approach. For Dandiset 000563, the assistant needed to provide guidance on how to properly load spike trains. Four of the dandisets proceeded without intervention, indicating that the process can operate autonomously in some cases. A key advantage of the chat-based approach is that it provides a clear record that could be used for training evaluation models to correct common errors and enhance prompts to reduce errors. Human notebook evaluation The generated notebooks were independently evaluated by four expert reviewers with backgrounds in data science and neurophysiology and nine reviewers with varying experience with NWB and DANDI. None of these reviewers are authors of this paper. Reviewers completed a structured questionnaire (Supplementary Table 2) designed to assess the notebooks based on correctness, usefulness, and clarity. The reviewers were given one hour to complete as many notebook reviews as they could in the allotted time and did not discuss the notebooks with anyone until after all their reviews were submitted. Each reviewer reviewed between 5 and 10 notebooks. Reviewers found the notebooks to be at least moderately effective across most criteria (Fig. 5). Reviewers generally reported success in accessing data, working with NWB files, and interpreting visualizations. There were very few reports of redundancy with the example explorations, and the representation was largely accurate with what was observed independently using Neurosift. Summary of reviews for AI-generated notebooks. Each subplot shows the distribution of responses to a survey question evaluating aspects such as usefulness, accuracy, visualization quality, and reviewer expertise. The final 5 questions relate to the reviewer’s experience level. See Supplementary Table 2 for details on the questions and the response options. Figure 6 shows the breakdown of “Overall Helpfulness” ratings by Dandiset ID and by reviewer experience with NWB and DANDI. Notebooks could be rated “not helpful”, “moderately helpful”, or “very helpful”. Most notebooks received a mix of “moderately helpful” and “very helpful” ratings, with only one dandiset (000690) receiving any “not helpful” responses. Notably, reviewers across different levels of experience with NWB and DANDI gave similar ratings, suggesting that the notebooks are broadly accessible and useful to a wide range of researchers. Left: Distribution of “Overall Helpfulness” ratings for each AI-generated notebook. Bars show the number of reviewers who rated each notebook as not helpful (0), moderately helpful (1), or very helpful (2). Note that one notebook was not reviewed by any of the reviewers. Right: Distribution of “Overall Helpfulness” ratings by reported experience with NWB/DANDI. LLM cost We recorded API costs for both the exploration and generation phases of notebook generation across all 12 dandisets. Chat sessions using GPT-4.1 ranged from $0.37 to $1.84 (median $1.03), while notebook generation with Claude Sonnet 4 ranged from $0.09 to $0.46 (median $0.17). The cost for one of the notebooks was not available due to technical issues. The average total cost per notebook was $1.15. Discussion Reducing barriers to data reuse The adoption of the NWB standard by DANDI has been transformative, establishing a common language for representing neurophysiology data across diverse experiments. This standardization facilitates collaboration and interoperability among researchers and lays the groundwork for AI-driven data exploration and analysis. This work introduces an AI-powered system that lowers barriers to reusing neurophysiology data from the DANDI Archive by combining LLM-driven automation with lightweight human oversight. The system automates early exploration of NWB datasets and generates ready-to-run, dataset-specific notebooks that help users get started with access, visualization, and analysis. These notebooks are designed to work both locally and in cloud environments such as Google Colab or DANDI Hub. By providing tailored, interactive support, the system enables faster onboarding for both new and experienced users, without requiring detailed knowledge of the NWB data standard or software. We distinguish between two modes of user interaction supported by the system. First, the AI-generated notebooks are intended as pre-built, curated resources to help users get started with a dandiset. These notebooks are generated following a structured exploration phase, during which expert reviewers inspect and correct issues arising in the AI’s analysis. Second, the chat interface allows end users to engage interactively with any dataset in real time, without expert supervision. While this mode provides flexible, user-driven exploration, it also carries the typical risks of automated systems, and outputs should be treated as provisional. Enhancing incentives for data sharing By lowering the technical and conceptual barriers to using open neurophysiology datasets, these tools have the potential to foster new forms of collaboration between data generators and data analysts. When reuse becomes easier and more visible, experimental datasets are more likely to serve as the foundation for secondary analyses and derivative publications that include or credit the data generators. This, in turn, creates a stronger incentive for researchers to adopt standardized formats like NWB and deposit their data in public archives. Future extensions of this work could further support this ecosystem by integrating notebook generation into the data sharing workflow itself, enabling contributors to review and refine AI-generated notebooks before they are published alongside the dataset. Balancing automation and human oversight While the system is largely autonomous, minimal human oversight during the exploration phase of notebook generation was essential for maintaining quality. Interventions were limited to clear issues such as broken visualizations, misread data structures, or flawed analyses rather than guiding the process throughout. These light corrections helped avoid misleading outputs, especially when dataset-specific quirks (e.g., missing units, misaligned ROIs) required expert judgment. Oversight was generally efficient and unobtrusive, but scaling to the full set of 400 + dandisets may require new strategies. Whether future models can fully replace this role remains an open question. LLM model selection We qualitatively assessed several LLMs for the two stages of our pipeline and selected different models for each. OpenAI’s GPT-4.1 was selected for the chat phase, reliably calling tools at the right times, avoiding overinterpretation, and keeping analysis appropriately introductory. For notebook generation, Claude Sonnet 4 was selected, demonstrating strong scientific reasoning and producing creative, well-structured notebooks with high-quality visualizations when constrained to analyses completed during exploration. Claude was less suitable for the exploratory chat phase, where it tended to overanalyze data, generate overly complex plots, and draw unsupported conclusions. Other models, including DeepSeek R1 and Google Gemini 2.0, were excluded due to weaker task adherence and inconsistent tool use. We did not test OpenAI’s most advanced models due to cost. Given the rapid pace of progress in the LLM ecosystem, our choices reflect a pragmatic snapshot of what was available and affordable at the time. Generalization to other domains While this work focuses on neurophysiology and the DANDI Archive, the underlying approach is broadly applicable to other scientific domains. There is nothing inherently unique about neurophysiology that makes this system viable; rather, it relies on two key prerequisites: (1) a library of openly accessible, structured datasets, and (2) standardized tools or APIs for efficient programmatic access. Fields such as genomics, earth sciences, materials science, and astronomy already meet these criteria to varying degrees. The system is particularly well-suited for exploratory analysis tasks that can be executed quickly, as the computational cost of notebook generation scales with execution time. We believe this framework can serve as a general model for accelerating data reuse and lowering entry barriers across a wide range of scientific disciplines. Risks and limitations While LLMs can accelerate scientific analysis, they also pose risks. These include hallucinations (confident but incorrect statements or code), misleading visualizations, overinterpretation of results without proper statistical support, and automation bias where users may trust AI outputs without verification. To reduce these risks, we rely on various safeguards when generating notebooks. A key element is the initial exploration phase, which allows the AI system to learn about the dataset’s structure and content prior to notebook generation. The agent catches and fixes code that produces an error. Before publishing a notebook for the community, an expert user reviews code to prevent scientifically incorrect analyses and help ensure trust and reliability of results. This phase not only improves the accuracy of the final notebook but also allows for the detection and correction of metadata inconsistencies or unsupported file types in the source dataset, providing valuable feedback for curation to enhance data quality. Additionally, all notebooks include a clear disclaimer noting they were AI-generated and should be critically reviewed before use. These notebooks were designed to serve as instructional guides, not to produce novel scientific insights. This narrower objective makes the task considerably more manageable for the AI system. Rather than interpreting raw neurophysiology data to uncover new findings, the agent assembles familiar scientific context and standard analysis code, much of which likely reflects patterns in its training data. By focusing on explanation and usability rather than discovery, the system can reliably generate useful, dataset-specific tutorials without needing deep scientific inference. Whether such systems could eventually support novel research on public neurophysiology data remains an open question for future research. The effectiveness of our system is inherently constrained by the current capabilities of LLMs. While these models can produce syntactically correct and often helpful code and commentary, their grasp of scientific nuance is limited and occasionally inconsistent. As a result, some generated notebooks may feature superficial analyses, overlook relevant methodological considerations, or offer interpretations that sound plausible but are not rigorously supported. We expect improvements in LLMs will reduce these concerns in the future. The notebook generator is optimized for datasets that can be explored within a modest computational budget. Many dandisets contain large or complex data files that exceed what is practical to process automatically. To address this, the agent typically selects a single NWB file for inspection and focuses on representative data arrays and metadata that are easily accessible. While this strategy ensures feasibility, it may miss important experimental conditions or data features not present in the sampled file. This work is intended to offer a qualitative understanding of how the system performs across a diverse set of real datasets and assess the system’s overall strengths or weaknesses. The number of reviewed notebooks and the size of the reviewer pool are appropriate for a preliminary evaluation, but are not sufficient to support statistically rigorous conclusions. Finally, LLM-generated code raises unresolved questions around intellectual property and licensing. Although the code produced by our system is created dynamically in response to specific prompts, it may reflect patterns from training data that include content under various open-source licenses. Attribution is generally not possible, and the legal status of such outputs remains uncertain. We acknowledge this as an important area for future guidance and community standards. Future improvements Our results show that even current-generation LLMs can produce helpful, dataset-specific notebooks with minimal supervision. Looking ahead, the most impactful improvements will likely come from continued advances in LLM capabilities, especially in scientific reasoning, tool use, and contextual understanding. Reviewer feedback highlighted areas such as clearer scientific framing, better visualization choices, and more nuanced interpretation of dataset structure. Prompt design and metadata integration have evolved over the course of this project and continued adjustment has the potential to offer incremental gains. However, the greatest progress is expected as models become more reliable and better aligned with scientific tasks. Conclusion We have presented an AI-powered system that lowers the barrier to analyzing public neurophysiology datasets from the DANDI Archive. By combining a conversational LLM agent with an automated notebook generation pipeline, the system helps users navigate dataset contents, access NWB files, and generate meaningful visualizations. Human review of notebooks demonstrated that the approach can be effective and helpful. While challenges remain, this work illustrates how LLMs, when combined with robust data standards like NWB, can support more accessible, scalable, and reproducible use of open scientific data. The framework may be generalizable to other domains where standardized formats and programmatic access are available. Methods Model selection We used OpenAI’s GPT-4.1 for the exploration phase and Anthropic’s Claude Sonnet 4 was used in the notebook generation phase. Prompts Prompts for the LLM agents were iteratively developed through trial and error, based on observed performance across datasets. Prompts for the exploration phase and the notebook generation are provided in Appendix A. The system prompt for the Dandiset Explorer application was omitted here for brevity and is available on our public GitHub repository. Agent framework All AI agents were implemented using custom logic that utilized API calls to OpenRouter (https://openrouter.ai/). Code execution Code execution during both phases of notebook creation was performed primarily on DANDI Hub, with some computation being performed on the author’s laptop. Notebook execution was limited to 600 seconds; if a timeout occurred, the LLM agent was notified and a modified version, presumably with reduced data loading, was generated and executed. Dandiset selection Recently published Dandisets were selected for inclusion in this study as described in the Results section. Notebook generation In the second phase of notebook generation, an LLM is tasked with generating a complete Jupyter notebook based on the chat conversation from the first phase (Appendix A, first prompt). This includes summaries, visual outputs, and code produced during exploration. The LLM is guided by structured instructions (Appendix A, second prompt) to produce a Jupytext-formatted Python script (https://jupytext.readthedocs.io/), which is more natural for LLMs to generate than notebook files. This script is then converted to a standard.ipynb notebook file. The notebook begins with a heading, a brief overview of the dandiset, and a disclaimer reminding users to critically assess AI-generated content. Subsequent sections walk the user through accessing the dataset via the DANDI Python API, exploring its NWB files using PyNWB, and visualizing example data. The script is executed in a controlled environment, and the agent engages in an execution and error correction loop, similar to that of the exploration phase, until a fully functioning notebook is produced without runtime errors. Because the code is adapted from already debugged exploration code, fewer errors and correction cycles are expected. Chat application system architecture The Dandiset Explorer was implemented as a web-based application using React and TypeScript, with a custom chat interface. The system integrates with LLMs through the OpenRouter API. Code execution is handled through integration with Jupyter kernels, allowing real-time Python code execution in either local Jupyter servers or remote JupyterHub instances. The application implements a custom tool system that enables the LLM agents to programmatically interact with the DANDI API for dataset metadata retrieval, access NWB file structure information through a specialized get_nwbfile_info tool (see below), and execute Python scripts with real-time output capture including text, error messages, and generated visualizations. Chat conversations are persisted in a cloud database. Agent access to NWB file information A key component enabling the system’s functionality is a Python tool called get_nwbfile_info, which is available to the Dandiset Explorer agent. This function streams a specified NWB file from the DANDI Archive and extracts detailed information about its contents. It automatically generates usage instructions for accessing each neurodata object in the file and produces structured textual summaries, including object attributes, descriptions, and dataset shapes. This information allows the agent to reason about the dataset structure and generate appropriate code and visualizations during exploration. Volunteer notebook reviewers For notebook reviewers, we contacted students, post-docs, and other scientists in the NWB/DANDI/Neurophysiology research community, including attendees at an NWB hackathon event. Individuals who chose to participate were provided with access to our review website in order to read the notebooks and anonymously complete the reviews. Participants were allowed to select which notebooks to review and were instructed to review as many notebooks as they could within the allotted time. Supplementary information Appendix A Supplementary Table 1 Supplementary Table 2 Acknowledgements This work was supported by the NIH under awards NIH U24NS120057 and NIMH 1R24MH117295. Author contributions All authors contributed to the conception and design of the study, software development, data collection and analysis, and writing and reviewing of the manuscript. All authors read and approved the final version. Data availability The AI-generated Jupyter notebooks analyzed in this study are available on Zenodo28. The spreadsheet of notebook reviewer scores with accompanying comments/rationale is also available in the same Zenodo record. Code availability The source code for both the notebook generator and the chat, at the time of writing, is available on GitHub: https://github.com/dandi-ai-notebooks/dandi-ai-notebooks-study. https://github.com/dandi-ai-notebooks/dandiset-explorer. Competing interests Benjamin Dichter is the Founder and CEO of CatalystNeuro, a consulting company that specializes in open science in neurophysiology. The other three authors declare no competing interests. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Supplementary information The online version contains supplementary material available at 10.1038/s41597-025-06285-x. References Associated Data Data Citations Supplementary Materials Appendix A Supplementary Table 1 Supplementary Table 2 Data Availability Statement The AI-generated Jupyter notebooks analyzed in this study are available on Zenodo28. The spreadsheet of notebook reviewer scores with accompanying comments/rationale is also available in the same Zenodo record. The source code for both the notebook generator and the chat, at the time of writing, is available on GitHub: https://github.com/dandi-ai-notebooks/dandi-ai-notebooks-study. https://github.com/dandi-ai-notebooks/dandiset-explorer.