GFlowState: Visualizing the Training of Generative Flow Networks Beyond the Reward
Florian Holeczek, Andreas Hinterreiter, Alex Hernandez-Garcia, Marc Streit, and Christina Humer · 2026
Read GFlowState: Visualizing the Training of Generative Flow Networks Beyond the Reward by Florian Holeczek, Andreas Hinterreiter, Alex Hernandez-Garcia, Marc Streit, and Christina Humer (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-21830.txt.
Introduction
Holeczek and four colleagues present GFlowState, a visual analytics system for the training of Generative Flow Networks, which sample in proportion to a reward function. Standard tools track metrics but hide how models explore sample space, build trajectories, or shift sampling odds during training. GFlowState offers a candidate-ranking chart, a state projection, a node-link diagram of the trajectory network, and a transition heatmap. Case studies show it helps developers debug training, spot underexplored regions, and judge model quality.
Catalog record
| CREATOR | Florian Holeczek, Andreas Hinterreiter, Alex Hernandez-Garcia, Marc Streit, and Christina Humer |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55341 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.21830
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GFlowState: Visualizing the Training of Generative Flow Networks Beyond the Reward Florian Holeczek, Andreas Hinterreiter, Alex Hernandez-Garcia, Marc Streit, and Christina Humer Abstract We present GFlowState , a visual analytics system designed to illuminate the training process of Generative Flow Networks (GFlowNets or GFNs). GFlowNets are a probabilistic framework for generating samples proportionally to a reward function. While GFlowNets have proved to be powerful tools in applications such as molecule and material discovery, their training dynamics remain difficult to interpret. Standard machine learning tools allow metric tracking but do not reveal how models explore the sample space, construct sample trajectories, or shift sampling probabilities during training. Our solution, GFlowState , allows users to analyze sampling trajectories, compare the sample space relative to reference datasets, and analyze the training dynamics. To this end, we introduce multiple views, including a chart of candidate rankings, a state projection, a node-link diagram of the trajectory network, and a transition heatmap. These visualizations enable GFlowNet developers and users to investigate sampling behavior and policy evolution, and to identify underexplored regions and sources of training failure. Case studies demonstrate how the system supports debugging and assessing the quality of GFlowNets across application domains. By making the structural dynamics of GFlowNets observable, our work enhances their interpretability and can accelerate GFlowNet development in practice. \onlineid 0 \vgtccategory Analytics & Decisions \authorfooter Florian Holeczek, Andreas Hinterreiter, and Marc Streit are with Johannes Kepler University Linz. E-mail: [email protected], [email protected], [email protected]. Christina Humer is with Eidgenössische Technische Hochschule Zürich. E-mail: [email protected]. Alex Hernandez-Garcia is with Mila – Quebec AI Institute. E-mail: [email protected]. \teaser GFlowState allows users to analyze the training behavior of GFlowNets based on four views: (A) A ranking of generated objects based on their reward. (B) A visualization of the sample space of generated objects relative to a reference dataset. (C) A directed acyclic graph representing the generated trajectories. (D) A transition heatmap indicating transition probabilities. While components A and B depend on the samples produced during training, components C and D analyze the graph formed by their trajectories after training has completed. Introduction keywords Generative Flow Networks, Training Visualization, Directed Acyclic Graph Scientific discovery increasingly relies on generative AI models to suggest novel candidates, such as molecules or materials [ 31 , 12 ] . Accelerating this discovery process is critical to addressing global challenges such as climate change and global health by advancing battery technologies, electrocatalysts for hydrogen generation [ 23 ] , and the design of therapeutic molecules such as antibodies for combating infectious diseases [ 5 ] . Large-scale initiatives in machine learning for scientific discovery have demonstrated the potential of data-driven approaches to explore vast chemical and material spaces [ 4 ] . However, these spaces are combinatorially large, making an exhaustive search infeasible, and motivating algorithms that can efficiently explore and prioritize promising candidates. Generative Flow Networks (GFlowNets or GFNs for short) [ 1 ] were designed to tackle this task by learning to sample objects proportionally to a reward function. This principle, combined with the generalization potential of deep learning, enables an effective exploration of the search space and the generation of many diverse high-value solutions. GFlowNets have shown promising results in electrocatalyst [ 20 ] and antibody design [ 34 ] , offering advantages over other generative algorithms. For example, compared to Markov Chain Monte Carlo (MCMC) methods, GFlowNets can learn the structure of the sample space rather than relying solely on local transitions, amortizing the sampling cost during training. Compared to standard reinforcement learning, GFlowNets are particularly well-suited for tasks that require generating many diverse, high-reward solutions instead of a single optimal one. However, GFlowNet training remains difficult to interpret and diagnose. Key characteristics of GFlowNet training, such as sample space coverage, state transition dynamics, and the relationship between sampling probability and reward, are difficult to analyze with existing visualization platforms. The underlying structures rapidly grow combinatorially large, exceeding the limits of manual inspection and conventional monitoring tools. This creates a critical need for tailored visual analytics methods that make the structure and dynamics of GFlowNet training observable. In this work, we introduce GFlowState , a tool designed for interactive exploration and diagnosis of the GFlowNet training processes. To the best of our knowledge, no prior work has addressed the specific analytical needs for understanding and diagnosing GFlowNet training dynamics. Although recent advances in GFlowNet research have focused on improving learning algorithms and applications [ 25 , 18 , 12 ] , tools for interactive analysis of their learning behavior remain largely unexplored. As a result, practitioners currently rely on aggregate metrics or ad-hoc inspection methods that provide limited insight. In contrast, the visualization community has demonstrated the effectiveness of model-specific visual analytics solutions for understanding complex machine learning models. In reinforcement learning, interactive visualization systems enable the analysis of trajectories, policy decisions, and the comparison between agents during training [ 16 , 36 ] . Similar tailored tools have also been proposed for other model classes, including sequence and language models [ 29 , 28 , 24 ] , as well as convolutional neural networks [ 33 ] . These approaches demonstrate that visualization tools designed to capture the structure and training dynamics of a specific model class can reveal behaviors that are difficult to observe with standard metrics alone. However, these approaches are tightly coupled to the structure of the respective learning paradigms and cannot be directly applied to GFlowNets. To address this gap, we collaborated with GFlowNet developers at Mila, the Quebec AI Institute, to identify analysis tasks to rigorously assess the quality of GFlowNet training runs, inspect model shortcomings, and aid debugging. Key tasks include ( i ) the analysis of sampling trajectories, ( ii ) the exploration of the space of generated training samples, and ( iii ) the analysis of training dynamics throughout episodes. Based on an iterative feedback loop with our collaborators, we designed GFlowState to address these and other tasks effectively. Our main contributions are: • an identification of analysis tasks for improving the understanding and diagnosis of GFlowNet training; • GFlowState , an interactive prototype addressing these tasks; and • an evaluation of the approach based on case studies from two application domains. 1 A Generative Flow Networks Primer GFlowNets provide a generative framework for sampling proportionally to a reward function. GFlowNets treat sampling as a sequential decision problem: Generating the final sample is a process of constructing it from smaller building blocks. To illustrate these concepts, we first introduce the grid environment used in the original GFlowNet paper [ 1 ] . Throughout the paper, we will use this grid environment as a guiding example. Consider a Markov decision process (MDP) where the states are cells of a two-dimensional grid of side length H . The agent starts at position s_{0}=(0,0) and moves by sampling actions a=(a_{x},a_{y});a_{x},a_{y}\in[0,1] . The current state transitions into the next state according to the sampled action: s_{i+1}=s_{i}+a_{i} . The agent can also sample a stop action to terminate the sampling trajectory. In this example setup, a fixed reward distribution is defined to place high-reward modes near the corners of the grid (see Fig. 1 ). Figure 1 : A heatmap of the true rewards