WINFlowNets: Warm-up Integrated Networks Training of Generative Flow Networks for Robotics and Machine Fault Adaptation

Zahin Sufiyan, Shadan Golestan, Yoshihiro Mitsuka, Shotaro Miwa, Osmar Zaiane · 2026

Read WINFlowNets: Warm-up Integrated Networks Training of Generative Flow Networks for Robotics and Machine Fault Adaptation by Zahin Sufiyan, Shadan Golestan, Yoshihiro Mitsuka, Shotaro Miwa, Osmar Zaiane (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17301.txt.

Introduction

Sufiyan and co-authors present WINFlowNets, a generative flow network framework that trains its two networks together. Flow networks learn stochastic policies for sequential decisions, but the continuous variant needs a pretrained retrieval network. That pretraining is a burden in shifting robot settings. WINFlowNets warms up the retrieval network first, then co trains both networks with a shared architecture and replay buffer. In simulated robot tasks, it beat the older framework and strong RL baselines on reward and stability. It also adapted well in fault settings.

Catalog record

CREATORZahin Sufiyan, Shadan Golestan, Yoshihiro Mitsuka, Shotaro Miwa, Osmar Zaiane
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43843
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.17301

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

WINFlowNets: Warm-up Integrated Networks Training of Generative Flow Networks for Robotics and Machine Fault Adaptation Zahin Sufiyan, Shadan Golestan, Yoshihiro Mitsuka, Shotaro Miwa, Osmar Zaiane Abstract Generative Flow Networks for continuous scenarios (CFlowNets) have shown promise in solving sequential decision-making tasks by learning stochastic policies using a flow and a retrieval network. Despite their demonstrated efficiency compared to state-of-the-art Reinforcement Learning (RL) algorithms, their practical application in robotic control tasks is constrained by the reliance on pre-training the retrieval network. This dependency poses challenges in dynamic robotic environments, where pre-training data may not be readily available or representative of the current environment. This paper introduces WINFlowNets, a novel CFlowNets framework that enables the co-training of flow and retrieval networks. WINFlowNets begins with a warm-up phase for the retrieval network to bootstrap its policy, followed by a shared training architecture and a shared replay buffer for co-training both networks. Experiments in simulated robotic environments demonstrate that WINFlowNets surpasses CFlowNets and state-of-the-art RL algorithms in terms of average reward and training stability. Furthermore, WINFlowNets exhibits strong adaptive capability in fault environments, making it suitable for tasks that demand quick adaptation with limited sample data. These findings highlight WINFlowNets’ potential for deployment in dynamic and malfunction-prone robotic systems, where traditional pre-training or sample inefficient data collection may be impractical. † † paperid: 123 1 Introduction Generative flow networks (GFlowNets) [ 3 ] is a class of generative algorithms designed to learn a stochastic policy for generating (sampling) discrete structured objects (such as graphs, and sequences) x\in\mathcal{X} , in the space of objects \mathcal{X} to maximize a reward function R(x) . To construct an object x , GFlowNets starts from an initial null state s_{0} and generates a trajectory by iteratively taking actions that add simple building blocks, passing through intermediate states until it reaches the final state x with a probability proportional to R(x) . The process can be shown using directed acyclic graphs (depicted in Figure 1(a) ), where nodes and edges denote the states and actions, respectively. This problem can be framed as a sequential decision-making problem, making GFlowNets analogous to Reinforcement Learning (RL) [ 22 ] . GFlowNets learns a distribution over trajectories, enabling efficient exploration and reducing the risk of local optima. GFlowNets has demonstrated potential in solving sequential decision-making problems [ 8 , 7 , 26 , 25 , 13 ] and has been compared with RL algorithms across various problem settings, such as combinatorial optimization problems [ 12 , 39 ] , HyperGrid modeling [ 3 , 23 ] , and operation scheduling [ 38 ] . In several real-world domains, where the space of objects \mathcal{X} is typically continuous, a variation of GFlowNet, known as Continuous Flow Networks (CFlowNets) [ 15 ] has been proposed. CFlowNets generates a distribution over all possible trajectories, and sample from it in proportion to the trajectories’ performance. As shown in various studies [ 15 , 3 , 32 ] , GFlowNet/CFlowNets can be more sample-efficient than certain state-of-the-art RL algorithms such as Proximal Policy Optimiztion (PPO) and Soft Actor Critic (SAC). (a) Directed acyclic graph representation of the decision-making process in GFlowNet. (b) An overview of our proposed decision-making framework. Figure 1 : Main concepts of WINFlowNets. Nevertheless, significant challenges remain in the training of CFlowNets [ 30 ] . While CFlowNets is effective for continuous spaces [ 14 ] , its application can be impractical particularly in dynamic robotic control tasks [ 17 , 32 ] such as object manipulation, where evolving environments require continual adaptation and where the space of objects \mathcal{X} is exponentially large or composed of long trajectories [ 3 , 19 ] , making repeated re-training computationally expensive and inefficient. To generate an object x , CFlowNets operates using two networks, i.e., a flow network F_{\theta} , parametrized by \theta , and a retrieval network G_{\phi} , parametrized by \phi . F_{\theta} computes the flow, i.e., the probabilistic transitions between consecutive states conditioned on actions (inflows and outflows in Figure 1(a) ). This computation relies heavily on G_{\phi} , which estimates the possible parent states of a given state. In the original CFlowNets training architecture, G_{\phi} is pre-trained prior to training F_{\theta} , assuming a fixed data distribution. However, this assumption limits the practical applicability of CFlowNets in continuous robotic control tasks. In robotic control tasks, environments often change due to evolving conditions and unexpected factors—particularly hardware faults and malfunctions [ 9 ] —resulting in out-of-distribution (OOD) scenarios. Without adaptation, robots are prone to errors in OOD situations, as they encounter previously unseen states more frequently. Such errors can slow down the learning process or gradually degrade the overall performance of CFlowNets. This is especially problematic because G_{\phi} requires a separate pre-training dataset, which may not be readily available in novel or dynamic environments. If G_{\phi} is pre-trained on data that does not fully represent the space of objects \mathcal{X} , it can lead to both incorrect inflow and outflow estimations. Moreover, pre-training G_{\phi} introduces an extra step into the workflow, increasing resource requirements such as time, memory, and computational power. Integrating the training of G_{\phi} with F_{\theta} is essential for enabling G_{\phi} to dynamically adapt to OOD situations. This paper proposes WINFlowNets (Figure 1(b) ), a novel CFlowNets framework that enables simultaneous training of both G_{\phi} and F_{\theta} networks within a shared architecture. To improve the training efficiency, both networks utilize a shared reply buffer. WINFlowNets begins with a Warm-Up phase, wherein G_{\phi} independently interacts with the environment to gather initial experience and refine its policy. The Warm-Up phase is followed by a Dual-Training phase, where both G_{\phi} and F_{\theta} actively interact with the environment, collect experience, and contribute to the shared replay buffer to support collaborative learning. Experiments in simulated robotic environments demonstrate that WINFlowNets outperforms standard CFlowNets in both training efficiency and adaptation performance under OOD situations. We also find that while WINFlowNets surpasses state-of-the-art RL algorithms (PPO, SAC) in performance, its training efficiency remains comparatively lower. Our contributions are: 1) We propose WINFlowNets, a novel training framework tailored for dynamic, continuous robotic control tasks. 2) We introduce and validate a two-phase training strategy—comprising a Warm-Up and Dual-Training phase—that enables efficient adaptation without the need for pre-training G_{\phi} . 3) We empirically demonstrate that WINFlowNets outperforms standard CFlowNets and state-of-the-art RL algorithms in both standard and OOD (faulty) environments. All code and data are publicly available at [link will be provided in the camera-ready version]. 2 Related Work Reinforcement Learning (RL) for adaptation in robotic tasks . There is a rich body of literature proposing various RL [ 36 , 5 ] and Meta-RL [ 1 , 16 , 21 , 20 , 10 ] . algorithms for adapting to OOD situations in robotic tasks. In a previous research, Meta-reinforcement learning (Meta-RL) frameworks have been developed to enable adaptation to unseen tasks using minimal data from task distributions. For instance, Neghabadi et al. [ 21 ] proposed a model-based Meta-RL approach for fault adaptation tasks, utilizing two adaptive learners: a gradient-based adaptive learner (GrBAL) and a recurrence-based adaptive learner (ReBAL). GrBAL, based on model-agnostic meta-learning (MAML), uses neural networks to learn generalized dynamics, while ReBAL employs recurrent neural networks for task adaptation.

Read the complete text