Offline Materials Optimization with CliqueFlowmer

Jakub Grudzien Kuba, Benjamin Kurt Miller, Sergey Levine, Pieter Abbeel · 2026

Read Offline Materials Optimization with CliqueFlowmer by Jakub Grudzien Kuba, Benjamin Kurt Miller, Sergey Levine, Pieter Abbeel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-06082.txt.

Introduction

Kuba, Miller, Levine, and Abbeel propose an offline model-based tuning route to code-based matter finding that fuses direct trait tuning into making. Popular making methods, trained by maximum chance, explore nice regions of matter space too shyly to find the best picks. Their domain-set model, CliqueFlowmer, combines clique-based tuning with trans-net and flow making. The model's matter beat those from making baselines by a clear margin, and the authors release code, model weights, and project means to support expert finding work.

Catalog record

CREATORJakub Grudzien Kuba, Benjamin Kurt Miller, Sergey Levine, Pieter Abbeel
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86790
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.06082

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Offline Materials Optimization with CliqueFlowmer Jakub Grudzien Kuba, Benjamin Kurt Miller, Sergey Levine, Pieter Abbeel Abstract Recent advances in deep learning have inspired neural network-based approaches to computational materials discovery (CMD). A plethora of problems in this field involve finding materials that optimize a target property. Nevertheless, the increasingly popular generative modeling methods are ineffective at boldly exploring attractive regions of the materials space due to their maximum likelihood training. In this work, we offer an alternative CMD technique based on offline model-based optimization (MBO) that fuses direct optimization of a target material property into generation. To that end, we introduce a domain-specific model, CliqueFlowmer , that incorporates recent advances in clique-based MBO into transformer and flow generation. We validate CliqueFlowmer’s optimization abilities and show that materials it produces strongly outperform those from generative baselines. To enable its use in specialized materials discovery problems and support interdisciplinary research, we open-source our code and provide additional project information at znowu/CliqueFlowmer \faXTwitter @kuba_AI 1 Introduction Large neural network models have recently enabled solving challenging artificial intelligence tasks such as language modeling, automated coding, and image generation ( 1 ; 55 ; 17 ; 6 ) . Meanwhile, scientific problems that deal with the world of atoms, rather than the world of bits, are yet to benefit from this revolution ( 56 ) . Indeed, exploration of new physical breakthroughs continues to take place in physical wet labs, and is driven by costly physical experimentation ( 63 ; 7 ; 51 ) . One such problem is computational materials discovery (CMD), wherein scientists strive to invent new chemical structures that display properties absent in known materials ( 5 ; 23 ; 65 ; 39 ) . A systematic, sample-efficient method for CMD has the potential to deliver structures that serve as catalysts for key energy-conversion reactions and as functional components of advanced biomaterials. Such advances would greatly accelerate the development of clean-energy technologies and medical therapies, effectively extending the recent AI-driven progress from the world of bits to the world of atoms ( 21 ; 44 ; 11 ; 13 ) . Having realized the importance of this problem, AI researchers have been increasingly turning their attention to AI-driven CMD. Most commonly, the introduced methods have been utilizing the celebrated diffusion and flow models that had become very successful in the domain of image generation ( 17 ; 40 ; 43 ; 22 ) . While effective at harnessing the distribution of viable materials presented to them in the dataset, likelihood-based generative models do not actively explore the relation between the materials and their properties. This is a major impediment in settings where CMD is expected to optimize materials with respect to a specified metric ( 67 ; 14 ) . In the meantime, a new paradigm, known as offline model-based optimization (MBO) has made steps towards techniques that optimize scientific designs by bootstrapping models trained entirely on offline data ( 34 ; 57 ; 32 ) . Nevertheless, MBO methods have been mainly deployed in standard benchmark tasks, free, for example, of the challenging intricacies of CMD in which data is inherently of the hybrid discrete-continuous nature, irregular shape, and subject to physical constraints. To address the need for AI-driven CMD and the limitations of generative approaches, in this work, we introduce CliqueFlowmer —a model that renders materials data tractable by MBO and enables optimizing material structures. The core of the model is an auto-encoder that converts the multi-modal, irregular materials data into structured, finite-dimensional vectors that can be optimized with clique-based MBO ( 32 ) , and mapped back into the material form. These abilities are attained by a carefully engineered neural network architecture that utilizes transformers and combines them with generative learning techniques such as next-token prediction and flow matching, and MBO techniques such as clique decomposition ( 12 ) . Empirically, utilizing data from Materials Project ( 23 ) as well as M3GNet ( 3 ) and MEGNet ( 3 ) as oracles for the studied properties, we show that materials optimized by CliqueFlowmer vastly outperform those sampled by generative baselines. Additionally, we hope that this work will attract attention to the field of MBO and will open up possibilities for more effective CMD methods. Figure 1 : The unit cell of a hypothetical material. The cell has a shape of a parallelepiped determined by three axes, \vec{{\textnormal{a}}} , \vec{{\textnormal{b}}} , and \vec{{\textnormal{c}}} . The angles between the axes are \text{ang}(\vec{{\textnormal{b}}},\vec{{\textnormal{c}}})=\alpha , \text{ang}(\vec{{\textnormal{c}}},\vec{{\textnormal{a}}})=\beta , \text{ang}(\vec{{\textnormal{a}}},\vec{{\textnormal{b}}})=\gamma . In this cell, there are five atoms, whose type sequence is {\mathbf{a}}=[\text{N, Cl, C, O, S}] . 2 Background This section provides the necessary background on the key notions discussed by our work. First, we lay down foundations of material modeling that enables us to use machine learning methods in CMD. Then, we introduce the basic ideas of continuous normalizing flows ( i.e., flow matching). Lastly, we state the problem that we aim to solve (CMD) and formulate it through the lens of offline MBO. 2.1 Modeling Materials We characterize a material \mathsf{M} by its unit cell —the material’s unit amount. The cell has a shape of a parallelepiped determined by the lengths of its axes, (a,b,c)\in\mathbb{R}_{+}^{3} , and angles between them, (\alpha,\beta,\gamma)\in(0,\pi)^{3} . The content of the cell is a set of N_{\text{atom}} (which varies between materials) atoms that is represented by the sequence {\mathbf{a}}\in\mathcal{A}^{N_{\text{atom}}} of their types, as well as their positions {\mathbf{X}}\in[0,1)^{N_{\text{atom}}\times 3} expressed in the basis induced by (a,b,c) and (\alpha,\beta,\gamma) . Thus, the space of materials \mathsf{M} can be embedded in the product \mathcal{M}=\mathbb{R}_{+}^{3}\times(0,\pi)^{3}\times(\mathcal{A}\times[0,1)^{3})^{*} , and a single material can be characterized as a tuple [(a,b,c),(\alpha,\beta,\gamma),{\mathbf{a}},{\mathbf{X}}] . 1 1 1 \mathcal{S}^{*} denotes the countable union of the products of the set \mathcal{S} with itself. That is, \mathcal{S}^{*}=\cup_{n=1}^{\infty}\mathcal{S}^{n} . We denote the geometrical part of the material as {\mathbf{G}}=[(a,b,c),(\alpha,\beta,\gamma),{\mathbf{X}}] , and thus can write [{\mathbf{a}},{\mathbf{G}}] to represent material \mathsf{M} . 2.2 Flow Matching The goal of a continuous normalizing flow ( 40 ) is to learn sampling from a data distribution p_{\text{data}}({\mathbf{x}}) . To that end, one sets the target distribution p_{1}({\mathbf{x}})=p_{\text{data}}({\mathbf{x}}) , as well as source distribution p_{0}({\mathbf{x}})=p_{\text{source}}({\mathbf{x}}) , such as the standard-normal. To learn to turn a sample from p_{\text{source}}({\mathbf{x}}) into one from p_{\text{data}}({\mathbf{x}}) , one interpolates noise and data, \displaystyle{\mathbf{x}}_{t}=(1-t)\cdot{\mathbf{x}}_{0}+t\cdot{\mathbf{x}}_{1},\quad\text{where}\quad{\mathbf{x}}_{0}\sim \displaystyle p_{0}({\mathbf{x}}),\ {\mathbf{x}}_{1}\sim p_{1}({\mathbf{x}})\ \ \&\ \ t\sim p_{\text{time}}(t), and passes the mixture, together with timestep information, to a velocity neural network that minimizes \displaystyle\mathbb{E}_{{\mathbf{x}}_{0}\sim p_{0},{\mathbf{x}}_{1}\sim p_{1},t\sim p_{\text{time}}}\big[\big(v_{\theta}({\mathbf{x}}_{t},t)-({\mathbf{x}}_{1}-{\mathbf{x}}_{0})\big)^{2}\big]. Once learned, the velocity network can be used to sample from the target distribution by solving the following ordinary differential equation (ODE) with initial conditions, \displaystyle\text{d}{\mathbf{x}}_{t}=v_{\theta}({\mathbf{x}}_{t},t)\text{d}t,\ \quad{\mathbf{x}}_{0}\sim p_{0}({\mathbf{x}}), from t=0 to t=1 , which results in a sample {\mathbf{x}}_{1}\sim p_{1}({\mathbf{x}}) . Typically, the ODE is solved numerically, by discretizing the time interval [0,1] into N_{\text{step}} steps and using, e.g., the Euler method. In this paper, we will use continuous normalizing flows to model the geometry of materials. 2.3 Model-Based Optimization in Computational Materials Discovery We consider a function f(\mathsf{M})\in\mathbb{R} , often referred to as target property , and assume that we are given a finite dataset \mathcal{D}=\{\mathsf{M}^{i},{\textnormal{y}}^{i}=f(\mathsf{M}^{i})\}_{i=1}^{N} of examples of materials and their values of the property. Given this dataset, our goal is to discover materials that optimize

Read the complete text