CORDS: Continuous Representations of Discrete Structures

Tin Hadži Veljković, Erik Bekkers, Michael Tiemann, Jan-Willem van de Meent · 2026

Read CORDS: Continuous Representations of Discrete Structures by Tin Hadži Veljković, Erik Bekkers, Michael Tiemann, Jan-Willem van de Meent (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21583.txt.

Introduction

Tin Hadži Veljković, Erik Bekkers, Michael Tiemann, and Jan-Willem van de Meent give learning systems a smooth way to predict sets of unknown size. Many tasks need a set as output: objects in a scene, molecules, or sources in the sky. The count is part of what must be predicted.

Older methods pad outputs to a fixed size or guess the count as a separate step. Both routes are clumsy and error-prone. The authors build continuous representations of discrete sets, so learning can flow smoothly. A cleaner set representation helps detection, science, and design tools that think in collections rather than single answers.

Catalog record

CREATORTin Hadži Veljković, Erik Bekkers, Michael Tiemann, Jan-Willem van de Meent
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE108 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21583

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CORDS: Continuous Representations of Discrete Structures Tin Hadži Veljković, Erik Bekkers, Michael Tiemann, Jan-Willem van de Meent Abstract Many learning problems require predicting sets of objects when the number of objects is not known beforehand. Examples include object detection, molecular modeling, and scientific inference tasks such as astrophysical source detection. Existing methods often rely on padded representations or must explicitly infer the set size, which often poses challenges. We present a novel strategy for addressing this challenge by casting prediction of variable-sized sets as a continuous inference problem. Our approach, CORDS ( Continuous Representations of Discrete Structures ), provides an invertible mapping that transforms a set of spatial objects into continuous fields: a density field that encodes object locations and count, and a feature field that carries their attributes over the same support. Because the mapping is invertible, models operate entirely in field space while remaining exactly decodable to discrete sets. We evaluate CORDS across molecular generation and regression, object detection, simulation-based inference, and a mathematical task involving recovery of local maxima, demonstrating robust handling of unknown set sizes with competitive accuracy. 1 Introduction In problems where we wish to reason about discrete structure, we often need to reason about a set of objects without knowing the cardinality of this set in advance. Examples include object detection ( 33 ; 38 ) , scientific inference tasks, such as reconstructing catalogs of astrophysical sources ( 35 ; 4 ) , or conditional molecular generation, where the conditioned property does not uniquely determine the number of atoms ( 10 ; 22 ) . Inferring cardinality directly from data is often difficult, making sampling in conditional generation or inference tasks inefficient. Reasoning about unknown cardinality is a challenge that has been around for a long time. Classic approaches include model selection with variational inference ( 1 ) , reversible jump MCMC ( 28 ) , and Bayesian nonparametrics ( 12 ) . In modern approaches based on deep learning, a common strategy is to pre-allocate capacity beyond what is typically needed, and then suppress or ignore unneeded capacity ( 44 ) . In the sciences, similar ideas appear when combining variational inference and empirical Bayes estimation, where learning the prior can serve to prune unneeded degrees of freedom 36 . These examples reflect a pervasive pattern: rather than modeling the distribution over cardinalities p(N) explicitly, many methods sidestep the issue by way of user-specified truncations or paddings. In parallel, continuous representations have become increasingly common across domains, offering flexible ways to encode signals and structures and partly addressing the challenges of variable cardinality. Neural fields and coordinate-based models ( 18 ; 31 ; 42 ) showed how images and scenes can be embedded in continuous domains, and this perspective has since been extended to molecules and proteins 23 ; 14 ; 10 . These approaches remove the need to specify the number of objects in advance, since atoms or components are represented as smooth densities that can, in principle, be sampled or inpainted flexibly 10 . Yet cardinality is still only inferred indirectly, and object attributes are often added afterwards through auxiliary classifiers or peak-detection heuristics, rather than being built into the representation itself. As a result, continuous fields provide flexibility but not a unified treatment of both counts and features. To address this gap, we introduce CORDS ( Continuous Representations of Discrete Structures ). Here, discrete objects are mapped into continuous fields , smooth functions defined over an ambient domain (e.g. space, time, or grids), where a density field encodes object counts and positions, and a feature field carries their attributes. The total density mass then acts as a continuous, differentiable quantity that implicitly encodes the number of objects. The construction is bijective, so models can operate directly in field space and still recover discrete sets without auxiliary mechanisms such as fixed slots or padding. This provides a systematic alternative to existing workarounds, offering a single representation that applies across domains. Figure 1: An image with N MNIST digits (top) is encoded with CORDS into a density field \rho(\mathbf{r}) (middle) and per-class feature fields h_{k}(\mathbf{r}) (bottom). The number of objects is encoded directly in the density mass, N=\int\rho(\mathbf{r})\,d\mathbf{r} . 2 Related Work Generative modeling on graphs. Graph generative modeling is typically framed in either discrete or continuous terms. Discrete methods treat molecules as graphs of nodes and edges encoded through adjacency matrices, as in DiGress ( 37 ) , which applies discrete denoising diffusion, variational flow matching ( 9 ) , which casts flow matching as variational inference over categorical states, and MoFlow ( 47 ) , which uses invertible flows to generate atoms and bonds with exact likelihoods. Continuous methods instead generate atomic coordinates and features directly in 3D space. Examples include G-SchNet ( 11 ) , which models molecular conformations with equivariant GNNs, ENF ( 30 ) , which extends normalizing flows with equivariant dynamics, and diffusion-based approaches such as EDM ( 13 ) , EDM-Bridge ( 41 ) , and GeoLDM ( 45 ) . More recent architectures such as Ponita ( 3 ) and Rapidash ( 34 ) further improve scalability with multi-scale equivariant representations. Alongside these approaches, several works have explored representing molecular graphs in continuous fields rather than explicit graphs. VoxMol ( 23 ) represents molecules as voxelized density grids processed by convolutional networks, while Ragoza et al. ( 25 ) introduced one of the earliest 3D molecular generative models based on atomic density fields, reconstructing molecules through peak detection and bond heuristics. FuncMol ( 14 ) proposed a neural field parameterization of molecular occupancy, and ProxelGen ( 10 ) extended this idea to conditional generation and inpainting. While these methods remove the need to fix graph size in advance, they ultimately recover atoms and features through thresholding or auxiliary classifiers, leaving cardinality and features only indirectly modeled. Object detection. Two-stage detectors such as Faster R-CNN ( 27 ) generate region proposals followed by refinement, while one-stage models like YOLO ( 38 ) and RetinaNet ( 16 ) predict boxes and classes densely, balancing speed and accuracy. EfficientDet ( 32 ) further improves this trade-off via compound scaling. Transformer-based approaches, exemplified by Deformable DETR ( 50 ) and the real-time RT-DETR family ( 39 ) , accelerate convergence and improve small-object handling. Methods that operate through heatmaps or density maps, e.g., CenterNet ( 6 ) , crowd counting ( 43 ) , and microscopy detection ( 15 ) , are most closely related to our setting, as they localize objects from continuous spatial representations. However, unlike CORDS, these approaches focus purely on localization and do not model or recover object-level features from the underlying fields. Simulation-based inference. Simulation-based inference (SBI) is used in domains such as cosmology, astrophysics, and particle physics where the likelihood is intractable but simulators are available. Early approaches such as Approximate Bayesian Computation (ABC) ( 2 ) relied on handcrafted summary statistics, while modern neural methods—neural posterior estimation (NPE) and neural ratio estimation (NRE) ( 19 ; 17 ) —provide flexible and scalable inference. Recent advances include flow matching posterior estimation (FMPE) ( 5 ) , which leverages flow matching to improve scalability and accuracy, achieving state-of-the-art results in gravitational-wave inference. Yet,

Read the complete text