StippleDiffusion: Capacity-Constrained Stippling using Controlled Diffusion
Ofir Gilad, Aleksander Plocharski, Przemyslaw Musialski, Andrei Sharf · 2026
Read StippleDiffusion: Capacity-Constrained Stippling using Controlled Diffusion by Ofir Gilad, Aleksander Plocharski, Przemyslaw Musialski, Andrei Sharf (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15816.txt.
Introduction
Ofir Gilad and three colleagues present a diffusion-based sampler for stippling, where point density must track a target image. Earlier learned methods handled only unconditional point generation, while traditional optimizers rerun slowly for each new density. Their ControlNet branch conditions an optimal-transport point-set diffusion model on the target density map and a high-resolution image, restricted to late-stage denoising from a density-weighted rejection sample. On the Icons-50 benchmark it matched per-density optimizers on every reported metric while staying differentiable.
Catalog record
| CREATOR | Ofir Gilad, Aleksander Plocharski, Przemyslaw Musialski, Andrei Sharf |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 41031 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.15816
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
StippleDiffusion: Capacity-Constrained Stippling using Controlled Diffusion Ofir Gilad, Aleksander Plocharski, Przemyslaw Musialski, Andrei Sharf Abstract . Stipple patterns—point sets whose local density tracks a target image—are traditionally produced by per-density iterative optimizers, which are slow, non-differentiable, and must be re-run from scratch for each new target. Learned alternatives have so far addressed only unconditional point generation; capacity-constrained, image-conditioned stippling has remained out of reach. We present the first diffusion-based sampler that simultaneously satisfies a learned local point-distribution prior and a continuous, image-defined capacity constraint at inference. The method is a ControlNet branch built on top of an optimal-transport-grid point-set diffusion baseline, conditioned on the target density map and a high-resolution image. Two design choices make the combination tractable: training and inference are restricted to the late-stage denoising regime, initialized from a density-weighted rejection sample, and the standard zero-convolution injection is replaced with a sigmoid-gated 1 × 1 1{\times}1 projection that preserves the base model’s blue-noise structure under hard density signals. A single trained checkpoint accepts arbitrary target densities at inference, generalizes to point budgets that were not seen during training, and produces stipples in time nearly independent of the output point count. On the Icons-50 benchmark, our learned sampler reaches parity with per-density-optimized baselines on every reported metric while remaining differentiable end-to-end. Figure 1 . StippleDiffusion generates capacity-constrained stipple patterns by conditioning a U-Net diffusion process on a target density image. (Left) The controlled denoising process progressively refines the point distribution while preserving local point repulsion. (Right) Examples on diverse target densities, showing the target density maps, the generated stipple results, and the corresponding Optimal Transport (OT) grid-based representation used by our model. 1. Introduction The generation of high-quality 2D point sets with prescribed spatial properties is a fundamental problem in computer graphics, with applications in rendering, object placement, and image stippling. Classical approaches typically rely on iterative optimization or carefully designed sampling strategies to produce structured distributions such as blue-noise patterns. While effective, these methods are often computationally expensive, depend on specialized numerical solvers, and are difficult to extend to multiple simultaneous constraints. Crucially, each new target density requires a fresh optimization run, ruling out interactive use or large-scale generation. Recently, denoising diffusion probabilistic models have emerged as a powerful alternative for learning complex geometric distributions directly from data ( 10 ) . However, applying diffusion models to scattered point sets remains challenging due to their convolutional structure, which is inherently designed for grid-based data rather than unordered sets. A common workaround introduces a coupling between points and a regular grid via optimal transport, where each point is assigned to a stratum center and represented as a local offset. While this enables the use of convolutional architectures, it still struggles to faithfully capture fine-grained spatial constraints and structured point distributions. Extending diffusion-based samplers to spatially controlled settings—such as grayscale-conditioned stippling or capacity-constrained point distributions—introduces additional challenges. While standard conditioning architectures ( 43 ) can be applied to geometric optimal transport (OT) grids, they often lead to instability in practice. In particular, jointly enforcing global point allocation from pure Gaussian noise and satisfying high-frequency local density constraints is difficult to balance. This mismatch can lead to uneven within-silhouette density, residual point clustering, and loss of fine geometric structure in the generated distributions. In this paper, we bridge the gap between example-based geometric sampling and image-conditioned generation by introducing a diffusion-based framework for capacity-constrained stippling. Instead of forcing the model to generate valid point configurations directly from pure noise over the full diffusion trajectory—an approach that often leads to geometric collapse under strict density constraints—we reformulate the problem as a guided refinement process. Our method begins with a structured initialization obtained via density-weighted rejection sampling, which provides a spatially informed prior over point locations. Starting from this initialization, we apply a truncated diffusion process focused primarily on late-stage denoising, inspired by editing-based diffusion strategies ( 20 ) . This design allows the model to progressively refine point positions while preserving global structure and maintaining local density (capacity) consistency. We further extend the framework to grayscale-conditioned stippling by introducing image-based spatial guidance within the diffusion process. This enables the point distribution to adapt to underlying image intensity while still respecting capacity constraints. Overall, our approach stabilizes diffusion-based point generation under structured constraints and enables high-quality, spatially controlled stippling. The resulting model accepts an arbitrary target density at inference, and we further observe that a single checkpoint generalizes across point budgets that were not seen during training. Our contributions are as follows: • We propose the first diffusion-based framework for capacity-constrained stippling under explicit spatial control; a single trained model accepts arbitrary target densities at inference, in contrast to optimization-based stipplers that re-run per density. • We reinterpret structured point generation through an optimal transport formulation to enable learning of blue-noise-like distributions under constraints, with inference cost nearly independent of the output point count and gradients available end-to-end through the sampler. • We introduce a quantitative and qualitative evaluation protocol adapted from blue-noise and capacity-constrained sampling literature for comparing capacity-aware and spatially conditioned point generation methods against prior unconditional samplers. Figure 2 . Illustration of the model architecture and inference flow. Starting from the conditioning input (top block), features are extracted, aggregated, and encoded (blue blocks), and then used to control the stippled diffusion process (gray blocks). 2. Related Work 2.1. Blue Noise Blue-noise distributions are characterized by a power spectrum whose low frequencies vanish and whose remaining energy concentrates in an isotropic high-frequency annulus. They are widely used in Monte Carlo integration ( 34 ; 27 ) , digital halftoning ( 37 ) , and image stippling ( 9 ) , and they also describe biologically optimized arrangements such as the retinal cone mosaic ( 42 ) . Their construction is typically expensive: classical methods rely on kernel-based formulations ( 12 ) and Gaussian variants ( 2 ) , on pair-correlation matching ( 25 ) , on optimal transport ( 7 ; 28 ; 26 ) , or on fast approximations thereof ( 21 ) . Tile-based synthesis instead trades runtime for memory by pre-computing reusable patches ( 23 ; 24 ; 1 ; 13 ; 38 ) . Consequently, a number of researchers focused on developing methods to compute point sets with high-quality blue noise properties, typically by evenly distributing points over a domain via Lloyd-based iterations ( 19 ; 8 ; 32 ; 3 ; 16 ; 40 ; 5 ) , electrostatic forces ( 31 ) , statistical-mechanics interacting Gaussian particle models ( 12 ) , farthest-point optimization ( 30 ) , or Gaussian blue noise ( 2 ) . These iterative methods consistently generate much improved point distributions, albeit at sometimes excessive computational complexity. The optimal-transport formulation of capacity-constrained Voronoi tessellation introduced by ( 7 ) casts blue-noise generation as a continuous optimization in the space of power diagrams, providing the analytic baseline that learning-based samplers seek to approximate. Stippling has further been extended to the color domain by treating it as a multi-class blue-noise sampling problem ( 18 ) . What all of these classical methods share is a per-density optimization step: a fresh iterative run is required for each new target density, with no inference-time conditioning