Proportion and Perspective Control for Flow-Based Image Generation
Julien Boudier, Hugo Caselles-Dupré · 2025
Read Proportion and Perspective Control for Flow-Based Image Generation by Julien Boudier, Hugo Caselles-Dupré (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21763.txt.
Introduction
Julien Boudier and Hugo Caselles-Dupré give artists two missing dials for AI image generation: proportion and perspective. Text-to-image models render beautifully but place objects at whatever size and geometry they please, leaving illustrators and designers without the spatial authority they have in every traditional medium.
Two specialized ControlNet modules add that authority. One reads bounding boxes and honors the position and scale they dictate for each object; the other reads vanishing lines and bends the scene's three-dimensional geometry to match. Vision-language models help annotate the training data, and custom algorithms synthesize the conditioning images. Experiments show both dials work, though complex combined constraints still strain them, and both modules are released publicly for others to build on.
Catalog record
| CREATOR | Julien Boudier, Hugo Caselles-Dupré |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 10 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21763
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Proportion and Perspective Control for Flow-Based Image Generation Julien Boudier, Hugo Caselles-Dupré Abstract While modern text-to-image diffusion models generate high-fidelity images, they offer limited control over the spatial and geometric structure of the output. To address this, we introduce and evaluate two ControlNets specialized for artistic control: (1) a proportion ControlNet that uses bounding boxes to dictate the position and scale of objects, and (2) a perspective ControlNet that employs vanishing lines to control the 3D geometry of the scene. We support the training of these modules with data pipelines that leverage vision-language models for annotation and specialized algorithms for conditioning image synthesis. Our experiments demonstrate that both modules provide effective control but exhibit limitations with complex constraints. Both models are released on HuggingFace: https://huggingface.co/obvious-research . (a) The Proportion ControlNet uses bounding boxes to guide object placement with proportions principles. (b) The Perspective ControlNet uses vanishing lines to define the 3D geometry and viewpoint of the scene. Figure 1 : Overview of the proposed ControlNets for high-level artistic control. 1 Introduction Text-to-image diffusion models [ 2 ] have demonstrated remarkable capabilities in synthesizing complex images. However, their reliance on a single text prompt provides only coarse, global control, hindering their use in applications requiring compositional accuracy. The ControlNet architecture [ 9 ] introduced a paradigm for adding fine-grained, spatial conditioning to pre-trained models. While existing ControlNets for tasks like edge detection (Canny), line art, and depth provide low-level structural control, they don’t provide control for all applications. Our work focuses on providing artists with control based on fundamental principles of proportion and perspective. We extend this paradigm to a Flow Matching-based diffusion model, FLUX.1-dev [ 3 ] , and introduce two ControlNets, which are available open-source: the Proportion ControlNet and the Perspective ControlNet . A contribution is the development of data pipelines that generate the required training triplets from unannotated datasets. We evaluate each ControlNet individually and explore the dynamics of applying them simultaneously. 2 Method For each control task, a dedicated ControlNet is trained while the pre-trained FLUX model remains frozen. The modules are initialized from a pre-trained LineArt ControlNet to provide a foundational understanding of lines as structural guides. Training minimizes the conditional Flow Matching loss [ 4 ] , where the velocity field is conditioned on both text c_{\text{text}} and a visual condition c_{\text{cond}} : \mathcal{L}=\mathbb{E}_{x_{0},t,x_{t},c_{\text{text}},c_{\text{cond}}}\left[\|v_{t}(x_{t}\mid x_{0})-v_{\theta}(x_{t},t,c_{\text{text}},c_{\text{cond}})\|^{2}\right] 2.1 High-Level Conditioning Design Our choice of conditioning inputs is motivated by the desire for artist-friendly, high-level control. Proportion Control via Bounding Boxes. Our goal is to control the abstract proportions of a scene—the placement and visual weight of its semantic elements. • Comparison to Regional Prompting: Unlike regional prompting, which rigidly assigns different textual prompts to specific masked areas, our method uses a single global prompt. The bounding boxes define regions where elements described in the global prompt should appear, giving the model freedom to interpret the scene and decide how to fill those spaces. This decouples proportion from content segmentation. • Comparison to Low-Level Controls: This approach also differs from pixel-level controllers like Canny or LineArt, which dictate the exact contours of an object. Bounding boxes are a higher level of abstraction, defining the semantic space an object should occupy, not its specific shape. This allows artists to easily apply proportion rules like the rule of thirds without pre-visualizing detailed outlines. Perspective Control via Vanishing Lines. Our aim is to provide an intuitive method for defining the 3D geometry of a scene. • Problem with Vanishing Points (VPs): While VPs mathematically define perspective, they are a problematic conditioning input. VPs for parallel lines (1-point perspective) are at infinity and cannot be represented in a fixed-size image. Other VPs can be located far outside the canvas, making their encoding imprecise and difficult for a user to manipulate. • Solution with Vanishing Lines: We therefore condition on vanishing lines. These lines are intuitive for an artist to draw, mimicking the sketching process of defining convergence. Crucially, they are always contained within the canvas, providing a direct, unambiguous, and spatially grounded proxy for VPs. This makes them a more robust and user-friendly input for controlling perspective. 2.2 Data Pipelines To train our models, we designed two distinct, fully automated data pipelines, shown in Figure 2 . Proportion Pipeline. This pipeline processes the WikiArt dataset ( \sim 80k images). After filtering for aesthetic quality (score > 5.0) using a SigLIP-based [ 8 ] predictor [ 1 ] to 73k images, Florence-2 [ 7 ] generates a short caption (for the prompt) and a detailed caption. Grounding DINO [ 5 ] then uses the detailed caption to detect object bounding boxes, which are rendered to create the conditioning image. Perspective Pipeline. This pipeline processes a 1M image subset of OpenImages v7. After aesthetic filtering (score > 3.5), a geometric filter using the 2-Line Exhaustive Search algorithm [ 6 ] identifies images with a strong perspective structure. This yielded a final dataset of 373,632 images. The data distribution is heavily skewed: the vast majority have a reliable 1-point perspective, a smaller fraction have a 2-point perspective, and 3-point perspectives are extremely rare. WikiArt Dataset 1. Aesthetic Filtering ( > 5.0) 2. Captioning (Florence-2) 3. Object Detection (Grounding DINO) Training Triplets FLUX ControlNet Proportion Training (a) Proportion pipeline. OpenImages v7 (1M subset) 1. Aesthetic Filtering ( > 3.5) 2. Vanishing Point Detection 3. Captioning (Florence-2) Training Triplets FLUX ControlNet Perspective Training (b) Perspective pipeline. Figure 2 : Overview of the data pipelines. Both produce structured training triplets from unannotated datasets. 3 Experiments and Results (a) Bounding box control for object placement. (b) Emergent generalization to non-rectangular shapes. Figure 3 : Results from the Proportion ControlNet. 3.1 Qualitative Results of Individual Modules Proportion Control. The model effectively adheres to bounding box constraints (Figure 3 a). Training on WikiArt induced a strong "pictorial" style bias, whose intensity correlates with higher ControlNet guidance strength (Figure 4 ). Due to its LineArt initialization, the model exhibits an emergent ability to interpret non-rectangular shapes as proportional guides (Figure 3 b). Figure 4 : Illustration of the stylistic bias. The horizontal axis represents the ControlNet guidance strength (0.5 to 0.9). The vertical axis represents the end percentage of the diffusion process during which the ControlNet is active (0.1 to 0.9). The "pictorial" effect intensifies as both values increase. (a) Successful control for a 1-point perspective. (b) Failure on a 3-point perspective ("axis dropping"). Figure 5 : Results from the perspective ControlNet. Perspective Control. The model successfully generates scenes respecting 1- and 2-point perspectives (Figure 5 a). However, it consistently fails to render 3-point perspectives, often exhibiting "axis dropping" by ignoring vertical convergence (Figure 5 b). This failure is directly attributable to the skewed data distribution. Furthermore, the model shows a strong prior for straight horizons, likely inherited from both its training data and the base FLUX model. Overcoming this for non-standard views (e.g., Dutch angles) requires explicit textual prompting (Figure 6 ). (a) Default prompt ignores tilted perspective. (b) Explicit "Top view" prompt forces adherence. Figure 6 : Overcoming model priors for non-standard perspectives requires textual guidance. 3.2 Multi-ControlNet Composition We investigated applying both ControlNets simultaneously. Using guidance strengths optimal for a single module (e.g., 0.8) leads to image degradation. While