CompArt: Operationalizing Aesthetic Alignment in Text-to-Image Generation via Principles of Art

Zhe JIN, Tat-Seng CHUA · 2025

Read CompArt: Operationalizing Aesthetic Alignment in Text-to-Image Generation via Principles of Art by Zhe JIN, Tat-Seng CHUA (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12018.txt.

Introduction

Zhe Jin and Tat-Seng Chua tackle a gap in 2025 in how image generators handle the look of a picture, not just its content. Today's models can draw what a prompt names, yet users have weak control over how the parts of the image are arranged. Past work treats a good look as one vague trait, like high quality, which does not match what a user asks for.

The pair define the task as match to clear rules for how an image is put together. They borrow those rules from art classes, such as balance, rhythm, and emphasis. To test models, they built a data set of more than 80,000 art images with captions and notes on those rules. Their method gives users a way to state the look they want in plain terms. It also gives builders a firm test for that skill.

Catalog record

CREATORZhe JIN, Tat-Seng CHUA
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE54 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-12018

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CompArt: Operationalizing Aesthetic Alignment in Text-to-Image Generation via Principles of Art Zhe JIN, Tat-Seng CHUA Abstract . Text-to-Image (T2I) diffusion models have made rapid progress on semantic alignment (generating what is described in the prompt), yet users still lack reliable control over aesthetic composition (how visual elements are put together). Prior work often treats aesthetics as a single, preference-driven notion (e.g., “high quality”, “detailed”, “breathtaking”), which does not map cleanly to compositional intent. We propose Aesthetic Alignment : aligning generated images to explicit, user-specified compositional constraints . We operationalize these constraints using the Principles of Art (PoA)—e.g., Balance, Rhythm, and Emphasis—commonly used in art education to describe composition. To support this task, we introduce CompArt , a dataset of 80,032 WikiArt images augmented with captions and PoA analyses produced by a multimodal LLM under structured prompting. We further propose ArtDapter , a lightweight and disentangled adapter that enables steering a pretrained T2I model along 10 PoA dimensions while retaining the base model’s semantic capability. Experiments on CompArt show improved adherence to PoA controls over strong baselines under a dual evaluation protocol. Conference: International Conference on Multimedia Retrieval; June 16–19, 2026; Amsterdam, Netherlands International Conference on Multimedia Retrieval (ICMR ’26), June 16–19, 2026, Amsterdam, Netherlands DOI: 10.1145/3805622.3810794 ISBN: 979-8-4007-2617-0/2026/06 CCS: Applied computing Fine arts CCS: Computing methodologies Neural networks CCS: Computing methodologies Supervised learning CCS: Computing methodologies Computer vision CCS: Computing methodologies Natural language processing Context : A snowy forest scene with bare trees surrounding a dark, frozen pond. Art-Style : Post-Impressionism. Symmetric balance is evident in the composition, with the dark pond centrally placed and flanked by similar arrangements of snow-covered trees on both sides. Context : A coiled snake with a detailed pattern on its body, against a plain background with handwritten text. Art-Style : Symbolism. Harmony is achieved through the consistent color palette of browns, blacks, and yellows, which unifies the snake with the background. Context : A vibrant scene of houses surrounded by trees with a colorful sky in the background. Art-Style : Expressionism. Variety is present in the diverse range of colors, shapes, and textures used throughout the composition. Context : A lush countryside scene with trees, a small pond, a house, and cows grazing. Art-Style : Impressionism. Unity is achieved by the consistent theme of a rural landscape, with all elements fitting together seamlessly. Context : Landscape with rolling hills, a winding river, and ruins of a castle under a colorful sky. Art-Style : Impressionism. Contrast is created by the dark shadows of the hills and the bright colors of the sky. Context : A landscape with blooming trees and a grassy foreground under a blue sky. Art-Style : Impressionism. Emphasis is placed on the blooming trees, which are the focal point due to their central placement. Context : Mountainous landscape with layered hills and silhouettes of structures under a cloudy sky. Art-Style : Symbolism. Proportion is used to create a sense of depth, with larger structures in the foreground and smaller mountains. Context : A pastoral scene with a woman, cows grazing, and a winding path through a green landscape. Art-Style : Impressionism. Movement is suggested by the winding path leading the viewer’s eye through the landscape. Context : A stormy seascape with crashing waves and rocks on the shore under a cloudy sky. Art-Style : Realism. Rhythm is present in the repetitive patterns of the waves and the clouds, creating continuity. Context : A lush garden with abundant white flowers and green foliage, with water in the background. Art-Style : Impressionism. Pattern is present in the consistent arrangement of the flowers and leaves, adding a sense of order. Figure 1 . Samples generated with our ArtDapter model, showcasing its ability to adhere to the respective context, art-style and compositional controls (e.g., Emphasis, Variety) across the 10 Principles of Art (PoA). † † cc-license: by-nc-nd Table 1 . The Vocabulary of Composition: We operationalize aesthetics using these 10 Principles of Art (PoA). Each principle serves as a distinct control dimension in our framework. Principle Definition Visual effect & function Balance Distribution of visual weight (symmetric, asymmetric, or radial). Brings equilibrium and stability; prevents the image from appearing “heavy” on one side. Harmony Cohesiveness of elements appearing coordinated. Creates a sense of order and organization; prevent haphazardness. Variety Diversity and complexity of visual elements. Adds interest, excitement, and dynamism by inviting visual exploration of different areas; prevents monotony. Unity Coherence or oneness of elements in serving the context, theme, or message. Ensures elements belong together semantically and visually; brings meaning and completeness to the work. Contrast Use of opposing visual elements (e.g. light/dark, big/small) to highlight differences. Adds drama and focus by creating strong distinctions that direct attention to key areas. Emphasis Creation of focal points that draw attention to important parts of the artwork. Guides the viewer to the main subject(s), strengthening communication; prevents a directionless viewing experience. Proportion Relationship between elements based on their relative sizes. Establishes perceived depth, realism and importance of elements through scale; cues relationships between objects. Movement Suggestion of motion or directional and sequential flow. Guides the gaze sequentially through the visual narrative; adds dynamism, narrative flow, and sometimes tension. Rhythm Visual tempo created through repetition, progression, or alternation of elements. Creates continuity and flow (calm or energetic); strings distinct parts into a connected sequence. Pattern Consistent, organized repetition of elements, often in decorative arrangements. Adds texture and structure; creates predictability and decorative appeal. 1. Introduction Diffusion Models (DMs) have driven remarkable advances in text-to-image (T2I) generation, producing high-fidelity outputs ( Ho et al., 2020 ; Podell et al., 2024 ) . While the community has achieved significant success in semantic alignment , which ensures the model generates what is described in the prompt (e.g., “a dog”), the challenge of aesthetic alignment , which controls how the image is composed, remains largely unsolved. In this work, we turn compositional art theory into a practical control interface for T2I diffusion: we (i) define PoA-based constraints, (ii) build a large-scale PoA-annotated dataset, and (iii) learn a lightweight adapter that maps those constraints into diffusion guidance. Existing T2I approaches on improving the visual appeal of outputs typically sidestep the complexity of aesthetics, treating it as a universal property inferred from large-scale user preferences. Methods often rely on “prompt engineering” with vague keywords (e.g., “masterpiece”, “trending on artstation”) or fine-tuning on curated datasets like LAION-Aesthetics ( Murray et al., 2012 ; Datta et al., 2006 ) . While effective for improving general visual appeal, these methods suffer from a fundamental limitation: they conflate “aesthetics” with a specific, popular visual style, ignoring the complexities of artistic intent. Addressing visual literacy. Current T2I interfaces provide strong control over entities and sometimes relations , but they offer no explicit handle for global compositional attributes. The core issue lies in a lack of visual literacy in current T2I control mechanisms. As a result, users must rely on vague aesthetic keywords or trial-and-error prompt edits that do not correspond to stable, interpretable controls. Efforts to decompose image aesthetics into interpretable dimensions are not new. Prior work ( Jin et al., 2024a ; Liu et al., 2026 ; Jin et al., 2024b ) on aesthetics assessment has proposed various

Read the complete text