Synthia : Novel Concept Design with Affordance Composition
Hyeonjeong Ha, Xiaomeng Jin, Jeonghwan Kim, Jiateng Liu, Zhenhailong Wang, Khanh Duy Nguyen, Ansel Blume, Nanyun Peng, Kai-Wei Chang, Heng Ji · 2025
Read Synthia : Novel Concept Design with Affordance Composition by Hyeonjeong Ha, Xiaomeng Jin, Jeonghwan Kim, Jiateng Liu, Zhenhailong Wang, Khanh Duy Nguyen, Ansel Blume, Nanyun Peng, Kai-Wei Chang, Heng Ji (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-17793.txt.
Introduction
Hyeonjeong Ha and co-authors present, in a 2025 paper, a system that designs new objects by blending what their parts can do. Text-to-image models can vary a design's look, but the result often makes no functional sense. Their system, SYNTHIA, starts from a chart that breaks objects into parts and affordances, the uses each part affords. New designs must join several uses into one coherent form.
Training follows a staged plan. The model first learns simple part-use links, then harder blends that join uses from unlike objects. A contrastive push keeps results visually fresh, away from known designs. In human ratings, SYNTHIA beat top image models on both novelty and functional sense by wide margins. The work points toward AI design tools that respect how things must actually work.
Catalog record
| CREATOR | Hyeonjeong Ha, Xiaomeng Jin, Jeonghwan Kim, Jiateng Liu, Zhenhailong Wang, Khanh Duy Nguyen, Ansel Blume, Nanyun Peng, Kai-Wei Chang, Heng Ji |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-17793
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Synthia : Novel Concept Design with Affordance Composition Hyeonjeong Ha, Xiaomeng Jin, Jeonghwan Kim, Jiateng Liu, Zhenhailong Wang, Khanh Duy Nguyen, Ansel Blume, Nanyun Peng, Kai-Wei Chang, Heng Ji Abstract Text-to-image (T2I) models enable rapid concept design, making them widely used in AI-driven design. While recent studies focus on generating semantic and stylistic variations of given design concepts, functional coherence —the integration of multiple affordances into a single coherent concept—remains largely overlooked. In this paper, we introduce Synthia , a framework for generating visually novel and functionally coherent designs based on desired affordances. Our approach leverages a hierarchical concept ontology that decomposes concepts into parts and affordances, serving as a crucial building block for functionally coherent design. We also develop a curriculum learning scheme based on our ontology that contrastively fine-tunes T2I models to progressively learn affordance composition while maintaining visual novelty. To elaborate, we (i) gradually increase affordance distance, guiding models from basic concept-affordance association to complex affordance compositions that integrate parts of distinct affordances into a single, coherent form, and (ii) enforce visual novelty by employing contrastive objectives to push learned representations away from existing concepts. Experimental results show that Synthia outperforms state-of-the-art T2I models, demonstrating absolute gains of 25.1% and 14.7% for novelty and functional coherence in human evaluation, respectively. Code is available at https://github.com/HyeonjeongHa/SYNTHIA . 1 Introduction Imagine a coffee machine with wheels that brews your morning coffee and delivers it directly to your bed. This example illustrates a novel concept that is atypical and dissimilar to everyday objects we regularly encounter. Such novel concept design is defined as the synthesis of concepts that are visually unique and functionally coherent through the effective fusion of disparate concepts (e.g., coffee machine , trolley ) based on the user’s desired affordances (e.g., brew , deliver ). This process mirrors how humans blend ideas across cognitive domains to generate creative innovations Fauconnier and Turner 2002 ; Han et al. 2018 . (a) Generated concepts with similar affordances. (b) Generated concepts with distant affordances. Figure 1: Effect of Affordance Sampling on Novel Concept Generation. Our affordance sampling strategy selects disparate affordance pairs within our ontology, promoting novel functional coherence rather than redundant combinations. Baseline models tend to generate existing concepts for close affordances (Fig. 1(a) ) but struggle with distant pairs, often introducing multiple objects or omitting functions (Fig. 1(b) ). In contrast, our models consistently generate functionally coherent novel concepts, achieving higher novelty scores for distant affordance pairs. Existing studies on conceptual design using T2I models have enabled rapid ideation of novel visual concepts ( Cai et al. 2023 ; Ma et al. 2023 ; Wang et al. 2024 ; Lin et al. 2025 ) by identifying user challenges such as interpreting abstract concepts in language to help visualize a novel design concept ( Lin et al. 2025 ) , or using large language models (LLMs) to bootstrap initial ideation in texts ( Cai et al. 2023 ; Zhu and Luo 2023 ) . However, they often naively feed LLM-generated textual prompts into T2I models, relying on simple key phrases or semantic variations of concept description ( Cai et al. 2023 ; Wang et al. 2024 ) . While existing works show that T2I models can generate images that seem to correctly reflect complex human-formulated textual descriptions (e.g., “beautiful rendering of neon lights in futuristic cyberpunk city” ), they do not focus on whether a model can synthesize a novel concept when given a set of affordances as input, while ensuring these affordances are preserved. An important aspect lacking in existing approaches to concept synthesis is their focus on pixel-based control, overlooking the structural and functional roles embedded in design. Many real-world concepts are naturally “decomposable” into parts, where each part signals a specific functionality. To address this, we propose Synthia , a framework for Concept Synth es i s with A ffordance composition that generates functionally coherent and visually novel concepts given a set of desired affordances. Unlike prior works relying on complex descriptive text to generate stylistic variations or aesthetic features Richardson et al. 2024 ; Vinker et al. 2023 , Synthia leverages affordances–defined as “the functionality offered by an object or its parts”—as a structural guide for novel concept synthesis. By aligning textual descriptions with affordances as control signals, our models implicitly learn to “decompose and reassemble” functional parts, ensuring that, for instance, a hybrid of a coffee machine and a trolley not only appears novel but also retains its brewing and mobility functions, achieving functional coherence . To facilitate structured affordance composition, we construct a hierarchical concept ontology that decomposes visual concepts (e.g., Sofa ) into their constituent parts (e.g., leg , cushion ) and associated affordances (e.g., support , rest ). It provides a structured representation of concept-affordance associations, serving as the foundation for generating functionally meaningful designs. Inspired by the theory of combinational creativity in humans Han et al. 2018 , which suggests novel concepts emerge from disparate ideas, we propose an affordance sampling mechanism that strategically selects affordances associated with sufficiently different concepts using our novel similarity-based metric (§ 3.1 ). This ensures that generated designs integrate novel functionalities, avoiding trivial combinations, whereas random sampling yields similar affordances (e.g., cook , heat ) that result in redundant outputs (Fig. 1(a) ). We also introduce a new curriculum learning scheme that fine-tunes T2I models to progressively learn affordance composition while maintaining visual novelty. Our curriculum gradually increases the affordance distance, allowing models to first learn basic concept-affordance associations from close affordance pairs before tackling complex affordance compositions that integrate multiple affordances into a single, coherent form. To further ensure novelty, we employ contrastive objectives to push learned representations away from existing concepts in our ontology. This addresses a critical limitation of existing T2I models, which struggle to generate coherent multi-functional concepts (Fig. 1(b) ). Without structured affordance composition, models tend to default to familiar objects—e.g., when prompted with drive and vacuum affordances, Stable Diffusion models simply generate a car with missing vacuum functions (Fig. 1(b) ), rather than blending both affordances into an integrated design. Importantly, unlike existing AI-driven design frameworks that rely on detailed LLM-generated descriptions, Synthia enables direct affordance-based prompting, e.g., “ a new design that has functions of { desired affordances } . ”. Our model implicitly learns concept-affordance associations, producing novel, structured designs without redundant textual prompting. To evaluate our framework, we uniformly sample 500 unseen affordance pairs from our ontology and assess generated concepts using automatic and human evaluation. We design evaluation metrics (§ 4.2 ) that measure faithfulness, novelty, practicality, and coherence. Experiments show that Synthia significantly outperforms baselines, creating designs that are visually novel and functionally coherent, with consistently higher scores across all metrics. Our contributions are as follows: • We introduce a hierarchical concept ontology that encodes concept-affordance associations, serving as crucial building blocks for novel concept synthesis with functional coherence. • We propose an affordance sampling strategy that guides disparate affordance selection, avoiding redundant functionalities while ensuring coherent concept synthesis. • We develop a curriculum-based optimization for affordance composition that fine-tunes T2I models, enabling T2I models to fuse multiple affordances into a single coherent concept. 2 Related Work Text-to-Image Models. The advancement of text-to-image