ColorGPT: Leveraging Large Language Models for Multimodal Color Recommendation
Ding Xia, Naoto Inoue, Qianru Qiu, Kotaro Kikuchi · 2025
Read ColorGPT: Leveraging Large Language Models for Multimodal Color Recommendation by Ding Xia, Naoto Inoue, Qianru Qiu, Kotaro Kikuchi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08987.txt.
Introduction
Ding Xia and three co-authors ask whether large language models can pick colors for graphic design, in a 2025 arXiv paper. Color choice lifts a design's appeal, clarity, and ease of use. When one or more colors are missing, a good pick is hard, and old tools were weak at it.
Their pipeline, ColorGPT, tests ways to write colors as text and ways to prompt the model. Its main task is to complete a palette from a few given colors plus context. The same setup can also build a whole palette from a text brief. In tests the model beat older methods on pick accuracy and on how well the palette spread matched. It also gave richer, closer palettes for the full build task.
Catalog record
| CREATOR | Ding Xia, Naoto Inoue, Qianru Qiu, Kotaro Kikuchi |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-08987
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ColorGPT: Leveraging Large Language Models for Multimodal Color Recommendation Ding Xia, Naoto Inoue, Qianru Qiu, Kotaro Kikuchi Abstract Colors play a crucial role in the design of vector graphic documents by enhancing visual appeal, facilitating communication, improving usability, and ensuring accessibility. In this context, color recommendation involves suggesting appropriate colors to complete or refine a design when one or more colors are missing or require alteration. Traditional methods often struggled with these challenges due to the complex nature of color design and the limited data availability. In this study, we explored the use of pretrained Large Language Models (LLMs) and their commonsense reasoning capabilities for color recommendation, raising the question: Can pretrained LLMs serve as superior designers for color recommendation tasks? To investigate this, we developed a robust, rigorously validated pipeline, ColorGPT , that was built by systematically testing multiple color representations and applying effective prompt engineering techniques. Our approach primarily targeted color palette completion by recommending colors based on a set of given colors and accompanying context. Moreover, our method can be extended to full palette generation, producing an entire color palette corresponding to a provided textual description. Experimental results demonstrated that our LLM-based pipeline outperformed existing methods in terms of color suggestion accuracy and the distribution of colors in the color palette completion task. For the full palette generation task, our approach also yielded improvements in color diversity and similarity compared to current techniques. 1 Introduction Color is a fundamental element in visual media, playing a pivotal role in graphic design by evoking specific emotions in viewers and significantly influencing their comprehension of the media [ 20 , 39 , 17 ] . For the color recommendation task, once the overall structure of a design is fixed, designers typically need to adjust colors to ensure harmony and alignment with the semantic concepts of the design. This complex process requires a deep understanding of color theories and heuristics, making it difficult for novice designers. Various approaches to color modeling have been investigated to support such designers. These approaches can be classified into heuristic-based [ 27 ] and machine-learning-based methods [ 31 , 24 , 35 , 7 , 26 , 33 , 34 ] . While heuristic methods often failed to capture complex priors, machine-learning methods typically relied on labor-intensive data collection processes, limiting their scalability. One of the most remarkable and promising trends in tackling data inefficiency was to utilize priors embedded in the pretrained LLMs, as investigated in knowledge graph [ 8 ] and layout domains [ 13 ] . We can also see similar trends in color research, with some studies obtaining the semantic description of a single color without expensive manual annotations [ 5 , 19 , 25 , 30 ] . Despite significant advances in LLMs, their application in color recommendation tasks remained underexplored. This raises a critical research question: Can pretrained LLMs serve as superior designers for color recommendation tasks? To answer this question, we need two main steps: First, we investigated effective color representations and prompt structures. Based on these insights, we developed a robust, rigorously validated pipeline tailored for the task. Second, we assessed the superiority and effectiveness of our approach across various types of LLMs. We introduced ColorGPT , a robust and thoroughly validated pipeline that leverages the commonsense reasoning capabilities of LLMs to tackle the complexities of color design in graphic documents. Our approach was evaluated on two primary tasks: color palette completion and full palette generation. Full palette generation entails creating a harmonious set of colors based solely on textual descriptions, while color palette completion involves suggesting appropriate colors for a vector graphics document by considering both existing colors and the accompanying textual context. We compared our approach with existing methods, and our results demonstrated that ColorGPT can effectively harness LLMs’ commonsense reasoning to address complex, real-world color design challenges. Additionally, our analysis evaluated the impact of different color representations and prompts. Our findings indicated that the choice of color representation significantly influences performance and that prompts with similar in-context exemplars yield better results. Our contributions are summarized as follows: • We presented ColorGPT , a robust and thoroughly validated pipeline that leveraged the commonsense knowledge of large language models for color design tasks involving multi-modal data. • We conducted extensive experiments on two color recommendation tasks, including color palette completion and full palette generation, confirming the superior performance and effectiveness of ColorGPT . • We evaluated various component variants, including different color representations and prompt structures, and analyzed their impact on the outcomes. 2 Related Works 2.1 Color Palette Completion Color palette completion aims to suggest appropriate colors that harmonize with existing palettes and align with human aesthetic preferences. Traditional methods [ 38 , 31 , 24 ] relied on hand-crafted features and regression models to predict new colors based on given palettes. Later, deep learning methods have become prominent in this field. Kim et al. [ 22 ] developed a model that predicts and recommends colors based on seed colors, leveraging a word-embedding-like approach. Yuan et al. [ 43 ] utilized Variational AutoEncoder with Arbitrary Conditioning (VAEAC) to dynamically recommend color palettes based on user-defined conditions. Kikuchi et al. [ 21 ] proposed a new dataset consisting of mobile e-commerce web pages and introduced several Transformer-based methods for generative colorization. Qiu et al. [ 33 , 35 ] proposed Transformer-based models for color recommendation in landing pages and vector graphic documents, and they further improved the performance of the color recommendation method by incorporating the information of text contents and image labels [ 34 ] . Their work utilizes multiple color palettes extracted from multiple design elements in the Crello dataset [ 41 ] and recommends colors corresponding to the existing ones in multiple palettes. Providing multiple palettes for a graphic document with various elements presents a challenging task that further explores the capabilities of LLMs in color recommendation and understanding. 2.2 Full Palette Generation Full palette generation is a special case of text-aware color recommendation, involving the creation of an entire color palette aligned with a given text. Some developers offer online encyclopedias that provide a comprehensive resource to identify and understand various color names [ 2 , 3 , 1 ] . These resources are valuable for designers and researchers working with color theory. Early studies on full palette generation often relied on predefined palettes based on specific themes, such as magazine cover design [ 15 , 42 ] . These methods were limited by their lack of flexibility and inability to adapt to varying contexts. Later, neural network models were introduced to predict colors for specific design elements [ 18 , 29 , 21 ] . Maheshwari et al. [ 26 ] and Bahng et al. [ 7 ] proposed conditional generative models for generating color palettes for image colorization using text inputs like ‘warm sunshine’ and ‘cute dog’. Their work introduced the Palette-And-Text (PAT) dataset, which offers a benchmark in this field. Recently, Qiu et al. [ 34 ] proposed a multimodal masked color model using CLIP-based text representations to connect semantic content with palette suggestions, improving the performance of color generation tasks. The generation of color palettes can generally reflect LLMs’