Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization
Connor Dunlop, Matthew Zheng, Kavana Venkatesh, Pinar Yanardag · 2025
Read Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization by Connor Dunlop, Matthew Zheng, Kavana Venkatesh, Pinar Yanardag (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-05616.txt.
Introduction
Connor Dunlop, Matthew Zheng, Kavana Venkatesh, and Pinar Yanardag present a method for editing images to suit one person's taste. Text-to-image models are generic by design. They cannot learn that one user likes moody light while another wants bright color, so edits often miss the mark.
Their framework, Collaborative Direct Preference Optimization, places each user in a graph of people with like tastes. A small graph network learns user embeddings, and those embeddings steer the diffusion model's editing objective. The aim is edits that fit the person and stay coherent with their taste neighbors. In benchmarks and user studies, the method beat baselines at matching what each user preferred.
Catalog record
| CREATOR | Connor Dunlop, Matthew Zheng, Kavana Venkatesh, Pinar Yanardag |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-05616
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Personalized Image Editing in Text-to-Image Diffusion Models via Collaborative Direct Preference Optimization Connor Dunlop, Matthew Zheng, Kavana Venkatesh, Pinar Yanardag Abstract Text-to-image (T2I) diffusion models have made remarkable strides in generating and editing high-fidelity images from text. Yet, these models remain fundamentally generic, failing to adapt to the nuanced aesthetic preferences of individual users. In this work, we present the first framework for personalized image editing in diffusion models, introducing Collaborative Direct Preference Optimization (C-DPO), a novel method that aligns image edits with user-specific preferences while leveraging collaborative signals from like-minded individuals. Our approach encodes each user as a node in a dynamic preference graph and learns embeddings via a lightweight graph neural network, enabling information sharing across users with overlapping visual tastes. We enhance a diffusion model’s editing capabilities by integrating these personalized embeddings into a novel DPO objective, which jointly optimizes for individual alignment and neighborhood coherence. Comprehensive experiments, including user studies and quantitative benchmarks, demonstrate that our method consistently outperforms baselines in generating edits that are aligned with user preferences. Figure 1: Our framework performs personalized image editing aligned with user’s preference via a novel DPO objective that learns user preferences while leveraging collaborative signals from like-minded individuals. 1 Introduction Text-to-image (T2I) diffusion models have achieved remarkable success in visual content generation, enabling high-quality image synthesis from textual descriptions [ 32 , 31 , 18 ] . Building on this success, recent advancements have begun to repurpose diffusion models for image editing – allowing users to modify a given image via text prompts or other guidance [ 42 , 5 ] . Despite progress, achieving a desired edit often demands substantial manual effort. Existing editing approaches do not account for the individual user’s preferences. They treat image editing as a one-size-fits-all problem: the model does not adapt to a particular user’s style, and it assumes the same definition of a ‘good’ edit for everyone. In other domains like natural language processing, it is well recognized that different users have unique tastes and requirements, and models can be adapted accordingly [ 38 , 30 ] . Analogously, in image editing each user may exhibit distinct preferences - one user might favor images with brighter, more saturated colors, while another prefers muted tones and centered composition. Current text-to-image models and editors ignore these nuances, effectively optimizing for an average preference that may not align with any particular user. This lack of personalization means users must continuously correct or fine-tune outputs that do not suit their aesthetic, which limits the practicality of these AI editors in real creative workflows. In this work, we take the first step towards personalized image editing by proposing a framework that learns and adapts to an individual user’s editing preferences. Our method models how each user prefers their images to be modified, capturing aspects such as stylistic choices, color palettes, object attributes, and other consistently favored visual traits. By learning from user-specific preferences, our framework aligns image edits with the user’s intent, producing results that better reflect their personal aesthetic. Importantly, we also recognize that collaboration across users can amplify personalization. Users often fall into clusters with overlapping tastes; by sharing preference information among like-minded users, we can generalize better from limited data. Consider a home-decor hobbyist who frequently edits interior photos to give them a rustic living-room feel. They always add a stone fireplace and distressed-leather sofa, but they have never thought to include exposed wooden ceiling beams. Several like-minded users in our preference graph routinely pair the fireplace-and-sofa combo with rough-hewn ceiling beams to complete the rustic look. Even though the target user has never requested the beams explicitly, our collaborative mechanism learns this object-level association from neighboring users’ histories and can automatically insert the beams in future edits. The result is an image that remains true to the user’s rustic touches while enriching the scene with an additional detail they are likely to appreciate. In this paper, we propose a novel Direct Preference Optimization (DPO) framework for collaborative personalized editing. Specifically, we introduce a DPO-based learning approach that aligns a diffusion model’s image edits with individual user preferences while simultaneously leveraging feedback from a community of users. To enable this capability, we propose a novel synthetic dataset of user editing preferences; a rich corpus of editing examples with annotations linking them to specific users and their satisfaction. We represent the users in this dataset as nodes in a graph, which captures similarities in their editing behavior. This graph-based collaborative learning, together with the DPO objective, allows our model to learn a wide spectrum of personalized editing styles within a single unified model. In summary, our contributions are three-fold: • To the best of our knowledge, we introduce the first formulation of personalized text-to-image editing. We define a new problem setting where an editing model is tailored to an individual user’s preferences, moving beyond the one-size-fits-all paradigm in image editing. • We propose a novel training framework called, Collaborative Direct Preference Optimization, introducing a graph-structured regularization term into the DPO loss, explicitly modeling and leveraging collaborative relationships among user embeddings. This structured collaboration allows the model to capture nuanced preferences implicitly shared among like-minded users. • We curate a novel synthetic dataset comprising 144K editing preferences which includes annotated examples of edits grouped by user. This dataset provides a valuable benchmark for studying personalization in image editing and facilitates quantitative evaluation of models in this setting. • Our source code and dataset are publicly available at http://personalized-editing.github.io . 2 Related Work Text-to-Image Editing Text-to-image diffusion models [ 36 , 14 , 34 , 32 ] are frequently used for both image generation and editing because of their high-fidelity generation capabilities. Instruct Pix2Pix [ 5 ] enables flexibility across edit types with user prompts without requiring task-specific supervision, however, its performance can degrade for highly detailed or spatially complex edits. Blended Latent Diffusion [ 3 ] and SDEdit [ 23 ] provide more fine-grained control with diffusion models whereas ControlNet [ 42 ] conditions the model on additional structural inputs to maintain the spatial consistency while also allowing for localized edits. User Preference Optimization Reinforcement Learning from Human Feedback (RLHF) [ 26 ] has been used to optimize Large Language Models (LLMs) [ 6 , 29 , 37 ] and multimodal LLMs [ 2 , 22 , 8 , 1 ] to better align with human preferences. It involves training a reward model based on pairs of preferred and rejected data to guide the fine-tuning of the language model. A simpler alternative to RLHF is Direct Preference Optimization (DPO) [ 30 ] , which directly optimizes the model parameters on preference data rather than the need for a separate reward model or reinforcement learning loop. The applications of DPO have extended beyond LLMs to vision domains. Adaptations such as Diffusion-DPO [ 38 ] , have been proposed to similarly align diffusion models to generate images in accordance with human preferences. DPO has been used to further align models with individual users rather than an entire population [ 20 ] . Personalized Image Generation The expressive capabilities of T2I models have spurred research in user personalization. Recent methods have aimed to fit the preferences of