DiscoverLLM : From Executing Intents to Discovering Them Tae Soo Kim, Yoonjoo Lee, Jaesang Yu, John Joon Young Chung, Juho Kim Abstract To handle ambiguous and open-ended requests, Large Language Models (LLMs) are increasingly trained to interact with users to surface intents they have not yet expressed (e.g., ask clarification questions). However, users are often ambiguous because they have not yet formed their intents: they must observe and explore outcomes to discover what they want. Simply asking "what kind of tone do you want?" fails when users themselves do not know. We introduce DiscoverLLM , a novel and generalizable framework that trains LLMs to help users form and discover their intents. Central to our approach is a novel user simulator that models cognitive state with a hierarchy of intents that progressively concretize as the model surfaces relevant options—where the degree of concretization serves as a reward signal that models can be trained to optimize. Resulting models learn to collaborate with users by adaptively diverging (i.e., explore options) when intents are unclear, and converging (i.e., refine and implement) when intents concretize. Across proposed interactive benchmarks in creative writing, technical writing, and SVG drawing, DiscoverLLM achieves over 10% higher task performance while reducing conversation length by up to 40%. In a user study with 75 human participants, DiscoverLLM improved conversation satisfaction and efficiency compared to baselines. Figure 1 : DiscoverLLM Framework: A simulated user with a latent intent hierarchy (1) interacts with a model. The user can only articulate discovered intents (2), and model responses (3) that successfully probe or satisfy undiscovered intents trigger state updates (4). The framework computes rewards based on discovery progress (5), which are used for fine-tuning of the model (6). 1 Introduction Large Language Models (LLMs) have emerged as effective conversational assistants, capable of following complex user requirements to generate fluent and high-quality outputs. However, this effectiveness relies on a critical assumption: users begin with fully formed intents. In reality, users often approach tasks with ill-defined intents , not knowing precisely what they want ( 50 ) —which is common in diverse open-ended or creative tasks like writing and design ( 45 ; 13 ; 11 ; 16 ) . Consider a user who asks an LLM to “write a personal essay about a hard lesson.” The model produces a full essay with a neutral formal tone. After reading a few lines, the user feels it is not right but cannot pinpoint why so they ask “maybe a unique tone?” Noting the ambiguity, the assistant tries to clarify: “What type of unique tone do you want?” The user is unsure so they hesitantly say "not sure, something different" to which the assistant generates a highly personal and emotional essay, which feels like “too much.” A revision tones down the intimacy and emotion, making the user feel that it is “too distant” now. Only after several more turns does the user realize what they want: a narrative that is emotionally restrained but still personally revealing. Current approaches to improving LLM interaction do not address this challenge of ill-defined, unformed intents . Benchmarks mainly assess single-turn performance ( 32 ) and fine-tuning techniques (e.g., RLHF ( 41 ) ) reward single-turn full outputs—assuming all crucial details are present in the users’ initial requests. Recent work on improving models’ multi-turn capabilities, whether through prompting ( 34 ; 38 ) or fine-tuning ( 64 ; 47 ; 57 ) , assumes users possess well-defined intents that they have simply not articulated, which the model can surface by asking clarifying questions. But when intents are not yet discovered or formed, there is nothing to surface—like how the example user could not answer what tone they wanted. Research on the cognitive process of tackling open-ended, ill-defined problems offers an alternative: people’s understanding of a problem and its solutions co-evolve —people discover what they need ( problem space ) by exploring possible outcomes ( solution space ) ( 8 ; 11 ; 45 ) . By creating and examining options, even incomplete ones, people gradually discover what they need or want—like the example user discovering their desired tone by seeing opposite options. This process of discovering intent through exploration demands a fundamentally different role for LLMs: instead of simply eliciting and executing users’ intents, models should help users explore, discover, and form their intents. Inspired by this, we formalize intent discovery as a distinct problem from intent elicitation and propose a novel user simulator that operationalizes this formalization. Unlike prior simulators where users have hidden but fully formed goals ( 57 ; 52 ) , ours represent goals as a hierarchy of intents—from broad to specific—that progressively concretize when a model successfully satisfies or directly asks about them. The simulator can reward model responses based on how effectively they help uncover more specific intents, enabling fine-tuning by synthesizing high-quality conversations or Reinforcement Learning (RL) ( 46 ; 48 ) . With this, we propose a training framework, DiscoverLLM , that teaches language models to help users discover and refine their intents. Returning to our example: a simulated user possesses the hierarchy of unique tone \rightarrow personal but composed \rightarrow personally revealing but emotionally restrained . When only the top-level intent is formed, asking “what type of unique tone?” fails, but generating an overly personal draft leads the user to discover “personal tone.” Using our framework, we propose three challenging multi-turn tasks across diverse open-ended, creation domains: creative writing, technical writing, and SVG drawing. On these tasks, we apply DiscoverLLM to fine-tune Llama-3.1-8B-Instruct and Qwen3-8B. Across models and tasks, our approach improved intent discovery by around 10%, increased interactivity scores by 83% (as rated by LLM judges), and reduced conversation length by 32%—all compared to the best baselines. We also verified that these gains generalized to unseen domains (e.g., travel planning, web development). In a user study with 75 crowdworkers, who completed writing tasks with anonymized models, DiscoverLLM achieved higher user satisfaction while reducing task completion time—with participants noting how the model appeared to “anticipate” their latent intents. 2 Problem Formulation We consider multi-turn conversations where a user collaborates with an AI assistant to perform an open-ended task. Instead of assuming that users start with fully-formed intents, we formalize a setting where the user’s intents are discovered through the conversation . 2.1 Intent Discovery Through Interaction At turn t of a conversation \mathcal{C}=(u_{1},r_{1},u_{2},r_{2},\ldots,u_{T},r_{T}) , the user sends message u_{t} and receives assistant response r_{t} . Let I_{t} denote the user’s intent state at turn t : a collection of requirements and constraints that must be satisfied in this task. We model intent formation as progressive refinement: I_{0}\subseteq I_{1}\subseteq\cdots\subseteq I_{T} (1) In a successful conversation, the initial state I_{0} contains a few abstract requirements (e.g., “a poem” , “includes an animal” ) while the final state I_{T} has multiple concrete ones (e.g., “a haiku” , “includes a tabby cat” )—each transition adding or concretizing requirements. Response-Driven Progressive Discovery. Drawing on cognition research that describes how people develop understanding of their goals by exploring or creating outcomes ( 45 ; 16 ) , we formalize intent discovery as dependent on the assistant’s responses. Let \mathcal{R}(I_{t}) denote the set of potential refinements : directions to concretize or expand I_{t} that the user has not yet formed but could adopt if surfaced in the interaction. For instance, if I_{t} includes “includes a pet,” then \mathcal{R}(I_{t}) might contain “includes a cat” or “pet in an indoor setting.” Let \phi:(r_{t},I^{\prime})\to\{0,1\} indicate whether the assistant’s response r_{t} successfully engages with refinement I^{\prime}\in\mathcal{R}(I_{t}) : directly asks about that aspect or reveals it through a concrete artifact that satisfies it. The intent state updates as: I_{t+1}=I_{t}\cup\{I^{\prime}\in\mathcal{R}(I_{t}):\phi(r_{t},I^{\prime})=1\} (2) When a response engages refinements in \mathcal{R}(I_{t}) , the user may recognize them as matching latent preferences. For example, while initially only knowing they want a poem featuring “an animal” , a response featuring “a dog” leads the user to realize they want “a pet” —adding this specification to I_{t+1} . A response that fails to engage any refinement leaves the intent state unchanged. User Expressiveness Constraint. In u_{t} , users can only articulate intents in I_{t} and cannot request undiscovered refinements in \mathcal{R}(I_{t}) . A user who knows “pet” but not the refinement “cat” can only vaguely request “maybe another type of pet?” This asymmetry distinguishes our setting from standard intent elicitation: clarifying question like “what type of pet?” fails to progress the conversation as the user has not realized the refinement and cannot yet answer. 2.2 Objectives The assistant must balance two objectives: (1) Intent Discovery: Help the user discover concrete, specific intents by exploring and surfacing possibilities that engage with latent, undiscovered intents in \mathcal{R}(I_{t}) . (2) Intent Satisfaction: Produce outcomes that satisfy the user’s discovered intents I_{T} , tracking and satisfying increasingly detailed requirements as they emerge. Effective assistance requires balancing divergence (i.e, explore options to probe around abstract intents) and convergence (i.e., refine outputs to integrate and satisfy concrete intents) ( 19 ) . 2.3 Underlying Assumptions Our formalization assumes that users possess latent intents they would recognize if surfaced—a direction they are striving toward, but not fully formed. In practice, users may have no direction and fully construct intents through interaction, meaning that the AI assistant may exert greater influence on what intents are formed. Our formalization does emulate this: as users cannot articulate intents unless surfaced by the assistant, the user appears to construct new intents. However, as we model discovery rather than construction, the assistant can only surface possibilities that users are already disposed towards rather than direct them toward new directions. We also assume monotonic refinement: once discovered, intents remain discovered, excluding cases where users abandon intents or backtrack to explore alternative refinements. Despite these assumptions, our formalization captures common observed patterns of intent concretization ( 16 ; 45 ) —relaxing these assumptions remains future work. 3 DiscoverLLM : General Training Framework for Intent Discovery We propose DiscoverLLM , a training framework that operationalizes the intent discovery formulation. The central challenge is obtaining a training signal: in natural conversations, we cannot observe a user’s intent state I_{t} or refinement space \mathcal{R}(I_{t}) . Our solution is a user simulator with an explicit latent intent structure, enabling reward computation for training. Details and prompts in Appendix A . Figure 2 : (A) Intent tree construction : (1) Given an artifact type and its content, we generate a specific intents list, (2) iteratively abstract them across levels, and (3) organize all resulting intents into a tree hierarchy. (B) Simulation Example: The user simulator begins ( t=0 ) with only a few abstract intents discovered and provides an initial request based on these. At t=1 , the model’s response fails to probe or satisfy intents in the refinement space so no state updates occur and the user remains vague. At t=2 , the model provides various options where one probes at an undiscovered intent, updating its state and enabling the user to articulate it. 3.1 Operationalizing Intents as a Hierarchy We represent intents as a tree hierarchy \mathcal{H}=(V,E) , where each node v\in V is a requirement, and edges connect parents to more specific children. This hierarchical structure is grounded in cognitive process theory that posits that people “create a hierarchical network of goals” ( 16 ) . The hierarchy serves as ground truth for our simulator: it defines what the user would recognize as matching their preferences if surfaced by the assistant, enabling reward computation for intent discovery. A simulated user may have multiple trees \{\mathcal{H}_{1},\ldots,\mathcal{H}_{K}\} for different intent dimensions (e.g., subject, tone, structure). For simplicity, we use V to denote all nodes across these trees. The user’s intent state I_{t}\subseteq V is the set of discovered nodes by turn t , and the refinement space consists of undiscovered children of discovered nodes: \mathcal{R}(I_{t})=\{v\in V:\text{parent}(v)\in I_{t},\,v\notin I_{t}\} (3) This captures progressive discovery (Sec. 2.1 ): a node can only be discovered once its parent is discovered. Branches are independent, so discovering intents along one path does not require resolving others. In Figure 2 , the user begins with “includes an animal” from which “includes a pet” can be discovered, and discovering “pet” enables discovery of the pet type ( “cat” ) and setting ( “indoors” ). Discovery States. Each node maintains a discovery state: undiscovered , emerging , or discovered . Emerging nodes represent intents the user has begun to recognize but not fully realized—they may only be vaguely referenced (e.g., “maybe a smaller animal?” for an emerging “pet” intent). Only discovered nodes can be directly articulated, satisfying the expressiveness constraint (Sec. 2.1 ). 3.2 Response Evaluation & State Transitions ( Figure 2 B) At each turn, the simulator evaluates how the response r_{t} engages with nodes in \mathcal{R}(I_{t}) . If r_{t} is an artifact (e.g., draft, samples), whether it satisfies the intent. If it is a dialogue act (e.g., question, suggestion), whether it probes the intent. State Transitions. We model transitions to capture how both direct and indirect exposure can trigger intent discovery. Direct engagement: If r_{t} explicitly asks about or satisfies the intent, the node becomes discovered . Tangential engagement: If r_{t} provides related but non-matching options, each contributes toward a cumulative score, where exceeding a randomly sampled threshold advances the node one state ( undiscovered \to emerging \to discovered ). This reflects how people discover preferences not only through exact matches, but by narrowing possibilities and seeing what they do not want ( 45 ; 49 ) . 3.3 Constructing Intent Trees ( Figure 2 A) We construct intent trees automatically from existing artifacts (e.g., stories, code, etc.), enabling generalization across tasks and domains. (1) Initial Intent Synthesis: Given artifact a , we prompt an LLM to list all requirements satisfied by the artifact, which become the most concrete intents of the hierarchy. (2) Gradual Abstraction: An LLM iteratively abstracts or generalizes the intent list through multiple levels by abstracting intents or removing them at each level. (3) Hierarchy Organization: Given the intents at multiple abstraction levels, an LLM organizes them into a tree by identifying abstract intents that subsume concrete ones. 3.4 Reward Function We design the per-turn reward: \text{R}(r_{t})=\text{R}_{\text{d}}(r_{t})+\text{R}_{\text{e}}(r_{t}) . Discovery Progress ( \text{R}_{\text{d}} ). This reward measures progress toward complete intent specification: \text{R}_{\text{d}}(r_{t})=|I_{t+1}|-|I_{t}| . This captures both discovery and satisfaction: satisfying an intent also discovers it, and an intent must be discovered to be expressed and subsequently satisfied. Efficiency Penalty ( \text{R}_{\text{e}} ). To encourage concise interactions, we penalize excessive response length: \text{R}_{\text{e}}(r_{t})=-\min(\lambda\cdot\max(0,\text{tokens}(r_{t})-\tau),1) , where \tau is a token threshold and \lambda controls penalty severity. Design Rationale. Our reward design prioritizes discovery first, then efficiency: each discovered intent contributes +1 while efficiency penalty is capped at 1. We avoided normalizing \text{R}_{\text{d}} by remaining intents, as this yielded inconsistent signals: length penalties negated discovery gains, and later turns became unstable as the denominator shrunk. 3.5 Training With our user simulator and the reward function, our framework supports multiple training paradigms. We can simulate interactions between the user simulator and assistant models, sampling multiple responses per turn and selecting the top-ranked ones via the reward function. This yields a high-quality dataset of synthetic conversations for Supervised Fine-Tuning (SFT) . This can also yield pairwise comparisons at each conversation turn, which can be used for Offline Reinforcement Learning (RL) , like DPO ( 42 ) . Finally, models can be optimized directly on the reward signal through Online RL methods, like PPO ( 46 ) or GRPO ( 48 ) . 4 Experimental Setup We apply the DiscoverLLM framework to create multi-turn datasets for both fine-tuning and evaluation across three diverse domains: creative writing, technical writing, and visual design. Details in Appendix B.3 . 4.1 Tasks and Datasets We focus on open-ended tasks that require substantial user-assistant collaboration and iteration. Writing—among the most common uses of AI assistants ( 53 ; 61 ) —involves iterative composition and revision ( 16 ) . Visual creation tasks like SVG drawing share similar properties: users often concretize preferences through exploration ( 33 ) . We construct datasets from existing artifact sources: Creative Writing ( 14 ) 1 1 1 https://huggingface.co/datasets/euclaise/WritingPrompts_preferences (i.e., user posts in the r/WritingPrompts subreddit), Technical Writing ( 44 ) (i.e., existing journalistic articles), and SVG Drawing ( 58 ) . From each, we sample 500 artifacts for training and 100 for evaluation. We use Claude Sonnet 4.5 to construct intent trees (Sec. 3.3 ) and Gemini 3 Flash for the user simulator (i.e., state transition and response generation in Sec. 3.2 ). Each conversation runs 5 turns. In evaluation, we repeat each conversation 3 times and average scores across trials. Details in Appendix B.1 . 4.2 Evaluation Metrics We simulate conversations between each user simulator and evaluated model to assess performance on four metrics. Intent Discovery Score. We measure the proportion of intents discovered by the end of each conversation. To account for variation in difficulty across artifacts, we normalize scores using per-instance bounds: exclude intents discovered by all evaluated models (i.e., trivially easy) and those discovered by none (i.e., unreachable within the conversation length). This focuses evaluation on the discriminative range where model behavior meaningfully differs: \textit{Discovery}=\frac{1}{N}\sum_{i=1}^{N}\frac{|I_{T}^{(i)}|-|I_{\text{all}}^{(i)}|}{|I_{\text{any}}^{(i)}|-|I_{\text{all}}^{(i)}|} (4) where I_{T}^{(i)} is the set of discovered intents at conversation end for artifact i , I_{\text{all}}^{(i)} is the set discovered by all models, and I_{\text{any}}^{(i)} is the set discovered by at least one model. Intent Satisfaction Score. To assess how intent discovery leads to better intent satisfaction, we append a message at the end of each conversation prompting the model to generate a final complete artifact. We then use an LLM-as-a-judge ( 65 ) (GPT-5.1) to assess whether the artifact satisfies each leaf node in the intent trees (e.g., most specific intents from the original artifact). The satisfaction score is the proportion of satisfied leaf nodes, but excluding those that were not satisfied by any evaluated model. Interactivity Score. Following 57 , we use an LLM judge (GPT-5.1) to rate how well the assistant collaborates and engages with the user in each conversation—scores rescaled to 0-1. Average Token Count. We compute the mean number of tokens generated by each model across all turns. 4.3 Training DiscoverLLM s We apply the DiscoverLLM framework to two base models: Llama-3.1-8B-Instruct ( 20 ) and Qwen3-8B ( 59 ) , using LoRA fine-tuning ( 24 ) . We train four model variants with progressively more optimization: (1) SFT : Supervised fine-tuning on synthesized conversation histories. (2) DPO : Starting from the base model, we apply Offline DPO on preference pairs from the same synthesized conversations. (3) SFT+DPO : We apply Offline DPO but starting with the SFT model. (4) SFT+DPO+GRPO : For the Qwen3-8B-based models, we additionally apply Online GRPO. Details in B.2 . 4.4 Baselines We compare DiscoverLLM models against three baselines: (1) Llama-3.1-8B-Instruct and Qwen3-8B ( Base ), (2) the base models with the same system prompt (Appendix E.10 ) that instructs them to support the user’s intent discovery ( Prompted Base ), and (3) CollabLLM ( 57 ) , a fine-tuning of Llama-3.1-8B-Instruct trained to collaborate with users through follow-ups and clarifications. Creative Writing Technical Writing SVG Drawing Discover \uparrow Satisfy \uparrow ITR \uparrow #Tok (k) \downarrow Discover \uparrow Satisfy \uparrow ITR \uparrow #Tok (k) \downarrow Discover \uparrow Satisfy \uparrow ITR \uparrow #Tok (k) \downarrow Llama-3.1-8B-Instruct Base 38.2 30.0 20.1 3.09 49.1 36.0 21.2 3.32 45.6 32.5 21.6 3.59 Prompted Base 37.7 26.4 26.0 2.97 43.6 33.5 24.2 3.05 40.0 30.9 25.1 3.18 CollabLLM 37.3 28.0 32.6 2.93 45.8 33.7 24.9 3.13 43.0 29.9 30.8 3.18 SFT 40.7 33.4 92.3 1.71 47.1 35.2 81.6 2.09 45.4 34.9 66.9 2.92 DPO 40.5 29.2 33.1 2.91 47.2 34.2 27.3 3.11 45.3 32.5 29.2 2.89 SFT+DPO 42.4 28.4 32.9 2.77 49.0 35.9 31.3 2.94 51.6 37.0 44.6 2.61 Rel. Improv. 11.0% 11.3% 183% 44.7% -0.0% -0.0% 227% 11.4% 13.2% 13.8% 117% 27.3% Qwen3-8B Base 35.2 30.4 36.2 3.41 40.7 33.7 35.3 3.39 47.0 32.0 54.4 3.96 Prompted Base 39.0 30.8 62.9 3.01 41.3 33.8 64.0 2.79 47.0 35.6 75.1 2.83 SFT 34.9 31.0 90.4 1.59 41.6 33.7 81.0 1.90 48.9 38.8 70.2 2.36 DPO 42.6 33.5 70.5 3.10 42.2 33.3 67.2 2.76 46.9 35.1 75.9 2.70 SFT+DPO 44.0 33.4 72.1 2.87 47.5 36.4 69.1 2.78 42.6 38.7 32.7 1.81 SFT+DPO+GRPO 45.2 33.7 83.1 2.05 48.2 35.5 55.0 2.63 48.7 38.6 46.3 2.46 Rel. Improv. 13.7% 9.4% 43.7% 31.9% 14.3% 7.7% 26.6% 31.9% 4.0% 9.0% 1.1% 40.4% Table 1 : Evaluation results across tasks and models: baselines, DiscoverLLM variants , and relative improvement of the best DiscoverLLM variant over the best baseline. For token length, we compare the DiscoverLLM with the lowest Discovery score that still exceeds all baselines against the highest-Discovery baseline, as a model can be highly efficient but completely ineffective. 5 Experimental Results We present the main results in Table 1 . Prompting showed inconsistent gains. For Qwen, prompting improved both performance and efficiency—e.g., Discovery score increased from 35.2% to 39.0% while conversation length dropped from 3.41k to 3.01k in Creative Writing. For Llama, prompting decreased performance across tasks—e.g., Discovery dropped from 45.6% to 40.0% in SVG Drawing. This stemmed from behavioral differences: prompted Llama mainly asked clarifying questions that simulated users could not answer, whereas prompted Qwen provided concrete options. The same pattern also explained CollabLLM ’s lower performance, as it was trained primarily to ask clarifying questions. While helpful in cases, prompting was limited as models failed to adapt to user states across turns—e.g., the model diverged initially but then fixated on revising a single option despite the user’s vague feedback—behavioral analysis in Section 5.1 . DiscoverLLM achieves superior intent discovery and satisfaction while being more interactive and efficient. Across models and tasks, DiscoverLLM generally improved intent discovery and satisfaction while reducing conversation length and enhancing interactivity. While the optimal training recipe varied by setting, SFT+DPO generally yielded the best results, with GRPO providing further gains. We observed that SFT alone tended to overfit toward divergent behaviors every turn, as reflected by high interactivity scores (e.g., 92.3 for Llama in Creative Writing) and behavioral pattern analysis (Sec. 5.1 ). However, effective intent discovery required balancing divergence with convergence, as refining an option can “unlock” new, more specific intents—DPO following SFT encouraged this adaptive behavior. Even when discovery scores are similar, DiscoverLLM achieves comparable performance more efficiently—e.g., SFT+DPO on Llama in Technical Writing matches performance with 11.4% fewer tokens. Supporting intent discovery is challenging. The overall low satisfaction scores indicate that fully satisfying intents within five turns is difficult when intents are not formed at the outset. This challenge was compounded by the limited diversity in LLM outputs ( 7 ; 62 ) : even when models diverged, their options lacked sufficient variety to help the users. 5.1 Behavioral Patterns We used an LLM to classify all turns from Qwen variants in Creative Writing as divergent (D) (e.g., multiple options, questions) or convergent (C) (e.g., single artifact). We then analyzed consecutive turn trigrams: 3-convergent (CCC), 3-divergent (DDD), 2-consecutive (e.g., CCD), and alternating (i.e., CDC, DCD). Figure 3 shows that the non-prompted base model and SFT show opposite tendencies: base almost exclusively used consecutive convergent turns, and SFT mostly used consecutive divergent. The prompted base model and the other DiscoverLLM variants showed more balanced patterns and higher use of alternating behaviors, with SFT+DPO+GRPO—which achieved the highest Discovery score—showing the most balanced use of patterns. Turn-by-turn analysis of Discovery score in Appendix B.4 . Figure 3 : Behavioral patterns across Qwen3-8B variants in Creative Writing. Turns are classified as divergent (D) or convergent (C) and analyzed as trigrams. Base is almost entirely convergent (91% CCC), SFT overfits toward divergence (43% DDD), and DiscoverLLM variants show more balanced patterns. Figure 4 : Case study on Creative Writing shows how Qwen3-8B (left) fails to move the conversation forward as it only continues to revise the same story, while DiscoverLLM (right) notices ambiguity and diverges, providing options that help the user discover new intents. 5.2 Case Study Figure 4 presents a specific conversation that illustrates DiscoverLLM ’s behavior on a story request with only a broad topic (i.e., corporate fugitive who is actually the CEO). Both models initially provide a draft to which the user vaguely expresses uncertainty about setting and technology. The base LLM only revises minor details, failing to progress the conversation. DiscoverLLM instead provides three distinct options, which helps the user realizes they want a “cyberpunk setting” with “stolen tech” —moving the conversation forward where the next turn integrates these ideas. 5.3 Task Generalization To test generalization, we evaluate the Llama variants on a diverse set of unseen tasks: travel plans, data visualization code, research abstracts, website components, and text-to-image prompts (details in 2 ). As seen in Table 2 , DiscoverLLM outperforms baselines with only a slight decrease in efficiency, demonstrating that its collaborative behaviors generalize across tasks. Llama-3.1-8B-Instruct Discover \uparrow #Tok (k) \downarrow Base 47.8 3.64 Prompted Base 48.8 3.19 CollabLLM 45.3 3.13 SFT 35.0 1.81 DPO 54.6 3.37 SFT+DPO 51.8 3.16 Rel. Improv. 11.9% 0.9% Table 2: Evaluation results of baselines and DiscoverLLM variants trained on Creative Writing on simulated experiments with a diverse set of unseen artifacts (N=50, 10 artifacts per task). 6 User Study Setup. We conducted a user study by recruiting 75 participants via Prolific 2 2 2 https://www.prolific.com/ , each assigned a random writing task (i.e., story, poem, or personal essay) and selected a topic from a given set. No additional priming was provided to reflect real-world scenarios where users begin without fully formed intents. Participants were randomly assigned to a condition: Base (Qwen3-8B), Prompted Base, or DiscoverLLM (SFT+DPO for Creative Writing). Following 57 , participants interacted for at least 8 turns, rating their interaction experience every three turns. At the end, participants rated their final satisfaction (1-10) with the interaction and the final writing artifact. Details in C . Figure 5 : User study results. Participants rated interaction satisfaction (a) and final writing satisfaction (b) higher with DiscoverLLM , while spending less time (c). Interaction ratings every three turns (d) show DiscoverLLM achieves higher satisfaction in early turns. Quantitative Results ( Figure 5 ). DiscoverLLM outperformed baselines in both interaction and writing satisfaction. For interaction satisfaction, 84% of DiscoverLLM participants gave a rating of 7 ("good") or higher, compared to 78% for Base and 72% for Prompted Base. DiscoverLLM also matched or exceeded these baselines in efficiency (i.e., time spent) and with notably lower variance—suggesting more consistent efficiency across participants. Notably, DiscoverLLM reached higher satisfaction early (turn 3) and maintained it throughout, while the baselines started lower and only improved after further back-and-forth. Qualitative Results. We also collected participants’ comments on each model’s strengths and weaknesses. Participants praised Base for reliably executing “commands” , but noted it required substantial back-and-forth as it “repeated things” , only made minor changes, and produced “cliched” and “generic” outputs. Prompted Base improved on exploration by guiding participants “one step at a time” with options that “triggered thoughts and ideas of my own.” However, participants felt it “wasted time” before starting the task and was poor at “following instructions.” DiscoverLLM combined both strengths: participants found it “creative and helpful” with “great suggestions,” , while it also executed and expanded on requests by “creating something amazing from the start” and “turning brief thoughts into fluent sections.” Interestingly, some noted that it seemed to understand their latent preferences: “anticipated what I was thinking” and “knew my needs.” However, some found its generated options to be “overwhelming” and “a bit standardised” —diversity was also an issue with Prompted Base. Unlike Base, participants also noted that the model sometimes made overly aggressive changes: “the AI removed all of an idea, when I asked it to adjust it.” Overall, these comments suggest that DiscoverLLM successfully balances exploration and execution, though the models must be further trained to enhance diversity in exploration. 7 Related Work LLMs for Multi-turn Interaction. LLMs are predominantly trained to optimize single-turn response quality ( 41 ) , resulting in models that fail to adequately collaborate with users ( 60 ; 30 ; 50 ) . Recent work has explored enhancing multi-turn interaction through prompting ( 27 ; 10 ; 38 ; 34 ; 9 ; 63 ) and training techniques ( 2 ; 4 ; 57 ; 52 ; 67 ; 47 ; 18 ) . For example, CollabLLM ( 57 ) simulates future turns to reward responses by their downstream impact, while PPP ( 52 ) rewards responses holistically, considering productivity, proactivity, and personalization. However, these approaches assume users possess fully formed but hidden intents that can be surfaced through clarification—they do not address cases where intents have not yet been formed. User Simulators for Training and Evaluating LLMs User simulators have become central to evaluating ( 35 ; 66 ; 29 ; 32 ; 3 ) and training ( 23 ; 31 ; 57 ; 52 ; 25 ) LLM-based assistants. Due to this, recent work has also explored how to improve the fidelity of the user simulators through fine-tuning ( 39 ; 3 ; 55 ) , prompting ( 36 ) , and by proposing novel evaluation methods ( 12 ; 66 ) . Our work extends this line by designing user simulators with internal cognitive states, focusing on the cognitive process through which users develop and discover intents through interaction. AI Systems for Open-Ended Problems Cognitive theories in design ( 11 ; 45 ; 8 ) and writing ( 16 ) show that in open-ended and ill-defined problems , people do not begin with clear intents but discover them through action. Considering the inherent difficulty of these tasks but also their prevalence, research in Human-Computer Interaction (HCI) has long proposed interactive systems to support these problems ( 17 ) . Recent efforts leverage the capabilities of AI models to further support these tasks by enhancing iteration ( 28 ; 43 ; 5 ) and exploration ( 1 ; 6 ; 51 ) . Our work proposes that, beyond integrating AI into such systems, the AI models should be trained for collaborative iteration and exploration to support open-ended, ill-defined problems. 8 Conclusion Current AI assistants are predominantly trained to execute and elicit instructions, yet users often approach tasks without fully formed goals. We introduce a formalization of intent discover and a novel user simulator design grounded in cognitive theories of how people discover intents through action. By designing a reward from this user simulator, we introduce DiscoverLLM , a training framework that teaches models to help users form their intents rather than simply elicit them. Across three domains, DiscoverLLM achieves substantial improvements in intent discovery and satisfaction while increasing interaction efficiency—gains that transfer to unseen domains and hold with real users. Our work encourages a shift in how the field conceptualizes human-centered AI: from models that execute and elicit instructions to proactive partners that explore and shape the problem and solutions with users. 9 Impact Statement Towards human-centric AI. This paper presents work aimed at making AI models more human-centric, which we believe yields positive societal impact. Current research increasingly focuses on AI models or agents that can autonomously complete tasks with minimal user input, making decisions without inviting user feedback. While increasing efficiency and productivity, this risks diminishing user agency, increasing deskilling ( 15 ) , and introduces friction when users aim to steer or revise AI outcomes ( 28 ; 60 ) . This problem is exacerbated by fixation effects, where users struggle to diverge from complete, high-fidelity outcomes from the AI model ( 13 ; 54 ) . These concerns extend beyond the open-ended, creation domains studied here: a computer-using agent may book an optimized itinerary but miss unconventional options the user would have appreciated; a deep research AI may retrieve comprehensive sources but overlook adjacent subfields that could have reshaped the user’s understanding. Our work addresses this by training models to collaboratively explore with users in small steps: inviting input throughout the process and helping users discover new requirements or constraints before producing complete outcomes. User simulation. Our framework relies on LLM-based user simulators to generate training data, provide reward signals, and evaluate models. We acknowledge that our user simulator design takes various assumptions that may not fully capture real human behavior: intent refines monotonically (excluding backtracking), and intent discovery depends solely on assistant responses (excluding external factors). Furthermore, an LLM may not simulate behaviors that reflect real human diversity. Despite these limitations, user simulators enable scalable training toward desirable behaviors, and our human study confirmed that simulated improvements transfer to real users. We view our approach not as a complete representation of human intent formation, but as a step forwards to models that can collaborate with users while considering this complex cognitive process. Human participants. Our user study involved human participants recruited through Prolific. Participants were compensated £3.00, and with an average duration of 15.14 minutes. This corresponds to approximately £12.00 per hour, which matches the minimum wage in the country that most participants were located in. To ensure participant privacy, we implemented two precautions: (1) participants were given a disclaimed that their data would be released in a public dataset and had to manually confirm that they read and agreed with this disclaimer, and (2) participants were instructed not to include personally identifiable information (PII) and could use fictional details in their conversations. Potential safety misalignment. We acknowledge a potential risk of our approach: models trained to collaboratively explore user intents may be more susceptible to engaging with malicious or harmful requests, particularly when such intents are vague or ambiguous. In these cases, as DiscoverLLM models are inclined toward exploration, they might unintentionally generate harmful content while attempting to help users clarify and form their intents. To assess this risk, we conducted a small-scale safety evaluation using established benchmarks (Appendix D ). Our results show no significant degradation in safety scores between base models and their DiscoverLLM variants, suggesting that our training approach does not compromise the models’ underlying safety alignment. However, we recognize that this evaluation is limited in scope, and more comprehensive safety analyses with diverse scenarios and attacks are needed before deployment of these models. Appendix A User Simulator Details Our user simulator consists of two main pipelines: (1) initial intent tree construction, which builds a hierarchical intent structure from any given artifact, and (2) the conversation simulation, which models user behavior during multi-turn interactions based on the intent trees. A.1 Intent Tree Construction We construct intent trees automatically from existing artifacts in four stages. Each stage uses Claude Sonnet 4.5. Figure 6 provides an example of an intent hierarchy synthesized for an artifact. Stage 1: Initial Intent Synthesis. Given an artifact a and its type (e.g., “short story,” “news article”, “svg drawing”), we prompt the LLM to extract a list of the specific requirements and constraints that the artifact satisfies (prompt in E.1 ). We instruct the model to focus on requirements that are not obvious or straightforward given only the artifact type—e.g., for a “haiku poem”, a requirement like “has three lines” should not be included. This requirement list represents the most developed and specific intents of the user. Stage 2: Intent Abstraction. Given the artifact type and the specific intent list, we instruct an LLM to iteratively abstract the intents to create progressively more general and abstract versions in a single completion (prompt in E.2 ). In a single completion, the LLM processes the artifact type and the intent list to produce multiple abstraction levels. At each level, the model is instructed to: (1) generalize an intent by removing details (e.g., “includes three different statistics” \to “includes statistics”) or abstracting them (e.g., “tabby cat” \to “cat” \to “pet”), (2) combine related intents into a single broader intent, or (3) remove intents. This produces a sequence of progressively more general and abstract list of intents. Stage 3: Hierarchy Organization. Given the intents at all abstraction levels, we instruct an LLM to organize them into tree structures by (1) grouping intents across abstraction levels that address similar aspects, (1) merging redundant intents, and (3) identifying parent-child relationships by determining which abstract intents subsume which concrete ones (prompt in E.3 ). The output is a set of intent trees, where each tree represents an independent dimension of the user’s intent (e.g., subject matter, stylistic choices, structural requirements). Stage 4: Initial Request Generation. Finally, given the artifact type and the root nodes of all intent trees, we instruct an LLM to generate the user’s initial request that begins the conversation (prompt in E.4 ). Here, the LLM is also instructed to select a subset of the root nodes that should be mentioned in the initial request, which are set as discovered from the start of the conversation. This models how users can begin tasks with only a broad sense of some their goals—e.g., a user might request “write me a poem about nature” while not having formed preferences regarding the specific tone, structure, or imagery. Figure 6 : Example intent hierarchy for a story artifact presents a structure consisting of three intent trees. A.2 Conversation Simulation During conversation simulation, the user simulator maintains the intent hierarchy and updates discovery states based on assistant responses. Three modules operate at each conversation turn, where we use Gemini 3 Flash for the LLM modules. A.2.1 Assistant Response Evaluation The evaluation module assesses how the assistant’s response engages with the user’s latent intents. To model realistic limitations in users’ cognitive load, we provide the evaluator with only a subset of the full intent hierarchy: specifically, the first root node (in tree order) that has any undiscovered descendant. This reflects how users typically only have the cognitive capacity to focus on one dimension of the task at a time (e.g., refine tone first and then move to structure). The evaluation proceeds in three steps within a single LLM call (prompt in E.10 ): (a) Response Classification. The evaluator first classifies the assistant’s response as either an artifact response (e.g., a single or multiple drafts or samples) or a dialogue act (e.g., a clarification question, suggestion, or explanation). This distinction determines the latter evaluation criteria: artifact responses are assessed on whether they satisfy intents, while dialogue acts are assessed on whether they probe or surface intents. (b) Recursive Intent Evaluation. Starting from the provided root node, the evaluator assesses each intent by cascading through the tree structure. For each intent node, the evaluator provides reasoning and a binary judgment of whether the response probed or satisfied. If the response probed or satisfied the intent, the evaluator then evaluates the node’s children. If not, the children are not evaluated and the evaluator moves to the node’s siblings. (c) Tangential Alternatives Identification. For intents that were not directly probed or satisfied, the evaluator identifies tangential alternatives : distinct but related intents that the response did satisfy or probe. For example, if the intent is “includes a cat” but the response features a dog, the evaluator notes “dog” as a tangential alternative. These alternatives are used in the state update mechanism to model how exploring adjacent options in the space of possibilities can help users to recognize their preferences. A.2.2 State Updater The updater module processes evaluation results to update the discovery state of each intent. Unlike other modules, we implement the updater through heuristics rather than an LLM. Direct Engagement Updates. If an intent was evaluated as probed or satisfied, it transitions immediately to the discovered state. Additionally, if the intent was satisfied (not just probed), we mark it as satisfied to track task completion. Tangential Engagement Updates. For intents that were not directly engaged but have tangential alternatives, we compute a cumulative exposure score: \text{score}_{v}=p\cdot|\text{tangential alternatives for }v| (5) where p is a fixed probability parameter representing the chance that each alternative helps the user recognize their preference. If this score exceeds the intent’s threshold \theta_{v} , the intent advances one discovery state: undiscovered \to emerging , or emerging \to discovered , and the threshold resets to its initial value. If the score does not exceed the threshold, we decrease the threshold by the score: \theta_{v}\leftarrow\theta_{v}-\text{score}_{v} . This models how repeated exposure to tangential alternatives accumulates over multiple turns, gradually bringing the user closer to recognizing their latent preference even when no single turn provides sufficient signal. During intent tree initialization, we set \theta_{v} for each node to a random value sampled uniformly from [0,1] . This reflects that some preferences are easier to recognize than others: a low threshold indicates an intent that the user can quickly discover through minimal exploration, while a high threshold represents a preference that requires more extensive exposure to alternatives before forming. A.2.3 User Response Generation The response generation module produces the simulated user’s reply based on their current intent state. The key constraint is that users can only articulate intents that are consistent with their discovery state, operationalizing the User Expressiveness Constraint from Section 2.1 (prompt in E.6 ). The LLM receives the conversation history and a filtered view of the intent hierarchy. The filtering logic determines what information the user can access and express: Always Included: Satisfied Intents. The user always knows which requirements have been satisfied to this stage of the conversation. We include the lowest descendants in each tree that are both discovered and satisfied , representing concrete achievements the user is aware of and can acknowledge (e.g., “I like that it’s about a cat”). Conditionally Included: Unsatisfied Intents. For unsatisfied intents, we include only those at the “frontier” of the user’s awareness, following a priority order: 1. If there are any intents that are discovered but unsatisfied, include only these. The user can explicitly state what needs to change and how (e.g., “I want it to be about a cat, not a dog”). 2. Else if there are any intents that are emerging and unsatisfied, include only these. The user can indicate what is wrong, but only vaguely hint at the direction of how that should be changed (e.g., “maybe a smaller, more domestic animal?”). 3. Else if there are any intents that are undiscovered and unsatisfied, include only these. The user can express dissatisfaction, but can only vaguely articulate what to change without providing any hints about how to change it (e.g., “the subject doesn’t feel quite right, but I’m not sure why”). Appendix B Experiment Details B.1 Dataset Generation Details We construct training datasets by simulating conversations between our user simulator, and two different assistant LLMs: (1) GPT-4o-Mini without prompting, and (2) GPT-4.1, which we prompt to encourage collaborative and exploratory interaction with the user (prompt in E.9 ). At each turn, each assistant generates a response, which are then evaluated against the intent hierarchy to compute the discovery reward (Sec. 3.2 ). The higher-ranked response is designated as Chosen and lower-ranked as Rejected . The Chosen response is then used to continue the conversation to which the user simulator responds based on its current discovery state, and the conversation continues until the maximum turn limit is reached. The collected data is then used for Supervised Fine-Tuning (SFT) with the full conversation trajectories, and training through Direct Preference Optimization (DPO), with the pairwise preference data at each turn. Table 3 summarizes the statistics of our generated training datasets. Regarding cost, a single synthetic five-turn conversation across our datasets required a total cost of $0.189: (1) Intent Tree Construction : $0.127 with claude-sonnet-4-5 through the Amazon Bedrock API, and (2) Assistant Response Evaluation & User Response Generation (5 turns) : $0.056 with gemini-3-flash-preview through the Google AI Studio API. Reward Scores Win Rates Dataset # Artifacts # Turns Chosen Rejected GPT-4.1 GPT-4o-Mini Creative Writing 500 2500 1.59 \pm 2.21 0.41 \pm 1.44 73.1% 24.0% Technical Writing 500 2476 1.91 \pm 2.50 0.89 \pm 1.79 54.0% 41.3% SVG Drawing 500 2497 1.85 \pm 2.66 0.55 \pm 1.54 63.6% 33.6% Table 3 : Statistics of synthesized training datasets. Chosen and Rejected indicate the mean and standard deviation of the reward scores for the chosen and rejected responses. Win rates indicate the proportion of turns in which each model’s response was chosen. B.2 Training Details Table 4 summarizes the LoRA configurations, fine-tuning hyperparameters, and our framework-specific hyperparameters. Fine-Tuning Hyperparameters Parameters SFT DPO GRPO LR 2e-5 5e-6 5e-6 Batch size 16 16 32 Epochs 3 3 1 Num Generations - - 4 LoRA Configurations Rank r 32 Alpha \alpha 64 Dropout 0.1 Bias None Framework Hyperparameters Parameters SFT & DPO GRPO Tangential prob. p 0.25 0.25 Token thresh. \tau 250 500 Penalty coeff. \lambda 1e-3 1e-3 Table 4 : Hyperparameters for fine-tuning, the LoRA configurations, and the framework. B.3 Evaluation Details During evaluation, the DiscoverLLM variants and the prompted models were given the system prompt in E.10 . Our experiments use three main evaluation metrics: Intent Discovery , Intent Satisfaction , and Interactivity . We provide additional details below: Intent Discovery Score. This metric measures the proportion of intents in the user simulator’s intent hierarchy that transition to a discovered state by the end of the conversation. For each artifact, all evaluated models begin with identical intent hierarchy states, and intents that were initially set as discovered are excluded from the calculation. Intents in the emergent state at conversation end are counted as 0.5 discovered. Intent Satisfaction Score. To assess whether discovered intents translate to satisfactory outputs, we append a final user message prompting the model to generate a complete artifact: “Okay, now generate a complete output artifact considering the conversation so far. Return only the artifact without any other text or explanation.” We then use GPT-5.1 as an LLM judge (prompt in E.7 ) to evaluate the artifact against the leaf nodes of the intent hierarchy (i.e., the most specific intents). For each intent, the judge provides a justification and a score from 1 to 5. We consider an intent satisfied if it receives a score of 4 or higher, and compute the final score as the proportion of satisfied intents—but excluding intents that were satisfied by all assistants. Interactivity Score. Following 57 , we use an LLM judge to rate how interactive and engaging the AI assistant is throughout the conversation (prompt in E.8 ). The judge returns a score from 1 to 3, which we normalize to [0,1] by computing (\text{score}-1)/2 . B.4 Turn-by-Turn Discover Score To classify the model turns as divergent or convergent, we used GPT-5-Nano with the prompt in E.11 . Figure 7 shows the turn-by-turn average Intent Discovery score for each model across tasks. We compare the best-performing DiscoverLLM variant against baselines of the same base model. DiscoverLLM consistently gains an advantage in early turns, compared to the baselines, and maintains and extends this lead in subsequent turns, while the baselines are unable to match its performance or even show diminishing performance. Even in the case where DiscoverLLM only matches baseline performance (i.e., Technical Writing with Llama), we observe that this was due to how DiscoverLLM starts with a lower initial score but is able to recover and match Base by turn 5. Figure 7 : Turn-by-turn intent discovery scores for baselines and the best DiscoverLLM variant for each task and base model. B.5 Generalization Experiment To test the generalizability of DiscoverLLM , we construct a smaller dataset composed of diverse artifacts. We sampled or generated 10 artifacts for each type below: • Travel Plans : We prompted different LLMs to compose travel plans for different countries or cities with different durations for the travels. • Data Visualization Code : We randomly sampled code artifacts from the VisCode-200K dataset ( 40 ) . • Research Paper Abstracts : We retrieved the abstracts from 10 papers from ML, NLP, and HCI venues. • Text-to-Image Prompts : We randomly sampled text prompts from the DiffusionDB dataset ( 56 ) . • Website Components : We prompted different LLMs to generate self-contained web components. Then, we applied our framework to initialize user simulators with these artifacts and evaluated the models on simulated conversations against these user simulators. Similar to the main experiments, for each artifact, we conducted three conversations for each evaluated model and averaged the resulting scores. Table 5 shows the performance for each model evaluated in generalization on each of the artifact or task types (10 artifacts per type). As shown, DiscoverLLM mostly outperforms all baselines across all task types, aside from the data visualization task. Travel Plan Data Viz Paper Abstract T2I Prompt Web Component Discover \uparrow #Tok (k) \downarrow Discover \uparrow #Tok (k) \downarrow Discover \uparrow #Tok (k) \downarrow Discover \uparrow #Tok (k) \downarrow Discover \uparrow #Tok (k) \downarrow Llama-3.1-8B-Instruct Base 57.8 3.86 52.6 4.20 40.4 3.11 53.2 2.45 35.2 4.58 Prompted Base 46.5 3.23 55.6 3.91 51.5 2.64 53.7 2.21 37.0 3.98 CollabLLM 48.7 3.19 59.4 3.56 44.3 2.52 42.9 2.07 31.0 4.33 SFT 39.0 2.39 30.8 1.93 34.8 1.39 41.6 1.24 28.8 2.10 DPO 63.2 3.22 57.7 3.91 59.1 2.79 57.8 2.45 35.0 4.49 SFT+DPO 54.7 3.24 58.8 3.91 51.4 2.64 55.1 2.30 39.0 3.68 Rel. Improv. 9.3% 16.6% -1.0% N/A 14.8% -5.7% 7.6% -4.1% 5.4% 7.5% Table 5 : Detailed results for the generalization evaluation shows performance of each model on each task/artifact type. B.6 Behavioral Patterns To analyze patterns, we use GPT-5-Nano to annotate all evaluation conversations using the prompt in E.11 . Specifically, the LLM annotates each turn whether it is convergent (i.e., provides only one option) or divergent (i.e., provides multiple options). B.7 Additional Example Figure 8 presents an additional example of an evaluated conversation comparing Prompted Base against DiscoverLLM . As seen, while the prompted model presents positive collaborative behaviors (e.g., step-by-step creation, clarification questions at the end of responses), it fails to adequately probe and satisfy the users’ latent intents as it mainly focuses on satisfying the user’s vague intents rather than explore them further. In contrast, DiscoverLLM first provides and revises a draft that directly responds to the user’s initial request. However, as the conversation continues and the user’s expressed intents fail to concretize, the model decides to expand and explore: proposing multiple plausible solutions to the vague issue that the user identified. Figure 8 : Example evaluation conversation (3 turns) of Prompted Base (Qwen3-8B) against DiscoverLLM (SFT+DPO+GRPO) in a Technical Writing task. Appendix C User Study Details Figure 9 shows the interface used for the user study conducted on the crowdsourcing platform, Prolific. Participants were recruited based on the following prescreening criteria: (1) English as their first and primary language, (2) an approval rate above 95%, (3) at least 100 prior submissions, (4) usage of AI in their work at least once a week, and (5) residence in either the UK or the USA. The participants had an average age of 41.9 years, with 24 identifying as female and 51 as male. 63 participants were based in the UK, and 12 in the US. Figure 9 : Interface used during the user study: (1) Initial page explaining the task and providing disclaimers to participants. (2) Page showing the assigned task to the participant and presenting them with possible topics or intents for the writing task. (3) Multi-turn chat interface. (4) Prompt requesting participants to provide interaction ratings every three turns. (5) Prompt indicating to participants that they have completed the minimum number of turns for the task. (6) Final screen asking participants to provide overall interaction ratings, satisfaction with the final writing, and describeobserved strengths and weaknesses of the models. Appendix D Safety Evaluation DiscoverLLM models are trained to help users discover and form their intents through collaborative exploration. This collaborative disposition raises a potential concern: these models may be more susceptible to responding to malicious requests, particularly when user intents are vague or ambiguous. For example, for an ambiguous but potentially adversarial or harmful user request, the model may be inclined to provide multiple options that explore that intent—unintentionally generating harmful content. While the base models underlying DiscoverLLM have been fine-tuned for safety alignment, we conduct a small-scale safety evaluation to assess whether DiscoverLLM training increases vulnerability to safety attacks or the generation of harmful content. We use the Ai2 safety-eval 3 3 3 https://github.com/allenai/safety-eval tool ( 26 ; 21 ) to evaluate base models and their DiscoverLLM variants on three safety benchmarks: WildGuard ( 21 ) , HarmBench ( 37 ) , and ToxiGen ( 22 ) . Table 6 presents the results of the safety evaluation. Both base models and their DiscoverLLM variants achieve high safety scores, with no significant degradation in the DiscoverLLM models. These results suggest that the collaborative behavior trained into DiscoverLLM does not compromise the underlying safety alignment of the base models. Model WildGuard \uparrow HarmBench \uparrow ToxiGen \uparrow Llama-3.1-8B-Instruct 98.1 89.1 99.9 DiscoverLLM (Llama) 97.9 90.6 100 Qwen3-8B 88.7 84.7 100 DiscoverLLM (Qwen) 88.8 83.4 100 Table 6 : Safety evaluation results for the base models and their fine-tuned DiscoverLLM variants (SFT+DPO in Creative Writing). We report the following scores for each benchmark: inverted harm score for WildGuard, inverted attack success rate for HarmBench, and safety score for ToxiGen (higher scores are better for each benchmark). Appendix E LLM Prompts E.1 Prompt for Initial Intent Synthesis ⬇ 1 You will receive a text artifact and its type . Your task is to : 2 3 ( a ) Identify the very broad or general topic of the artifact , which you will represent in a short phrase . 4 ( b ) Comprehensively describe the specific characteristics and attributes of the artifact , which are not a given from the artifact type and the artifact topic . This description should contain significant detail so that a person that follows this description can create an artifact that captures the general essence of the original artifact ( but not necessarily the same artifact ). Treat this description as a guideline or set of requirements / constraints for creating an artifact that resembles the original . 5 ( c ) Decompose the description into a checklist that represents the various sub - requirements , constraints , or sub - guidelines that are included in the description , and must all be fulfilled in order to create an artifact that satisfies the description . 6 7 ** Guidelines :** 8 9 1. ** Broad Topic :** The topic should be broad and general , capturing the main idea or direction of the artifact without revealing specific details . 10 2. ** Key Characteristics :** The description should capture the key or main characteristics of the original artifact . Ensure that you include the most essential , important , and / or representative aspects of the original artifact that make it unique and distinct from other artifacts of the same type or with the same topic . Be specific and selective when deciding what to include in the description . 11 3. ** Independent of Artifact Type and Topic :** The description should focus on the characteristics that go beyond what is already implied or given by the artifact ' s type and topic . Avoid restating generic features that would apply to any artifact of that type or any artifact with that topic . 12 4. ** Positive Framing :** Phrase the description in positive or neutral terms . Avoid phrasing that suggests the artifact is deficient or deviates from an assumed standard . Avoid prescribing errors or mistakes . It is acceptable to slightly reinterpret the original artifact if needed to keep the framing neutral or appreciative . 13 - Wrong : " Switches inconsistently between professional , academic tone to more colloquial , informal tone " 14 - Right : " Blends multiple registers by alternating professional and casual language " 15 5. ** Description and Checklist are Equal **: For any detail , if it is included in a checklist item , it should have also been included in the artifact description . Ensure that the artifact description itself includes all the details . 16 6. ** Independent Checklist Items :** Each checklist item should represent a distinct sub - requirement . Avoid creating checklist items that overlap ( i . e ., satisfying one item automatically leads to another item being satisfied by default ). 17 18 ** Examples :** 19 20 { examples } 21 22 ** Return your output in this format :** 23 ` ` ` yaml 24 internal_thinking : | 25 < think about the broad topic of the artifact > 26 27 < think in - depth about what type of constraints or requirements must be met to recreate the artifact closely matching the original > 28 29 < verify that you have captured all the key characteristics of the original artifact > 30 31 < check whether any of these characteristics are trivial or redundant when considering the artifact type and topic ; if so , remove them from the description or modify them to not be trivial > 32 33 < think about how to decompose the description into checklist items , and verify that the items are independent and one does not automatically satisfy another > 34 35 artifact_topic : < short phrase of 1-3 words describing the broad topic of the artifact > 36 description : < description of the artifact ' s key characteristics and attributes > 37 checklist : 38 - < checklist item 1> 39 - < checklist item 2> 40 - ... 41 ` ` ` E.2 Prompt for Intent Abstraction ⬇ 1 ## What is Progressive Abstraction ? 2 3 Progressive abstraction means gradually making a criterion less specific and more general , so that more artifacts can satisfy it . Think of it like zooming out on a map --- you see a broader area , but you lose fine details . 4 5 ** Simple Example :** 6 - Start : " Uses Python 3.9 with pandas library " 7 - Abstract once : " Uses Python with data analysis tools " 8 - Abstract again : " Uses a programming language " 9 10 ** Key principle :** If artifact X satisfies the specific version , it MUST also satisfy all more general versions . 11 12 --- 13 14 ## Task Overview 15 16 You will receive criteria that assess artifacts . Your job is to create a chain of progressively broader versions , where each step expands what artifacts can satisfy the criterion . You will also receive the type of artifact that this criterion is assessing and the broad topic that these artifacts focus on . 17 18 For each criterion , you should progressively and gradually abstract it until you reach the number of times specified . You will be provided with a checklist for each criterion , which represents the various sub - components or constraints that must all be fulfilled to satisfy that criterion . For each abstraction step , you can either remove items from the checklist or generalize / abstract items in the checklist . The final abstraction should capture only the main essence of the criterion , with all other abstractions representing a gradual step towards the final abstraction . However , ensure that this final abstraction is not trivial --- i . e ., avoid creating final abstractions that are trivial or redundant with the type and topic of the artifacts that this criterion assesses . 19 20 --- 21 22 ## Guidelines 23 24 ### Apply Multiple Abstraction Strategies 25 26 At each abstraction step , use these techniques to broaden the items in the criterion ' s checklist : 27 28 1. ** Broaden the scope ** ( expand the domain ) 29 - " technical details " -> " details " 30 - " Puerto Rican culture " -> " Latin American culture " -> " culture " 31 32 2. ** Generalize categories ** ( move up the hierarchy ) 33 - " Python " -> " programming language " 34 - " neon colors " -> " bright colors " 35 36 3. ** Remove specific instances or constraints ** ( names , numbers , dates , brands ) 37 - "10 research papers " -> " research papers " 38 - " three colors " -> " multiple colors " -> " colors " 39 40 ### Make Abstractions Distinct 41 42 Each abstraction should be meaningfully different from the previous version . Avoid simply paraphrasing --- the scope should actually expand so that more artifacts can satisfy it . 43 44 ** Example of what NOT to do :** 45 - " Uses three neon colors " -> " Employs three neon colors " ( just different words , same scope ) 46 47 ** Example of what TO do :** 48 - " Uses three neon colors " -> " Uses bright colors " ( removed quantity constraint , expanded scope ) 49 50 ### Guarantee Superset Expansion 51 52 ** The Superset Rule :** Each abstraction must be a " superset "- a larger set that contains the previous one . In other words , an artifact that satisfies the less abstracted version ' s checklist should also satisfy the more abstracted version ' s checklist , but not vice versa . 53 54 ** Test :** Can you think of an artifact that satisfies the new version ' s checklist but NOT the old one ? If yes , you ' ve successfully abstracted . 55 56 Check that your abstraction : 57 - Actually expands the scope significantly ( not just rephrasing ) 58 - Allows artifacts satisfying the previous version to also satisfy this one 59 - Allows NEW artifacts to satisfy this version that couldn ' t satisfy the previous one 60 61 ### Stop at the Requested Number 62 63 Abstract each criterion for the exact number of times specified . The final abstraction should be the most general form . 64 65 ### Gradual Abstraction until Maximum Generality in the Final Abstraction , Without Losing Distinctiveness 66 67 You should gradually abstract the criterion at each stage so that the final abstraction will be a minimal checklist that contains the essential requirement at a meaningful level , while discarding all forms of unnecessary specificity . Ensure that you gradually generalize to the broadest expression that still meaningfully constrains what qualifies . 68 69 For this , you must consider two critical constraints at each abstraction step : 70 - ** Non - triviality **: Ensure that you avoid abstracting a checklist item so much that it becomes trivially satisfied by all artifacts of the given type or topic . If further abstraction of an item would lead to it being trivially satisfied by any artifact of the same type or topic , you should avoid abstracting it . 71 - ** Key essence **: Ensure that , at each abstraction step , you keep the key essence of the criterion , while only abstracting or removing the supplementary details . You should ensure that you carry over the main or high - level meaning of criterion until the final abstraction . 72 73 --- 74 75 ## Common Pitfalls to Avoid When Abstracting 76 77 ** Pitfall 1: Paraphrasing Instead of Abstracting ** 78 - Wrong : " Uses three neon colors " -> " Employs three neon colors " ( just different words ) 79 - Right : " Uses three neon colors " -> " Uses neon colors " ( removed quantity constraint ) 80 81 ** Pitfall 2: Jumping Too Far in One Step ** 82 - Wrong : " Summarizes 10 papers on deep learning for protein folding " -> " Summarizes papers " 83 - Right : " Summarizes 10 papers on deep learning for protein folding " -> " Summarizes papers on computational methods for protein folding " 84 85 ** Pitfall 3: Breaking the Superset Rule ** 86 - Wrong : " Uses dark colors " -> " Uses bright colors " ( these are DIFFERENT sets , not superset ) 87 - Right : " Uses dark colors " -> " Uses colors " ( dark colors are a subset of colors ) 88 89 ** Pitfall 4: Not Actually Expanding the Scope ** 90 - Wrong : " Cites 10 peer - reviewed papers " -> "