RPS: Information Elicitation with Reinforcement Prompt Selection

Tao Wang, Jingyao Lu, Xibo Wang, Haonan Huang, Su Yao, Zhiqiang Hu, Xingyan Chen, Enmao Diao · 2026

Read RPS: Information Elicitation with Reinforcement Prompt Selection by Tao Wang, Jingyao Lu, Xibo Wang, Haonan Huang, Su Yao, Zhiqiang Hu, Xingyan Chen, Enmao Diao (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13817.txt.

Introduction

Wang and seven colleagues help chatbots draw out facts users hold back. People hide details for privacy, uncertainty, or shyness, leaving models with gaps. Their method, Reinforcement Prompt Selection, learns which prompt to ask next, treating prompt choice as a sequence of decisions. In a synthetic test it beats random questioning. On IELegal, a benchmark built from real legal cases, adaptive prompts beat fixed prompts.

Catalog record

CREATORTao Wang, Jingyao Lu, Xibo Wang, Haonan Huang, Su Yao, Zhiqiang Hu, Xingyan Chen, Enmao Diao
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE76487
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13817

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

RPS: Information Elicitation with Reinforcement Prompt Selection Tao Wang, Jingyao Lu, Xibo Wang, Haonan Huang, Su Yao, Zhiqiang Hu, Xingyan Chen, Enmao Diao Abstract Large language models (LLMs) have shown remarkable capabilities in dialogue generation and reasoning, yet their effectiveness in eliciting user-known but concealed information in open-ended conversations remains limited. In many interactive AI applications, such as personal assistants, tutoring systems, and legal or clinical support, users often withhold sensitive or uncertain information due to privacy concerns, ambiguity, or social hesitation. This makes it challenging for LLMs to gather complete and contextually relevant inputs. In this work, we define the problem of information elicitation in open-ended dialogue settings and propose Reinforcement Prompt Selection (RPS), a lightweight reinforcement learning framework that formulates prompt selection as a sequential decision-making problem. To analyze this problem in a controlled setting, we design a synthetic experiment, where a reinforcement learning agent outperforms a random query baseline, illustrating the potential of policy-based approaches for adaptive information elicitation. Building on this insight, RPS learns a policy over a pool of prompts to adaptively elicit concealed or incompletely expressed information from users through dialogue. We also introduce IELegal, a new benchmark dataset constructed from real legal case documents, which simulates dialogue-based information elicitation tasks aimed at uncovering case-relevant facts. In this setting, RPS outperforms static prompt baselines, demonstrating the effectiveness of adaptive prompt selection for eliciting critical information in LLM-driven dialogue systems. 1 Introduction Large language models (LLMs) have demonstrated impressive capabilities in dialogue generation, reasoning, and information synthesis. Despite these strengths, their ability to effectively elicit concealed or uncertain information from users in open-ended conversations remains a significant challenge. In real-world domains such as health interviews ( 19 ; 25 ; 32 ) , financial advising ( 36 ; 18 ) , and legal consultation ( 4 ; 17 ; 28 ) , users often hesitate to disclose sensitive or uncertain information due to privacy concerns, self-presentation goals, or contextual ambiguity ( 30 ) . As a result, interactions become less informative and effective, often leading to suboptimal recommendations, missed diagnoses, or incomplete legal assessments. Making matters worse, such information is inherently difficult to annotate or collect, since it is purposefully withheld by users. To address this, a key step toward building more robust and context-aware conversational agents is the design of systems that can adaptively elicit concealed information without relying on explicit supervision or handcrafted rules. With the advent of instruction-tuned LLMs, there is increasing interest in leveraging prompt engineering to guide these models toward more effective information-seeking behavior ( 21 ; 3 ; 23 ; 31 ) . However, crafting optimal prompts remains labor-intensive and highly context-dependent. Moreover, without a principled formulation of the information elicitation problem, existing methods lack mechanisms to learn or adapt prompt strategies over time. Recent approaches to automated prompt search, such as RLPrompt ( 5 ) and GRIPS ( 24 ) , highlight the potential of reinforcement and gradient-free optimization respectively. Yet, both face practical limitations: RLPrompt introduces significant computational overhead due to token-level sequential generation and reward evaluation. In addition to being inefficient, token-level generation can lack prompt diversity and makes it difficult to integrate domain-specific knowledge. GRIPS, though edit-based and gradient-free, lacks reinforcement-driven adaptation and requires frequent reward lookups during inference. These shortcomings motivate our proposed approach: a lightweight, reinforcement-based prompt selection framework that adaptively elicits concealed information with high efficiency, ensures prompt diversity, and facilitates integration of domain-specific knowledge, making it suitable for real-time open-ended dialogue. In this paper, we define the problem of Information Elicitation (IE) in open-ended dialogue, where a language model must adaptively elicit information that users know but may conceal or only partially express during interaction. To analyze this problem in a controlled setting, we design a synthetic experiment using a Gaussian Mixture Model environment that highlights the core challenges of adaptive information elicitation. We then propose Reinforcement Prompt Selection (RPS), a lightweight reinforcement learning framework that formulates prompt selection as a sequential decision-making task. Unlike static or handcrafted approaches, RPS learns a policy over a pool of prompts and adapts its strategy based on user feedback to uncover information users may initially withhold. This lightweight and domain-agnostic approach enables scalable deployment in real-world applications. By adaptively selecting prompts from a predefined pool, RPS reduces reliance on static prompts, handcrafted rules, and costly token-level generation, while also promoting prompt diversity and supporting the integration of domain-specific knowledge. The key contributions of this paper are summarized as follows: • We define the problem of Information Elicitation (IE) in open-ended dialogue, where a language model must uncover user-known but initially concealed information. To address this, we propose Reinforcement Prompt Selection (RPS) , a lightweight reinforcement learning framework that formulates prompt selection as a sequential decision-making task. RPS adaptively selects prompts from a predefined pool to promote user disclosure during conversation. • We introduce IELegal , a benchmark dataset constructed from real legal case documents. It simulates an interactive dialogue in which the user represents the subject of a legal case, and the LLM must elicit key factual information. IELegal is designed to reflect the challenge of information elicitation in settings where users may deliberately withhold or obscure sensitive facts, making it a suitable testbed for evaluating models in complex and realistic legal consultation scenarios. • We conduct experiments in both synthetic and real-world settings. In a controlled Gaussian Mixture Model environment, an RL agent outperforms a random query baseline, validating the challenge of adaptive information elicitation. In IELegal, RPS significantly outperforms static prompt baselines, demonstrating its effectiveness in uncovering relevant and potentially concealed information. 2 Related Works Information Elicitation. Early works on information elicitation are largely rule-based, relying on predefined dialogue triggers or policies ( 14 ; 29 ) . 14 model elicitation as an interactive dialogue for decision tree construction, treating it as a heuristic search process. 29 explore multi-round dialogue for eliciting user requirements, using knowledge graphs and granular computing to reduce conversational burden. Learning-based approaches use deep neural networks to model and optimize information elicitation ( 26 ; 38 ; 13 ; 12 ) . 26 propose coached conversational preference elicitation, which guides users in expressing structured preferences through dialogue. 38 introduce hierarchical bandit algorithms that balance between asking about categories and recommending items, improving elicitation efficiency by modeling category and item-level rewards. 13 explore reinforcement learning methods for training dialogue systems to gather information through natural user interaction. 12 propose a novel strategy for conversational recommendation by asking usage-oriented questions rather than directly requesting item attributes, addressing the challenge that users may lack domain knowledge. Recent advances in Large Language Models (LLMs) have enabled more powerful strategies for information elicitation ( 21 ; 23 ; 7 ; 31 ) . 21 introduce a virtual human system that combines LLM-driven semantic parsing, context-aware dialogue management, and adaptive question generation to extract user information in real-time interactions. 23 propose a framework that empowers LLMs to actively reduce uncertainty through strategic questioning. 7 present a benchmark designed to evaluate the ability of seeker agents to resolve uncertainty through clarification and information seeking questions. 31 introduce a meta-training approach that models uncertainty in information elicitation by simulating future observations. Discrete Prompt Optimization. Prompt generation

Read the complete text