RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplars WARNING: This paper may contain examples that are offensive, non-inclusive, or biased.
Yuncheng Hua, Lizhen Qu, Zhuang Li, Hao Xue, Flora D. Salim, Gholamreza Haffari · 2025
Read RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplars WARNING: This paper may contain examples that are offensive, non-inclusive, or biased. by Yuncheng Hua, Lizhen Qu, Zhuang Li, Hao Xue, Flora D. Salim, Gholamreza Haffari (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-11681.txt.
Introduction
Yuncheng Hua, Lizhen Qu, Zhuang Li, Hao Xue, Flora Salim, and Gholamreza Haffari improve the behavior of language models by restyling the examples shown to them, in a 2025 paper. Teaching models to be helpful and safe usually takes costly human labeling and heavy training. Showing a model good examples in the prompt — in-context learning — is far cheaper, if the examples are right.
The team studied what makes demonstration examples work and found style to be a key lever. They rewrote examples under a style framework, then blended sets to balance truthfulness against safety. The restyled prompts lifted alignment scores past the best rival method in their tests. Prompt style alone, it seems, can do part of the work of full training.
Catalog record
| CREATOR | Yuncheng Hua, Lizhen Qu, Zhuang Li, Hao Xue, Flora D. Salim, Gholamreza Haffari |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 121 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-11681
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
RIDE: Enhancing Large Language Model Alignment through Restyled In-Context Learning Demonstration Exemplars WARNING: This paper may contain examples that are offensive, non-inclusive, or biased. Yuncheng Hua, Lizhen Qu, Zhuang Li, Hao Xue, Flora D. Salim, Gholamreza Haffari Abstract Alignment tuning is crucial for ensuring large language models (LLMs) behave ethically and helpfully. Current alignment approaches require high-quality annotations and significant training resources. This paper proposes a low-cost, tuning-free method using in-context learning (ICL) to enhance LLM alignment. Through an analysis of high-quality ICL demos, we identified style as a key factor influencing LLM alignment capabilities and explicitly restyled ICL exemplars based on this stylistic framework. Additionally, we combined the restyled demos to achieve a balance between the two conflicting aspects of LLM alignment— factuality and safety . We packaged the restyled examples as prompts to trigger few-shot learning, improving LLM alignment. Compared to the best baseline approach, with an average score of 5.00 as the maximum, our method achieves a maximum 0.10 increase on the Alpaca-eval task (from 4.50 → \to 4.60), a 0.22 enhancement on the just-eval-instruct benchmark (from 4.34 → \to 4.56), and a maximum improvement of 0.32 (from 3.53 → \to 3.85) on the MT-Bench dataset. We release the code and data at https://github.com/AnonymousCode-ComputerScience/RIDE . 1 Introduction Figure 1: Overview of our framework for constructing the optimal ICL exemplar set, designated as R estyled I n-context-learning D emonstration E xemplars ( RIDE ), to achieve In-Context Alignment (ICA). Alignment tuning helps bridge the gap between raw model capabilities and the nuanced requirements of different tasks, such as delivering accurate information, maintaining user safety, and handling sensitive topics with care ( Shneiderman, 2020 ; Wang et al., 2023b ; Qi et al., 2024b ) . The mainstream alignment tuning methods, such as supervised fine-tuning and reinforcement learning with human feedback (RLHF), rely on a large mount of annotated data and significant computing resources ( Ouyang et al., 2022 ; Sun et al., 2023 ; Dai et al., 2024 ; Rafailov et al., 2024 ; Zhou et al., 2024 ; Wu et al., 2024 ) . They potentially leading to catastrophic forgetting of pre-trained knowledge Wang et al. (2023a) . In contrast, In-Context Alignment (ICA) provides a low-cost, flexible alternative by employing a handful of selected demonstration exemplars for In-Context Learning (ICL), enabling LLMs to align with user intent without changing model parameters Lin et al. (2024) . The majority of the prior works on ICA investigate selecting demonstration exemplars Liu et al. (2022) ; Min et al. (2022) ; Ye et al. (2023) ; Peng et al. (2024) ; Choi and Li (2024) ; Wang et al. (2024) , while Lin et al. (2024) only utilize three manually designed exemplars with customized styles, referred to as URIAL, across all tasks. These handcrafted ICL exemplars complement each other, achieving a delicate balance between factuality and safety , which effectively enhance LLM alignment capabilities empirically. However, URIAL lacks quantitative analyses to explain why these specific manually crafted ICL demos are effective and what is the impact of each style factor. In addition to styles, what are the other key factors may influence the selection and combination of ICL exemplars? Zhao et al. (2024) identify the importance of the source of ICL exemplars, while Zhou et al. (2024) investigate the impact of labels, input-label mappings, and distribution of inputs. Moreover, ICA seems to impose two conflicting demands: on one hand, LLMs need to provide more in-depth, informative, and helpful content ( factuality ) ( Shen et al., 2023 ) ; on the other hand, for safety reasons, LLMs must refuse to answer inappropriate queries ( safety ) ( Ji et al., 2024 ) . Balancing these factors is crucial to effectively leveraging ICL exemplars. In this work, we conduct the first quantitative study to assess the impact of individual style factors. In particular, we select and rank ICL candidates from a candidate tool in terms of a metric, termed value impact . Our detailed analysis of these exemplars reveals to what extent distinctive stylistic factors in ICL exemplars influence LLM alignment capabilities (see Section 2 ). Based on the insights from our study, we propose to automatically restyle selected ICL demonstrations using an LLM with a customized prompt (Section 3 , RQ1). To address the trade-off between factuality and safety , we systematically explore different exemplar combinations while maintaining stylistic consistency across those exemplars (Section 3 , RQ2). Through this process, we identify a handful of optimal exemplar combinations in terms of both styles and content across various tasks (Section 3 , RQ3). As illustrated in Figure 1 , we refer to these optimized sets as R estyled I n-context-learning D emonstration E xemplars ( RIDE ). In summary, our contributions are three-fold: (i) Through a systematic analysis of ICL exemplars, we identify specific stylistic factors that improve LLM alignment capabilities. By evaluating these features using the value impact metric, we provide insights into how different styles influence the effectiveness of ICL demonstrations. (ii) We propose an automatic restyling approach that systematically modifies ICL demonstrations to enhance alignment. By exploring different style configurations, we identify the optimal stylistic composition that balances the trade-off between factuality and safety , leading to the development of RIDE as the most effective ICL demo set. (iii) We conduct a series of experiments across different datasets and LLM models, demonstrating the effectiveness and superiority of our proposed method. The experimental results show that, across the three benchmarks, our method achieves improvements of 2.22\% , 4.28\% , and 9.07\% compared to the SOTA methods, respectively. 2 Impact of Styles on LLM Alignment In this section, we address one research question: What styles in in-context learning (ICL) examples can influence LLM alignment? Recent studies have demonstrated that the style of in-context examples significantly affects the few-shot online learning performance of LLMs ( Chen et al., 2024 ) . However, the specific impact of different ICL example styles on various facets of LLM alignment has not been thoroughly explored in the literature ( Millière, 2023 ; Anwar et al., 2024 ) . To fill this gap, we propose a novel metric, termed value impact, to quantify the positive or negative influence that an ICL demonstration example exerts on an LLM’s alignment capabilities. Value Impact Computation. For a given user query q , our approach proceeds as follows. First , we generate an output o=P(q) using an LLM P that has not undergone any alignment tuning. Next , we introduce an ICL demonstration example c alongside the query q and generate a new output o_{c}=P(q,c) . Then , we employ an LLM-as-a-judge framework to score both o and o_{c} on six distinct dimensions (as the metrics shown in Table 1 ) that capture different aspects of LLM alignment. For any given dimension v , we define a score as: \delta^{v}_{c}=v(o_{c})-v(o) . Here, \delta^{v}_{c} represents the effect of the demonstration example c on the LLM’s performance in dimension v when answering the query q . Finally , for a validation dataset Q comprising various queries, we calculate the average \delta^{v}_{c} for each dimension v as \overline{\delta_{c}^{v}}=\frac{1}{|Q|}\sum_{q\in Q}\delta_{v}^{c}(q). We define the \overline{\delta_{c}^{v}} as value impact , which reflects the overall positive or