Filtering for Creativity: Adaptive Prompting for Multilingual Riddle Generation in LLMs

Duy Le, Kent Ziti, Evan Girard-Sun, Sean O’Brien, Vasu Sharma, Kevin Zhu · 2025

Read Filtering for Creativity: Adaptive Prompting for Multilingual Riddle Generation in LLMs by Duy Le, Kent Ziti, Evan Girard-Sun, Sean O’Brien, Vasu Sharma, Kevin Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-18709.txt.

Introduction

Duy Le and co-authors teach language models to write better riddles in a 2025 paper. Their method, called Adaptive Originality Filtering, throws out drafts that hew too close to ones already kept. Standard prompts tend to yield riddles that repeat stock patterns and miss the culture they aim at, across the many languages such models now serve.

Alongside the filter, the team built a scoring tool that rates each riddle on novelty, spread, fluency, and whether the answer fits. Tests showed clear gains in languages such as Arabic and Japanese, and human judges agreed the riddles read better and felt more at home in their cultures. Scores rose most where the filter had the most dull material to cut. The authors note the same filter could serve other creative tasks without retraining the model.

Catalog record

CREATORDuy Le, Kent Ziti, Evan Girard-Sun, Sean O’Brien, Vasu Sharma, Kevin Zhu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE67 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-18709

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Filtering for Creativity: Adaptive Prompting for Multilingual Riddle Generation in LLMs Duy Le, Kent Ziti, Evan Girard-Sun, Sean O’Brien, Vasu Sharma, Kevin Zhu Abstract Multilingual riddle generation challenges large language models (LLMs) to balance cultural fluency with creative abstraction. Standard prompting strategies—zero-shot, few-shot, chain-of-thought—tend to reuse memorized riddles or perform shallow paraphrasing. We introduce Adaptive Originality Filtering (AOF), a prompting framework that filters redundant generations using cosine-based similarity rejection, while enforcing lexical novelty and cross-lingual fidelity. Evaluated across three LLMs and four language pairs, AOF-enhanced GPT-4o achieves 0.177 Self-BLEU and 0.915 Distinct-2 in Japanese, signaling improved lexical diversity and reduced redundancy compared to other prompting methods and language pairs. Our findings show that semantic rejection can guide culturally grounded, creative generation without task-specific fine-tuning. 1 Introduction Large Language Models (LLMs) excel across many natural language processing tasks but often falter in creative generation within multilingual settings ( Zhang 2025 ; Ismayilzada et al. 2024 ). This limitation is especially apparent in bilingual riddle generation, where outputs are frequently rote and easily found online. To address this, we propose Adaptive Originality Filtering (AOF)—a prompting framework that promotes cultural specificity and lexical novelty without fine-tuning. AOF employs a semantic rejection mechanism to discard formulaic outputs and iteratively prompt the model until it produces a response that is both original and culturally coherent. In contrast to existing strategies like zero-shot, few-shot Brown et al. 2020a , and chain-of-thought prompting Wei et al. 2023 , which often yield paraphrased common riddles, AOF actively steers generation toward novelty and cultural fidelity. Figure 1: This diagram illustrates the end-to-end pipeline for generating and validating riddles using LLMs(GPT-4o, R1, LLaMA). The process begins with enforcing constraints on novelty and structure. Model outputs are then checked for semantic similarity using MiniLM, with a threshold score of < 0.75 determining acceptance. Outputs that fail are regenerated (Retry), while accepted samples undergo final evaluation. We ask: Can prompting alone enable culturally aware, original bilingual generation in LLMs? To explore this, we apply AOF to three state-of-the-art LLMs (GPT-4o, LLaMA 3.1, DeepSeek) across four language pairs, benchmarking against four standard prompting baselines. We evaluate using metrics for lexical diversity (Distinct-n), redundancy (Self-BLEU), bilingual semantic alignment (Cross-Lingual BERTScore), and syntactic validity. Results show that AOF significantly improves both originality and cross-cultural alignment. By injecting an external filtering loop into prompting, AOF offers a lightweight, scalable path toward more creative and culturally attuned multilingual generation (see Figure 1 ). 2 Related Work Most prior research emphasizes riddle solving rather than generation. Panagiotopoulos et al. 2024 and Heavey et al. 2024 developed multilingual inference models for riddle solving but did not extend to generative tasks. Xu et al. 2022 incorporated cultural embeddings for Chinese riddle comprehension, while Smith et al. 2022 evaluated NLG models on static benchmarks without multilingual or generative scope. Multilingual representation efforts like MUSE Lample & Conneau 2019 , LASER Chen & Avgustinova 2021 , and XLM-R Conneau et al. 2019 support cross-lingual transfer but not figurative or culturally grounded generation. Sentence-level encoders Xia et al. 2019 ; Gritta & Iacobacci 2021 improve alignment but lack capacity for creative language. Dufter 2021 noted that such embeddings often collapse metaphorical nuance essential to cultural generation. Creative Language Generation Creative generation research has largely focused on ideation and open-domain tasks Ma et al. 2025 . Cox et al. 2021 ; Cox et al. 2023 used structured prompts for idea diversity, while Huang et al. 2023 improved multilingual output via cross-lingual-thought prompting without filtering. Constraint-based methods like Yang et al. 2022 and Laverghetta Jr et al. 2024 use decoding control or heuristics to boost diversity. Structural prompting has also gained traction: de Wynter et al. 2023 introduced recursive meta-prompts, Verma et al. 2024 identified abstraction limits in ReAct prompting, and Liu et al. 2024 applied paraphrasing under semantic constraints. Atmakuru et al. 2024 benchmarked narrative creativity under fixed templates, highlighting rejection-based filtering as a viable originality enhancer. Prompting for Originality and Structure Standard approaches prioritize structure over creativity. Few-shot prompting Brown et al. 2020a ; Brown et al. 2020b and CoT Wei et al. 2023 improve reasoning but often replicate training artifacts. Iterative methods like Self-Refine Madaan et al. 2023 , Tree-of-Thought Yao et al. 2023 , and Reflexion Krishna et al. 2023 enhance fluency and factuality through retries but rarely enforce novelty. Constraint-driven prompting offers stronger generative control. COLD decoding Mou et al. 2022 , EditCoT Wang et al. 2024 , and Sketch-of-Thought Aytes et al. 2025 use template-based completions, while DeLorean Liu et al. 2023 integrates symbolic constraints. These techniques target structure but overlook cultural fidelity. In contrast, our method enforces novelty during decoding via cosine-based filtering and supports cross-lingual generation of culturally resonant riddles. 3 Methodology This section describes the components of our multilingual riddle generation framework, including our originality-enforcing prompting method, a centralized automatic evaluation metric, human annotation protocols, experimental design, and fine-tuning our model. 3.1 Adaptive Originality Filtering (AOF) Prompting Traditional prompting strategies such as Chain-of-Thought (CoT) and Few-Shot improve logical coherence but frequently fall short in producing structurally novel and semantically diverse riddles. Prior work has shown that LLMs often reproduce internet riddles or paraphrase entries from BiRdQA Zhang & Wan 2022 , defaulting to overused scaffolds such as “I have…” and “What am I?”. To address this, we propose Adaptive Originality Filtering (AOF) , a prompting method designed to enforce semantic novelty , syntactic variation , and cross-lingual fidelity . Semantic Filtering. Each generated riddle r_{\text{gen}} is compared against a reference set \mathcal{D}=\{r_{i}\}_{i=1}^{N} using cosine similarity in embedding space. A candidate is accepted only if: \mathcal{S}(r_{\text{gen}},\mathcal{D})<\theta,\text{ with }\theta=0.75. The full formulation and reasoning for .75 appears in Appendix J.1 . Rejection Sampling Loop. AOF employs rejection sampling: if a candidate fails the novelty filter, it is discarded and regenerated. This process repeats until a valid output is produced or a retry cap is reached. See Appendix J.2 for pseudocode. Prompt Constraints. Prompts used in AOF enforce specific closure formats, grammatical variation, and discourage common answers. Template rules are detailed in Appendix J . 3.2 Experimental Setup We evaluate generation across multiple LLMs and prompting strategies, comparing structural, semantic, and fluency metrics in five languages. Models Evaluated. We benchmark GPT-4o, LLaMA 3.1, and DeepSeek Reasoning (R1), all queried under identical settings (temperature 0.7, token limit 3000). Prompting Strategies. We compare Zero-Shot, Few-Shot, Chain-of-Thought, Adversarial Wallace et al. 2019 ; Ribeiro et al. 2018 , and AOF prompting. Full details are provided in Appendix K . Dataset. BiRdQA Zhang & Wan 2022 provides a bilingual corpus of 15,365 riddles in English and Chinese. We sample balanced subsets and align multilingual answers. Metrics. Automatic metrics include Self-BLEU (repetition), Distinct-2 (diversity), BERTScore (alignment). Human validation is included in Appendix A . 3.3 Fine-Tuning of the GPT-4o Model Objective and Motivation This fine-tuning aimed to improve GPT-4o-2024-08-06’s performance on riddle comprehension and generation across multiple languages. Riddles involve more than surface-level matching—they require metaphor understanding, logical contradiction, and creative misdirection. Our goal was not only to raise answer accuracy but also to instill structural reasoning abilities. Methodological Overview We framed the task as a supervised multi-class classification problem using the BiRdQA dataset. Riddles were presented in a multiple-choice format, and the model

Read the complete text