Shakespearean Sparks: The Dance of Hallucination and Creativity in LLMs’ Decoding Layers
Zicong He, Boxuan Zhang, Lu Cheng · 2025
Read Shakespearean Sparks: The Dance of Hallucination and Creativity in LLMs’ Decoding Layers by Zicong He, Boxuan Zhang, Lu Cheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-02851.txt.
Introduction
Zicong He, Boxuan Zhang, and Lu Cheng wrote this 2025 paper on large language models. These models are known to hallucinate, which means they produce text that is not true. That flaw is often linked to creativity. Earlier work rested mostly on theory or on close looks at single cases. The authors instead test the link in a planned, step-by-step way.
They set a narrow meaning of creativity for these models and build a tool called HCL. The tool scores hallucination and creativity at each layer used during decoding. A tradeoff between the two holds across layer depth, model type, and model size. For each size, one layer gives the best balance. In larger models, that layer tends to sit early, with much higher model confidence there. The team shares its code and test data.
Catalog record
| CREATOR | Zicong He, Boxuan Zhang, Lu Cheng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 35 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-02851
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Shakespearean Sparks: The Dance of Hallucination and Creativity in LLMs’ Decoding Layers Zicong He, Boxuan Zhang, Lu Cheng Abstract Large language models (LLMs) are known to hallucinate, a phenomenon often linked to creativity. While previous research has primarily explored this connection through theoretical or qualitative lenses, our work takes a quantitative approach to systematically examine the relationship between hallucination and creativity in LLMs. Given the complex nature of creativity, we propose a narrow definition tailored to LLMs and introduce an evaluation framework, HCL, which quantifies H allucination and C reativity across different L ayers of LLMs during decoding. Our empirical analysis reveals a tradeoff between hallucination and creativity that is consistent across layer depth, model type, and model size. Notably, across different model architectures, we identify a specific layer at each model size that optimally balances this tradeoff. Additionally, the optimal layer tends to appear in the early layers of larger models, and the confidence of the model is also significantly higher at this layer. These findings provide a quantitative perspective that offers new insights into the interplay between LLM creativity and hallucination. The code and data for our experiments are available at https://github.com/ZicongHe2002/HCL-Spark . 1 Introduction LLMs have demonstrated exceptional performance across various aspects, often rivaling or even surpassing those of humans ( Luo et al., 2024 ; Trinh et al., 2024 ; OpenAI, 2024 ) . Among these, creativity is a highly recognized capability of LLM, which allows it to be used in a variety of domains, including text generation ( Radford et al., 2019 ) , reasoning ( Brown et al., 2020 ) , and image synthesis ( Ramesh et al., 2021 ) . However, the enhanced creativity usually comes with an increased propensity for hallucination ( Jiang et al., 2024 ) , i.e., generating misleading information and risky behaviors ( Orgad et al., 2024 ) , which significantly hinders their application especially in high-stakes scenarios such as finance ( Wu et al., 2023 ) and healthcare ( Singhal et al., 2025 ; Zhou et al., 2025 ) . To address this concern, a considerable body of research has been dedicated to detecting ( Farquhar et al., 2024 ; Manakul et al., 2023 ) and mitigating ( Chuang et al., 2023 ; Du et al., 2023 ; Li et al., 2024 ) hallucinations. Figure 1: Illustration of our HCL evaluation criteria. Givena question with multiple correct answers, we instruct the LLM to generate various responses several times. Correct responses are shown in various shades of green, and creativity is defined as the diversity represented by distinct types grouped based on semantic similarities. Red boxes depict hallucinatory answers that are factually incorrect. Recently, some efforts begin to delve into the connection between the two characteristics in LLMs ( Lee, 2023 ; Jiang et al., 2024 ) . From a philosophical perspective, as The Creativity Hidden in Hallucination suggests, what is often dismissed as “wrong” may harbor unexpected creativity. For example, Copernicus’s heliocentric theory was initially regarded as heresy, yet it eventually revolutionized the field of astronomy ( Jiang et al., 2024 ) . Although promising progress has been achieved, existing studies are still limited in theoretically or qualitatively exploring the relationship between creativity and hallucination, lacking a empirical and systematic study of this connection in LLMs. Simultaneously, current efforts centered on creativity assessments primarily explore on specific tasks such as storytelling ( Gómez-Rodríguez and Williams, 2023 ) , poetry ( Chakrabarty et al., 2024 ) , and artistic ideation ( Lu et al., 2024 ) , lacking a general and accurate definition and quantification method for the creativity tailored to LLMs. More specifically, traditional approaches typically rely on predefined criteria (e.g., originality, content fluency, and character similarity) or comparisons against other generations. However, the inherently stochastic (i.e., generations vary across instances) and unpredictable hallucinations (i.e., false or inaccurate information) of LLM outputs make it difficult for established methods to accurately measure the creative capabilities of LLMs. To fill the above gaps, we propose a novel framework to conduct the first empirical analyses of the interplay between creativity and hallucinations from the inner structure of LLMs, i.e., layer to layer. We refer to this framework as HCL ( H allucination and C reativity across L ayers). Since the outputs directly generated by the early layers of LLM are usually unstable or even invalid ( Elhoushi et al., 2024 ) , we adopt the Layer-Skip ( Elhoushi et al., 2024 ) to ensure the generated content are consistently meaningful during layer-wise response sampling. Each response is then subjected to factual and diversity verification and categorized into two classes: creativity and hallucination. Following prior works ( Orgad et al., 2024 ) , the hallucination indicator is assigned with the error rates among the generated responses. For the creativity metric, we provide a narrow definition tailored to the LLM that quantifying it as the diversity of correctness among sampled responses for each layer (Figure 1 ). We conduct extensive empirical analyses to examine their connections and identify a broadly consistent tradeoff between hallucination and creativity across different layer depths and sizes of LLMs. The combination of these two dimensional metrics consequently yields a hallucination-creativity balanced (HCB) score for each layer, assisting in locating the optimal decoding layer for different model architectures that tend to produce accurate and varied outputs. Our contributions are summarized as follows: 1. Conceptually, we study a new perspective to explore LLMs’ inner structure regarding the relationship between creativity and hallucination in LLMs during generating responses in common question-answering domains. 2. Technically, we propose a new evaluation framework, namely, HCL, to analyze the layer-wise evolution of creativity and hallucination in LLM’s responses and the trade-offs between the two concepts. 3. Empirically, Our experiments show several inspiring findings, including the observation that creativity always comes with hallucination in LLMs. Furthermore, from the perspective of balancing creativity and hallucination, we find that relying on the final layer’s output is not always optimal. Instead, early-exiting at intermediate layers yields better performance. Figure 2: Overview of our HCL framework. We employ the layer_skip method, where each layer of the LLM is queried with the same prompt multiple times, generating diverse responses. The responses are then categorized into correctness and hallucination . Next, the correct responses undergo a secondary classification, where each color represents a distinct category of responses, collectively referred to as a type of creativity . Finally, we compute the HCB score by integrating the creativity score ( \mathbf{S_{c}} ) and the hallucination score ( \mathbf{S_{H}} ). 2 Related Work LLMs have demonstrated remarkable abilities in various domains, yet they still suffer from inherent issues such as hallucination and creativity uncertainty. While previous research has explored these two aspects separately, little attention has been given to their interplay. This section reviews existing work on hallucination and creativity in LLMs, highlighting the research gap that our study aims to address. Hallucination in Large Language Models Hallucination in LLMs refers to the generation of misleading, or incorrect content, which poses a significant challenge in high-stakes scenarios such as finance ( Wu et al., 2023 ) and healthcare ( Singhal et al., 2025 ) . Extensive research has been conducted to detect and mitigate hallucinations