Large Language Models as Innovators: A Framework to Leverage Latent Space Exploration for Novelty Discovery
Mateusz Bystroński, Mikołaj Hołysz, Grzegorz Piotrowski, Nitesh V. Chawla, Tomasz Kajdanowicz · 2025
Read Large Language Models as Innovators: A Framework to Leverage Latent Space Exploration for Novelty Discovery by Mateusz Bystroński, Mikołaj Hołysz, Grzegorz Piotrowski, Nitesh V. Chawla, Tomasz Kajdanowicz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-13874.txt.
Introduction
Can a language model be made more inventive without retraining it? The head of this 2025 work answers yes, by steering the model through the shape of its own knowledge space. The catalog gives a different title and a byline with two extra names; the head names Mateusz Bystroński, Doheon Han, Nitesh Chawla, and Tomasz Kajdanowicz. Their claim is that word sense lies along smooth sheets in the model's inner space, and a walk on those sheets can widen what the model can say.
A small set of varied seed answers is pooled into a steering spread, then fed to the model through a bridge tool as it writes. In trials the steered model ranged wider than its plain self, with gains in the split-path thinking that marks inventive work. No weights were touched. The team frame the sheet walk as a cheap add-on for any big model.
Catalog record
| CREATOR | Mateusz Bystroński, Mikołaj Hołysz, Grzegorz Piotrowski, Nitesh V. Chawla, Tomasz Kajdanowicz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-13874
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Large Language Models as Innovators: A Framework to Leverage Latent Space Exploration for Novelty Discovery Mateusz Bystroński, Mikołaj Hołysz, Grzegorz Piotrowski, Nitesh V. Chawla, Tomasz Kajdanowicz Abstract Innovative idea generation remains a core challenge in AI, as large language models (LLMs) often struggle to produce outputs that are both novel and relevant. Despite their fluency, LLMs tend to replicate patterns seen during training, limiting their ability to diverge creatively without extensive prompt engineering. Prior work has addressed this through domain-specific heuristics and structured prompting pipelines, but such solutions are brittle and difficult to generalize. In this paper, we propose a model-agnostic latent-space ideation framework that enables controlled, scalable creativity by navigating the continuous embedding space of ideas. Unlike prior methods, our framework requires no handcrafted rules and adapts easily to different domains, input formats, and creative tasks. This paper introduces an early-stage prototype of our method, outlining the conceptual framework and preliminary results highlighting its potential as a general-purpose co-ideator for human-AI collaboration. 1 Introduction Creative idea generation is a cornerstone of innovation across domains, from product design and writing to scientific research. Recent advances in large language models (LLMs) have opened new possibilities for AI-assisted brainstorming, as these models can produce fluent and contextually relevant ideas on demand. However, LLMs often struggle to generate truly original or imaginative outputs. Because they are trained on vast repositories of existing data, LLMs tend to follow familiar patterns, leading to repetitive or “safe” ideas that lack genuine novelty Mizrahi et al. 2025 . Simply increasing a model’s randomness (e.g., the temperature setting) yields only modest gains in originality Mizrahi et al. 2025 , and instructing an LLM to “be creative” can backfire by producing invalid or hallucinated results Mizrahi et al. 2025 . In short, despite their fluency, current LLMs often fall short in creative divergence, necessitating new approaches to unlock their ideation potential. One promising direction is to augment LLMs with heuristic seeding and structured idea transformations. Recent studies have shown that providing an LLM with carefully crafted seeds or combining existing concepts in novel ways can markedly improve the creativity of its outputs. For example, Mizrahi et al. (2024) introduce a pipeline that parses textual inputs into structured representations and recombines them to generate creative outcomes Mizrahi et al. 2025 . In the culinary domain, their system merges disparate recipes—e.g., combining a pizza preparation method with the flavors of alfajores cookies yields a novel “alfajores pizza” concept Mizrahi et al. 2025 . Complementing this line of work, Holysz et al. (2025) propose a multi-stage LLM pipeline for synthesizing realistic medical datasets: the system first seeds generation with domain-specific “chief complaints,” then incrementally expands them into full patient profiles and simulated clinical dialogues, using tightly crafted schemas to preserve medical plausibility Holysz et al. 2025 . Such heuristic-seeding approaches—though effective—illustrate the cost of hand-tailoring rules and representations for each domain to unlock higher-level creativity. However, these methods rely on hand-crafted rules and domain-specific representations. In the above example, bespoke steps (parsing recipes into ingredient trees, using a specialized recombination algorithm, etc.) were tailored to the cooking domain. Applying this technique to a different domain (e.g., product design or story writing) would require designing new rules or structured schemas from scratch, limiting the scalability of structured prompting solutions. This highlights a need for a more general and automated creativity enhancement framework that does not depend on extensive prompt engineering or domain-specific heuristics. In this work, we propose a novel solution: a model-agnostic latent-space ideation framework that automatically explores variations of seed ideas in a continuous latent “idea space.” The key insight is that modern generative models implicitly organize concepts in a high-dimensional latent space, where semantic relationships and potential combinations are encoded as vector operations. By navigating this space, we can uncover imaginative combinations that would be hard to reach through direct prompting. The latent space connecting different concepts can serve as a wellspring of creativity: blending disparate ideas along latent dimensions produces new concepts that are both novel and coherent. By harnessing latent-space exploration, our method synthesizes high-quality ideas without extensive human prompt engineering, instead letting the model’s learned conceptual manifold drive the creative generation. Our framework is designed to act as a versatile “co-ideator” for humans, adapting to different domains and input formats with minimal effort. It exhibits strong adaptability and compositionality: the ideation process can begin from a short textual brief (e.g., a problem description or theme) or from a handful of seed ideas provided by the user. In either case, the system encodes the inputs and then recursively explores their latent neighbors to expand the idea space. This means the system can start with a simple concept and iteratively build more complex, unexpected ideas by combining and varying elements in latent space. However, our method shows, that textual outputs can be opitimized in semantic latent space, which can be generalized to various NLP related tasks. 2 Related Work Inspired by human brainstorming, recent work explores using multiple LLMs in tandem to enhance creativity. LLM Discussion (Lu et al., 2024) proposes a multi-agent discussion framework in which several LLM instances assume diverse roles and collectively tackle open-ended prompts Lu et al. 2024 . The approach employs a three-phase interaction (divergent idea generation, debate, and convergence) and a role-playing setup (e.g., one model acts as a “visionary” and another as a “devil’s advocate”) to mimic a group brainstorming session Lu et al. 2024 . This diversity of perspectives mitigates the homogeneous thinking of a single model and leads to more original responses. Empirical evaluations on classic divergent thinking tasks—e.g., Alternative Uses Tests and other creativity benchmarks—show that the role-play discussion framework produces significantly more original and elaborated answers than both single-LLM baselines and prior multi-LLM strategies Lu et al. 2024 . Notably, it outperformed simpler multi-LLM baselines like parallel brainstorming with voting or LLM debate on metrics such as originality and flexibility Lu et al. 2024 . These findings align with those of Zhao et al. (2025), who also observed that collaboration among multiple LLMs can boost creative output, with especially marked gains in originality (novelty) of ideas Zhao et al. 2025 . Multi-agent setups have also been used to evaluate creativity in interactive settings. Hejabi et al. (2024) introduce a simulation based on the Balderdash game to assess LLMs’ creative deception abilities Hejabi et al. 2024 . In this game, several LLM “players” invent plausible-sounding fake definitions for obscure terms to fool a guesser, while one provides the true definition. A referee LLM judges the similarity of definitions to detect the truth Hejabi et al. 2024 . Through this framework, the authors measure metrics like the True Definition Ratio (how often the model finds the real meaning) and Deception Ratio (how often others are misled by the fake definition) Hejabi et al. 2024 . The results yield insights into LLMs’ capacity for creative deception and strategy, highlighting strengths and failure modes (e.g., models struggled when confronted with very rare vocabulary). This multi-agent evaluation underscores that even when each model individually is a large pretrained LLM, engaging them in adversarial or cooperative roles can reveal and potentially enhance creative behaviors that would not manifest in a single-turn setting. Measuring “creativity”