From State Changes to Creative Decisions: Documenting and Interpreting Traces Across Creative Domains Conference: Workshop paper submitted to ’Herding CATs: Making Sense of Creative Activity Traces’ at the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Herding CATs: Making Sense of Creative Activity Traces Workshop of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain CCS: Human-centered computing Interactive systems and tools CCS: Applied computing Arts and humanities
Xiaohan Peng, Sotiris Piliouras, Carl Abou Saada Nujaim · 2026
Read From State Changes to Creative Decisions: Documenting and Interpreting Traces Across Creative Domains Conference: Workshop paper submitted to ’Herding CATs: Making Sense of Creative Activity Traces’ at the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Herding CATs: Making Sense of Creative Activity Traces Workshop of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain CCS: Human-centered computing Interactive systems and tools CCS: Applied computing Arts and humanities by Xiaohan Peng, Sotiris Piliouras, Carl Abou Saada Nujaim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-07184.txt.
Introduction
Peng and colleagues study how to record and read creative work traces without losing intent. Current tools log raw state changes but drop the intent and links that define higher-level creative moves. That loss looks different in each domain: generative AI tools lose non-linear exploration, visualization tools hide representational intent, and code tools flatten interaction bounds. The team offers three fixes. A node-based interface serves stateful generative artifacts, a vocabulary of visual cues captures visualization moves, and a programming model embeds semantic histories in interaction state.
Catalog record
| CREATOR | Xiaohan Peng, Sotiris Piliouras, Carl Abou Saada Nujaim |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 10969 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.07184
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From State Changes to Creative Decisions: Documenting and Interpreting Traces Across Creative Domains Conference: Workshop paper submitted to ’Herding CATs: Making Sense of Creative Activity Traces’ at the 2026 CHI Conference on Human Factors in Computing Systems; April 13–17, 2026; Barcelona, Spain Herding CATs: Making Sense of Creative Activity Traces Workshop of the 2026 CHI Conference on Human Factors in Computing Systems (CHI ’26), April 13–17, 2026, Barcelona, Spain CCS: Human-centered computing Interactive systems and tools CCS: Applied computing Arts and humanities Xiaohan Peng, Sotiris Piliouras, Carl Abou Saada Nujaim Abstract . Analyzing creative activity traces requires capturing activity at appropriate granularity and interpreting it in ways that reflect the structure of creative practice. However, existing approaches record state changes without preserving the intent or relationships that define higher-level creative moves. This decoupling manifests differently across domains: GenAI tools lose non-linear exploration structure, visualization authoring obscures representational intent, and programmatic environments flatten interaction boundaries. We present three complementary approaches: a node-based interface for stateful GenAI artifact management, a vocabulary of visual cues as higher-level creative moves in visualization authoring, and a programming model that embeds semantic histories directly into interaction state. † † cc-license: by 1. Introduction Analyzing creative activity traces requires capturing activity at appropriate levels of granularity and interpreting them in ways that reflect the structure of creative practice. Existing approaches such as version histories, interaction logs, or collaborative records tend to record state changes without preserving the higher-level intent or semantic context that give those changes meaning. Traces thus remain structurally decoupled from the creative decisions they result from across creative domains and tool paradigms: losing non-linear exploration structure in GenAI tools, obscuring representational intent in visualization authoring, and reducing meaningful actions to low-level operations in programming environments. In this paper, we present a multi-faceted approach to documenting and interpreting creative traces in GenAI, visualization, and programming contexts. First, we introduce a node-based interface design that treats generative artifacts as stateful, manipulable units, enabling designers to preserve alternative versions, branch explorations, and link semantic attributes to visual content. Second, we examine how 3D authoring traces such as material appearance, perspective, deformation, and dynamics can be mapped to higher-level “creative moves” that reflect representation and framing choices in expressive 3D visualizations. Finally, we propose a programming model that embeds semantic histories directly into interaction state, supporting both single-user and collaborative environments. 2. Documenting GenAI Creative Traces: Tracking Interactive Generative Visual Artifacts across Scales While prior creative history management systems introduce git-style version control, semantic grouping ( 2 ) , and node-based workflows ( 1 ) , they are primarily oriented toward capturing discrete, linear iterations and state transitions as the system evolves ( 11 ) , or often focus on relatively simple graphic elements ( 6 ) rather than higher-abstraction, context-dependent and hierarchically structured design assets. We argue for more visual, dynamically applicable and retrievable, and non-linear histories tailored to iterative, GenAI-supported design, particularly as GenAI images become a new class of design assets. Figure 1 . DesignTrace: a GenAI design tool for exploring alternatives and tracking design progress. Designers can extract or create semantic attributes then apply them to visuals, branch multiple alternatives under shared semantic categories, make localized edits inside individual canvas node while navigating editing history and different versions. 2.1. Stateful Nodes for Dynamic Visual Interaction and Documentation Existing node-based systems typically treat nodes as parametric operators in a data-flow pipeline. However, in DeisngTrace ( 4 ) , we explored nodes as interactive and stateful artifacts placed on a freeform canvas, each containing an editable image canvas and a local history of generation and editing actions. They can be independently resized, collapsed, duplicated, and spatially arranged to externalize design states as discrete, manipulable units. This structure enables designers to keep alternatives visible, compare them side by side, branch manually or intelligently with semantics, and reuse prior states without clutter or overlap. By linking semantic attributes to visual content and preserving branching histories, Nodes support non-linear exploration while documenting the evolving rationale behind design decisions. 2.2. Multi-Scale GenAI Traces Combine Commands, Artifact States, and Exploration Paths Current chat-based GenAI tools’ logs can grow long easily since they weigh minor edits and major design shifts equally, while canvas-based ones can get cluttered without proper design on the prompt-image relationship. DesignTrace supports multi-scale trace management. Each node maintains a local history of edits, while persistent, reusable nodes represent intermediate states that can be spatially arranged and revisited. By connecting these nodes into single or multiple exploration paths, the system provides an overview of design progression while preserving flexibility for branching and reuse. This structure enables designers to trace minor temporal-proximal changes, but also trace “design versions,” while keeping a broader overview of different exploration paths. 3. Visualizing Creative Traces: Representation and Framing Choices in Expressive 3D Visualization Authoring 3D creators iterate through intertwined decisions about geometry, materials, lighting, camera/viewpoint, and motion expressive in 3D visualization authoring. Yet creative activity traces are typically captured as sequences of parametric edits but not the higher-level representational intent behind those creative choices. Seemingly small edits can correspond to high-level representation and framing choices with substantial novelty, such as scattering objects in 3D space to communicate accumulation of microplastics in the ocean ( 8 ) or using particle motion simulations to convey air pollution ( 7 ) . Reconstructing creative intent from parametric edits is challenging because most 3D software is representation-agnostic, and logs lack the semantic and representational structure needed to relate actions to visualization goals, especially for unconventional representations. We propose that interpreting creativity traces in expressive 3D visualization can benefit from an intermediate vocabulary that translates tool actions into representation-level “creative moves”. Our ongoing work on physically-inspired visualization ( 5 ) provides a toolkit for this translation by describing (i) the visual elements that compose a scene, (ii) the physically-inspired properties being manipulated, and (iii) their semantic relationship to the data variables and theme (such as literal, iconic, or symbolic relationships). In this workshop, we want to explore how this structured design space of material-based visual representation choices can be used to make sense of creative activity traces in expressive 3D visualization authoring, supporting the analysis of 3D exploration trajectories and informing future creativity-support tools. 4. Programming Creative Traces: Embedding Semantic Histories in Interaction State Digital creative activity traces—ranging from changing the color of a shape to generating a new design alternative in a GenAI tool—remain difficult to analyze because they are typically captured as external logs rather than produced by the system’s interaction model. Even in collaborative environments based on Conflict-Free Replicated Data Types ( 9 ) (CRDTs), histories record operational changes but not the structure of user activity, requiring developers to manually reconstruct intent by grouping operations into higher-level transactions. Inspired by Object-Oriented Drawing ( 10 ) , we introduce a model for embedding histories directly in application state and take a first step toward representing semantics in both single-user and collaborative environments. Figure 2 . (1) A reactive signal records value changes in a persistent history. (2) A semantic action groups a segment of history changes recorded between the start and end of an interaction, such as a drag. Modern front-end frameworks increasingly rely on reactive signals