Towards Physical Intuitions for Alignment Dynamics: A Case Study With Randomness Crystallization

Kunal Samanta, Ari Holtzman, Peter West · 2026

Read Towards Physical Intuitions for Alignment Dynamics: A Case Study With Randomness Crystallization by Kunal Samanta, Ari Holtzman, Peter West (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-29933.txt.

Introduction

Samanta, Holtzman, and West argue that alignment research should borrow ideas from physics, in particular phase transitions, to describe how language models change during post-training. Their case study uses crystallization, the freezing of a liquid into a crystal.

For tasks such as random number generation, they describe three phases. The pretrained model is a high-entropy liquid, with many sampling behaviours reachable by prompting. Supervised finetuning nucleates behaviour onto a single seed distribution already present in the model. Reinforcement learning then settles that collapsed distribution, shifting probability among mostly the same options. The authors propose simple metrics for spotting the transitions and validate the picture across a range of random tasks.

Catalog record

CREATORKunal Samanta, Ari Holtzman, Peter West
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.29933

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Towards Physical Intuitions for Alignment Dynamics: A Case Study With Randomness Crystallization Kunal Samanta, Ari Holtzman, Peter West Abstract The alignment of language models is typically studied through the lens of capability benchmarks, but the dynamics of how models change during post-training remain poorly understood. We argue that the physical sciences, and thermodynamic phase-transition theory in particular, offer a principled and underexplored vocabulary for reasoning about these dynamics. As a case study, we instantiate this position through the lens of material Crystallization , which is a well-studied thermodynamic phase transition. For tasks like random number generation, this breaks into 3 phases: (1) the high entropy liquid phase in the pretrained model, with many distinct sampling distributions promptable from the model; (2) the nucleation phase caused by supervised finetuning, in which behavior collapses onto a single seed distribution present in the pretrained LLM; and (3) a settling phase in which reinforcement learning techniques redistribute probability of the collapsed distribution, but largely keep it concentrated on the same options as the seed distribution. We propose intuitive metrics to verify the transitions between these phases, and validate the idea across a range of random tasks. Crystallization is one instance of a broader class of physical frameworks we believe alignment research should import to answer questions about where alignment-induced structure comes from, why it converges where it does, and what it fundamentally cannot change. 1 Introduction The field of NLP has made remarkable empirical progress in understanding how to align large language models to be useful, safe, and instruction-following ( 12 ; 2 ) . What remains considerably less understood is how these changes unfold — the structural processes by which a pretrained model transitions to an aligned one with near deterministic behavior, strong instruction following, and idiosyncratic tendencies. Where does the structure that alignment imposes come from? Why does convergence occur at the specific behavioral attractors it does, rather than others? The dominant vocabulary of alignment research is essentially taxonomic. They describe fruitful recipes and their end-states: aligned/unaligned, diverse/collapsed, capable/incapable, but lack a coherent framework for the transitions between them. The physical sciences have grappled with closely analogous questions for over a century, resulting in rich and potentially useful theories. Energy-based models and statistical mechanics have already proven their value in ML contexts ( 8 ) , and phase transition theory in particular offers a vocabulary precisely suited to the dynamics of complex systems moving between ordered and disordered states. Concepts like nucleation, supercooling, metastability, and tempering encode rich predictions about how transitions unfold, not just that they occur. This has already borne fruit in ML: grokking has been analyzed as a phase transition ( 10 ) and diffusion models grounded in non-equilibrium thermodynamics ( 4 ) . In each case, the physical lens revealed structure that purely empirical characterization had missed and helped point toward new interventions. Following in this tradition, we argue that physical theories should be actively imported to provide a rich, predictive framework for understanding the alignment process itself. Adopting such theories as a new mission for NLP would reorient the field from cataloguing outcomes to understanding processes; and crucially, from post-hoc description to a priori prediction of which behaviors alignment will amplify, which it will suppress, and why. Figure 1: Crystallization , illustrated on the digit task. Pretrained base LLMs exhibit a high-entropy liquid phase: varied prompts elicit varied output distributions, much like atoms in a liquid moving in many directions at once (left). Supervised fine-tuning (SFT) acts as a nucleation event: regardless of prompt variation, the model’s output distribution collapses rapidly onto a single latent seed distribution already present in the base model (center). This mirrors the behavior of nucleation, where diverse atoms suddenly align to a seed crystal. Subsequent preference optimization stages (DPO, RLHF) drive a settling process, sharpening and redistributing probability mass within the crystalline support already established by SFT (right). As a case study illustrated in figure 1 , we develop the crystallization framework as a prototypical example of what physically-grounded alignment theories can look like. Physical crystallization describes how a liquid (a high-entropy system with many degrees of freedom) undergoes a phase transition into a rigid crystal, a low-entropy structure organized around a seed. We show that post-training alignment follows precisely this pattern when examined on tasks with finite support , where we can compare model behavior/fingerprint on a limited and shared set of possible outputs. Given a stochastic task such as “ Give me a single random digit from 0 to 9 ”, the crystallization analogy can be understood as a transition over 3 phases (figure 1 ). The first is the liquid phase : the pre-trained (or base ) LLM harbors a superposition of latent distributions, each accessible through different prompts; a high entropy state whose diversity has received little systematic attention. The second is the nucleation phase : we find that SFT alone is sufficient to trigger a distributional collapse, with output distributions collapsing abruptly onto one seed distribution already latent in the base model, with prompt-sensitivity vanishing almost entirely, well before RL enters the picture. This snap-to-grid behavior mirrors nucleation in physical crystals, crystallizing around what already exists in the model. The third is the settling phase , driven by DPO and RL: probability mass concentrates further onto the peaks established at nucleation, without escaping the crystalline support SFT created, a pattern similar to tempering, and one that we predict will set firm limits on what later alignment stages can change. We develop this framework formally in Section 2 . We empirically test the crystallization framework across 15 stochastic tasks, finding a similar pattern over two different strong public models, OLMo 2 ( 11 ) and Tulu 3 ( 7 ) , for which checkpoints are available over the entire process. We also discover that seed distributions can be shared across different LLMs and further show where the framework’s current instantiation breaks down (large output spaces, open-ended generation) and argue these failures point to extensions: developing richer metrics and deeper formal connections with physics. We finish off by mapping out future physical analogies to be explored and their potential implications (§ 5 ). Crystallization is a proof of concept for a broader program. By showing that a well-characterized physical process maps onto alignment dynamics, we make the case that reducing aspects of LLM training to known phenomena from physics can act as a route to mathematical structure, testable predictions and ultimately to principled interventions. Importing physical reasoning into ML such as statistical mechanics into generalization or thermodynamics into diffusion has produced new descriptions and algorithms. Crystallization is a first step towards this direction for alignment research as a whole. (a) OLMo (digit) (b) Tulu (int 1-50) Figure 2: Across models and tasks, we see a huge variety of the base model distributions over prefixes. SFT latches onto a particular "seed" base distribution and DPO/Instruct concentrates probability mass within certain pre-existing peaks in the SFT nucleation. 2 Crystallization through Alignment While past works have noted mode collapse in aligned LLMs ( 18 ; 16 ) , the underlying dynamics of this process are not well understood. In this section, we work through an example of reasoning about alignment dynamics with

Read the complete text