Grokking in Linear Models for Logistic Regression Nataraj Das, Atreya Vedantam, Chandrasekhar Lakshminarayanan Abstract Grokking, the phenomenon of delayed generalization, is often attributed to the depth and compositional structure of deep neural networks. We study grokking in one of the simplest possible settings: the learning of a linear model with logistic loss for binary classification on data that are linearly (and max margin) separable about the origin. We investigate three testing regimes: (1) test data drawn from the same distribution as the training data, in which case grokking is not observed; (2) test data concentrated around the margin, in which case grokking is observed; and (3) adversarial test data generated via projected gradient descent (PGD) attacks, in which case grokking is also observed. We theoretically show that the implicit bias of gradient descent induces a three-phase learning process—population-dominated, support-vector-dominated unlearning, and support-vector-dominated generalization—during which delayed generalization can arise. Our analysis further relates the emergence of grokking to asymmetries in the data, both in the number of examples per class and in the distribution of support vectors across classes, and yields a characterization of the grokking time. We experimentally validate our theory by planting different distributions of population points and support vectors, and by analyzing accuracy curves and hyperplane dynamics. Overall, our results demonstrate that grokking does not require depth or representation learning, and can emerge even in linear models through the dynamics of the bias term. † † affiliationnotice: Equal contribution 1 Introduction One of the most striking properties of modern machine learning systems is their ability to generalize from finite training data ( Neyshabur et al. 2017 ; Zhang et al. 2017 ; Papyan et al. 2020 ; Ma et al. 2018 ; Kaplan et al. 2020 ) . Recent work has revealed a counterintuitive phenomenon in which models achieve near-perfect performance on the training set long before exhibiting meaningful improvements on test data. This delayed generalization behavior is now commonly referred to as grokking ( Junior et al. 2025 ) . Despite growing empirical and theoretical evidence across architectures and tasks ( Tan & Huang 2024 ; Fan et al. 2024 ; Mohamadi et al. 2024 ) , a complete principled theoretical understanding of grokking remains incomplete. Grokking was first reported by ( Power et al. 2022 ) in Transformer models trained on modular arithmetic tasks. Subsequent work has documented grokking across a wide range of architectures, datasets, and optimization regimes, and has proposed various mechanistic explanations ( Nanda et al. 2023 ; Lyu et al. 2024 ) . Recently, there has been increasing interest in understanding grokking ( Davies et al. 2023 ; Doshi et al. 2024 ; DeMoss et al. 2025 ) through analytically tractable models that isolate the essential ingredients responsible for delayed generalization. In this work, we study grokking in one of the simplest possible settings for binary classification: a linear model trained with logistic loss on data that are linearly separable (and max margin) through the origin. While the data admit a separating hyperplane with zero bias, the model we train includes a learnable bias parameter. This seemingly mild mismatch turns out to be sufficient to induce rich training dynamics and, under appropriate test distributions, delayed generalization. These results indicate that delayed generalization need not rely on depth or representation learning, and can already arise in simple linear models through optimization dynamics alone. A key insight of our analysis is that grokking in this setting is driven primarily by the dynamics of the bias term, rather than by continued evolution of the weight vector. While the weight follows the classical max-margin trajectory and stabilizes in direction early in training, the bias undergoes a slower, support-vector-driven evolution that governs delayed generalization. Our contributions can be summarized as follows. 1. Three-phase learning dynamics. We show in Theorem 4.5 that, in our setting, the weight vector follows the same asymptotic trajectory characterized by Soudry et al. 2024 : its norm grows as \log t while its direction converges to the maximum-margin separator. We refer to this regime as Phase 1 . We then characterize the evolution of the bias term after Phase 1 in Theorem 4.5 . Using this characterization, we show that learning proceeds through two additional phases: in Phase 2 , the bias deviates away from the generalizing solution, and in Phase 3 , it recovers and converges back. We show that when grokking occurs, it arises during Phases 2 and 3. 2. Factors influencing grokking. While Phase 1 is driven by the entire training dataset, Phases 2 and 3 are governed solely by the support vectors. Using Theorem 4.5 , we show that asymmetries in the data—both class imbalance in the population and imbalance among the support vectors—directly influence whether grokking occurs and shape the resulting dynamics. 3. Grokking time. We formally define grokking in Definition 3.4 . In Theorem 4.6 , we provide a characterization of the grokking time, quantifying the delay between the attainment of near-perfect training accuracy and near-optimal test accuracy, and explicitly relate it to properties of the support vectors and the test distribution. 4. Numerical experiments. We validate our theoretical predictions empirically. For clarity of exposition, we consider three types of test data: (i) standard , where the test distribution matches the training distribution; (ii) concentrated , where the data are separable with the same margin as the training data but concentrated near the decision boundary; and (iii) adversarial , generated using projected gradient descent attacks. We do not observe grokking in the standard setting, but observe clear grokking behavior in both the concentrated and adversarial settings. Moreover, whenever grokking occurs, the observed dynamics align closely with the predictions of our theory. 2 Related Work The phenomenon of grokking has attracted considerable attention since its discovery, yet prior work has largely remained empirical or confined to specific architectural assumptions. Our work provides an understanding of grokking in a single learnable hyperplane on linearly separable data. When ( Power et al. 2022 ) introduced grokking, they observed that optimization required for generalization increases as dataset size decreases but provided no theoretical justification. We derive closed-form expressions for grokking time as a function of the support vector imbalance and the sensitivity of the concentration around the margin (Theorem ( 4.6 )). ( Liu et al. 2022 ) empirically linked generalization to structured embeddings and identified a critical training size beyond which grokking vanishes, while ( Gromov 2023 ) made similar observations. We find a connection between the number of support vectors and learning in Sec. ( 5 ). Several studies have proposed mechanisms for grokking: ( Thilak et al. 2022 ) suggested a ‘slingshot’ mechanism, ( Liu et al. 2023 ) proposed an LU mechanism involving a “goldilocks zone” in weight space, and ( Kumar et al. 2024 ) linked grokking to transitions from lazy to rich training dynamics. Our work shows that grokking in our setting arises from implicit bias in gradient descent, manifesting as three distinct learning phases—a population dominated phase, a support vector dominated unlearning phase and a support vector dominated generalization phase (Theorem 4.5 ). Recent efforts have analyzed tractable models: ( Levi et al. 2023 ) studied linear regression, ( Beck et al. 2025 ) attributed grokking to data dimensionality, and ( Žunkovič & Ilievski 2022 ) calculated grokking probabilities for specific datasets. The connection between double descent ( Nakkiran et al. 2019 ) and grokking has been explored ( Davies et al. 2023 ) , but without theoretical justification. Theorem 4.5 explains why we should expect this behavior in our setting. ( Humayun et al. 2024 ) empirically studied adversarial robustness in deep networks. We observe empirically adversarial robustness even in our scenario, asserting that this phenomenon is not restricted to deep networks. Our theoretical machinery builds upon the late-time gradient dynamics studied by ( Ji & Telgarsky 2019 ) and ( Soudry et al. 2024 ) . From asymptotic behavior, we study finite time gradient dynamics in different phases. 3 Model Setup and Notation We consider a linearly separable training dataset \mathcal{D}=\{(\mathbf{x}_{1},y_{1}),(\mathbf{x}_{2},y_{2}),\ldots,(\mathbf{x}_{N},y_{N})\} where features x_{i}\in\mathbb{R}^{d} and labels y_{i}\in\{-1,1\} and datapoint (x_{i},y_{i})\sim\mathcal{P} . The data has a separating hyperplane that passes through the origin. Moreover, the dataset has a margin of separation \gamma>0 , meaning that all points lie at least \gamma/2>0 on either side of this separating hyperplane. Formally, Assumption 3.1 (Separability and margin) . The data is linearly separable about the origin and the maximum margin SVM solution passes through the origin. That is, \exists\mathbf{w}_{*}\in\mathbb{R}^{d} such that ||\mathbf{w}_{*}||=1 and \forall\mathbf{x}_{i},y_{i}\mathbf{w}_{*}^{\top}\mathbf{x}_{i}\geq\gamma/2 . Also, for any \mathbf{w}_{s}\in\mathbb{R}^{d} and b_{s}\in\mathbb{R} such that \forall\mathbf{x}_{i},y_{i}(\mathbf{w}_{s}^{\top}\mathbf{x}_{i}+b_{s})\geq 1 and ||\mathbf{w}_{s}|| is minimum, b_{s}=0 . Assumption 3.2 . Let \mathcal{S}=\{\mathbf{x}_{s_{1}},\mathbf{x}_{s_{2}},\ldots,\mathbf{x}_{s_{k}}\}\subset\{\mathbf{x}_{1},\ldots,\mathbf{x}_{N}\} be the support vectors such that \mathbf{w}_{*}=\sum_{i=1}^{k}\alpha_{i}y_{s_{i}}\mathbf{x}_{s_{i}} . Finally, we assume that the distribution has finite support. Assumption 3.3 (Finite Support) . There exists a fixed R>0 such that ||\mathbf{x}_{i}||0 define T_{tr}(\epsilon) and T_{te}(\epsilon) to be the time taken for the train and test accuracies to reach a (1-\epsilon) fraction of their suprema. \displaystyle T_{tr}(\epsilon)=P^{-1}((1-\epsilon)\sup_{t}P(t)) (1) \displaystyle T_{te}(\epsilon)=Q^{-1}((1-\epsilon)\sup_{t}Q(t)) (2) whenever the inverses exist and the least of all the pre-images otherwise. Then the model is said to grok with a delay \zeta>0 if \exists\epsilon_{0}>0 such that \forall 0<\epsilon<\epsilon_{0} , T_{te}(\epsilon)>\zeta\cdot T_{tr}(\epsilon) . The grokking time is defined as T_{gr}(\epsilon)=T_{te}(\epsilon)-T_{tr}(\epsilon) . Remark 1: To the best of our knowledge, there is no single definition of grokking in the literature. However, in the light of our definition for grokking ( Levi et al. 2023 ; Manning-Coe et al. 2025 ) can be said to use \epsilon=0.05 and ( Žunkovič & Ilievski 2022 ) use \epsilon=0 . Remark 2: Similarly, the usage of the term ‘delay’ is very subjective. ( Power et al. 2022 ) observed \zeta=1000 , ( Gromov 2023 ) observed \zeta=4 , ( Liu et al. 2022 ) observed grokking on MNIST with \zeta=100 and ( Thilak et al. 2022 ) observed \zeta=300 . The above definition encompasses all observations and makes precise our intended ‘delay’ threshold. In this paper, unless \zeta is of the order of 100 or more we do not consider the training dynamics as representing grokking. Figure 1 : Grokking with a learnable bias . We analyze grokking in three scenarios: (1) when the test data is drawn from the same distribution as the test data (right); (2) when the test data is distributed around the optimal separator (sensitive distribution) (left); (3) adversarial test data generated via Project Gradient Descent attacks (center). We see that the model groks on the concentrated and adversarial test data whereas this is not observed in the standard test data. Model: We consider a linear model with bias whose prediction for input \mathbf{x}_{i}\in\mathbb{R}^{d} is given by \hat{y}(\mathbf{x}_{i};w,b)=w^{\top}\mathbf{x}_{i}+b , where w\in\mathbb{R}^{d} and b\in\mathbb{R} are the weight vector and bias respectively. Loss: We are interested in minimizing the following loss function \displaystyle\mathcal{L}(\mathbf{w},b)=\sum_{i=1}^{N}l(y_{i}(\mathbf{w}^{\top}\mathbf{x}_{i}+b)) (3) where l(x)=\log(1+\exp({-x})) is the logistic loss. Gradient Update: We use gradient descent to optimize this loss using \displaystyle\mathbf{w}(t+1)=\mathbf{w}(t)-\eta\nabla_{\mathbf{w}}\mathcal{L}(\mathbf{w},b) (4) \displaystyle b(t+1)=b(t)-\eta\nabla_{b}\mathcal{L}(\mathbf{w},b). (5) 4 Model Bias Learns in Three Phases We begin by recalling a key aspect of our setup: while the training data are linearly separable through the origin and hence admit a separator with zero bias, the model we train includes a learnable bias parameter. When optimizing the logistic loss with gradient descent under this mismatch, several natural and interrelated questions arise about the resulting training dynamics. First, how does the weight vector \mathbf{w}(t) evolve over time? This question has been thoroughly analyzed by Soudry et al. 2024 , and we briefly recall the relevant results in Section ( 4.1 ). Second, how does the bias term b(t) evolve under gradient descent? Our main theoretical contribution answers this question; we characterize the dynamics of b(t) and its asymptotic behavior in Section ( 4.2 ), culminating in Theorem ( 4.5 ). Third, how does the interaction between the dynamics of \mathbf{w}(t) and b(t) influence generalization and give rise to grokking? In Sections ( 4.3 ), ( 4.4 ) and ( 4.5 ), we address this question by arguing—using Theorem ( 4.5 )—that learning proceeds through three distinct phases: an initial population-dominated phase, a support-vector-dominated unlearning phase, and a final phase in which both \mathbf{w}(t) and b(t) converge toward the generalizing solution. Finally, how long does this process take? In particular, what determines the grokking time, defined as the delay between the attainment of near-perfect training accuracy and near-optimal test accuracy? We show that grokking time is governed by the late-time dynamics of the bias term during the support-vector-dominated phases, and we make this dependence explicit in Section ( 4.6 ). 4.1 Prior Results from Soudry et al. 2024 Theorem 4.1 (Rephrased from Theorem 3 of ( Soudry et al. 2024 ) ) . For any smooth monotonically decreasing loss function with an exponential tail, and for small learning rate, gradient descent iterates follow \displaystyle\mathbf{w}(t)=\hat{\mathbf{w}}\log(t)+\rho(t) (6) where \hat{w} is given by \displaystyle\hat{\mathbf{w}}=\arg\min||\mathbf{w}||^{2}+b^{2}\text{ s.t. }y_{i}(\mathbf{w}^{\top}\mathbf{x}+b)\geq 1 (7) and \rho(t) has a bounded norm. We would like to emphasize that this is not the hard margin SVM solution \mathbf{w}_{SVM} in general. However with assumption ( 3.1 ), they are equal. To analyze the training dynamics and show grokking, we first describe the convergence of \rho(t) . Theorem 4.2 (Rephrased from Theorem 4 of ( Soudry et al. 2024 ) ) . If the support vectors span the dataset, and \hat{\mathbf{w}}=\sum_{\mathcal{S}}\alpha_{i}y_{s_{i}}\mathbf{x}_{s_{i}} then \lim_{t\to\infty}\rho(t)=\tilde{\mathbf{w}} where \displaystyle\eta\exp(-\x_{s_i}^\top\tilde{\w})=\alpha_{i} (8) 4.2 Our Theoretical Result On Learning Phases Theorem 4.3 . Under assumptions ( 3.1 ) and ( 3.2 ), the SVM solution given by \displaystyle\mathbf{w}_{SVM}=\arg\min||\mathbf{w}||^{2}\text{ s.t. }y_{i}(\mathbf{w}^{\top}\mathbf{x}+b)\geq 1 (9) is equal to \hat{\mathbf{w}} . Proof. From ( 3.1 ) and ( 3.2 ), the solution to ( 9 ) is some (\mathbf{w}_{SVM},0) . Hence for all feasible (\mathbf{w},b) from the optimality of (\mathbf{w}_{SVM},0) , we have ||\mathbf{w}||^{2}\geq||\mathbf{w}_{SVM}||^{2} . Note that both problems ( 7 ) and ( 9 ) have the same feasible region. Hence for all feasible (\mathbf{w},b) for ( 7 ), we also have ||\mathbf{w}||^{2}+b^{2}\geq||\mathbf{w}||^{2}\geq||\mathbf{w}_{SVM}||^{2}+0^{2} implying that (\mathbf{w}_{SVM},0) also optimizes ( 7 ). Since ( 7 ) is a strictly convex optimization problem it has a unique solution. Hence \hat{\mathbf{w}}=\mathbf{w}_{SVM} . ∎ As discussed above, the learning dynamics can be naturally divided into three phases. Phase 1 is dictated by the entire training dataset and is fully characterized by the results of Soudry et al. 2024 , which we recalled in Section ( 4.1 ). Phases 2 and 3 occur after the convergence of the residual term \rho(t) to its limiting value \tilde{\mathbf{w}} . While in reality the dynamics evolve continuously and the transitions between these phases are gradual, for pedagogical clarity we idealize the process by assuming the existence of a finite time t_{0} that marks the end of Phase 1. This idealization is captured in the following assumption. Assumption 4.4 (Late-time dynamics) . There exists a time t_{0} such that the following hold: 1. \rho(t)=\tilde{\mathbf{w}} for all t\geq t_{0} . 2. For all \mathbf{x}_{i}\notin\mathcal{S} , we have \exp\!\big(-\mathbf{w}(t)^{\top}\mathbf{x}_{i}\big)=0\qquad\text{for all }t\geq t_{0}, where \mathcal{S} denotes the set of support vectors. 3. The weight vector has converged in direction: \frac{\mathbf{w}(t)}{||\mathbf{w}(t)||}=\frac{\hat{\mathbf{w}}}{||\hat{\mathbf{w}}||}. Theorem 4.5 . Suppose \mathcal{S}^{+} and \mathcal{S}^{-} represent all the support vectors which belong to class 1 and -1 respectively. Suppose we use the exponential loss l(x)=\exp({-x}) . Define A_{S}^{+}=\sum_{i\in S^{+}}\exp({-\tilde{\w}^{\top}\x_i}) and A_{S}^{-}=\sum_{i\in S^{-}}\exp({\tilde{\w}^{\top}\x_i}) where \tilde{\mathbf{w}} is defined above. Define \delta=\sqrt{\frac{A_{S}^{-}}{A_{S}^{+}}} and g(t) as \displaystyle g(t)=2\sqrt{A_{S}^{+}A_{S}^{-}}\log\left(\frac{t}{t_{0}}\right) where t_{0} is the time at which the late time dynamics begin. In this regime, we assume that \rho(t) has converged to \tilde{\mathbf{w}} . Further, only support vectors contribute to the gradient. Then the late time gradient flow dynamics of the bias follows \displaystyle\derivative{b}{t}=\frac{1}{t}\left(A_{S}^{+}\exp({-b})-A_{S}^{-}\exp({b})\right). (10) Integrating, b(t) evolves as \displaystyle b(t)=\log\!\left(\frac{(1+\delta e^{b_{0}})\exp({g(t)})-(1-\delta e^{b_{0}})}{(1+\delta e^{b_{0}})\exp({g(t)})+(1-\delta e^{b_{0}})}\right)-\log\delta. (11) Consequently, b(t) converges to b_{\infty}=-\log\delta . Proof sketch: . We now provide a sketch of the proof (the complete proof can be found in the Theorem A.1 ). The exponential loss \mathcal{L}(\mathbf{w},b)=\sum_{i}\exp({-y_i(\w^\top\x_i + b)}) approximates the logistic loss in the late time dynamics. Then gradient descent ( 4 ) takes the form \displaystyle b(t+1)=b(t)+\eta\sum_{i}y_{i}e^{-y_{i}(\mathbf{w}(t)^{\top}\mathbf{x}_{i}+b)} For small enough learning rates, this approaches gradient flow given by \displaystyle\derivative{b}{t} \displaystyle=\sum_{i}y_{i}e^{-y_{i}(\mathbf{w}(t)^{\top}\mathbf{x}_{i}+b)} (12) \displaystyle=\sum_{y_{i}=+1}e^{-(\mathbf{w}(t)^{\top}\mathbf{x}_{i}+b)}-\sum_{y_{i}=-1}e^{(\mathbf{w}(t)^{\top}\mathbf{x}_{i}+b)} (13) \displaystyle=\left(\sum_{y_{i}=+1}e^{-\mathbf{w}(t)^{\top}\mathbf{x}_{i}}\right)e^{-b}-\left(\sum_{y_{i}=-1}e^{\mathbf{w}(t)^{\top}\mathbf{x}_{i}}\right)e^{b} (14) For late time dynamics, \rho(t) has converged to \tilde{\mathbf{w}} . We also have \mathbf{w}(t)=\hat{\mathbf{w}}\log t+\rho(t) . Substituting, we get equation ( 10 ) (since only support vectors contribute). Integrating ( 10 ) gives us ( 11 ). ∎ Observation: Equation 14 carries rich meaning. Define A^{+}=\sum_{y_{i}=+1}e^{-\mathbf{w}(t)^{\top}\mathbf{x}_{i}} and A^{-}=\sum_{y_{i}=-1}e^{\mathbf{w}(t)^{\top}\mathbf{x}_{i}} . For a dataset with balanced classes, A^{+}\approx A^{-} . This is because the training population is symmetric on an average and adding the cost e^{\pm\mathbf{w}(t)^{\top}\mathbf{x}_{i}} over each class for every data point will yield similar values. Following a similar argument, for a dataset with class imbalance towards +1 ( -1 ), A^{+}>A^{-} ( A^{+}>A^{-} ). Equipped with the above observation we now discuss the Phases 1, 2 and 3. 4.3 Phase 1: Population dominated In the early dynamics of ( 14 ), A^{+} and A^{-} change with time as \mathbf{w} evolves. We also observe from Theorem ( 6 ) that since ||\mathbf{w}|| evolves approximately as \mathcal{O}(\log t) , and both A^{+} and A^{-} depend on (largely positive) inner products with \mathbf{w} , on an average both decrease with time. Now for different class sizes in the dataset, we have different possible evolutions of the early time dynamics. 1. Classes balanced : In this case, A^{+}\approx A^{-} . Hence, on an average \derivative{b}{t}\propto e^{-b}-e^{b} . This quickly brings b to 0 regardless of where it is initalized. Hence throughout the early time dynamics, b remains near zero. 2. Class imbalance towards +1 : In this case A^{+}>A^{-} . Now the early time dynamics no longer force b to 0 . Instead, the steady state solution is b=0.5\log(A^+/A^-)>0 . Hence regardless of where it is initalized, the bias is pulled to this value. 3. Class imbalance towards -1 : In this case A^{+}A_{S}^{-} . The late time dynamics therefore force b to settle at b_{\infty}=0.5\log(A_S^+/A_S^-)>0 regardless of the early time dynamics. Note that unlike the steady state achieved in the early time dynamics, the bias need not necessarily reach this value (the gradient could vanish before then). 3. Asymmetrically more -1 class support vectors : In this case, A_{S}^{+}>T_{tr}(\epsilon) . This implies that T_{gr}(\epsilon)\approx T_{te}(\epsilon) . To demonstrate the validity of the above theorem we use the dataset with balanced classes and different values of \log\delta . We control this by planting a different ratio of support vectors in each class which is a measure of \delta . The plots are shown in Figure ( 2 ). The implementation details can be found in Appendix. • We observe that a higher imbalance in the support vectors (which corresponds to a higher \delta ) causes the bias to stagnate at a higher value as theoretically predicted. • This causes the hyperplane to move further from the origin before learning the generalizing solution. This increases grokking time as theoretically predicted. Therefore we theoretically show that grokking time depends on the support vector imbalance (related to \delta ) and the sensitivity parameter ( \alpha ). 5 Dataset Choice and Experimental Results The passage of the learning process through the two phases and regimes takes time. This is why generalization is delayed in grokking. Now we are interested in finding the dependency of grokking time on the structure of the data. To this end we consider specific distributions \mathcal{P} and \mathcal{P}_{conc} , empirically observe the accuracy curves and theoretically derive an expression for grokking time. Figure 3 : Dataset . Chosen training and test distribution sampled \mathcal{P} and \mathcal{P}_{conc} in d=2 dimension (exaggerated for visibility). Since \mathcal{P}_{conc} is concentrated around the seperator, it is sensitive to poorly generalizing solutions. 5.1 Dataset distribution details • Test distribution \mathcal{P} [Standard Test]: Samples \mathbf{x}\sim\mathcal{P} are drawn uniformly from the d -dimensional hypercube [-5,5]^{d} . The separating hyperplane is defined by \mathbf{w}_{\text{true}}^{\top}\mathbf{x}=0 where \mathbf{w}_{\text{true}}=\frac{1}{\sqrt{n}}[1,\ldots,1]^{\top}\in\mathbb{R}^{d}. The dataset satisfies a margin condition with margin \gamma=10^{-3} . Labels are assigned as y=\begin{cases}+1,&\text{if }\mathbf{w}_{\text{true}}^{\top}\mathbf{x}\geq\gamma/2,\\ -1,&\text{if }\mathbf{w}_{\text{true}}^{\top}\mathbf{x}\leq-\gamma/2.\end{cases} • Sensitive distribution \mathcal{P}_{conc} [Concentrated Test]: The distribution \mathcal{P}_{conc} concentrates mass near the decision boundary. Samples satisfy \gamma/2\leq\mathbf{w}_{\text{true}}^{\top}\mathbf{x}\leq\alpha\gamma\quad\text{(class }+1), -\alpha\gamma\leq\mathbf{w}_{\text{true}}^{\top}\mathbf{x}\leq-\gamma/2\quad\text{(class }-1), where \alpha controls the severity of the distribution shift and is set to \alpha=10 . We refer to \alpha as the sensitivity parameter . • Adversarial distribution \mathcal{P}_{\text{PGD}} : Starting from samples drawn from the base distribution \mathcal{P} , we generate adversarial examples using a projected gradient descent (PGD) attack. Each sample is perturbed within a bounded \ell_{p} -ball around the original input while preserving its label, resulting in a distribution \mathcal{P}_{\text{PGD}} that captures adversarially shifted data. We conduct studies on these distributions as the sources for our train and concentrated test sets for d=2 (small dimension, easy to visualize hyperplane movement) and d=1024 (large dimension, asserts usage of same concepts in high dimensional models). Figure 1 presents the simulation results for the above setup. Figure 4 : Evolution of (a) the bias b(t) (left), (b) the distance of the separating hyperplane from the origin |b(t)|/\|\mathbf{w}(t)\| (middle), and (c) training and concentrated test accuracies (right), as functions of training epochs, for varying degrees of positive and negative class imbalance,where D(1000,1000) mean 1000 positive class points and 1000 negative class points. Similarly S(350,350) means 350 support vectors for positive class and 350 support vectors for negative class.The three shaded regions refer to phase 1, phase 2, phase 3 respectively in all the plots. 5.2 Analysis of results d Test \epsilon_{0} T_{tr} T_{te} \zeta 2 Std. 0 5\times 10^{5} 2\times 10^{6} 4 2 Conc. 0 4\times 10^{2} 7\times 10^{6} 1.75\times 10^{4} 2 Conc. 0.001 \leq 10 \sim 10^{6} \geq 1.75\times 10^{4} 1024 Std. 0 10^{6} 10^{6} 1 1024 Conc. 0 10^{2} 5\times 10^{6} 5\times 10^{4} 1024 Conc. 0.001 10 4\times 10^{6} \geq 5\times 10^{4} Table 1 : Training and test convergence times under standard and concentrated test distributions. No grokking is observed for the vanilla test set, where training and test accuracies track closely. In contrast, the concentrated (sensitive) test distribution exhibits persistent grokking with large delay \zeta , independent of dimension. For the standard test distribution (right panel), we do not observe grokking. In contrast, for the concentrated test distribution (left panel), we observe clear grokking. This is true for both n=2 and n=1024 . See Table ( 1 ) for the results. The learning also follows the three phases as predicted: Phase 1 ( 10^{2} – 5\times 10^{3} epochs), Phase 2 ( 5\times 10^{3} – 5\times 10^{4} epochs) and Phase 3 ( 5\times 10^{4} – 10^{7} epochs). This non-monotonic test behavior is reminiscent of double descent ( Davies et al. 2023 ) , and our theory explains its origin for linear classifiers. However, the three learning phases are less distinct in high dimensions. This is because A_{S}^{+} and A_{S}^{-} , which depend on inner products involving \mathbf{x}_{i} , become smaller, reducing the magnitude of g(t) and consequently the variation in b(t) . Experimental validation of Theorem ( 4.5 ). Using the same setup, we simulate cases with population class imbalance and support vector imbalance. Figure ( 4 ) reports b(t) , |b(t)|/\|\mathbf{w}(t)\| , and train and concentrated test accuracies. Implementation details are provided in Appendix ( B ). • Balanced classes : The bias remains near zero in early dynamics (up to 10^{2} epochs) and drifts in late dynamics when support vectors are imbalanced. The distance from the origin first increases (unlearning, up to 2\times 10^{3} epochs) and then decreases (generalization), after which test accuracy improves. • Imbalance toward +1 : The bias drifts upward in early dynamics and then diverges in late dynamics under support vector imbalance. The distance from the origin peaks around 7\times 10^{2} epochs before decreasing, again leading to generalization. • Imbalance toward -1 : This case is symmetric to the previous one. 6 Grokking with PGD attacks We generate adversarial test samples after each training update by using PGD attacks on the test data ( Madry et al. 2019 ) . We monitor this accuracy and empirically observe grokking in this setting as well. Figure ( 1 ) shows that the model groks adversarial test data with and without a learnable bias in both low and high dimensional data. Implementation details are provided in Appendix ( C ). We observe empirically that the beginning of the generalization phase in the model with a learnable bias is coincident with the grokking of the adversarial test samples. This shows that ‘delayed robustness’, a type of grokking in deep networks first introduced by ( Humayun et al. 2024 ) is also prevalent in the learning of a single hyperplane. 7 Conclusion In this paper, we explored grokking in logistic regression trained with gradient descent on linearly separable data. Our main theoretical result (Theorem ( 4.5 )) shows that the model bias proceeds through three distinct phases: an early population-dominated phase, a support-vector-dominated unlearning phase, and a late-time generalization phase. For standard test distributions, no grokking is observed (Figure ( 1 ) and Table ( 1 )). In contrast, for concentrated or margin-sensitive test distributions, test performance depends critically on the bias dynamics. In this regime, the late-time dynamics are entirely controlled by the support vectors, and we make this dependence explicit by relating the grokking delay to support-vector (Figure ( 4 )). The resulting delay between training and test convergence is quantified explicitly by the grokking time bound in Theorem ( 4.6 ) (Figure ( 2 )). Taken together, our results demonstrate that grokking can arise in the simplest linear models, without representation learning or overparameterization. The phenomenon is fully explained by implicit bias and slow optimization dynamics, and persists under adversarial and concentrated testing. Impact Statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here. Appendix A Proofs of Theorems Theorem A.1 . Suppose \mathcal{S}^{+} and \mathcal{S}^{-} represent all the support vectors which belong to class 1 and -1 respectively. Define A_{S}^{+}=\sum_{i\in S^{+}}e^{-\tilde{\mathbf{w}}^{\top}\mathbf{x}_{i}} and A_{S}^{-}=\sum_{i\in S^{-}}e^{\tilde{\mathbf{w}}^{\top}\mathbf{x}_{i}} where \tilde{\mathbf{w}} is defined above. Define \delta=\sqrt{\frac{A_{S}^{-}}{A_{S}^{+}}} and g(t) as \displaystyle g(t)=2\sqrt{A_{S}^{+}A_{S}^{-}}\log\left(\frac{t}{t_{0}}\right) where t_{0} is a constant. Then the late time gradient flow dynamics of the bias follows \displaystyle\derivative{b}{t}=\frac{1}{t}\left(A_{S}^{+}e^{-b}-A_{S}^{-}e^{b}\right). (17) Integrating, b(t) evolves as \displaystyle b(t)=\log\left(\frac{(1+\delta\exp(b_{0}))\exp(g(t))-(1-\delta\exp(b_{0}))}{(1+\delta\exp(b_{0}))\exp(g(t))+(1-\delta\exp(b_{0}))}\right)-\log\delta. (18) Consequently, b(t) converges to b_{\infty}=-\log\delta . Proof. From ( 6 ) we have \displaystyle\mathbf{w}(t)=\hat{\mathbf{w}}\log t+\rho(t) and from ( 4.4 ), \exists t_{0} such that \rho(t)\ =\tilde{\mathbf{w}} for all t\geq t_{0} . In the rest of the proof we write \mathbf{w}(t) as \mathbf{w} and b(t) as b to reduce clutter. When the time dependence is important to stress, we revert to showing the explicit time dependence. Gradient flow on the bias term gives us \displaystyle\Rightarrow\derivative{b}{t}=-\nabla_{b}\sum_{i}e^{-(\mathbf{w}^{\top}\mathbf{x}_{i}+b)y_{i}} \displaystyle\Rightarrow\derivative{b}{t}=-\sum_{i}e^{-(\mathbf{w}^{\top}\mathbf{x}_{i}+b)y_{i}}(-y_{i})=\sum_{i}y_{i}e^{-y_{i}(\mathbf{w}^{\top}\mathbf{x}_{i}+b)}=\sum_{y_{i}=+1}e^{-(\mathbf{w}^{\top}\mathbf{x}_{i}+b)}-\sum_{y_{i}=-1}e^{(\mathbf{w}^{\top}\mathbf{x}_{i}+b)}. Substituting for \mathbf{w} from ( 6 ), we have \displaystyle\derivative{b}{t}=\sum_{i\in I^{+}}e^{-\hat{\mathbf{w}}^{\top}\mathbf{x}_{i}\log t}\,e^{-\rho(t)^{\top}\mathbf{x}_{i}}\,e^{-b}\;-\;\sum_{i\in I^{-}}e^{\hat{\mathbf{w}}^{\top}\mathbf{x}_{i}\log t}\,e^{\rho(t)^{\top}\mathbf{x}_{i}}\,e^{b}. We now analyze the differential equation in the late time dynamics. In this regime, by assumption ( 4.4 ), only the support vectors contribute for t\geq t_{0} . \displaystyle\derivative{b}{t}=\sum_{i\in S^{+}}e^{-\log t}\,e^{-\rho(t)^{\top}\mathbf{x}_{i}}\,e^{-b}\;-\;\sum_{i\in S^{-}}e^{-\log t}\,e^{\rho(t)^{\top}\mathbf{x}_{i}}\,e^{b},\qquad\text{since }\hat{\mathbf{w}}^{\top}\mathbf{x}_{i}=1. We also have \rho(t)=\tilde{\mathbf{w}} . Therefore \displaystyle\derivative{b}{t}=\sum_{i\in S^{+}}t^{-1}\,e^{-\tilde{\mathbf{w}}^{\top}\mathbf{x}_{i}}\,e^{-b}\;-\;\sum_{i\in S^{-}}t^{-1}\,e^{\tilde{\mathbf{w}}^{\top}\mathbf{x}_{i}}\,e^{b}=t^{-1}\left(\sum_{i\in S^{+}}e^{-\tilde{\mathbf{w}}^{\top}\mathbf{x}_{i}}\,e^{-b}-\sum_{i\in S^{-}}e^{\tilde{\mathbf{w}}^{\top}\mathbf{x}_{i}}\,e^{b}\right). Define A_{S}^{+}=\sum_{i\in S^{+}}e^{-\tilde{\mathbf{w}}^{\top}\mathbf{x}_{i}} and A_{S}^{-}:=\sum_{i\in S^{-}}e^{\tilde{\mathbf{w}}^{\top}\mathbf{x}_{i}} . Then \displaystyle\derivative{b}{t}=t^{-1}\left(A_{S}^{+}e^{-b}-A_{S}^{-}e^{b}\right) \displaystyle\implies\frac{\differential{b}}{A_{S}^{+}e^{-b}-A_{S}^{-}e^{b}}=t^{-1}\,\differential{t} We integrate from the beginning of the idealized late time dynamics t_{0} where the bias is b_{0}=b(t_{0}) . \displaystyle\int_{b_{0}}^{b}\frac{\differential{b}}{A_{S}^{+}e^{-b}-A_{S}^{-}e^{b}}=\int_{t_{0}}^{t}t^{-1}\,\differential{t}. Now let e^{b}=u , so that e^{b}\,\differential{b}=\differential{u} . \displaystyle\implies\int_{e^{b_{0}}}^{e^{b}}\frac{\differential{u}}{A_{S}^{+}-A_{S}^{-}u^{2}}=\left.\log t\right|_{t_{0}}^{t} \displaystyle\implies\int_{e^{b_{0}}}^{e^{b}}\frac{\differential{u}}{(\sqrt{A_{S}^{+}})^{2}-(\sqrt{A_{S}^{-}}\,u)^{2}}=\left.\log t\right|_{t_{0}}^{t} \displaystyle\implies\frac{1}{2\sqrt{A_{S}^{+}}\sqrt{A_{S}^{-}}}\ln\left|\frac{\sqrt{A_{S}^{+}}+\sqrt{A_{S}^{-}}\,u}{\sqrt{A_{S}^{+}}-\sqrt{A_{S}^{-}}\,u}\right|\Bigg|_{e^{b_{0}}}^{e^{b}}=\log\frac{t}{t_{0}} \displaystyle\implies\log\Biggl|\frac{\sqrt{A_{S}^{+}}+\sqrt{A_{S}^{-}}\,e^{b}}{\sqrt{A_{S}^{+}}-\sqrt{A_{S}^{-}}\,e^{b}}\Biggr|+\log\Biggl|\frac{\sqrt{A_{S}^{+}}-\sqrt{A_{S}^{-}}e^{b_{0}}}{\sqrt{A_{S}^{+}}+\sqrt{A_{S}^{-}}e^{b_{0}}}\Biggr|=2\sqrt{A_{S}^{+}A_{S}^{-}}\left(\log\frac{t}{t_{0}}\right) \displaystyle\implies\log\left|\frac{1+\sqrt{\frac{A_{S}^{-}}{A_{S}^{+}}}\,e^{b}}{1-\sqrt{\frac{A_{S}^{-}}{A_{S}^{+}}}\,e^{b}}\cdot\frac{1-\sqrt{\frac{A_{S}^{-}}{A_{S}^{+}}}\,e^{b_{0}}}{1+\sqrt{\frac{A_{S}^{-}}{A_{S}^{+}}}\,e^{b_{0}}}\right|=2\sqrt{A_{S}^{+}A_{S}^{-}}\left(\log\frac{t}{t_{0}}\right). Let g(t)=2\sqrt{A_{S}^{+}A_{S}^{-}}\left(\log\frac{t}{t_{0}}\right) and \delta=\sqrt{\frac{A_{S}^{-}}{A_{S}^{+}}} . Then, \displaystyle\implies\log\left|\frac{1+\delta e^{b}}{1-\delta e^{b}}\cdot\frac{1-\delta e^{b_{0}}}{1+\delta e^{b_{0}}}\right|=g(t) \displaystyle\implies\frac{1+\delta e^{b}}{1-\delta e^{b}}\cdot\frac{1-\delta e^{b_{0}}}{1+\delta e^{b_{0}}}=e^{g(t)} \displaystyle\implies 1+\delta e^{b}=\frac{1+\delta e^{b_{0}}}{1-\delta e^{b_{0}}}\,e^{g(t)}-\delta\,\frac{1+\delta e^{b_{0}}}{1-\delta e^{b_{0}}}\,e^{g(t)}\,e^{b}. Therefore \displaystyle\delta e^{b}\Bigl(1-\delta e^{b_{0}}+e^{g(t)}(1+\delta e^{b_{0}})\Bigr)=\Bigl((1+\delta e^{b_{0}})e^{g(t)}-(1-\delta e^{b_{0}})\Bigr) \displaystyle\implies b(t)=\log\!\left(\frac{(1+\delta e^{b_{0}})\exp({g(t)})-(1-\delta e^{b_{0}})}{(1+\delta e^{b_{0}})\exp({g(t)})+(1-\delta e^{b_{0}})}\right)-\log\delta. ∎ Theorem A.2 . There exists a small enough \epsilon_{0} such that \forall\epsilon<\epsilon_{0} we have the following bounds on T_{te}(\epsilon) : \displaystyle T_{te}(\epsilon)\leq\exp\left(\frac{\gamma B}{2}+\frac{|\log\delta|}{1+2\epsilon(2\alpha-1)}\right) (19) \displaystyle T_{te}(\epsilon)\geq\exp\left(-\frac{\gamma B}{2}+\frac{|\log\delta|}{1+2\epsilon(2\alpha-1)}\right) (20) Therefore for a small enough \gamma we have \displaystyle T_{te}(\epsilon)=\exp\left(\frac{|\log\delta|}{1+2\epsilon(2\alpha-1)}\right) Proof. In the late time dynamics, from assumption ( 4.4 ) the hyperplane movement is parallel to that of the true seperator \hat{\mathbf{w}} . The test accuracy in this regime can be written as \displaystyle Q(t)=1-\frac{|b(t)|/||\mathbf{w}(t)||-\gamma/2}{2(\alpha\gamma-\gamma/2)} From the definition of T_{te}(\epsilon) , we have \displaystyle Q(T_{te}(\epsilon))=(1-\epsilon)\cdot 1\implies\frac{|b(T_{te}(\epsilon))|}{||\mathbf{w}(T_{te}(\epsilon))||}=\frac{\gamma}{2}+\epsilon(2\alpha-1)\gamma The evolution of the bias is given by \displaystyle b(t)=\log\!\left(\frac{(1+\delta e^{b_{0}})\exp({g(t)})-(1-\delta e^{b_{0}})}{(1+\delta e^{b_{0}})\exp({g(t)})+(1-\delta e^{b_{0}})}\right)-\log\delta In the late time dynamics, \exp(g(t)) becomes large and we can approximate the above evolution by \displaystyle b(t)=\log\!\left(\frac{1-\frac{(1-\delta e^{b_{0}})}{(1+\delta e^{b_{0}})}\exp(-{g(t)})}{1+\frac{(1-\delta e^{b_{0}})}{(1+\delta e^{b_{0}})}\exp(-{g(t)})}\right)-\log\delta\approx-\frac{2(1-\delta e^{b_{0}})}{(1+\delta e^{b_{0}})}\exp(-{g(t)})-\log\delta Substituting for g(T) , b(t) decays to b_{\infty} as t^{-2\sqrt{A_{S}^{+}A_{S}^{-}}} . However, ||\mathbf{w}(t)|| increases as \mathcal{O}(\log t) . Hence, for a small enough \epsilon , the numerator becomes close enough to -\log\delta and we can write \displaystyle\frac{|\log\delta|}{\frac{\gamma}{2}+\epsilon(2\alpha-1)\gamma}=||\mathbf{w}(T_{te}(\epsilon))|| Now, for this to occur, we bound T_{te}(\epsilon) by using the Cauchy-Schwarz inequality: \displaystyle\frac{2}{\gamma}\log T_{te}(\epsilon)-B\leq\frac{|\log\delta|}{\frac{\gamma}{2}+\epsilon(2\alpha-1)\gamma}\leq\frac{2}{\gamma}\log T_{te}(\epsilon)+B \displaystyle\implies\exp\left(\frac{\gamma B}{2}+\frac{|\log\delta|}{1+2\epsilon(2\alpha-1)}\right)\geq T_{te}(\epsilon)\geq\exp\left(-\frac{\gamma B}{2}+\frac{|\log\delta|}{1+2\epsilon(2\alpha-1)}\right) This concludes the proof. Additional comments for Remark ( 4.7 ) : Going through similar steps for T_{tr}(\epsilon) , we can write \displaystyle\frac{2}{\gamma}\log T_{tr}(\epsilon)+B\geq\frac{|\log\delta|}{\frac{\gamma}{2}+\epsilon(2(R/\gamma)-1)\gamma}\geq\frac{2}{\gamma}\log T_{tr}(\epsilon)-B \displaystyle\implies\exp\left(-\frac{\gamma B}{2}+\frac{|\log\delta|}{1+2\epsilon(2(R/\gamma)-1)}\right)\leq T_{tr}(\epsilon)\leq\exp\left(\frac{\gamma B}{2}+\frac{|\log\delta|}{1+2\epsilon(2(R/\gamma)-1)}\right) Therefore for large R/\gamma we have T_{te}(\epsilon) is much larger than T_{tr}(\epsilon) . Hence we can say \displaystyle T_{gr}(\epsilon)\approx T_{te}(\epsilon). ∎ Appendix B Experimental Setup In this section, we provide a detailed description of the experimental setup used throughout the paper, including data generation, model specification, optimization procedure, and evaluation protocol. The goal of these experiments is to isolate and study grokking-like behavior in a minimal and fully controlled linear classification setting. B.1 Synthetic Data Generation We consider a binary classification problem in \mathbb{R}^{2} and \mathbb{R}^{1024} . Let the ground-truth separating hyperplane be defined by a unit-norm vector \mathbf{w}_{\text{true}}=\frac{1}{\sqrt{2}}[1,1,\ldots,1]\in\mathbb{R}^{d}. Labels are assigned according to the sign of the projection onto \mathbf{w}_{\text{true}} . We generate two datasets: a training distribution \mathcal{P} and a concentrated test distribution \mathcal{P}^{\prime} , which differ only in the margin at which data points are sampled. Training Data. For d=2 , the training dataset consists of N=2000 points, with 1000 samples per class. For d=1024 , the training dataset consists of N=10^{6} points, with 5\times 10^{5} samples per class. A positive-labeled point \mathbf{x} is accepted if \mathbf{w}_{\text{true}}^{T}\mathbf{x}\geq\frac{\gamma}{2}, and a negative-labeled point is accepted if \mathbf{w}_{\text{true}}^{T}\mathbf{x}\leq-\frac{\gamma}{2}. Here, \gamma denotes the base margin parameter, which we fix to \gamma=10^{-3} . Candidate points are drawn uniformly from the square [-5,5]^{d} and rejected until the margin constraint is satisfied. This procedure ensures that the training data are linearly separable with margin \gamma/2 . Test Data (Concentrated). To induce a controlled concentrated distribution, the test dataset is generated from a near-boundary region . Specifically, we introduce a scaling parameter \alpha>1 and define the test margin as \gamma_{\text{test}}=\alpha\gamma . In all experiments, we set \alpha=10 . Positive test points satisfy \frac{\gamma}{2}<\mathbf{w}_{\text{true}}^{T}\mathbf{x}\leq\alpha\gamma, while negative test points satisfy -\alpha\gamma\leq\mathbf{w}_{\text{true}}^{T}\mathbf{x}<-\frac{\gamma}{2}. As in the training set, points are sampled uniformly from [-5,5]^{2} and rejected until the corresponding constraints are met. The test set also contains 2000 points when d=2 and 10^{6} , balanced across classes. This construction ensures that the test data lie closer to the decision boundary than the training data, while remaining linearly separable by the same ground-truth classifier. B.2 Model We consider a linear model with bias whose prediction for input \mathbf{x}_{i}\in\mathbb{R}^{d} is given by \hat{y}(\mathbf{x}_{i};w,b)=w^{\top}\mathbf{x}_{i}+b , where w\in\mathbb{R}^{d} and b\in\mathbb{R} are the weight vector and bias respectively. We are interested in minimizing the following loss function \displaystyle\mathcal{L}(\mathbf{w},b)=\sum_{i=1}^{N}l(y_{i}(\mathbf{w}^{\top}\mathbf{x}_{i}+b)) (21) where l(x)=\log(1+\exp({-x})) is the logistic loss. B.3 Optimization Procedure We optimize the logistic loss using gradient descent. The parameters are initialized at \mathbf{w}_{0}=\mathbf{0} and b_{0}=0 . Gradient Update: We use gradient descent to optimize this loss using \displaystyle\mathbf{w}_{t+1}=\mathbf{w}_{t}-\eta\nabla_{\mathbf{w}}\mathcal{L}(\mathbf{w},b) \displaystyle b_{t+1}=b_{t}-\eta\nabla_{b}\mathcal{L}(\mathbf{w},b). We use a fixed learning rate of \eta=10^{-2} and train for a total of T=10^{7} gradient descent steps. No explicit regularization, early stopping, or learning-rate scheduling is used. B.4 Evaluation Metrics and Logging At every iteration, we compute both training and test loss as well as classification accuracy. To reduce storage overhead, metrics are logged once every 100 iterations. Accuracy is defined as the fraction of correctly classified examples under a 0.5 decision threshold. All experiments are run with a fixed random seed to ensure reproducibility. B.5 Implementation Details All experiments are implemented in Python using NumPy for numerical computation. Progress over training iterations is tracked using tqdm . No automatic differentiation frameworks are used; gradients are computed explicitly in closed form. Appendix C Adversarial Robustness Experiments In this appendix, we describe the experimental setup used to evaluate adversarial robustness in the high-dimensional linear classification setting. These experiments extend the main results by studying how delayed generalization interacts with adversarial vulnerability under distribution shift. The data generation process, model, optimization is identical to the previous setup. C.1 Adversarial Threat Model To evaluate robustness, we consider an untargeted \ell_{\infty} -bounded adversarial threat model. Given an input \mathbf{x} , an adversary may perturb it within an \ell_{\infty} ball of radius \varepsilon : \|\boldsymbol{\delta}\|_{\infty}\leq\varepsilon. We generate adversarial examples using Projected Gradient Descent (PGD). Starting from either the original input or a random point within the \ell_{\infty} ball, PGD iteratively updates adversarial examples via \mathbf{x}^{(t+1)}=\Pi_{\|\boldsymbol{\delta}\|_{\infty}\leq\varepsilon}\left(\mathbf{x}^{(t)}+\alpha\cdot\mathrm{sign}\left(\nabla_{\mathbf{x}}\ell(\mathbf{w}^{\top}\mathbf{x}^{(t)}+b,y)\right)\right), where \Pi denotes projection and \ell is the binary cross-entropy loss. In all experiments, we use: \varepsilon=1,\qquad\alpha=\frac{1}{4},\qquad\text{number of steps}=20. Perturbations are clipped to the range defined by the minimum and maximum values observed in the dataset. C.2 Evaluation Protocol At regular intervals during training (every 100 optimization steps), we evaluate Training loss and accuracy, Clean test loss and accuracy, and Adversarial test accuracy under PGD. Adversarial accuracy is computed by first generating PGD adversarial examples for the full test set and then evaluating classification accuracy on these perturbed inputs. All experiments are run with fixed random seeds. Computation is performed in PyTorch .