Fairness is Not Flat: Geometric Phase Transitions Against Shortcut Learning
Nicolas Rodriguez-Alvarez, Fernando Rodriguez-Merino · 2026
Read Fairness is Not Flat: Geometric Phase Transitions Against Shortcut Learning by Nicolas Rodriguez-Alvarez, Fernando Rodriguez-Merino (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-11704.txt.
Introduction
Rodriguez-Alvarez proposes a geometric method against shortcut learning in deep neural networks. Networks often memorize false correlations instead of causes, hurting robustness and adding demographic biases. A single-layer Topological Auditor isolates features hogging the gradient without human help. Once linear shortcuts are pruned, networks must use higher power to curve decision boundaries and learn ethical features, a Capacity Phase Transition. The approach beats L1 regularization, which collapses into demographic bias, and costs far less than Just Train Twice, cutting what-if gender risk from 21.18 percent to 7.66 percent.
Catalog record
| CREATOR | Nicolas Rodriguez-Alvarez, Fernando Rodriguez-Merino |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 11160 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.11704
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fairness is Not Flat: Geometric Phase Transitions Against Shortcut Learning Nicolas Rodriguez-Alvarez, Fernando Rodriguez-Merino Abstract . Deep Neural Networks are highly susceptible to shortcut learning, frequently memorizing low-dimensional spurious correlations instead of underlying causal mechanisms. This phenomenon not only degrades out-of-distribution robustness but also induces severe demographic biases in sensitive applications. In this paper, we propose a geometric a priori methodology to mitigate shortcut learning. By deploying a zero-hidden-layer ( N = 1 N=1 ) Topological Auditor, we mathematically isolate features that monopolize the gradient without human intervention. We empirically demonstrate a Capacity Phase Transition: once linear shortcuts are pruned, networks are forced to utilize higher geometric capacity ( N ≥ 16 N\geq 16 ) to curve the decision boundary and learn ethical representations. Our approach outperforms L1 Regularization—which collapses into demographic bias—and operates at a fraction of the computational cost of post-hoc methods like Just Train Twice (JTT), successfully reducing counterfactual gender vulnerability from 21.18% to 7.66%. 1. Introduction Biological intelligence is fundamentally economic. The human brain, acting as a ’cognitive miser’ ( 2 ) , evolved to prioritize fast, low-effort heuristics over exhaustive logical reasoning—a duality famously described as System 1 and System 2 ( 7 ) . This ancestral ’laziness’ is not a flaw, but a survival mechanism designed to conserve metabolic resources. Paradoxically, modern Machine Learning (ML) models have inherited this same evolutionary pressure. Despite their digital substrate, neural networks exhibit a profound simplicity bias ( 6 ) , naturally gravitating towards ’shortcuts’ that offer the path of least resistance to minimize the loss function at the expense of causal understanding. In tabular environments such as the Adult Census dataset, this algorithmic “System 1” manifests as a disproportionate reliance on direct data leakages. For instance, rather than learning the complex, multi-dimensional interactions between an individual’s education, age, and hours worked, a standard deep model will exploit financial anomalies like Capital-Gain . Much like a human cognitive miser who avoids complex mental arithmetic when a simple heuristic is readily available, the neural network collapses its hypothesis space onto the most accessible predictive feature, effectively bypassing true causal inference. To expose and neutralize this behavior, we propose the A Priori Geometric Auditor . By intentionally restricting the network’s topological capacity to a single linear boundary ( N=1 hidden neurons), we strip the model of its ability to “reason” geometrically. Forced into extreme cognitive starvation, the auditor is compelled to reveal the dataset’s path of least resistance, deterministically isolating spurious shortcuts and data leakages before the primary, high-capacity model is ever trained. Our main contributions are threefold: • The A Priori Geometric Auditor: We introduce a strictly linear probing methodology ( N=1 ) that successfully isolates topological shortcuts—such as the Capital-Gain data leakage—without requiring human prior knowledge. • Capacity Phase Transition: We empirically demonstrate that algorithmic fairness requires geometry. Once linear shortcuts are topologically pruned, the network is forced to utilize higher geometric capacity ( N\geq 16 ) to curve the decision boundary and learn meritocratic representations. • Counterfactual Bias Mitigation: Through stress testing, we prove that pruning low-dimensional shortcuts not only immunizes the model against direct data leakage but also drastically reduces secondary demographic biases, lowering gender vulnerability from 21.18% to 7.66%. 2. Related Work and Baselines Addressing shortcut learning often involves post-hoc data reweighting or standard regularization techniques. However, these methods suffer from significant computational or ethical drawbacks when compared to our geometric a priori approach. Standard Regularization (L1/Lasso): L1 regularization is traditionally used to induce sparsity. However, it is entirely “blind” to the ethical or causal value of the features. As demonstrated in our empirical baseline (Figure 1 A), applying an L1 penalty ( \lambda=0.05 ) catastrophically failed to mitigate bias. To minimize the regularization loss, the L1 model aggressively pruned true causal features (such as Education and Hours worked ) and ultimately collapsed the entire hypothesis space onto a single protected attribute: Husband . By contrast, our Topological Auditor successfully isolates the financial data leakage ( Capital-Gain ) while preserving ethical merit variables. Just Train Twice (JTT): Recent state-of-the-art approaches like JTT ( 5 ) attempt to mitigate spurious correlations by training an initial empirical risk minimization (ERM) model, identifying misclassified examples, and then training a second robust model. While effective, JTT doubles the computational cost and carbon footprint by requiring the training of multiple deep networks. Our Geometric Auditor offers a proactive alternative: by probing the dataset with N=1 hidden neurons, we mathematically isolate topological shortcuts at a fraction of the computational cost before the robust model is ever trained (Figure 1 B). Figure 1. State-of-the-Art (SOTA) Comparison. (A) Relative feature importance: Auditor vs L1 Regularization. (B) Relative computational efficiency compared to JTT. 3. Methodology: The Geometric Auditor Our approach relies on the principle that spurious correlations –such as direct data leakages– are fundamentally low-dimensional and linearly separable, whereas true causal relationships in human behavior demand higher geometric capacity to be approximated. We propose a three-phase methodology to proactively mitigate shortcut learning. 3.1. Linear Probing and Shortcut Isolation Following the intuition of linear classifiers to understand network representations, we deploy a zero-hidden-layer network ( N=1 ) directly on the input features. The prediction for a given instance x\in\mathbb{R}^{d} is given by standard logistic regression: (1) \hat{y}=\sigma\left(\sum_{i=1}^{d}w_{i}x_{i}+b\right) Lacking the geometric memory provided by hidden non-linearities, the optimizer is forced to exploit the simplest linear pathways to minimize the Binary Cross-Entropy loss. Consequently, features acting as spurious shortcuts monopolize the gradient, resulting in disproportionately large absolute weights ( 1 ) . 3.2. Topological Pruning Criterion Rather than relying on human domain knowledge to identify these leakages, we implement an automated statistical pruning mechanism. We define a pruning threshold \tau based on the mean absolute magnitude of the network’s weights: (2) \tau=2\times\frac{1}{d}\sum_{i=1}^{d}|w_{i}| Any feature x_{i} whose associated weight satisfies |w_{i}|>\tau is mathematically flagged as a topological shortcut and is subsequently pruned from the hypothesis space prior to training the primary model. 3.3. Capacity Phase Transition Once the low-dimensional shortcuts are removed, the dataset becomes strictly non-linear. To learn the underlying ethical representation (e.g., combining age, education, and hours worked), we train models of varying capacities. The robust model utilizes a deep architecture with non-linear ReLU activations: (3) f(x)=\sigma\left(W_{2}\cdot\max(0,W_{1}\cdot x+b_{1})+b_{2}\right) We search for the critical number of hidden neurons ( N ) where the model successfully curves the decision boundary to overcome the topological pruning, an event we define as the Capacity Phase Transition ( 3 ) . 4. Experiments and Results We validate our geometric approach using both synthetic environments (XOR with an artificial shortcut) and real-world tabular data (the Adult Census Income dataset). 4.1. Automated Shortcut Detection Figure 2. Automated Shortcut Detection using the Geometric Auditor ( N=1 ). As illustrated in Figure 2 , the linear auditor successfully identifies the dominant spurious correlations in both environments. In the Adult dataset, the Capital-Gain feature drastically exceeded the pruning threshold ( |w_{cap}|=23.63 , where \tau=8.71 ). This empirical finding confirms that standard models heavily rely on this specific financial variable as a direct shortcut to predict income, effectively ignoring educational and demographic merit. 4.2. The Geometric Phase Transition Figure 3. Phase transition between the biased and the pruned model. Figure 3 demonstrates the necessity of geometric capacity