Not All Explanations for Deep Learning Phenomena Are Equally Valuable
Alan Jeffares, Mihaela van der Schaar · 2025
Read Not All Explanations for Deep Learning Phenomena Are Equally Valuable by Alan Jeffares, Mihaela van der Schaar (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-23286.txt.
Introduction
Alan Jeffares and Mihaela van der Schaar argue in 2025 that not all work on odd deep learning effects is of equal worth. The field spends much time on strange effects such as double descent, delayed sudden learning, and lucky starting tickets. Each case tends to draw its own ad hoc story, built for that case alone.
The pair notes that many such effects have scant proof of showing up in real tools. Chasing each one as a lone puzzle may waste effort that could move the wider field. Yet the effects still hold worth as test beds where broad theories of learning can be honed. The paper looks back at what past chases yielded and gives plain advice for future work. The aim is work that serves the field's real end: better learning tools.
Catalog record
| CREATOR | Alan Jeffares, Mihaela van der Schaar |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-23286
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Not All Explanations for Deep Learning Phenomena Are Equally Valuable Alan Jeffares, Mihaela van der Schaar Abstract Developing a better understanding of surprising or counterintuitive phenomena has constituted a significant portion of deep learning research in recent years. These include double descent, grokking, and the lottery ticket hypothesis – among many others. Works in this area often develop ad hoc hypotheses attempting to explain these observed phenomena on an isolated, case-by-case basis. This position paper asserts that, in many prominent cases, there is little evidence to suggest that these phenomena appear in real-world applications and these efforts may be inefficient in driving progress in the broader field. Consequently, we argue against viewing them as isolated puzzles that require bespoke resolutions or explanations. However, despite this, we suggest that deep learning phenomena do still offer research value by providing unique settings in which we can refine our broad explanatory theories of more general deep learning principles. This position is reinforced by analyzing the research outcomes of several prominent examples of these phenomena from the recent literature. We revisit the current norms in the research community in approaching these problems and propose practical recommendations for future research, aiming to ensure that progress on deep learning phenomena is well aligned with the ultimate pragmatic goal of progress in the broader field of deep learning. 1 Introduction Deep learning phenomena – that is, surprising empirical observations that appear to disagree with our existing expectations of how neural networks function – have become a prominent topic in deep learning research. In recent years, there has been a remarkable interest and excitement around several of these phenomena within the research community ( Belkin et al. 2019 ; Frankle & Carbin 2019 , e.g.) , which has even begun to extend into the public consciousness ( Heaven 2024 ; Ananthaswamy 2024 , e.g.) . The sustained research focus on these idiosyncratic phenomena is heavily reflected in publications at leading machine learning conferences and community efforts such as workshop sessions. This work argues that many prominent deep learning phenomena discussed in the research literature are not representative of challenges encountered in real-world applications of deep learning. Thus, not all efforts to understand these phenomena are equal in value – we should focus on using them to refine our broad explanatory theories of important aspects of deep learning rather than developing narrow ad hoc hypotheses to describe them in isolation. However, this perspective is not consistently reflected in current research practices within the community. Position What type of “deep learning phenomena” does this work address? As this terminology could refer to a broad range of empirical observations, we will first specify the kind of phenomena we address throughout this work. We adopt the definition used in the workshop for “Identifying and Understanding Deep Learning Phenomena” at ICML 2019 which called for “interesting and unusual behavior observed in deep nets” that can be “isolated and analyzed”. We further narrow our scope to what we call edge cases i.e. phenomena that challenge our intuitions without appearing prominently in practical deep learning applications. For example, the emergence of grokking – that is, delayed generalization, long after perfect training performance (q.v. Section 2 ) – is included in our discussion as it has captured the interest of many in the research community despite not being a practical concern in training frontier models. We focus on these edge case phenomena as, despite their prevalence in research (which we later detail), it is not obvious how or why they should be studied. Where might research on these phenomena go wrong? Given that these edge case phenomena are, by definition, of limited relevance to practical settings, their role in deep learning research more broadly is worth revisiting. In particular, practitioners might ask when developing models for real-world applications (e.g. large language models in production settings), is this phenomenon something I should be concerned with? Consequentially, researchers might ask what is the value in studying this phenomenon? In this work, we posit that there is little value in pursuing a resolution-oriented approach to research in this setting. In other words, if a phenomenon presented a practical challenge then we might wish to resolve it (i.e. develop methods to mitigate its negative effects and amplify its positive effects) which could be supported by efforts to understand it isolation (by this we refer to the development of narrow ad hoc hypotheses that don’t necessarily generalize more broadly – see Section 3.2 for further details on this terminology). However, as these phenomena do not pose a significant practical challenge, it should not be assumed that this approach will yield valuable outcomes. Indeed, without broader implications “explaining” an inconsequential phenomenon may be no more useful than solving a puzzle. However, this view is not reflected in current research practices on deep learning phenomena. What is the role of these phenomena in deep learning research? When deep learning phenomena do not require resolution, we might be tempted to reason that research efforts are better directed elsewhere. However, in this position piece, we argue that these phenomena can still provide a valuable test bed for refining our broad explanatory theories 1 1 1 This term refers to the type of theories that provide insight more generally beyond the narrow setting which they describe. In Section 3.2 we make this distinction between narrow ad hoc hypotheses and broad explanatory theories more explicit and provide concrete examples. which encapsulate our fundamental understanding of core aspects of deep learning. In particular, they can act as extreme or synthetic settings that can stress test or provide counterexamples to our current beliefs about important mechanisms in deep learning. We argue that this can be achieved by integrating two key changes into our research practices. \blacktriangleright A more pragmatic approach: Although estimating the downstream impact of research is challenging, it is not entirely random. We suggest that a more critical consideration of the potential practical utility when pursuing research on a phenomenon can indeed result in more useful outcomes. While competing explanations for a phenomenon can clearly be evaluated based on their accuracy (i.e. are they both true ?), it can also be the case that they are equally accurate but unequal in their utility (i.e. does this explanation provide insights that enable us to improve performance in practice?) Thus, researchers should be pragmatic in how they select among the vast space of potential research directions, asking questions such as does this new theory provide generalizable intuition beyond the specific phenomenon in which it is developed? \blacktriangleright A more scientific approach: Paired with this pragmatic approach, we also advocate for the application of a more rigorous scientific methodology. The natural sciences, for example, have a long history of applying scientific practices, attempting to falsify and update competing theories of the world. We believe that better adapting these practices (e.g. preregistration) and working more collectively (e.g. greater efforts at reconciliation among different explanations), will result in research that is less disjoint and uncovers more general knowledge with broader utility. Goals: The objectives of this position paper are threefold. (1) It aims to spark a conversation within the community as to why we should study deep learning phenomena at all, suggesting