Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence
Moein Darman, Pedram Hassanzadeh, Laure Zanna, Ashesh Chattopadhyay · 2025
Read Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence by Moein Darman, Pedram Hassanzadeh, Laure Zanna, Ashesh Chattopadhyay (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-15487.txt.
Introduction
Moein Darman, Pedram Hassanzadeh, Laure Zanna, and Ashesh Chattopadhyay peer inside a neural network to learn why a trick called transfer learning rescues it on new ocean regimes. Their paper appeared in two thousand twenty-five. Climate models must fake the effect of swirls too small to simulate. Networks can learn that faking, but they fail on flows unlike their training data.
The team studied the network's filters through the lens of waves. Whatever the training flow, the filters sorted into smoothing, edge-spotting, and detail-keeping kinds. On new flows, deep layers shrank the signal's spectrum, so outputs came out too weak. Retraining a single layer on a little new data fixed the shrink. The fix explains transfer learning's power in plain physical terms. The lesson should carry to other data-driven models of moving fluids.
Catalog record
| CREATOR | Moein Darman, Pedram Hassanzadeh, Laure Zanna, Ashesh Chattopadhyay |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 47 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-15487
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fourier analysis of the physics of transfer learning for data-driven subgrid-scale models of ocean turbulence Moein Darman, Pedram Hassanzadeh, Laure Zanna, Ashesh Chattopadhyay Abstract Transfer learning (TL) is a powerful tool for enhancing the performance of neural networks (NNs) in applications such as weather and climate prediction and turbulence modeling. TL enables models to generalize to out-of-distribution data with minimal training data from the new system. In this study, we employ a 9-layer convolutional NN to predict the subgrid forcing in a two-layer ocean quasi-geostrophic system and examine which metrics best describe its performance and generalizability to unseen dynamical regimes. Fourier analysis of the NN kernels reveals that they learn low-pass, Gabor, and high-pass filters, regardless of whether the training data are isotropic or anisotropic. By analyzing the activation spectra, we identify why NNs fail to generalize without TL and how TL can overcome these limitations: the learned weights and biases from one dataset underestimate the out-of-distribution sample spectra as they pass through the network, leading to an underestimation of output spectra. By re-training only one layer with data from the target system, this underestimation is corrected, enabling the NN to produce predictions that match the target spectra. These findings are broadly applicable to data-driven parameterization of dynamical systems. 1 Introduction Improving the accuracy of climate, weather, and ocean models requires enhancing their resolution. This necessitates more computational power, which is restricted by limitations. Practical models that run on low spatio-temporal resolution with high predictive accuracy require an accurate representation of subgrid-scale (SGS) processes that occur below the numerical model’s grid size. Although these processes occur on such a small spatial and temporal scale that resolving them inside a model is computationally intractable, their effects on the large-scale dynamics is significant. SGS parameterization approximates these processes in models, allowing for more accurate simulations at lower computational costs. The accurate parameterization of SGS processes is essential for improving the accuracy of predictions of these nonlinear, multi-scale, and high-dimensional systems. Consequently, modeling SGS processes has been an active area of research for the past few decades [ Dipankar et al., 2015 , Meneveau and Katz, 2000 , Pressel et al., 2017 , Sagaut et al., 2013 , Sarlak et al., 2015 , Schneider et al., 2017 , Bracco et al., 2024 , Bracco et al., 2025 ] . In the early stages of global climate modeling, Smagorinsky [1963] introduced a physics-based SGS model, aiming to parameterize the effects of SGS eddies through a scale-selective dissipative approach, marked by positive eddy viscosity and second-order diffusion. This model and its variants since then, have found applications across a wide array of fields, including weather and climate simulation, combustion, and magnetohydrodynamics, amongst others [ Arakawa and Lamb, 1977 , Fox, 2012 , Fox-Kemper and Menemenlis, 2008 , Knaepen and Moin, 2004 , Piomelli, 1999 , Piomelli, 1999 , Sagaut, 2005 , Stevens et al., 2018 , Tan et al., 2017 ] . Despite their utility in ensuring numerical stability for Large Eddy Simulations (LES), these purely diffusive models often fail to accurately represent inter-scale physical processes like energy transfers, notably omitting backscattering [ Pope, 2000 ] . Backscattering – the transfer of energy from subgrid to resolved scales – is critical in various problems involving turbulent fluid flow and has prompted extensive research to incorporate it into physics-based SGS models [ Carati et al., 1995 , Domaradzki et al., 1987 , Kerr et al., 1996 , Khani and Waite, 2016 , Leslie and Quarini, 1979 , Mason and Thomson, 1992 , Mason and Thomson, 1992 , Shinde, 2020 , Thuburn et al., 2013 , Zhou, 1991 , Guan et al., 2024 , Jansen and Held, 2014 , Jansen et al., 2019 ] . Efforts have been made to refine these models for more accurate energy transfer depiction, such as the dynamic approach by Grooms [2023b] , Germano et al. [1991] that allows for negative eddy viscosity to account for backscattering [ Grooms, 2023a ] . However, these advancements often come with trade-offs in numerical stability, highlighting the ongoing challenge of developing SGS models that precisely capture both forward and backscatter energy transfers. This gap underscores the limitations of physics-based models in SGS parameterization, paving the way for exploring data-driven approaches in subsequent research efforts. The mathematical relationship between the large-scale dynamics of turbulent flows and the corresponding subgrid-scale forcing is generally nonlinear and poses a significant challenge for researchers. Therefore, neural networks (NNs), as universal approximators [ Hornik et al., 1989 ] , are attractive tools to establish such mapping and unveil more hidden knowledge from data, potentially providing better SGS models and even new insights into SGS physics. Nevertheless, some issues exist with relying solely on data-driven models to find the mapping between large-scale and small-scale processes. These models require a large training dataset that contains accurate SGS terms obtained from high-fidelity sources such as high-resolution observations or simulations. However, these sources are usually scarce, and the models only perform well on the dataset on which they were trained. They cannot adapt easily to new datasets, which reduces their effectiveness in predicting dynamics beyond their training dataset. On the other hand, the ability to generalize to newer dynamical regimes is crucial in both SGS modeling and the broader area of machine learning for the physical sciences. For example, NN-based SGS models must perform reliably across diverse climatic conditions to be effectively utilized in projections of global warming [ Rasp et al., 2018 , Larraondo et al., 2019 ] and its impact on extreme weather events, events that can only be estimated from the tails of the probability density function (PDF) of the variables. Additionally, NN-based SGS models are less interpretable, making it difficult to interpret the mappings they create, thereby challenging the trustworthiness and reliability of their predictions. Given the fact that extrapolation to different climate conditions is an out-of-distribution generalization problem and is challenging for NNs [ Krueger et al., 2020 ] , transfer learning (TL) is a flexible and robust framework that enables this [ Yosinski et al., 2014 , Zhuang et al., 2021 ] and can help effective blending of disparate training sets. It involves building a new NN called Transfer Learned NN (TLNN) from Base NN (BNN), which can achieve a similar level of accuracy for a target system that may have different statistical and dynamical properties compared to the base system. This is accomplished by re-training a few layers from the BNN using a small amount of data (usually orders of magnitude less data than what was used to train the BNN) from the target system. The process can produce a TLNN with comparable out-of-sample accuracy for the target system as the BNN, despite using only a small amount of re-training data. Numerous studies have been conducted on applying TL to improve NN generalizability for problems involving partial differential equations (PDEs). These investigations predominantly center around Physics-Informed Neural Networks (PINNs) [ Chen et al., 2021 , Desai et al., 2021 , Gao et al., 2022 , Goswami et al., 2020 , Guo et al., 2022 , Haghighat et al., 2021 , Hanna et al., 2022 , Li et al., 2021 , Mattheakis et al., 2021 , Xu et al.,