Addressing out-of-sample issues in multi-layer convolutional neural-network parameterization of mesoscale eddies applied near coastlines
2024
Read Addressing out-of-sample issues in multi-layer convolutional neural-network parameterization of mesoscale eddies applied near coastlines (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2411-01138.txt.
Description
A 2024 arXiv research paper by . This study addresses the boundary artifacts in machine-learned (ML) parameterizations for ocean subgrid mesoscale momentum forcing, as identified in the online ML implementation from a previous study Zhang et al. We focus on the boundary condition (BC) treatment within the existing convolutional neural network (CNN) models and aim to mitigate the ”out-of-sample” errors observed near complex coastal regions without developing new, complex network architectures. Our approach leverages two established strategies for placing BCs in CNN models, namely zero and replicate padding.
Catalog record
| YEAR | 2024 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2411-01138
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Addressing out-of-sample issues in multi-layer convolutional neural-network parameterization of mesoscale eddies applied near coastlines Abstract This study addresses the boundary artifacts in machine-learned (ML) parameterizations for ocean subgrid mesoscale momentum forcing, as identified in the online ML implementation from a previous study Zhang et al. (2023) . We focus on the boundary condition (BC) treatment within the existing convolutional neural network (CNN) models and aim to mitigate the ”out-of-sample” errors observed near complex coastal regions without developing new, complex network architectures. Our approach leverages two established strategies for placing BCs in CNN models, namely zero and replicate padding. Offline evaluations revealed that these padding strategies significantly reduce root mean squared error (RMSE) in coastal regions by limiting the dependence on random initialization of weights and restricting the range of out-of-sample predictions. Further online evaluations suggest that replicate padding consistently reduces boundary artifacts across various retrained CNN models. In contrast, zero padding sometimes intensifies artifacts in certain retrained models despite both strategies performing similarly in offline evaluations. This study underscores the need for BC treatments in CNN models trained on open water data when predicting near-coastal subgrid forces in ML parameterizations. The application of replicate padding, in particular, offers a robust strategy to minimize the propagation of extreme values that can contaminate computational models or cause simulations to fail. Our findings provide insights for enhancing the accuracy and stability of ML parameterizations in the online implementation of ocean circulation models with coastlines. † † journal: Journal of Advances in Modeling Earth Systems (JAMES) † † authors: Cheng Zhang, Pavel Perezhogin, Alistair Adcroft, Laure Zanna † † corresponding: Cheng Zhang, [email protected] Program in Atmospheric and Oceanic Sciences, Princeton University, Princeton, NJ 08542, USA Courant Institute of Mathematical Sciences, New York University, New York, NY 10012, USA keypoints This study validates specialized boundary condition treatments in CNN models to reduce boundary artifacts in ocean parameterizations. This approach can be applied directly to already trained CNN models to ensure accurate and stable implementation of mesoscale eddies parameterizations. Replicate padding outperforms zero padding by minimizing boundary artifacts and preventing extreme values that compromise simulations. Plain Language Summary This study focuses on improving machine learning (ML) models used to predict ocean forces near coastlines, where errors arise because these models lack information in the area. We investigated how boundary conditions are handled in existing convolutional neural network models to reduce these errors without creating complex new architectures. By using two methods, i.e., zero padding and replicate padding, we found that replicate padding significantly decreases prediction errors in coastal areas. While zero padding sometimes worsens issues in certain models, our results show that replicate padding is more reliable for effectively minimizing extreme value errors. This work highlights the importance of proper boundary condition treatment in ML models for coastal applications, ultimately aiming to enhance the accuracy and reliability of ocean circulation predictions. 1 Introduction Even with advances in computing over recent decades, climate models have finite resolution and must parameterize unresolved, subgrid-scale processes. Historically, these parameterizations employ a mix of theory and empirical approaches ( Gent et al. (1995) ; Griffies et al. (1998) ; Juricke et al. (2017) , e.g., for ocean circulation,) , but are imperfect so that the representation of subgrid processes continues to be a major source of bias and errors in climate projections Stevens & Bony (2013) ; Hewitt et al. (2020) . Recently, the use of machine learning methods has emerged as a promising tool for developing subgrid parameterizations in numerical models of both the atmosphere Rasp et al. (2018) ; Beucler et al. (2021) ; Yuval et al. (2021) ; Wang et al. (2022) ; Shamekh et al. (2023) and ocean Bolton & Zanna (2019) ; Zanna & Bolton (2020) ; Guillaumin & Zanna (2021) ; Sane et al. (2023) ; A. Ross et al. (2023) ; Bodner et al. (2023) ; Perezhogin et al. (2024) . Among machine learning architectures employed for parameterization of subgrid fluxes in climate models, convolutional neural networks (CNNs) have become increasingly popular due to their ability to connect local fluxes or tendencies to spatially non-local features ( Bolton & Zanna (2019) ; Zanna & Bolton (2020) ; Guillaumin & Zanna (2021) ; Bodner et al. (2023) ; Gregory et al. (2024) , e.g.,) . However, applying CNN-based ML parameterizations presents unique challenges in ocean circulation models which must account for complex boundary conditions and topographical features at the Earth’s surface. Unlike atmospheric models, ocean models must manage dynamic interactions of coastal water with shorelines, where CNNs often struggle to make a prediction, because they operate by sliding fixed-size kernels across images (fields) to extract features. This was highlighted in a recent study by Zhang et al. (2023) , where significant boundary artifacts were observed when a CNN, trained to parameterize mesoscale eddies in the open ocean Guillaumin & Zanna (2021) , was employed near coasts in an ocean circulation model. These artifacts are likely the ”out of sample” problem in ML parameterizations, wherein a network trained on limited data (open ocean) might extrapolate poorly beyond its knowledge base (near the coast). In their study, the out-of-sample predictions of the CNN applied near the boundaries lead to errors that can ultimately propagate across the entire computational domain as the circulation model evolves. Training a CNN model with global data including the near-coast regions should solve the out-of-sample problem near boundaries. However, this approach has significant challenges. Consider training a CNN to represent an open ocean (deep water) physical process that is unresolved in a coarse ocean model. Many of the eddy-resolving process-studies used to derive parameterizations of mesoscale turbulence do not consider eddy interaction with coasts, and so inherently may not rectify the parameterized physics and fluxes appropriately. Secondly, the shallow depth near the coast might modify the spatial scales of the process requiring even finer resolution in the high-resolution simulation used to obtain training data which could be too costly Hallberg (2013) . Thirdly, the training data near the coastline is limited because the majority of the grid points correspond to the open ocean. Finally, the modification of the process near coastlines (e.g. different dynamics, scales, etc.) likely needs more sophisticated CNN models to capture the additional complexity (i.e either or both deeper and wider networks). The machine learning models that we have seen developed so far to parameterize ocean processes are based either on data from idealized simulations Bolton & Zanna (2019) ; Zanna & Bolton (2020) ; A. Ross et al. (2023) or on regional data from open ocean areas in global General Circulation Model (GCM) simulations that exclude land points Guillaumin & Zanna (2021) ; Bodner et al. (2023) . All this to say, training a new ML model including coastlines remains challenging, and alternative strategies to mitigate the out-of-sample issues near shorelines should be explored. The border effect for CNNs has been extensively studied in image processing, where common remedies include filling values at the image edges, that is padding Innamorati et al. (2018) ; Nguyen et al. (2019) ; Huang et al. (2021) ; Yang et al. (2023) , rescaling the result of the convolution operation near the borders, that is partial convolution Liu et al. (2018) , or