Out-of-Sample Validation of MagNet

Aryiadna Yesmanchyk, Yan Xu, Jason T. L. Wang, Haodi Jiang, Chunhui Xu, Haimin Wang · 2026

Read Out-of-Sample Validation of MagNet by Aryiadna Yesmanchyk, Yan Xu, Jason T. L. Wang, Haodi Jiang, Chunhui Xu, Haimin Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-15926.txt.

Introduction

Aryiadna Yesmanchyk and five co-authors validate a solar physics model outside its training data in a 2026 arXiv paper. Machine learning is spreading through nearly every field, solar physics included. Their model, MagNet, was built for a hard data-mining task. It generates transverse magnetic fields of solar active regions. Those fields are hard to measure directly. A model that fills them in aids many studies.

Out-of-sample validation is the paper's focus. A model can look strong on data like its training set yet fail elsewhere. Testing on truly new data gives the honest score. Solar observations shift with instruments and solar cycles, so robustness matters. The validation tells users where MagNet can be trusted and where it should be doubted.

Catalog record

CREATORAryiadna Yesmanchyk, Yan Xu, Jason T. L. Wang, Haodi Jiang, Chunhui Xu, Haimin Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE10 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-15926

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Out-of-Sample Validation of MagNet Aryiadna Yesmanchyk, Yan Xu, Jason T. L. Wang, Haodi Jiang, Chunhui Xu, Haimin Wang Abstract Machine learning is starting to be used in almost every industry and academic research, and solar physics is no exception. A newly developed machine learning model named MagNet [ 1 ] helps us to tackle some of the most serious challenges in data mining by generating transverse fields of solar active regions. Being trained on line-of-sight magnetograms from Michelson Doppler Imager at Solar and Heliospheric Observatory (SOHO/MDI), H α \alpha maps from Big Bear Solar Observatory and Kanzelhöhe Solar Observatory and vector magnetograms from Helioseismic and Magnetic Imager at Solar Dynamic Observatory (SDO/HMI), this model provides vector magnetograms in active regions for SOHO/MDI data covering the strong solar cycle 23. In this study, we performed out-of-sample validation of the MagNet model with data from Imaging Vector Magnetograph (IVM) at Mees Solar Observatory, which was not included in the training process. Our results show good correlation between the AI generated data and the observed vector magnetograms and therefore strengthen the confidence of implementing MagNet to the entire SOHO/MDI archive and future scientific analysis of the AI generated data. I Introduction It has been well known that the topology and evolution of magnetic fields are crucial in providing energy storage and release of solar eruptive events. Magnetic field parameters, such as free energy, are derived from the photospheric vector magnetograms and used to study solar flares and CMEs, and eventually help in predicting solar eruptions. The most recent solar cycle 24 was relatively weak with few large flares, though it is the only solar cycle in which consistent time-sequence vector magnetograms have been available through the Helioseismic and Magnetic Imager [ 2 ] on board the Solar Dynamics Observatory [ 3 ] since its launch in 2010. On the other hand, the Michelson Doppler Imager [ 4 ] on board the Solar and Heliospheric Observatory [ 5 ] from 1996 to 2011 observed stronger events in Solar Cycle 23. However, MDI only collected line-of-sight (LOS) magnetograms, which are not suitable for comprehensive analysis of solar active regions (ARs). This created a need for a machine learning model that could be used to generate these missing but crucial data. We are specifically interested in reconstructing B x and B y components, as B z can be directly taken from the LOS magnetogram and does not require reconstruction. A schematic diagram of the MagNet workflow is shown in Figure 1 . This machine learning model takes a pair of co-aligned images H \alpha image and the LOS magnetogram as inputs. The outputs are B x and B y of a given solar AR. During the training and testing processes, SDO/HMI vector magnetograms were used as the “ground truth”. Fig. 1: MagNet workflow. H \alpha and LOS are MagNet’s inputs. B’ x and B’ y are fields generated by MagNet. Credit: [ 1 ] SOHO/MDI has spatial resolution is 4” and the full-disk images are collected on a 1024 × 1024 detector. H \alpha , 2032 × 2032 pixels, is collected at 6563 Å wavelength by the 10-inch telescopes at Big Bear Solar Observatory and Kanzelhöhe Solar Observatory [ 6 ] . H \alpha and SOHO/MDI magnetograms have different pixel resolutions due to the instruments’ differing aperture sizes. The H \alpha images are resized to the MDI’s resolution. Use of H \alpha maps allows us to take into account a missing directional component of magnetic fields on the horizontal solar surface direction [ 1 ] . In our study, we performed an out-of-sample (OOS) validation of MagNet. OOS validation is an important process in machine learning and statistical models that assesses accuracy of a model using brand new data to the model. Therefore, OOS provides a more realistic and unbiased evaluation in the real world. During the training process, an AI model exposed to a specific dataset and learned within the scope of the dataset. However, in a complex model, the training data might be “memorized” together with noise and fluctuations at certain levels. As a consequence, the underlying generalizable patterns are not fully understood by the model. Such a phenomenon is known as overfitting and it can lead to outstanding performance with training/testing samples, but less accurate on new data. By evaluating the model on a separate OOS data archive which has never been used during the training, we can get a true measure of its ability to generalize. This helps to identify and mitigate overfitting, ensuring that the model’s predictions are reliable and robust. II Preparing OOS Data The MagNet code takes a pair of co-aligned images, H \alpha + LOS magnetogram (B z ), as input and predicts the corresponding B x and B y in the same AR as output. The key component of OOS validation is to compare the output with new “ground truth” data, which is obtained from the Vector Magnetograph (IVM) at Mees Solar Observatory, Haleakala, Maui, Hawaii. II-A Generating Input Images for MagNet Each H \alpha -MDI pair contains a full disk H \alpha image and a full disk LOS MDI magnetogram. MDI LOS magnetograms with a cadence of 96 min were accessed through the Joint Science Operations Center (JSOC). For each of the MDI images, we located the H \alpha image obtained nearly simultaneously from Big Bear Solar Observatory (BBSO) and Kanzelhöhe observatory (KSO). Since MDI observations are taken in space and therefore are not affected by the turbulence of the Earth’s atmosphere like H \alpha data. We first truncated and aligned H \alpha image to the MDI image, which is used as reference. This alignment is to match the solar disk by their circular shape on both images regardless the magnetic features. After this alignment, all the ARs on disk are automatically aligned with minor shifts, which will be corrected using the cross correlation method, after a certain AR is selected. We used the IDL code align.pro (developed by the University of OSLO / Institute of Theoretical Astrophysics), which aligns two images by utilizing Fourier transform properties to compute their cross-correlation and thereby determine the shift needed to align one image with the reference. The location of ARs are identified by the Solar Region Summary (SRS) files, which are daily reports from the Space Weather Prediction Center (SWPC) at National Oceanic and Atmospheric Administration (NOAA). On each day, a NOAA SRS file is published around 00:30 UT. The major contents are the coordinates of ARs or brightenings at that time. Those coordinates can be transferred to other observing times by considering the solar differential rotation (such at the IDL function “drot_map.pro” developed by NASA/GSFC). After the center coordinates of a given AR is determined on the H \alpha -MDI pair, we crop the images with a field of view (FOV) of 256 x 256 pixels. As shown in Figure 2 , this new image pair with reduced FOV/size is the final inputs for the MagNet model. Fig. 2: AR extraction for AR 09643 on 2001-May-21. The left panel shows original full-disk H \alpha image taken at 16:01:04 UT and the selected AR image. Similar maps of the MDI magnetogram taken at 15:59:41 UT are shown in

Read the complete text