Uncovering Solar Wind Phenomena with iSAX, HDBSCAN, Human-in-the-loop and PSP Observations
Valmir P. Moraes Filho, Daniela Martin, Jasmine R. Kobayashi, Jinsu Hong, Connor O’Brien, Evangelia Samara, Joseph Gallego · 2025
Read Uncovering Solar Wind Phenomena with iSAX, HDBSCAN, Human-in-the-loop and PSP Observations by Valmir P. Moraes Filho, Daniela Martin, Jasmine R. Kobayashi, Jinsu Hong, Connor O’Brien, Evangelia Samara, Joseph Gallego (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21042.txt.
Introduction
Valmir P Moraes Filho leads six co-authors on uncovering solar wind phenomena in Parker Solar Probe data with a pipeline joining symbolic compression, density clustering, and human validation. The solar wind drives space weather, but high cadence probe data from twenty eighteen to twenty twenty four exceed manual review.
Over one hundred fifty gigabytes of magnetic and plasma measurements are processed efficiently across multiple time scales. Known structures are recovered, and uncatalogued coronal mass ejections and transient events are detected. The human in the loop design keeps expert judgment inside a scalable flow, a pattern other overflowing observatories can copy.
Catalog record
| CREATOR | Valmir P. Moraes Filho, Daniela Martin, Jasmine R. Kobayashi, Jinsu Hong, Connor O’Brien, Evangelia Samara, Joseph Gallego |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-21042
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Uncovering Solar Wind Phenomena with iSAX, HDBSCAN, Human-in-the-loop and PSP Observations Valmir P. Moraes Filho, Daniela Martin, Jasmine R. Kobayashi, Jinsu Hong, Connor O’Brien, Evangelia Samara, Joseph Gallego Abstract The solar wind is a dynamic plasma outflow that shapes heliospheric conditions and drives space weather. Identifying its large-scale phenomena is crucial, yet the increasing volume of high-cadence Parker Solar Probe (PSP) observations poses challenges for scalable, interpretable analysis. We present a pipeline combining symbolic compression, density-based clustering, and human-in-the-loop validation. Applied to 2018–2024 PSP data, it efficiently processes over 150 GB of magnetic and plasma measurements, recovering known structures, detecting uncatalogued CMEs and transient events, and demonstrating robustness across multiple time scales. A key outcome is the systematic use of the magnetic deflection angle ( θ B \theta_{B} ) as a unifying metric across solar wind phenomena. This framework provides a scalable, interpretable, expert-validated approach to solar wind analysis, producing expanded event catalogs and supporting improved space weather forecasting. The code and configuration files used in this study are publicly available to support reproducibility. 1 Introduction The solar wind is a continuous outflow of plasma from the Sun that shapes the heliosphere and drives space weather phenomena, which can affect satellites, astronauts, communications, and power grids ( Beedle_2022 ; Zank_2015 ; Kilpua_2017 ) . Key transient events within the solar wind include Coronal Mass Ejections (CMEs), High-Speed Streams (HSS), and Stream Interaction Regions (SIRs). CMEs are large-scale eruptions of magnetized plasma that can produce geomagnetic storms upon interacting with Earth’s magnetosphere ( Nieves-Chinchilla_2016 ; Kilpua_2015 ) . HSS arise from coronal holes and are often embedded within SIRs, regions where fast solar wind overtakes slower streams, leading to plasma compression, enhanced magnetic fields, and complex interface dynamics ( Jian_2006 ; Sanchez-Garcia_2024 ) . In addition, small-scale magnetic foldings known as switchbacks are ubiquitous in the near-Sun solar wind and exhibit rapid rotations of the magnetic field vector while maintaining near-constant plasma properties ( Kasper_2019 ) . Accurate detection and characterization of these diverse phenomena are crucial for space weather forecasting and for understanding solar wind dynamics. The increasing volume of high-cadence in situ observations from missions like the Parker Solar Probe (PSP) poses significant challenges for traditional analysis methods ( raouafi2023parker ) . Existing approaches include clustering and symbolic methods ( Roberts_2020 ; Camerra_2010 ) , deep learning-based detectors ( Nguyen_2019 ; Rudisser_2022 ; Nguyen_2025 ) , and physics- or threshold-based algorithms ( Hu_2018 ; Fargette_2021 ; Laker_2022 ; Raouafi_2023 ) . While these methods have demonstrated success in specific contexts, they are often limited to one or two types of events, computationally expensive, or difficult to interpret. Moreover, many techniques treat SIRs, CMEs, and switchbacks separately, missing opportunities to apply common diagnostics across phenomena. To overcome these limitations, we use a pipeline combining iSAX symbolic compression ( Camerra_2010 ) , HDBSCAN clustering ( campello-etal-2013 ) , and human-in-the-loop validation ( kobayashi2025cipherscalabletimeseries ) . PSP time series are segmented and encoded to preserve temporal patterns at multiple scales. Experts inspect representative windows, considering all relevant parameters, including the magnetic deflection angle \theta_{B} , which sharply rotates in switchbacks, varies gradually across SIRs, and rotates smoothly within CME ejecta. Labels from sampled windows are extrapolated to entire clusters, producing consistent, expert-validated catalogs. This approach enables scalable, interpretable identification of solar wind phenomena from 2018–2024 while highlighting \theta_{B} as a unifying diagnostic across event types. 2 Data PSP, launched in 2018, is the first spacecraft to approach the Sun as close as 10~R_{\odot} ( \approx 7 million km), providing unprecedented in situ observations of the solar wind. In this study, we use high-cadence magnetic field measurements from the FIELDS instrument and ion plasma data from SWEAP (SPC and SPAN-I instruments) ( Bale_2016 ; Kasper_2016 ) . We focus on the following key parameters: magnetic field components ( B_{r},B_{t},B_{n} ), solar wind speed ( V_{SW} ), proton density ( N_{P} ), proton thermal speed ( V_{P} ). The merged 2018–2024 dataset ( martin2025scalablemachinelearninganalysis ) , exceeding 150 GB, is stored in compressed Zarr format zarr-github2025 . Linear interpolation fills small gaps, with flags preserved to indicate whether each value is observed or interpolated, enabling reliable anomaly detection. Distributed computing is supported via Dask dask , facilitating scalable analysis of these solar wind structures across multiple time scales. 3 Methodology Our pipeline consists of four steps, beginning with the preprocessing of the time series data. This step includes an optional detrending process, which removes large-scale trends, followed by smoothing to eliminate high-frequency noise monke2020optimal . The second step involves compressing the time series using iSAX ( Camerra_2010 ) , which transforms Piecewise Aggregate Approximation (PAA) coefficients into symbolic “words” through statistically defined breakpoints. Each time series is divided into fixed-length windows, referred to as the chunk size , and each window is further partitioned into smaller segments, or word size , which define the temporal resolution of the symbolic encoding. This multi-resolution representation enables scalable indexing while preserving the essential temporal patterns. As the overall number of tree nodes becomes too large to be examined and categorized directly by an expert, clustering is performed on selected levels of the iSAX index using HDBSCAN ( campello-etal-2013 ) . HDBSCAN is a density-based algorithm that groups similar sequences while labeling low-density regions as noise. One hyperparameter, min_cluster_size , defines the minimum number of sequences required to form a valid cluster, while another, min_samples , controls how strictly the algorithm classifies points in low-density regions as noise. To handle unassigned points, the pipeline can optionally re-cluster them under relaxed density constraints. Figure 1 illustrates representative clusters obtained from our pipeline. Nine randomly selected clusters from a given experiment are shown, highlighting the mean and confidence intervals of the constituent windows. Figure 1: Representative summary of nine randomly selected clusters from one experiment, out of more than 150 total clusters. For each cluster, the mean time series is shown in blue, with the 5%–95% confidence intervals indicated in red shading. Cluster identifiers are indicated in the top-left corner, and the number of constituent windows (labeled as “segments” in the figure) is shown in the bottom-left corner. Across most clusters, variation around the mean is small, except for clusters 60 and 96, which exhibit slightly larger spread. Notably, the relationship between the mean and confidence intervals is clear, highlighting the characteristic temporal patterns captured within each cluster. For the final step, a domain expert evaluates representative windows from each cluster to validate physical consistency, resolve ambiguities, and assign meaningful labels. During this inspection, the expert considers all the corresponding PSP parameters for the same time intervals to ensure contextual coherence. The resulting labels are then propagated to all members of the cluster, yielding a robust expert-validated catalog of solar wind events. In addition to the primary PSP measurements ( B_{r} , B_{t} , B_{n} , V_{SW} , N_{P} , V_{P} ), we include the derived parameters the total magnetic field ( |B| ), the magnetic deflection angle ( \theta_{B} ), and the proton temperature ( T_{P} ) to support the identification of shocks, SIRs, CMEs, and magnetic switchbacks. Among these, we hypothesize that \theta_{B} , defined from the three field components in the Radial–Tangential–Normal coordinate system ( B_{R},B_{T},B_{N} )