Using the Open Science Data Federation for data distribution: Big Bear Solar Observatory use case Conference: Practice and Experience in Advanced Research Computing; July 20–24,2025; Columbus, Ohio CCS: Information systems Information storage technologies CCS: Networks Network performance analysis CCS: Networks Network measurement
Sydney Montiel, Alexsandra Guadarrama, Frank Würthwein, Fabio Andrijauskas · 2026
Read Using the Open Science Data Federation for data distribution: Big Bear Solar Observatory use case Conference: Practice and Experience in Advanced Research Computing; July 20–24,2025; Columbus, Ohio CCS: Information systems Information storage technologies CCS: Networks Network performance analysis CCS: Networks Network measurement by Sydney Montiel, Alexsandra Guadarrama, Frank Würthwein, Fabio Andrijauskas (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-15378.txt.
Introduction
Sydney Montiel, Alexsandra Guadarrama, and Fabio Andrijauskas describe using the Open Science Data Federation for Big Bear Solar Observatory data distribution. The federation extends StashCache into a global network with 20 origins and 30 caches plus new access methods and monitoring. Bringing observatory data into the federation gave standard, reliable worldwide access through local caches. A pipeline built on it can apply image processing to every observatory image from anywhere on the planet. It shows the federation's value for data-intensive solar science.
Catalog record
| CREATOR | Sydney Montiel, Alexsandra Guadarrama, Frank Würthwein, Fabio Andrijauskas |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 7771 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.15378
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Using the Open Science Data Federation for data distribution: Big Bear Solar Observatory use case Conference: Practice and Experience in Advanced Research Computing; July 20–24,2025; Columbus, Ohio CCS: Information systems Information storage technologies CCS: Networks Network performance analysis CCS: Networks Network measurement Sydney Montiel, Alexsandra Guadarrama, Frank Würthwein, Fabio Andrijauskas Abstract . The growing demand for extensive data processing is now a standard in many scientific fields. Efficiently distributing data to processing sites and enabling seamless sharing has become crucial. The Open Science Data Federation (OSDF) builds on the success of the StashCache project to establish a global data distribution network. By expanding StashCache, OSDF integrates additional data origins and caches, enhancing accessibility and performance (20 origins and 30 caches), new access methods, and monitoring and accounting mechanisms. Additionally, the OSDF has become essential to the US national cyber-infrastructure landscape due to the sharing requirements of recent NSF solicitations. One use case for the OSDF is the data access to the Big Bear Solar Observatory (BBSO). Integrating the BBSO data into the OSDF provided standard and reliable data access. Moreover, the OSDF caches provide local data worldwide. Using the OSDF and the BBSO data, creating a pipeline to apply image processing techniques to all images from BBSO anywhere on the planet was possible. Figure 1. Map featuring the locations of current OSDF architectural components (https://osg-htc.org/services/osdf). 1. Introduction Data sharing has become one of the most complex challenges researchers face in a technological age. It must be stored, managed, processed, and analyzed to generate scientific advances. Therefore, current research must satisfy processing speed and memory requirements. Furthermore, such data is shared by geographically distributed users, which increases network traffic due to access paths ( 2 ) . Taking this problem as a starting point, the development of the Open Science Data Federation (OSDF) emerged. Based on the Open Science Grid (OSG) project, StashCache, the OSDF is a data access framework that provides infrastructure and tools for accessing data globally, following the notion of "Any Data, Anytime, Anywhere," making data access more efficient for users. It supports large amounts of data from several independent experiments and funded projects by the National Science Foundation (NSF). This paper aims to show an OSDF application that can access data from the Big Bear Solar Observatory (BBSO) and efficiently perform image processing for the user. 2. Open Science Data Federation OSDF builds upon the OSG project, StashCache ( 7 ; 4 ) , creating a robust research environment in collaboration with OSG. They provide users with high computational throughput and efficient data access ( 6 ) . The OSDF environment continually enhances its tools and capacity to support growth and demand while providing reliable data delivery to compute sites. Additionally, it is essential to empower staff to identify and resolve potential issues early. To facilitate data access for computational workflows running on distributed computing infrastructure, the OSG leverages the Open Science Data Federation (OSDF) ( 7 ) . At the core of this data delivery system are "origins," "caches," and "redirectors," all implemented using the XRootD ( 8 ; 3 ) software framework, which enables low-latency and scalable data access. Figure 1 shows all the OSDF host locations. The Pelican Project created a new layer for the OSDF ( 5 ) , providing centralized services for registration, cache selection, and monitoring. Origins serve as the backend storage for project data. Within OSDF, an origin is an XRootD/Pelican configuration that provides access to storage through a data transfer node that mounts a project directory. Multiple origins form a hierarchical structure that connects to a redirector, which then communicates with the cache network. Applications typically access OSDF through the nearest cache to their computing site, with proximity determined using GeoIP ( 7 ) . A containerized approach on a federated Kubernetes infrastructure is ideal for deploying and managing OSDF software and services, whether at origins or caches. This federated model enables effective monitoring of data access and troubleshooting of issues. 3. BBSO data processing The Sun generates energy by fusing hydrogen atoms into helium. Various events can occur throughout this process, including solar storms, which are particularly interesting due to their potential impact on telecommunications. These storms can disrupt GPS signals, alter or destroy satellite trajectories, and cause electricity transmission issues. For instance, geomagnetically induced currents from solar storms can overload power grids. However, such catastrophic consequences can be mitigated if solar storms are detected in time, allowing for preventive measures such as shutting down satellites or reducing power flow in transmission lines. Solar storms exhibit distinct characteristics, one of which is the emergence of filaments on the Sun’s surface. These filaments, visible in H \alpha images, consist of dense plasma held in place by magnetic fields. They appear darker than the surrounding solar surface due to their lower temperature. Figure 3 illustrates a solar storm event featuring a filament, where the dark ribbons on the Sun represent these structures ( 1 ) . Figure 2 illustrates the flow of solar image data from the Big Bear Solar Observatory through the Open Science Data Federation (OSDF) to users and back. The process begins with the observatory collecting solar images in FITS format. These images are then gathered through a synchronization mechanism and transferred to OSDF, an intermediary data storage and distribution system. Once in OSDF, the data is transmitted via Internet/Internet2 to users, who access and process the images. The image processing stage enhances the solar images, as shown in Figure 2 . After processing, the refined data is returned to the origin, completing the cycle. This workflow ensures efficient handling, distribution, and enhancement of solar observational data. Figure 2. Data flowchart of the gathered images from the BBSO to perform processing. An example of a result produced by the diffusion filter is depicted in Figure 2 . We can observe that the image is more homogeneous than the original images, but its borders are well-defined. The other steps are threshold calculation, filament extraction, and labeling. The OSDF was used to get all the required images to run this detection. Figure 3. Sun image processed by the diffusion filter with a manually detached filament ( 1 ) . 4. Conclusion The Open Science Data Framework is crucial for sharing scientific data across the United States, featuring caches and origins located worldwide. This capability is vital for high-throughput and high-performance computing. With the addition of over 10 new caches and origins, we can deliver more data to a larger number of users. The BBSO use case demonstrates that it is possible to utilize the OSDF to establish an efficient processing pipeline. Acknowledgements.