opXRD: Open Experimental Powder X-ray Diffraction Database
Daniel Hollarek, Henrik Schopmans, Jona Östreicher, Jonas Teufel, Bin Cao, Adie Alwen, Simon Schweidler, Mriganka Singh, Tim Kodalle, Hanlin Hu, Gregoire Heymans, Maged Abdelsamie, Arthur Hardiagon, Alexander Wieczorek, Siarhei Zhuk, Ruth Schwaiger, Sebastian Siol, François-Xavier Coudert, Moritz Wo · 2025
Read opXRD: Open Experimental Powder X-ray Diffraction Database by Daniel Hollarek, Henrik Schopmans, Jona Östreicher, Jonas Teufel, Bin Cao, Adie Alwen, Simon Schweidler, Mriganka Singh, Tim Kodalle, Hanlin Hu, Gregoire Heymans, Maged Abdelsamie, Arthur Hardiagon, Alexander Wieczorek, Siarhei Zhuk, Ruth Schwaiger, Sebastian Siol, François-Xavier Coudert, Moritz Wo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05577.txt.
Introduction
Daniel Hollarek and co-authors release an open database of real X-ray diffraction patterns for materials science, in a paper from 2025. The catalog creator list is cut short; the text names a long author team. Powder diffraction is a core way to read a material's crystal structure. Yet sorting the patterns by machine is still slow, which blocks robot labs.
Models trained on fake patterns falter on real ones, above all on noisy, low-grade runs. The new set holds more than ninety thousand real patterns, with over two thousand labeled. They span many classes of materials. Labeled sets let teams grade their models on true data. Unlabeled sets can retrain models toward real-world mess. The authors hope the shared pool speeds full automation of materials checks.
Catalog record
| CREATOR | Daniel Hollarek, Henrik Schopmans, Jona Östreicher, Jonas Teufel, Bin Cao, Adie Alwen, Simon Schweidler, Mriganka Singh, Tim Kodalle, Hanlin Hu, Gregoire Heymans, Maged Abdelsamie, Arthur Hardiagon, Alexander Wieczorek, Siarhei Zhuk, Ruth Schwaiger, Sebastian Siol, François-Xavier Coudert, Moritz Wo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-05577
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
opXRD: Open Experimental Powder X-ray Diffraction Database Daniel Hollarek, Henrik Schopmans, Jona Östreicher, Jonas Teufel, Bin Cao, Adie Alwen, Simon Schweidler, Mriganka Singh, Tim Kodalle, Hanlin Hu, Gregoire Heymans, Maged Abdelsamie, Arthur Hardiagon, Alexander Wieczorek, Siarhei Zhuk, Ruth Schwaiger, Sebastian Siol, François-Xavier Coudert, Moritz Wolf, Carolin M. Sutter-Fella, Ben Breitung, Andrea M. Hodge, Tong-yi Zhang, Pascal Friederich Abstract Powder X-ray diffraction (pXRD) experiments are a cornerstone for materials structure characterization. Despite their widespread application, analyzing pXRD diffractograms still presents a significant challenge to automation and a bottleneck in high-throughput discovery in self-driving labs. Machine learning promises to resolve this bottleneck by enabling automated powder diffraction analysis. A notable difficulty in applying machine learning to this domain is the lack of sufficiently sized experimental datasets, which has constrained researchers to train primarily on simulated data. However, models trained on simulated pXRD patterns showed limited generalization to experimental patterns, particularly for low-quality experimental patterns with high noise levels and elevated backgrounds. With the Open Experimental Powder X-Ray Diffraction Database (opXRD), we provide an openly available and easily accessible dataset of labeled and unlabeled experimental powder diffractograms. Labeled opXRD data can be used to evaluate the performance of models on experimental data and unlabeled opXRD data can help improve the performance of models on experimental data, e.g. through transfer learning methods. We collected 92,552 diffractograms, 2179 of them labeled, from a wide spectrum of materials classes. We hope this ongoing effort can guide machine learning research toward fully automated analysis of pXRD data and thus enable future self-driving materials labs. 1 Introduction The advent of high-throughput experiments holds the prospect of significantly accelerating the speed of materials discovery 1 . The synthesis and characterization of novel materials are becoming increasingly efficient and automated, increasing the throughput of samples in experimentation pipelines 2 , 3 , 4 . After fabricating a new material, a number of analysis techniques can be used to characterize the sample. One method that can be used for phase identification, phase quantification, grain size characterization, and to determine the crystal structure of a new material is powder X-ray diffraction (pXRD). When using pXRD measurements, crystal structures are typically determined through Rietveld refinement. In Rietveld refinement, an initial crystal structure model is fitted to the observed diffractogram by iteratively updating the structural model. Each update of the structural model seeks to minimize the difference between the observed diffractogram and the diffractogram simulated from the current structural model 5 , 6 . As Rietveld refinement is a local optimization method, the result of the refinement procedure is generally only as good as the initial structural model the process started from. Manually performing Rietveld refinement is time-consuming and often requires expert knowledge. It is not scalable to the degree required to keep up with advances in throughput and efficiency in other steps of the experimentation pipeline. The refinement process requires the operator to determine an initial structural model from which the refinement can start and as well as initial values for parameters that characterize the background 7 . The structural model is usually obtained using search-match software, which identifies crystal structures with similar powder diffraction patterns from a database of crystal structures with accompanying powder diffraction patterns. However, an initial structural model obtained from such a database is not guaranteed to lead to an accurate structure solution through Rietveld refinement, especially not for novel structures. Additionally, attempting to refine all crystal structure parameters at once is known to lead to unphysical results 4 . Hence parameters are refined iteratively, with each iteration only refining a limited set of parameters. Finding the correct order in which to refine structure parameters and finding the correct values for initial background parameters both present problems that add to the difficulty of the refinement process. Machine learning has the potential to speed up the manual analysis of powder diffractograms and keep pace with an automated high-throughput experimentation environment 8 , 9 . Models can be either trained to predict crystal structure information directly given a diffractogram, or they can be used to automate the conventional refinement workflow. In the latter case a model would first predict an initial crystal structure 9 which is then refined by a second model trained to perform the refinement process 10 . So far due to an absence of labeled datasets with experimental diffractograms 11 , machine learning in this domain has largely relied on diffractograms simulated from known structures 12 , 13 or, most recently, from generated synthetic crystals 14 . Models trained on datasets with simulated diffractograms have already shown strong performance in predicting phases 12 , 15 , 16 , lattice parameters 17 , 18 , 19 , 20 , spacegroup 21 , 14 , 22 , 12 , 23 , 24 , 25 , 26 , 20 , and crystallite size 17 , 26 from simulated diffractograms. However, the performance substantially drops off when these models are applied to data originating from experiments 21 , 14 , 20 , 11 , 23 . This discrepancy in performance arises due to imperfections in experimental data which are not present in diffraction patterns modeled under ideal conditions. This is discussed in more detail below. Both labeled and unlabeled datasets of experimental powder diffractograms hold significant value for machine learning-based pXRD analysis, particularly with regard to bridging the performance gap between simulated and experimental domains. Labeled experimental data can be used to test and benchmark existing and new automated analysis approaches. This enables researchers to gauge how well a given model would perform under real-world conditions if integrated into an automated experimentation pipeline. Unlabeled experimental data enables machine learning researchers to evaluate how closely their simulations represent experimental data and modify their simulation algorithms accordingly. Unlabeled data can also find applications in transfer learning approaches to transfer model capabilities from the domain of simulated diffractograms to the domain of experimental diffractograms. While some experimental powder databases exist, their utility is limited by the fact that they are either small or not openly accessible. In this work, we introduce an open powder X-ray diffraction (opXRD) database featuring a broad range of patterns collected from experiments. With a total of 92,552 patterns collected from 6 contributing institutions, the opXRD database exceeds the size of the previously largest database of openly accessible experimental powder diffraction data by two orders of magnitude. To the best of our knowledge, the largest database of this type is the RRUFF database, containing 1290 experimental powder diffraction patterns 27 . Larger commerical datasets such as the PDF5+ 28 and the Linus Pauling File 29 exist, but their utility is limited by fees and restrictive licenses. License terms of commercial datasets, such as the PDF5+ and the Linus Pauling File, prohibit or restrict the publication of models trained on their data. In contrast, the opXRD database is both free and imposes no restrictions on how its data is used. Fig. ( 1 ) provides an overview of machine learning workflows enabled and supported by the opXRD database. Figure 1: Experimental powder X-ray diffraction (pXRD) patterns from several contributors are collected in the opXRD database. The proposed open-access database of experimental data aims to support each step in the pXRD-related machine learning