LA-UR-25-31801 Approved for public release; distribution is unlimited. Title: Data Analysis, Machine Learning, & More Author(s): Wolf, Carter Matthew Intended for: Report Issued: 2025-12-03 Los Alamos National Laboratory, an affirmative action/equal opportunity employer, is operated by Triad National Security, LLC for the National Nuclear Security Administration of U.S. Department of Energy under contract 89233218CNA000001. By approving this article, the publisher recognizes that the U.S. Government retains nonexclusive, royalty-free license to publish or reproduce the published form of this contribution, or to allow others to do so, for U.S. Government purposes. Los Alamos National Laboratory requests that the publisher identify this article as work performed under the auspices of the U.S. Department of Energy. Los Alamos National Laboratory strongly supports academic freedom and a researcher's right to publish; as an institution, however, the Laboratory does not endorse the viewpoint of a publication or guarantee its technical correctness. Data Analysis, Machine Learning, & More The Work I’ve Done at LANL through the SULI Program Carter Wolf At LANL, I have had the privilege and pleasure of working with Dr. Tika Kafle, an optics and condensed matter physicist at the MagLab in TA-35. Dr. Kafle’s research primarily involves the Angle Resolved Photoemission Spectroscopy (ARPES) technique of characterizing materials. More specifically, Dr. Kafle works with Time Resolved ARPES (Tr-ARPES), which extends this technique into the time domain. ARPES involves bombarding a sample with a high intensity beam of polarized and focused light which ejects the sample’s electrons via the photoelectric effect. Depending on the energy and momentum of the electrons, they will eject from the sample in a particular direction and land on the analyzer at a specific point. This allows us to visualize the electron and band structure of the sample. To extend this technique into the time domain, the sample is hit by another beam, called the pump beam, which excites the electrons. By varying the time delay between the pump beam and the photoemission beam, called the probe beam, we can visualize the electrons in their excited states as well as during the process of relaxation. This technique gives us an even more valuable insight into how electronic band structures and the electron dynamics affect certain materials such as superconductors, graphene, and topological insulators. While I have yet been unable to run a proper Tr-ARPES system at LANL, I have worked with the data that Dr. Kafle has taken in previous experiments and processed in certain ways such as plotting, cutting, fitting, and denoising. This data processing has taken most of my time here so far at LANL. While my initial expectation at LANL was to be able to work in the lab doing hands on laser experiments, this would not be the case. In order to obtain authorization to work unsupervised in the lab, I was required to fulfill the safety requirements for the lab which included training that took close to two months to complete. As a result, my work in the lab has unfortunately been limited. During this time, Dr. Kafle assigned me two primary projects: to plot and analyze ARPES data from his previous experiments and develop a neural network which could input noisy data from ARPES experiments and train a model which would mitigate or even completely remove any noise from the data. 1. Plot Fitting Many scientists who work with ARPES data standardly use the program IgorPro to input raw data, manage large databases, and generate plots. However, Dr. Kafle wants to transition to pyARPES, a Python module which can theoretically match the capabilities of IgorPro. The benefit of pyARPES is the added flexibility of running code in a Python notebook which allows for implementation of added features beyond the limitations of IgorPro. In particular, Dr. Kafle is interested in adding AI features to ARPES data analysis such as faster and automated graphics, more accurate characterization of the sample material, and most importantly denoising of experimental measurements. Figure 1: The two types of ARPES data to be analyzed. The left is from NaCd4As3, a topological insulator and the right is from YMn6Sn4Ge2, a Kagome material. In the left graph, the relevant structure is apparent. In the right, it is only faintly visible, showing as a slightly greener blob around 17.0 ev The relevant data looks like one of the two graph types shown in Figure 1. Since the data file itself only contains the intensity values, which is displayed via the colored gradient, the energy and momentum axes need to be input manually based on the specifications of the analyzer. By itself, this data cannot be directly fit to a curve. To remedy this, we can take a particular slice of the data to generate an Energy Density Curve (EDC). This is usually done by fixing a particular momentum and specifying an energy range to display a relationship between energy and intensity. In our case, we instead sum over a range of momenta to create a more accurate and smoother graph. Using this EDC, we can then fit the graph to a particular formula. In our case, we use a Fermi-Dirac distribution multiplied with a Lorentzian distribution, which is shown below. Here, I0 is constant background, I1 is linear background, A is amplitude, ELP is the Lorentizan peak energy, Γ is the curve width, EF is the Fermi energy, KB is the Boltzmann constant, and T is the electron temperature. We can experimentally derive I0, I1, A, ELP, and Γ, and thus we can solve for EF and T using our fit. Due to time constraints, implementation of solving for these specific parameters was not possible, but an EDC and its resulting fit and residual can be seen in Figure 2 below. Figure 2: An EDC of ARPES data and its resulting fit and residual. This EDC was taken from data resembling that in the right side of Figure 1. Note that axes are improperly labeled due to a bug in pyARPES code; the x axis is Energy and the y axis is intensity. 2. Machine Learning Due to hardware and experimental apparatus limitations, ARPES data can be quite noisy, as is seen in Figure 1. This results in EDCs that are not entirely accurate and which can significantly affect the parameters we want to solve for using curve fitting. Even small errors in the EDC can change, for example, the electron temperature to a degree that it resembles that of the material’s static state. The solution to this problem is denoising. Many denoising techniques exist which have been successfully applied to ARPES data, but recent papers1,2 have shown that machine learning techniques have the potential to outperform traditional methods. Dr. Kafle wants to apply machine learning on his own data to refine measurements and provide more accuracy. For the model’s architecture, I implemented a U-net architecture (see Figure 3) with a total of 15 training layers. The same model was used with success in a paper by Lehtinen et al¹. with a similar number of layers. Another paper by T. Takaai and M. Tsutsui² suggests that a different model architecture called Convolutional Auto-Encoder may better preserve the structure of the data, but that U-net still generally performs slightly better in terms of Mean Square Error. Most of the time, when a model is training to denoise data, it compares its denoising attempts to a clean version of the data it is using to train, but in our case no clean data exists to use for comparison. One solution would be to simulate clean data, but this is not feasible in our case. What I have done to remedy this issue is implement a training method called Noise2Noise², which uses pairs of independent noisy data to essentially pick out similarities and discard differences. With enough pairs of data, the Noise2Noise method becomes an effective tool despite the lack of clean data. Figure 3: The general structure of the U-net architecture. To train the model, I fed it a total of 28 pairs of data, allocating 20 pairs for training and 8 for validation. The data comes from two independent data acquisition times with similar time delays for each measurement. By making the delay time consistent between each data set in the pair, the model should theoretically perform the best, as the signal will be as close as possible while the noise is still independent. Since the size of the dataset is relatively small, I supplemented the number of training samples by patching each sample, that is cropping each data set to a smaller size, making sure the patch location is consistent between pairs. At first, this patching was done randomly, but I found that systematically patching data to ensure each data point is represented, then training on the patches in a random order provided the best results. For the training parameters, I varied each one to optimize performance, and found that a patch size of 64x64, 2 training sets (4 total patches) per batch, and 200 batches per epoch worked the best. I varied the number of epochs to compare, the best results of which are shown below in Figure 4. During some tests, I tried upwards of 50 epochs, but this result was very overfit and contained many artifacts, so I limited myself to 10 at the maximum. Figure 4: One measurement of YMn6Sn4Ge2 at -1500 ns fed through the trained model at various training lengths. 3 and 5 epochs has visible artifacts, though the artifacts in the 5 epoch result are very dampened and only visible in the residual. The 7 epoch result is very promising. Note the axes on the fit graph are still mislabeled as in Figure 2 Figure 5: A graph of each EDC for a given dataset arranged by time delay. The EDCs shown in this plot have had their background subtracted By taking EDCs of data shown in Figure 1 at each time delay and plotting them with respect to their time delay, a plot like that in Figure 5 can be created. We would also like to denoise this type of plot, but a different training protocol is required. Instead of training based on pairs, the initial idea was to selectively train the background towards zero. The hypothesis was that by then running the background and signal through the model that it would remove noise. However, as seen in Figure 6, this was not the case. Instead, I believe that the model is predicting the background noise for the data. What can be done then is to subtract this predicted background noise from the signal and then train a second model to denoise the background subtracted data using the same Noise2Noise training method. A potential problem with this method is that we currently only have six data sets of this type, so the model may be much more prone to overfitting than one with fifty six data sets. Figure 6: Time dependent EDC graphs before and after being run through a model which attempts to eliminate background. Seemingly, the model almost completely destroys the signal. The top two graphs show data from YMn6Sn4Ge2 while the bottom two show NaCd4As3. Note that the x axis should properly be labeled “time delay” In the near future, I plan to improve these models in a couple of different ways. First, the time dependent EDC denoising can be improved as mentioned previously by implementing a second model which can denoise background subtracted data. Additionally, it will be worthwhile to compare the results from denoising with machine learning to more traditional denoising methods such as Gaussian Denoising or Principal Component Analysis. One of our biggest issues is the small amount of data, so it may very well be the case that traditional methods will perform better than the neural network until we obtain more data. Once Dr. Kafle and I are satisfied with the level of denoising the model can deliver, I plan to return to work on the plot fitting of EDCs. References ¹Lehtinen, J., Munkberg, J., Hasselgren, J., Laine, S., Karras, T., Aittala, M. & Aila, T.. (2018). Noise2Noise: Learning Image Restoration without Clean Data. Proceedings of the 35th International Conference on Machine Learning, in Proceedings of Machine Learning Research. ²Takaai, T., & Tsutsui, M. (2021). Unsupervised Noise Reduction for Nanochannel Measurement Using Noise2Noise Deep Learning. In Trends and Applications in Knowledge Discovery and Data Mining