ANL/NSE-26/35 Large Language Model for Validation, Optical Calibration, and Learning (VOCAL) Distributed Temperature Sensing Interface SULI Report Nuclear Science and Engineering Division About Argonne National Laboratory Argonne is a U.S. Department of Energy laboratory managed by UChicago Argonne, LLC under contract DE-AC02-06CH11357. The Laboratory’s main facility is outside Chicago, at 9700 South Cass Avenue, Argonne, Illinois 60439. For information about Argonne and its pioneering science and technology programs, see www.anl.gov. Document availability Online Access: U.S. Department of Energy (DOE) reports produced after 1991 and a growing number of pre-1991 documents are available free at OSTI.GOV (http://www.osti.gov/), a service of the U.S. Dept. of Energy’s Office of Scientific and Technical Information Reports not in digital format may be purchased by the public from the National Technical Information Service (NTIS): U.S. Department of Commerce National Technical Information Service 5301 Shawnee Rd Alexandria, VA 22312 www.ntis.gov Phone: (800) 553-NTIS (6847) or (703) 605-6000 Fax: (703) 605-6900 Email: orders@ntis.gov Reports not in digital format are available to DOE and DOE contractors from the Office of Scientific and Technical Information (OSTI): U.S. Department of Energy Office of Scientific and Technical Information P.O. Box 62 Oak Ridge, TN 37831-0062 www.osti.gov Phone: (865) 576-8401 Fax: (865) 576-5728 Email: reports@osti.gov ANL/NSE-26/35 Large Language Model for Validation, Optical Calibration, and Learning (VOCAL) Distributed Temperature Sensing Interface SULI Report prepared by Evan Hong1,2 and Alexander Heifetz1 1 Nuclear Science Engineering Division, Argonne National Laboratory 2 Department of Nuclear Engineering, North Carolina State University July 30, 2026 Table of Contents Table of Contents .............................................................................................................................1 List of Figures ..................................................................................................................................2 List of Tables ...................................................................................................................................3 Abstract ............................................................................................................................................4 1. Introduction ................................................................................................................................5 2. Methodology ..............................................................................................................................6 2.1 Large Language Model Framework...........................................................................................6 2.2 VOCAL GUI Core Modules ......................................................................................................7 3. Results and Analysis ..................................................................................................................8 3.1 Validation of LLM Assistant Performance ..............................................................................10 3.2 Validation of RAG System Performance .................................................................................11 4. Conclusions ..............................................................................................................................13 References ......................................................................................................................................15 List of Figures Figure 1 – Flowchart of VOCAL GUI with LLM .......................................................................... 7 Figure 2 – Screenshot of the VOCAL GUI validation page from an example run after executing the LSTM autoencoder and the LLM ............................................................................................. 8 Figure 3 – Example of LLM responses......................................................................................... 11 Figure 4 – Agreement rates for different perturbations ................................................................ 12 Figure 5 – Agreement rates for different perturbations ................................................................ 12 Figure 6 – Prompt perturbation heatmap for each question .......................................................... 13 2 List of Tables Table 1 – Examples of questions in LLM performance validation study. ...................................... 9 Table 2 – LLM performance evaluation with BERTScore and SBERT metrics .......................... 10 3 Abstract Distributed temperature sensing (DTS) using fiber optic sensors (FOS) offers a promising method for temperature measurements in advanced reactors, such as sodium fast reactors and molten salt cooled reactors. To support the calibration and validation of DTS measurements, Argonne National Laboratory developed the Validation, Optical Calibration, and Learning (VOCAL) software package. This report describes the integration of a local large language model (LLM) with a retrieval-augmented generation (RAG) system into the VOCAL interface to serve as an interactive user assistant. The LLM framework enhances the VOCAL platform’s accessibility to users by explaining interface components, clarifying inputs and outputs, and answering user queries dynamically in real-time. The accuracy of the LLM assistant performance was evaluated with 20 queries regarding the interface and its parameters using experimental data from the Thermal Hydraulic Experimental Test Article (THETA) facility. Results demonstrate that the LLM achieved a 95% accuracy rate, with a BERTScore of 0.8816 and SBERT value of 0.7417. Furthermore, validation of the RAG system within the LLM framework showed optimal accuracy with k-values between 1 and 2 using the k-refinement convergence test. The prompt perturbation analysis demonstrated good initial consistency for the RAG system, exhibiting the highest accuracy under punctuation variations and the greatest sensitivity under query reordering. Notably, the model’s errors were limited to data retrieval failures rather than factual hallucinations, reinforcing its baseline reliability. The integration of LLM provides a highly accurate, user- friendly enhancement to the VOCAL platform without disrupting its core computational capabilities for FOS calibration and validation. 4 1. Introduction Coolant temperature is a key variable of interest for monitoring reactor power to ensure safety and optimal performance. The temperature of reactor fluid is conventionally measured using nuclear-grade thermocouples (TC) or resistance temperature detectors (RTD) [1-4]. Distributed temperature sensing (DTS) with fiber optic sensors (FOS) has emerged as a promising approach for temperature sensing in advanced nuclear reactors that operate with high-temperature fluids, such as sodium fast reactors (SFR) and molten salt cooled re-actors (MSCR) [5-9]. In contrast to multi-point TCs, FOS can measure reactor coolant temperature using fewer physical sensor units and fewer penetrations through the fluid vessel pressure boundary while having a higher spatial density. Furthermore, while multi-point TCs require extensive electrical cabling to connect each sensor to a data acquisition system, FOS requires significantly less wiring, thereby reducing the risk of fires caused by sparks. Therefore, using DTS is expected to contribute to reduction of operating and maintenance costs in advanced reactors [1]. While there exist several types of FOS, the highest spatial density of measurements using inexpensive telecommunication fibers can be accomplished with Rayleigh scattering fibers using optical frequency domain reflectometry (OFDR) [5-9]. Rayleigh scatterers consist of intrinsic subwavelength defects, such as nano-scale air bubbled trapped in the silica fiber during manufacturing. Temperature measurements are performed with a silica fiber without cladding inserted into a protective metallic capillary tube, so that the fiber is not wetted by the fluid. Coolant temperature transients are transmitted to the fiber through the wall of the capillary tube, and cause local silica thermal expansion, which changes the density of Raleigh scatterers [5-8]. The quantity measured with OFDR is an optical frequency shift, which can be converted to temperature via power law correlation. In a standard approach the power law coefficients are determined through a calibration procedure prior to fiber deployment in the thermal hydraulic system. We have introduced an alternative approach, where the power law coefficients are determined in-situ for the FOS installed in a thermal hydraulic systems using a spatially co-located reference TC. Because silica fiber has much lower resilience to ionizing radiation compared to the nuclear grade TCs, for early detection of degradation we introduce a procedure for validation of FOS measurements. This procedure involves reconstruction of FOS measurements using a co-located sparse multipoint TC array and a long-short term memory (LSTM) autoencoder [5-8]. To support calibration and validation of Rayleigh scattering FOS measurements in a high- temperature fluid advanced reactor, Argonne National Laboratory (ANL) developed the Validation, Optical Calibration, and Learning (VOCAL) graphical user interface (GUI) [5]. VOCAL provides users with an integrated platform for optical calibration, sensitivity anal-ysis, and machine learning (ML) validation of FOS measurements using a sparse, co-located thermocouple array [5]. The performance of this toolkit was previous demonstrated using experimental data obtained from the Thermal Hydraulic Experimental Test Article (THETA) of the Mechanism Engineering Test Loop (METL) facility at ANL, which provides a benchmark SFR environment for testing with reactor-grade sodium [5]. 5 This report describes the integration of a large language model (LLM) framework into the VOCAL GUI. The LLM serves as an interactive assistant that explains to the user the VOCAL interface components, clarifies input and output parameters, and answers user questions during operation [10]. Ultimately, this integration improves the platform’s accessibility, allowing users without extensive backgrounds in ML or uncertainty quantification (UQ) to effectively utilize the capabilities of the VOCAL software package. 2. Methodology 2.1 Large Language Model Framework To ensure strict data privacy and security, a local LLM framework was selected for the implementation of the VOCAL interface. The LLM assistant utilizes the Llama 3.2 model via the Ollama library [11,12]. This lightweight (2.0 GB), instruction-tuned, text-to-text model is optimized for agentic retrieval, context understanding, and summarization tasks, aligning directly with the requirements of the VOCAL platform. As a state-of-the-art lightweight model, it offers an optimal balance between low computational cost and high reasoning capability for interpreting the user prompts. To further enhance the performance, the local LLM incorporates a retrieval- augmented generation (RAG) system, which combines the model’s internal knowledge with external, verified knowledge relevant to the user’s query. The LLM is integrated into the VOCAL GUI as a modular, chatbot-style assistant, as shown in the flowchart in Figure 1. This modular implementation ensures that potential environ-ment conflicts with the LLM dependencies do not disrupt the core functionalities of the GUI’s computational toolkit. As shown in Figure 2, users can pose questions directly to the LLM through the interface sidebar. The LLM dynamically accesses the inputs and outputs displayed on the active page, interprets the query, and returns a context-specific response. To preserve continuity, the chat history remains intact as the user navigates across different pages of the GUI. However, maintaining this conversational history consumes computational resources, which may slow down primary VOCAL GUI tasks such as calibration or validation. To mitigate this performance latency and free up local memory, a "Clear chat" button is provided, allowing users to reset the context when transitioning to a new topic. 6 Figure 1 – Flowchart of VOCAL GUI with LLM 2.2 VOCAL GUI Core Modules The overall structure of the VOCAL GUI is comprised of five pages: the homepage, data loading, calibration, sensitivity analysis, and validation, as outlined within the red box in Figure 2. The homepage introduces the core capabilities of the toolkit, displays the current VOCAL status, and outlines the steps to get started. The data loading page serves as the critical entry point, enabling users to generate synthetic data, utilize real experimental data from the THETA facility, or upload custom datasets. Conversion of measured OFDR frequency shifts S(GHz) to temperature T(◦C) is accomplished using the power law correlation [5-7] T = aSb + c (1) The calibration page performs the optimization of a, b, and c parameters using the Markov Chain Monte Carlo (MCMC) Adaptive Metropolis method, a Bayesian inverse UQ method [5-7]. The sensitivity analysis page allows users to rank thermocouple sensor locations to optimize sensor placement for accurate temperature reconstruction. Users can apply qualitative Morris screening, which varies individual parameters to compute elementary effects, quantitative Sobol’ indices, which decompose output variance into parameter interactions, or both [5-7]. Finally, the validation page evaluates loaded data using a non-symmetric LSTM autoencoder that reconstructs the high spatial density temperature map from the co-located sparse thermocouple array 7 measurements to compare with the real-time temperature map obtained with FOS [5,8]. The interface displays the predicted temperature profile, measured FOS profile, and residual error map. Figure 2 – Screenshot of the VOCAL GUI validation page from an example run after executing the LSTM autoencoder and the LLM 3. Results and Analysis After integrating LLM into the GUI sidebar, we performed preliminary demonstration of the capability of the local LLM to answer general domain questions regarding VOCAL’s underlying methodologies and specific queries about real time interface data, such as validation test results. As indicated by the blue arrows in Figure 2, the LLM assistant dynamically accesses generated output values and incorporates them into its response. To validate the performance of the integrated assistant, the model was tested against a benchmark of 20 questions compiled in Table I. These questions are categorized by interface section and represent realistic queries a user might submit when navigating the VOCAL GUI. Each section includes a balanced mix of high-level operational questions and technical questions. The initial evaluation measured the accuracy of the integrated LLM based on a binary metric: whether it provided the correct answer relative to an established ground truth. This ground-truth baseline was determined by the authors. To evaluate performance of the LLM assistant, we utilized experimental data from DTS measurements in the THETA liquid sodium facility. This data was previously used in the development of the VOCAL interface [5]. To ensure independence between questions, the LLM was prompted individually for each question after clearing the chat history. This establishes a uniform baseline to ensure the preceding 8 outputs do not influence subsequent responses through the model’s context window. The evaluation results are displayed in Table I as either Yes (Y), indicating the LLM provided a response aligned with the ground truth, or No (N), indicating a failure to match the ground truth. This evaluation approach was selected because factual precision is essential for nuclear technologies because of strong emphasis on rigorous safety standards. Table 1 – Examples of questions in LLM performance validation study. Section Question Y/N Data How many thermocouple sensor locations are there in this dataset? Y Loading What is the time range for this dataset? Y What is the min and max temperature in this dataset? Y What is the fiber and thermocouple shape in this dataset? Y What dataset is currently loaded and how large is it? Y Calibration How does MCMC DRAM method work? Y What is the acceptance rate for MCMC calibration? Y What does the posterior distributions mean? Y What are the initial calibration parameters and its bounds? Y What are the calibration results? Y Sensitivity Which sensor location was the most influential in this dataset? Y Analysis What does Morris Screening mean? Y What is the difference between Morris Screening and Sobol’ Indices? Y What are the values of the sensitivity analysis results? Y What does S1 and ST stand for in Sobol’ Indices? Y Validation What are the test metrics values? Y What is the architecture of this LSTM autoencoder? N What are the final training and validation loss values? Y What is the best validation loss value and its corresponding epoch? Y What does the test metrics mean? Y Further evaluation was conducted using BERTScore, an established metric in literature for text generation [13-16]. Specifically, a 24-layer RoBERTalarge model was utilized to compute BERTScore for English language text generation [14,15]. The BERTScore ranges from 0 to 1 and evaluates text semantic correctness at the token level by comparing the response generated by the LLM against a reference answer [14,15]. To complement this analysis, Sentence-BERT (SBERT) similarity scores was also computed [15]. While BERTScore captures token-scale alignments, SBERT measures the overall semantic similarity of complete sentences, providing a comprehensive evaluation of the LLM’s performance [16]. To aid the interpretation of these metrics, qualitative evaluation thresholds were established for both scoring systems. Because no universally accepted thresholds have been defined for either metric, this study adopted a BERTScore threshold of 0.80. This value is chosen based on recent 9 work in a related clinical domain indicating that values above 0.80 are generally considered acceptable for clinical text evaluation [16]. For the SBERT metric, the threshold value of 0.75 was adopted [15]. Additionally, verification and validation (V&V) of the RAG-LLM system was conducted using methods established in our recent prior work, where prompt perturbation analysis and RAG k-refinement convergence testing were utilized [13]. The prompt perturbation analysis evaluates the consistency and sensitivity to query variations, such as synonym substitutions and typos [13]. The RAG k-refinement convergence test evaluates the accuracy of RAG system across varying values of k-value to optimize the RAG system. 3.1 Validation of LLM Assistant Performance As shown in Table I, the LLM correctly answered all five questions in the data loading section following dataset in-gestion. As summarized in Table II, this section achieved a mean BERTScore of 0.8534 and a mean SBERT score of 0.7194. For the calibration page, parameters were autotuned, and MCMC calibration was fully executed using the THETA dataset prior to prompting the LLM assistant. This section achieved a 100% correctness rate, with the LLM accurately answering all five validation questions against the ground-truth baseline. The average BERTScore for this section was 0.9070, and the mean SBERT score was 0.8297. In the sensitivity analysis section, automated parameter tuning was completed, and both Morris screening and Sobol’ indices were calculated before evaluating the model. This section similarly yielded a 100% correctness rate on its validation queries, achieving a mean BERTScore of 0.8700 and an average SBERT score of 0.6347. Finally, on the validation page, quick auto-tuning and LSTM validation were finalized before evaluating responses to the questions listed in Table I. This section matched the ground truth in four out of five evaluation queries, yielding an average BERTScore of 0.8962 and an average SBERT score of 0.7832. In summary, the preliminary evaluation demonstrates that the local LLM successfully answered 19 out of 20 questions listed in Table I, achieving an overall accuracy rate of 95%. The global mean BERTScore was 0.8816, and the overall mean SBERT score was 0.7417, with a section-by-section breakdown de-tailed in Table 2. Table 2 – LLM performance evaluation with BERTScore and SBERT metrics Section BERTScore SBERT Data Loading 0.8534 0.7194 Calibration 0.907 0.8297 Sensitivity Analysis 0.87 0.6347 Validation 0.8962 0.7832 Overall, the LLM exhibited moderate-to-high semantic similarity relative to ground-truth answers. An example of the LLM’s query response is shown in Figure 3. In the example case shown, the LLM correctly retrieved the minimum and maximum temperatures, but extracted the 10 final validation loss value rather than the best validation loss. Notably, the model consistently provided detailed reasoning and technical explanations for all its responses, even when the final output contained this minor retrieval error. Most importantly, the complete absence of false positives indicates that the model did not fabricate metrics or hallucinate data, reinforcing its baseline reliability for LLM deployment. Figure 3 – Example of LLM responses 3.2 Validation of RAG System Performance The RAG k-refinement convergence test evaluated the k-value ranging from 1 to 20, as illustrated in Figure 4, using the questions in Table 1. From this validation method, the optimal k- value range was 1 to 2, which yielded the highest selection accuracies. Prompt perturbation analysis was applied to the LLM framework within VOCAL to evaluate model sensitivity to variations in the queries listed in Table 1. The five changes were synonyms replacement, phrase reorder, punctuation changes, filler words, and typos. The agreement rates across these perturbation types are shown in Figure 5. The highest accuracy and the lowest sensitivity were observed for punctuation alterations, which achieved an 85.9% agreement rate. The lowest accuracy occurred with word reorder perturbations, yielding a 61.1% accuracy rate. Additionally, Figure 6 presents the agreement rate of each perturbation type for each individual question. 11 Figure 4 – Agreement rates for different perturbations Figure 5 – Agreement rates for different perturbations 12 Figure 6 – Prompt perturbation heatmap for each question 4. Conclusions This work presented a preliminary demonstration of an integrated LLM within the VOCAL GUI, designed to assist users in interpreting the interface inputs and outputs and answering context-specific queries with high accuracy. In the preliminary evaluation, the local LLM (llama3.2) achieved a 95% accuracy rate, with all errors restricted to retrieval failures rather than 13 factual hallucinations. Furthermore, text generation metrics confirmed the baseline performance of the assistant, yielding a global average BERTScore of 0.8816 and an SBERT score of 0.7417. In addition, the RAG system within the VOCAL local LLM framework displayed a promising initial performance through k-refinement convergence test and prompt perturbation analysis. The k-refinement convergence test indicated an optimal k-value range of 1 to 2 for this framework. Meanwhile, the prompt perturbation analysis showed that the RAG system maintained a good overall accuracy of 72.4% across five perturbation types, with the highest accuracy in the punctuation perturbation at 85.9%. These results establish a solid foundation with moderately good semantic alignment and consistency, though further refinement is required for deployment in a reactor test facility. Additionally, the modular architecture ensures that the VOCAL GUI maintains its high performance in calibrating and validating FOS measurements regardless of whether the LLM module is active. Future work will investigate the integration of AskSage, ANL’s internal suite of LLMs, to leverage powerful, frontier models that offer greater capabilities than the baseline local framework. Additionally, future work will also focus on the development of LLM evaluations (’evals’) and explore more quantitative evaluation methods to better assess and understand the performance of LLM framework within VOCAL for reactor test facility deployment. 14 References 1. A. Heifetz, M. Weathered, N. Hoyt, M. Anderson, S. Sanders, A. Cilliers, “New Sensing Technologies can Reduce Operating and Maintenance Costs to Ensure Advanced Reactors’ Economic Viability,” Nuclear News 64(18), 46-51 (2021). 2. S. Pantopoulou, A. Cilliers, L.H. Tsoukalas, A. Heifetz, “Transformers and Long Short-Term Memory Transfer Learning for GenIV Reactor Temperature Time Series Forecasting,” Energies 18(9), 2286 (2025). 3. S. Pantopoulou, V. Ankel, M.T. Weathered, D.D. Lisowski, A. Cilliers, L.H. Tsoukalas, A. Heifetz, “Monitoring of Temperature Measurements for Different Flow Regimes in Water and Galinstan with Long Short-Term Memory Networks and Transfer Learning of Sensors,” Computation 10, 108 (2022) 4. A. Heifetz and R.B. Vilim, “Eigendecomposition Model of Resistance Temperature Detector with Applications to S-CO2 Cycle Sensing,” Nuclear Engineering and Design 311, 60-68 (2017). 5. L. Kohler, M. Weathered, and A. Heifetz, “VOCAL: Validation, optical calibration and learning interactive graphical interface for distributed temperature sensing in advanced reactors,” Proceedings of Advances in Thermal Hydraulics (ATH 2024), 356-365 (2024). 6. L. Kohler, D. Lisowski, and A. Heifetz, “Global Sensitivity Analysis of Distributed Fiber Optic Temperature Measurements in a Thermal Hydraulic System,” Proceedings of International Congress on Advances in Nuclear Power Plants (ICAPP 2024), 16-19 (2024). 7. L. Kohler, D. Lisowski, M. Weathered, X. Wu, and A. Heifetz, “Bayesian calibration and sensitivity analysis of Rayleigh scattering fiber optic distributed temperature sensing in water flow loop,” Nuclear Science and Engineering 199, 2129–2142 (2025). 8. L. Kohler, D. Weathered, A. Cilliers, and A. Heifetz, “LSTM Autoencoder Prediction of Distributed Temperature in Liquid Sodium Using Measurements with Co-Located Fiber Optic Sensor and Sparse Multi-Point Thermocouple Array,” Transactions of American Nuclear Society 129(1), 95-97 (2023). 9. S. Pantopoulou, M. Weathered, D. Lisowski, L. H. Tsoukalas and A. Heifetz, "Temporal Forecasting of Distributed Temperature Sensing in a Thermal Hydraulic System with Machine Learning and Statistical Models," IEEE Access 13, 10252-10264 (2025). 10. J. Clifford, J. White, A. Heifetz, A. Hawari, X. Wu, “Enhancing Learning for Nuclear Reactor Operator Trainees with Large Language Models,” Proceedings of Nuclear Plant Instrumentation and Control & Human-Machine Interface Technology (NPIC&HMIT 2025). 11. “Ollama documentation,” https://docs.ollama.com/ (2024). 12. “Llama 3.2,” https://ollama.com/library/llama3.2 (2024). 13. P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.- T. Yih, T. Rocktäschel, et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” Advances in Neural Information Processing Systems (NeurIPS) 33, 9459–9474 (2020). 15 14. T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “BERTscore: Evaluating Text Generation with BERT,” ArXiv abs/1904.09675 (2019). 15. N. Reimers and I. Gurevych, “Sentence-BERT: Sentence Embeddings using Siamese BERT- Networks,” Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 3982–3992 (2019). 16. A. Z. Z. Aghasy, M. L. Lazuardi, and H. K. Josef, “Benchmarking Large Language Models for Dental Clinical Decision Support: A BERT Score Analysis of Claude Opus 4.5,” International Journal of Advanced Computer Science & Applications 17(1) (2026). 16 Nuclear Science and Engineering (NSE) Division Argonne National Laboratory 9700 South Cass Avenue, Bldg. 208 Argonne, IL 60439 www.anl.gov Argonne National Laboratory is a U.S. Department of Energy laboratory managed by UChicago Argonne, LLC