DOE Final Technical Report Award #: DE-SC0021167 Title: Constraining microphysical processes of warm rain formulation using advanced spectral separations, an ensemble retrieval framework and machine learning techniques Recipient: Colorado State University Principal Investigator: Jui-Yuan Christine Chiu (Colorado State University) Date of Report: 01/12/2026 Covering Period: 09/15/2020 - 09/14/2025 1. Executive Summary Drizzle, a common feature of marine boundary layer clouds formed through collision- coalescence, plays a key role in cloud microphysics and evolution. Yet, simultaneously retrieving cloud and drizzle properties from remote-sensing observations remains challenging because drizzle droplets often dominate radar signals, masking cloud contributions. The goal of the proposed research is to provide constraints for the process of autoconversion and accretion using ARM cloud measurements. Specifically, we provide concurrent retrievals of cloud and drizzle that allows users to derive corresponding autoconversion and accretion rates. Drizzle, a common feature of marine boundary layer clouds formed through collision– coalescence processes, plays a critical role in cloud microphysics and cloud evolution. However, the simultaneous retrieval of cloud and drizzle properties from remote-sensing observations remains challenging because drizzle droplets often dominate radar signals, obscuring contributions from cloud droplets. The goal of this project is to provide observational constraints on autoconversion and accretion processes using ARM cloud measurements. Specifically, we deliver concurrent retrievals of cloud and drizzle properties that enable users to derive corresponding autoconversion and accretion rates. This project achieved its goal through the following objectives: O1. Improve the Ensemble Cloud Retrieval (ENCORE) method for concurrent cloud and drizzle properties, and perform comprehensive evaluations using ARM field campaign data and other Value-Added Products. O2. Develop a machine-learning based model to predict autoconversion and accretion rates using cloud and drizzle microphysical properties retrieved from O1. O3. Establish the theoretical basis for uncertainty quantification in deep learning that can be applied to process rates derived from O2. This report provides an overview of the accomplishments associated with each of these key objectives. 1 2. Overview of Research Accomplishments 2.1 Major Updates to ENCORE and Key Scientific Findings Paper [1] documents the key updates to ENCORE, the performance evaluations against airborne in situ measurements from the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign, and independent ground-based observations from the ENA site. The evaluation includes both single-layer cloud cases and more complex cloud scenes, aiming to enhance the utility of these retrievals for model evaluation and process-level studies. ENCORE has undergone several updates since its original development. First, at each time step, clouds are now classified into three categories. In addition to the original two, in-cloud drizzling and non-drizzling, we introduced a sub-cloud drizzling category. Second, a refined first- guess state better represents the vertical structure of marine boundary layer clouds. Third, for drizzling clouds, we take a 2-stage approach and reserve some cloud radar observations at the 2- stage to allow the location of the cloud water content peak to be better constrained by observations. Fourth, The extended Ensemble Kalman Filter update equation now includes a term that accounts for constraints from the prior, helping to maintain realistic retrieval uncertainty estimates even after many iterations. Fifth, for non-drizzling clouds, the method uses the zenith radiance difference between 870 nm and 1640 nm rather than the absolute zenith radiances at 440, 870, and 1640 nm, alleviating the non-monotonic dependence of absolute radiances on cloud optical depth. Finally, the updated ENCORE defines cloud base height as the lower of the two estimates from the fixed-threshold and maximum-gradient methods. Compared with coincident in situ measurements and independent ground-based retrievals, ENCORE shows overall good agreement with the evaluation datasets. Relative to in situ observations, mean biases are 3–44% for cloud droplet number concentration, –21% to 19% for cloud effective radius, and –54% to 69% for cloud water content, which are comparable to in situ measurement uncertainty. For column-integrated quantities, ENCORE-retrieved liquid water path differs by 10–50% from microwave radiometer (MWR) retrievals, while cloud optical depth shows biases of –20% to 33% relative to estimates from the multifilter rotating shadowband radiometer (MFRSR) measurements. These retrieved cloud properties yield a reasonable shortwave broadband radiation closure, with mean biases below 15% for all cases. Performance in cloud properties is robust across the ten case days examined; however, accurate cloud-base identification is critical and can be challenging in multi-layer cloud scenes. To put retrieval performance in context, we compared ENCORE with the ARM value- added product NDROP, which estimates cloud droplet number concentration using microwave radiometer–derived liquid water path and surface flux–derived cloud optical depth. Error statistics relative to in situ data show that ENCORE performs significantly better than NDROP. NDROP has substantial variability across cloud types, including pronounced overestimation and underestimation, partly attributable to mismatches in the fields of view of the microwave and shortwave flux radiometers. Improved field-of-view alignment, for example, using narrow field- of-view shortwave radiometers for optical depth retrievals could improve the stability of NDROP retrievals. 2 Beyond bulk microphysical variables, the ENCORE evaluation also incorporates two integrated cloud diagnostics that reflect vertical cloud structure and precipitation processes. In particular, ENCORE retrievals span a similar range and show a comparable dependence of cloud adiabaticity on cloud geometric thickness to that inferred from in situ estimates. Relationships derived from ENCORE also show strong overlap with MWR retrievals. These findings have important implications, as many remote-sensing retrieval techniques rely on an assumed cloud adiabaticity, which can be a major source of uncertainty in retrieved cloud properties. In contrast, ENCORE does not require such an assumption and instead enables direct investigation of cloud adiabaticity, an important measure closely linked to both precipitation formation and entrainment– mixing processes. Evaluating drizzle retrievals remains challenging because drizzle is intermittent and spatial heterogeneous. In the non-single-layer cloud subset, only one well-sampled case is available. In that case, ENCORE biases (–33.0% in drizzle number concentration, –1.7% in drizzle effective radius, and –55.9% in drizzle water content) are comparable to the uncertainty of in situ measurements. For single-layer clouds, ENCORE shows larger negative biases in drizzle water content (–78% to –87%), partly associated with unusually large drizzle sizes near cloud top observed in the in situ data but absent from ENCORE retrievals. Within single-layer clouds, cases with very low drizzle number concentrations (<0.002 cm–3) perform similarly in drizzle water content compared to cases with higher concentrations, but compensating biases between drizzle number concentration and effective radius may occur. These issues likely reflect limited sensitivity of the measurements used in ENCORE to weak drizzle or an overly high radar reflectivity threshold for defining drizzling clouds. Because sample sizes are small, more extensive collocated in situ observations are needed to confirm these findings and identify the sources of error. Unexpectedly, airborne in situ measurements on several case days indicate an increase in drizzle size with height, which contradicts the conventional understanding of growth through collision–coalescence. To assess whether vertical transport could explain this behavior, we examined drizzle size as a function of vertical air motion using in situ data and found no systematic enhancement in regions of upward motion. We also analyzed KAZR Doppler spectra to assess whether ground-based observations support larger drizzle near cloud top. The spectra indicate stronger downward motion at lower cloud levels but more symmetric velocity distributions aloft. This pattern, consistent with ENCORE, does not support the vertical trend inferred from the airborne measurements. Given recent reports of potential biases in airborne in situ drizzle size distributions, further case studies and improved in situ data processing are needed to reconcile these differences and better understand drizzle size variations in marine boundary layer clouds. In conclusion, ENCORE retrievals provide reliable observations of cloud and drizzle properties and capture their co-variability with high vertical and temporal resolution. Evaluation results show that these retrievals are well suited for model assessment of bulk microphysical properties and key processes in marine warm clouds. They also support investigations of how these processes influence mesoscale cloud organization and Earth’s radiation balance. However, uncertainties in retrieved drizzle properties remain substantial in some cases. For applications that require direct computation of microphysical process rates (e.g., autoconversion and accretion) 3 from retrieved cloud and drizzle properties, retrieval uncertainty should be included in error propagation. Continued progress in constraining cloud microphysical process rates will require improved drizzle retrievals and dedicated field campaigns with collocated measurements spanning a wide range of drizzle conditions. 2.2 Machine learning in constraining warm rain processes Using ~93,000 in‐situ cloudy drop size distributions measured from the ACE‐ENA campaign and the stochastic collection equation, Paper [2] built two machine‐learning models to predict autoconversion and accretion process rates. The first model, dubbed “initiation model”, uses cloud water content and number concentration to predict autoconversion rate in drizzle‐absent conditions. Once drizzle is initiated, the second model, dubbed “standard model”, uses water content and number concentration of both cloud and drizzle to predict autoconversion and accretion rate in drizzling conditions. The uncertainty in the new estimates of autoconversion and accretion rates are about 15% and 5%, respectively, outperforming existing parameterizations. These machine learning models are available in the ARM Archive as a PI product. 2.3 Uncertainty Quantification to Deep Learning To quantify the uncertainty in machine-learning–based estimates, Paper [3] presents a comprehensive and statistically consistent framework for uncertainty quantification in deep learning. This framework accounts for all major sources of uncertainty, including input data, training and testing datasets, neural network weights, and machine-learning model imperfections. Each source of uncertainty is systematically quantified using Bayes’ theorem and conditional probability distributions, and a fast, practical implementation strategy is introduced. Applying this framework to the research outcomes from Paper [1], we find that uncertainty in the input data is the dominant contributor to the total uncertainty in the predicted autoconversion rates. This result suggests that reducing uncertainty in input cloud and drizzle properties, whether derived from in situ or ground-based observations, will substantially improve constraints on cloud process rates. While other cloud processes may exhibit different relative contributions from these uncertainty sources, the proposed framework is broadly applicable and can be used to identify the most critical areas for improving machine-learning predictions in real-world applications. 2.4 PI Product and Additional Accomplishments Once the submitted paper [1] is accepted and finalized, we will coordinate with ARM to submit the cloud and drizzle retrievals as a PI product. Although not included in [1], we have also extended ENCORE to incorporate brightness temperature measurements from microwave radiometers (MWR). These additional MWR measurements have little impact on ENCORE daytime retrievals, indicating that the current shortwave radiance measurements provide sufficient constraints. MWR observations are therefore used to extend ENCORE’s capability to nighttime cloud and drizzle retrievals, which will be included in the PI product. In addition, we have developed emulators for computing shortwave radiances and microwave brightness temperatures 4 to accelerate ENCORE retrievals. Pending interest, these emulators can also be shared with the ARM/ASR community. 2.5 Contributions to the wider ARM/ASR community We have continued to collaborate the ARM and ASR community on providing the expertise and research outcomes. These collaborations include scientific highlights of the ACE- ENA campaign dataset on advanced cloud and drizzle retrievals [4], impacts of autoconversion parameterizations on Earth system models considering the grid size and model time step [5], and evaluations of ARM cloud optical depth value-added products [6] that was checked with ENCORE retrievals. We have continued to collaborate with the ARM and ASR communities by contributing scientific expertise and sharing research outcomes. These collaborations include scientific highlights from the ACE-ENA campaign focused on advanced cloud and drizzle retrievals [4], investigations of the impacts of autoconversion parameterizations on Earth system models with explicit consideration of model grid spacing and time step [5], and evaluations of ARM cloud optical depth Value-Added Products [6], benchmarked against ENCORE retrievals. 3. Conclusion and Publications Supported by This Project We have improved our cloud and drizzle retrievals, developed a machine-learning–based model to predict autoconversion and accretion rates for warm clouds, and implemented rigorous uncertainty quantification for the resulting cloud process rate estimates. The research outcomes have been disseminated through many conferences and workshops, and the related journal publications are listed below. References [1] Hung, C.-S., Chiu, J. C., Blanchard, Yann, van Leeuwen, P. J., Chandrasekar, V., Biswas, S., Cadeddu, M. P., Flynn, C., Mei, F., & Wang, J. (2026). Concurrent cloud and drizzle observations from synergistic ground-based measurements. J. Geophys. Res. Atmos., submitted. [2] Chiu, J. C., Yang, C. K., Van Leeuwen, P. J., Feingold, G., Wood, R., Blanchard, Y., ... & Wang, J. (2021). Observational constraints on warm cloud microphysical processes using machine learning and optimization techniques. Geophysical Research Letters, 48(2), e2020GL091236. [3] van Leeuwen, P. J., Chiu, J. Y. C., & Yang, C. K. K. (2025). Uncertainty quantification for deep learning. Environmental Data Science, 4, e53. [4] Wang, J., Wood, R., Jensen, M. P., Chiu, J. C., Liu, Y., Lamer, K., ... & Zhang, Z. (2022). Aerosol and cloud experiments in the Eastern North Atlantic (ACE-ENA). Bulletin of the American Meteorological Society, 103(2), E619-E641. 5 [5] Zhang, Z., Mechem, D. B., Chiu, J. C., & Covert, J. A. (2024). A Comprehensive Analysis of Uncertainties in Warm-Rain Parameterizations in Climate Models Based on In Situ Measurements. Journal of the Atmospheric Sciences, 81(7), 1251-1269. [6] Sookdar, K., Giangrande, S. E., Rausch, J., Ma, L., Wang, M., Wang, D., ... & Chiu, J. C. (2025). Marine and continental stratocumulus cloud microphysical properties obtained from routine ARM Cimel sunphotometer observations. Atmospheric Measurement Techniques, 18(21), 6271-6289. 6