Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores
Benhour Amirian, Ashley S. Dale *, Sergei Kalinin, Jason Hattrick-Simpers · 2025
Read Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores by Benhour Amirian, Ashley S. Dale *, Sergei Kalinin, Jason Hattrick-Simpers (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01080.txt.
Introduction
Benhour Amirian, Ashley Dale, Sergei Kalinin, and Jason Hattrick-Simpers set out what trustworthy AI for materials discovery should mean in a 2025 paper. Artificial intelligence and machine learning now speed up the hunt for new materials, including in autonomous laboratories that run experiments themselves. Speed is worth little if scientists cannot trust the models.
The authors define a framework with named criteria, including whether methods are generalizable, interpretable, fair, transparent, explainable, robust, and stable. They tie evaluation to simple summary scores. A shared trust framework could help labs decide which machine learning claims to build on.
Catalog record
| CREATOR | Benhour Amirian, Ashley S. Dale *, Sergei Kalinin, Jason Hattrick-Simpers |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 63 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-01080
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Building Trustworthy AI for Materials Discovery: From Autonomous Laboratories to Z-scores Benhour Amirian, Ashley S. Dale *, Sergei Kalinin, Jason Hattrick-Simpers Abstract ABSTRACT . Accelerated material discovery increasingly relies on artificial intelligence and machine learning, collectively termed “AI/ML”. A key challenge in using AI is ensuring that human scientists trust the models are valid and reliable. Accordingly, we define a trustworthy AI framework GIFTERS for materials science and discovery to evaluate whether reported machine learning methods are generalizable , interpretable , fair , transparent , explainable , robust , and stable . Through a critical literature review, we highlight that these are the trustworthiness principles most valued by the materials discovery community. However, we also find that comprehensive approaches to trustworthiness are rarely reported; this is quantified by a median GIFTERS score of 5/7. We observe that Bayesian studies frequently omit fair data practices, while non-Bayesian studies most frequently omit interpretability. Finally, we identify approaches for improving trustworthiness methods in artificial intelligence and machine learning for materials science by considering work accomplished in other scientific disciplines such as healthcare, climate science, and natural language processing with an emphasis on methods that may transfer to materials discovery experiments. By combining these observations, we highlight the necessity of human-in-the-loop, and integrated approaches to bridge the gap between trustworthiness and uncertainty quantification for future directions of materials science research. This ensures that AI/ML methods not only accelerate discovery, but also meet ethical and scientific norms established by the materials discovery community. This work provides a road map for developing trustworthy artificial intelligence systems that will accurately and confidently enable material discovery. 1 Introduction Artificial intelligence and machine learning (AI/ML) methods are moving materials discovery towards data-driven approaches that yield more successes with fewer resources [ 30 , 71 , 174 , 33 ] . AI/ML methods promise to accelerate first principles computations [ 228 ] , predict experimental process parameters [ 63 ] , coordinate high-throughput laboratory experiments [ 2 ] , and more. However, sustaining rapid growth in these areas requires human scientists to trust that AI/ML methods are valid, reliable, and correct. Trustworthiness in artificial intelligence describes AI/ML models with properties that enable humans to trust the model’s predictions and has been identified as an important component of human-machine interaction [ 235 , 203 ] . Trustworthy artificial intelligence (TAI) frameworks help AI/ML practitioners address trustworthiness in applications such as healthcare [ 5 ] , criminal justice [ 159 ] , natural language processing [ 46 ] , computer vision [ 73 ] , and economics [ 50 ] . In these cases, TAI frameworks guide the development and implementation of AI by connecting discipline-based methodologies to principles such as fairness , robustness , and explainability . The purpose of this review is to contextualize current best practices within materials science and discovery using a proposed TAI framework for AI/ML methods. The journey toward TAI began with initiatives such as the European Commission’s High-Level Expert Group on Artificial Intelligence (HLEG AI) [ 84 ] , the “trust comparison index” proposed by Floridi et al. [ 49 ] , and early TAI frameworks [ 6 ] . In 2021, the Organization for Economic Co-operation and Development (OECD) formalized principles on artificial intelligence [ 25 ] . These frameworks guide AI systems beyond technical proficiency and towards ethical integrity, transparency, and reliability—qualities that support responsible AI/ML usage in technical domains. Proposed frameworks in disciplines beyond materials science vary which aspects of trustworthiness are emphasized. For example, a proposed TAI framework for computer vision identifies transparency, safety and security, privacy, fairness and justice, responsibility, freedom and autonomy, sustainability, and robustness as key elements [ 73 ] . Important elements of TAI for natural language processing include machine ethics, toxicity, privacy, fairness and justice, stereotyping, robustness on adversarial demonstrations, and out-of-distribution robustness [ 46 ] . TAI for healthcare includes explainable robotics, predictiveness, explainable decision support, trustworthy review of healthcare, data security, transparency, and trustworthy internet of medical things [ 5 ] . These technologically-driven frameworks share an emphasis on transparency/explainability/interpretability, social responsibility and fairness, a robustness to attack, and data privacy while emphasizing universal challenges in their respective disciplines. Importantly, these frameworks emphasize principles important to those affected by untrustworthy AI rather than performance metrics. Trustworthy AI for materials science and engineering represents stakeholder interests in these algorithms and models. Existing TAI frameworks may be adapted to materials science after acknowledging any unique challenges in the domain and the values of the community surrounding the use of these models, including the alignment of AI/ML models with fundamental scientific laws [ 158 , 169 ] and the integration of data with varying levels of fidelity [ 31 , 225 , 10 , 43 ] . This review shows that many principles of TAI are already part of common practice in the materials science field in the form of uncertainty quantification (UQ); a longer discussion of the distinction and connection between TAI and UQ is presented in § 2 . We explicitly clarify that the lack of methods to improve, verify, and explain trustworthiness is not unique to materials science and is shared by other disciplines that leverage AI/ML methods. However, because materials discovery intersects physical sciences, engineering, and data science, the materials discovery community possesses a unique perspective on TAI methods that may contribute towards a unifying theory of trustworthiness within the larger AI/ML community. Accordingly, we summarize the state-of-the-art for TAI practices in materials science by contextualizing existing best practices in the materials discovery community with the first proposed trustworthy AI framework for materials science. The review is organized in the following manner: in section § 2 , we provide concrete definitions and examples of the proposed framework principles: generalizability , interpretability , fairness , transparency , explainability , robustness , and stability (GIFTERS). We next provide case studies for various AI/ML models and applications in materials discovery in § 3 that highlight the relationship between the TAI framework and UQ in an experimental context. In § 4 , we summarize efforts towards TAI methods from outside the materials discovery community followed by a discussion in § 5 of directions where TAI will be expected. We hope that this provides a path towards improved AI/ML methodologies in general, and contributes new directions for AI/ML materials discovery research. 2 Components of a Trustworthy AI Framework for Materials Science In this section, we propose a trustworthy AI framework GIFTERS , grounded in the values and needs of the materials science community based on trends observed in the literature. The proposed principles of the framework are Generalizability, Interpretability, Fairness, Transparency, Explainability, Robustness, and Stability . In practice, these principles frequently overlap in their methodologies. For example, analysis examining a model’s interpretability may also provide information about its explainability [ 138 , 125 , 87 ] . A dataset preparation approach that addresses fairness will likely have implications for the model’s generalizability [ 79 , 90 , 121 ] . Accordingly, we emphasize that as the vocabulary surrounding these topics continues to evolve and that depending on usage, there may be overlap, ambiguity, and differences in the naming conventions proposed by various authors cited in this work for