Do AI Companies Make Good on Voluntary Commitments to the White House?

Jennifer Wang, Kayla Huang, Kevin Klyman, Rishi Bommasani · 2025

Read Do AI Companies Make Good on Voluntary Commitments to the White House? by Jennifer Wang, Kayla Huang, Kevin Klyman, Rishi Bommasani (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08345.txt.

Introduction

Jennifer Wang, Kayla Huang, Kevin Klyman, and Rishi Bommasani grade AI companies on promises they made to the White House in a 2025 paper. In 2023, sixteen firms signed eight voluntary commitments on safety and openness.

The team built a scoring rubric from those commitments and judged each firm only on what it has publicly shown. The spread is wide. The best firm scored eighty three percent, while the average was just fifty three. Guarding model weights was the weakest point, averaging seventeen percent, with eleven firms scoring zero. The authors urge that future pledges spell out how compliance will be disclosed and checked, and they offer three fixes aimed at vague promises, tangled supply chains, and transparency.

Catalog record

CREATORJennifer Wang, Kayla Huang, Kevin Klyman, Rishi Bommasani
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE157 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-08345

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Do AI Companies Make Good on Voluntary Commitments to the White House? Jennifer Wang, Kayla Huang, Kevin Klyman, Rishi Bommasani Abstract Voluntary commitments are central to international AI governance, as demonstrated by recent voluntary guidelines from the White House to the G7, from Bletchley Park to Seoul. How do major AI companies make good on their commitments? We score companies based on their publicly disclosed behavior by developing a detailed rubric based on their eight voluntary commitments to the White House in 2023. We find significant heterogeneity: while the highest-scoring company (OpenAI) scores a 83% overall on our rubric, the average score across all companies is just 52%. The companies demonstrate systemically poor performance for their commitment to model weight security with an average score of 17%: 11 of the 16 companies receive 0% for this commitment. Our analysis highlights a clear structural shortcoming that future AI governance initiatives should correct: when companies make public commitments, they should proactively disclose how they meet their commitments to provide accountability, and these disclosures should be verifiable. To advance policymaking on corporate AI governance, we provide three directed recommendations that address underspecified commitments, the role of complex AI supply chains, and public transparency that could be applied towards AI governance initiatives worldwide. Introduction The growing importance of artificial intelligence (AI) has rapidly catalyzed global policymaking efforts. Policymaking related to AI addresses many concerns including open innovation, market concentration, risk reduction, corporate governance, and geopolitics. Since 2023, many AI policy efforts have centered on the interplay between corporate governance, given that prominent AI systems are developed by the world’s most powerful companies, and risk reduction, due to the breadth of potential harms associated with AI systems. The approach to global AI policy varies significantly across jurisdictions. A key differentiator among jurisdictions that regulate AI companies is whether a policy imposes mandatory or voluntary obligations. Some jurisdictions have enacted mandatory requirements via legislative or executive action, such as the EU AI Act and the US Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence respectively. However, much of global AI policy centers on voluntary actions taken by major companies in line with recommendations by government bodies. Key examples include the NIST AI Risk Management Framework, the 2023 White House Voluntary Commitments on AI, the G7 International Code of Conduct, Canada’s Voluntary Code of Conduct, the 2024 White House Voluntary Commitments to Combat Image-Based Sexual Abuse, and the Frontier AI Safety Commitments secured at 2024 AI Seoul Summit. Voluntary measures offer flexibility in that they allow companies to pilot different approaches to meeting commitments, optimize for objectives other than minimizing legal risk associated with regulatory compliance, and harmonize approaches across jurisdictions despite different legal and political systems. But policy initiatives that rely on companies to take voluntary action have a number of pitfalls. Voluntary measures do not come with penalties for noncompliance, meaning that companies may choose to not participate, claim they are participating but not implement the government’s recommendations, only partially implement the measures, or implement the measures in ways that are opaque or not verifiable. Well-intentioned companies may have difficulty complying because voluntary measures are less likely to move markets and reorganize supply chains, meaning that measures that require coordinated action may be less likely to succeed if voluntary. Voluntary measures often lack any mechanism for monitoring implementation, presenting a potential loophole for noncompliance ( J. Alberto Aragòn-Correa and Vogel 2020 ) . A company’s statement that it will adhere to voluntary measures can give the illusion that the company is taking significant action to responsibly develop and deploy AI systems while it does not in fact carry out its commitments. To understand the impact and efficacy of voluntary commitments, we conduct the first comprehensive analysis of the first major commitments to governments made by top AI companies. In 2023, the White House secured voluntary commitments from 15 AI companies. 1 1 1 The commitments were secured in three phrases: (i) Amazon, Anthropic, Google, Inflection, Meta, Microsoft, and OpenAI committed in July 2023; (ii) Adobe, Cohere, IBM, Nvidia, Palantir, Salesforce, Scale AI, and Stability AI committed in September 2023; and (iii) Apple committed in July 2024, following the launch of its Apple Intelligence product. In announcing the commitments, the White House described their purpose as follows: “These commitments, which the companies have chosen to undertake immediately, underscore three principles that must be fundamental to the future of AI – safety, security, and trust – and mark a critical step toward developing responsible AI. As the pace of innovation continues to accelerate, the Biden-Harris Administration will continue to remind these companies of their responsibilities and take decisive action to keep Americans safe.” To reason about the companies and their behavior, we score companies based on how their public actions address their stated commitments. We design a scoring rubric that transforms the eight commitments specified by the White House on product safety, system security, and public trust into 30 indicators. Our rubric provides concrete and decidable criteria for determining if a company has satisfied its commitment. To score each of the 16 companies that signed the 2023 White House Voluntary Commitments on AI, for each of the 480 (indicator, company) pairs, we gather relevant public information, assign a score, and provide evidence for our decision. 2 2 2 Scores and justifications are released at . By compiling information about company practices and interpreting it via quantitative scores, we provide evidence for three key findings. First, the scores demonstrate significant heterogeneity in companies’ actions: the top-scoring company (OpenAI) scores 83% on our rubric, whereas the bottom-scoring company (Apple) scores 13%. Of the eight commitments, there are six commitments where at least one company scores 100%; at the same time, there are five commitments where at least one company scores 0%. Second, company-level scores demonstrate two clear, and interconnected, correlations: members of the Frontier Model Forum and earlier signatories tend to score higher. The six highest scoring companies are the six members of the Frontier Model Forum (OpenAI, Anthropic, Google, Microsoft, Meta, Amazon) and each score at least 60%. Third, model weight security is a commitment with distinctively poor performance, with companies scoring on average 17%. 11 companies score 0% on this commitment (Adobe, Apple, Cohere, IBM, Inflection, Meta, Nvidia, Palantir, Scale AI, Salesforce, Stability AI). Beyond providing empirical insight into the relationship between company practices and stated commitments, our work reveals a key design flaw in the 2023 White House Voluntary Commitments on AI: companies make public commitments to the White House, but no mechanism was created to monitor implementation or provide the public with information about implementation. To improve the design of future voluntary commitments related to corporate AI governance, we provide three recommendations to policymakers. 3 3 3 In contrast to the 2023 White House Voluntary Commitments on AI, we find that the 2024 White House Voluntary Commitments to Combat Image-Based Sexual Abuse implement some of our recommendations. 1. Commitments should be precise and specific. The wording of the 2023 White House Voluntary Commitments is often vague, leading to significant ambiguity over the intent of a commitment and the steps required to satisfy a commitment. Commitments should be precise, specifying

Read the complete text