Measuring the environmental impact of delivering AI at Google Scale

Cooper Elsworth, Keguo Huang, David Patterson, Ian Schneider, Robert Sedivy, Savannah Goodman, Ben Townsend, Parthasarathy Ranganathan, Jeff Dean, Amin Vahdat, Ben Gomes, James Manyika · 2025

Read Measuring the environmental impact of delivering AI at Google Scale by Cooper Elsworth, Keguo Huang, David Patterson, Ian Schneider, Robert Sedivy, Savannah Goodman, Ben Townsend, Parthasarathy Ranganathan, Jeff Dean, Amin Vahdat, Ben Gomes, James Manyika (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-15734.txt.

Introduction

Cooper Elsworth and a team of co-authors at Google gauge the footprint of serving AI answers at full scale. Their paper from twenty twenty-five notes that no prior study had measured AI serving in a live production setting. Their method counts the full stack: chip power, host energy, idle machines, and building overhead. The test bed is the fleet that serves the Gemini assistant.

The median text prompt used a small slice of a watt-hour. That is less energy than a few seconds of television, by their match. Water use per prompt came to a few drops. Over one year, software gains and clean power deals cut per-prompt energy sharply. Carbon per prompt fell even more. The authors urge full-stack gauges as the fair base for claims about AI's cost.

Catalog record

CREATORCooper Elsworth, Keguo Huang, David Patterson, Ian Schneider, Robert Sedivy, Savannah Goodman, Ben Townsend, Parthasarathy Ranganathan, Jeff Dean, Amin Vahdat, Ben Gomes, James Manyika
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-15734

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Measuring the environmental impact of delivering AI at Google Scale Cooper Elsworth, Keguo Huang, David Patterson, Ian Schneider, Robert Sedivy, Savannah Goodman, Ben Townsend, Parthasarathy Ranganathan, Jeff Dean, Amin Vahdat, Ben Gomes, James Manyika Abstract The transformative power of AI is undeniable—but as user adoption accelerates, so does the need to understand and mitigate the environmental impact of AI serving. However, no studies have measured AI serving environmental metrics in a production environment. This paper addresses this gap by proposing and executing a comprehensive methodology for measuring the energy usage, carbon emissions, and water consumption of AI inference workloads in a large-scale, AI production environment. Our approach accounts for the full stack of AI serving infrastructure—including active AI accelerator power, host system energy, idle machine capacity, and data center energy overhead. Through detailed instrumentation of Google’s AI infrastructure for serving the Gemini AI assistant, we find the median Gemini Apps text prompt consumes 0.24 Wh of energy—a figure substantially lower than many public estimates. We also show that Google’s software efficiency efforts and clean energy procurement have driven a 33x reduction in energy consumption and a 44x reduction in carbon footprint for the median Gemini Apps text prompt over one year. We identify that the median Gemini Apps text prompt uses less energy than watching nine seconds of television (0.24 Wh) and consumes the equivalent of five drops of water (0.26 mL). While these impacts are low compared to other daily activities, reducing the environmental impact of AI serving continues to warrant important attention. Towards this objective, we propose that a comprehensive measurement of AI serving environmental metrics is critical for accurately comparing models, and to properly incentivize efficiency gains across the full AI serving stack. 1 Introduction Artificial intelligence (AI) is reshaping industries and daily life, driven largely by the accelerating capabilities of Large Language Models (LLMs). While much of the initial focus on AI’s environmental impact rightly centered on the energy-intensive process of model training [ 1 , 2 , 3 ] , the surge in public adoption of generative AI applications has shifted attention toward the footprint of AI model inference and serving. With these AI models now serving billions of user prompts globally, the energy, carbon emissions, and water impacts associated with generating responses at scale represents a significant and rapidly growing component of AI’s overall environmental cost [ 4 , 5 ] . In response, several research efforts and disclosures have emerged to quantify the per-prompt energy consumption of inference (Wh/prompt). Early work provided coarse estimates of energy consumption per prompt, relying on high-level assumptions about hardware specifications and model parameters [ 6 ] . More recently, initiatives like the the AI Energy Score [ 7 ] and the ML.ENERGY [ 8 ] benchmarks have advanced the field by employing direct empirical measurements. These frameworks aim to standardize energy transparency by benchmarking models on specific tasks using consistent hardware. In addition, other studies have expanded the aperture to consider the carbon emissions and water consumption associated with serving AI models. Despite this progress, the field lacks first-party data from the largest AI model providers. Based on decades of deploying software at scale, Google has a unique perspective on the operational realities of maintaining a large-scale, globally-distributed AI production fleet, and serving software products at scale—such as web search. Characterizing and optimizing the environmental impact of AI model serving requires a comprehensive view of energy consumption—including the power drawn by the host machine’s CPU and DRAM, the significant energy consumed by idle systems provisioned for reliability and low latency, and the full data center overhead as captured by the Power Usage Effectiveness (PUE) metric [ 9 ] . The missing consensus on the energy-consuming activities to include in the measurement—known as the measurement boundary —has led to published estimates for similar AI tasks varying by an order of magnitude. A lack of agreed upon methodologies may have contributed to a lack of first-party data when it is needed most [ 10 ] . Contributions This paper presents the energy, emissions, and water impacts for a production AI product by establishing a comprehensive framework to measure critical aspects of serving AI at Google’s scale. First, we propose a full-stack measurement approach that accounts for all material energy sources. Second, we apply this methodology to Google’s Gemini Apps product to provide the first analysis of three AI serving environmental metrics: • Energy / prompt: the energy consumption required to serve an AI assistant text prompt • Emissions / prompt: the market-based (MB) emissions generated by grid electricity generation (including renewable energy procurement), and the embodied emissions of the AI accelerator hardware • Water consumption / prompt: the water consumed for cooling machines and associated infrastructure in data centers We demonstrate that existing—and often narrower—measurement approaches are missing material energy consumption activities for AI serving. Finally, we illustrate the compounding AI serving efficiency gains across the serving stack over a year of development, resulting in a 44x reduction in the total emissions generated for the median Gemini Apps prompt. Comprehensive environmental metrics—like those proposed in this paper—are critical to properly incentivize efficiency opportunities across a large-scale, globally distributed production fleet. 2 Related Work Efforts to quantify the environmental impact of AI inference can be broadly categorized into two approaches: model-based estimation and empirical measurement . Early work primarily rely on estimation, calculating energy consumption from publicly available hardware specifications, known model parameters, and a series of assumptions regarding usage patterns like prompt complexity and token length [ 11 ] . While these theoretical models are helpful for highlighting the potential scale of the problem, their results are highly sensitive to the underlying assumptions and may suffer from accumulating errors in the estimated values. In contrast, more recent initiatives have shifted toward empirical measurement approaches, using software tools [ 12 , 13 ] to directly measure energy consumption on standardized hardware during the execution of specific tasks. Moving to more empirical metrics has increased the precision of these metrics, but still suffers from large differences in the underlying methodologies and comparability between studies. In this section, we outline existing approaches to make the case for a more consistent and complete measurement approach. Figure 1: Existing and proposed boundaries for AI inference energy measurements. The existing approaches primarily consider the energy consumption of the active AI Accelerators. We propose including measured energy from all components of the serving stack to provide a more comprehensive measurement of AI inference energy consumption to inform reduction levers across the serving stack. 2.1 Estimated metrics Model-based estimations of AI inference environmental metrics are widespread and variable, due to their sensitivity on poorly-constrained input assumptions. Some of the most referenced and recent studies include: • De Vries, 2023 [ 6 ] : This work takes a model-based estimation approach to calculate prompt energy use based on publicly available hardware specifications (e.g., NVIDIA A100 GPUs), AI model parameters (e.g., 175B for GPT-3.5), and critical assumptions about typical usage patterns, such as input/output token lengths for a prompt. These results suggest that a single GPT-3.5 prompt may consume around 3 watt-hours (Wh) of energy. • Epoch.AI, 2025 [ 14 ] : This analysis employs a model-based estimation methodology

Read the complete text