The Algorithmic Gaze: An Audit and Ethnography of the LAION-Aesthetics Predictor Model CCS: Human-centered computing Empirical studies in HCI

Jordan Taylor, William Agnew, Maarten Sap, Sarah E. Fox, Haiyi Zhu · 2026

Read The Algorithmic Gaze: An Audit and Ethnography of the LAION-Aesthetics Predictor Model CCS: Human-centered computing Empirical studies in HCI by Jordan Taylor, William Agnew, Maarten Sap, Sarah E. Fox, Haiyi Zhu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-09896.txt.

Introduction

Jordan Taylor, William Agnew, Maarten Sap, Sarah E. Fox, and Haiyi Zhu audit the taste machine inside image AI. Their 2026 arXiv paper examines the LAION-Aesthetics predictor, which scores images for beauty. Generative models are trained on a single, one-size-fits-all measure of aesthetic appeal.

What counts as aesthetic is tied to personal taste and culture. The authors ask whose taste the predictor encodes. Their audit mixes measurement with ethnography of the model's use. The answer matters a lot because beauty scores quietly decide which images teach the next generation of models.

Catalog record

CREATORJordan Taylor, William Agnew, Maarten Sap, Sarah E. Fox, Haiyi Zhu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-09896

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Algorithmic Gaze: An Audit and Ethnography of the LAION-Aesthetics Predictor Model CCS: Human-centered computing Empirical studies in HCI Jordan Taylor, William Agnew, Maarten Sap, Sarah E. Fox, Haiyi Zhu Abstract . Visual generative AI models are trained using a one-size-fits-all measure of aesthetic appeal. However, what is deemed “aesthetic” is inextricably linked to personal taste and cultural values, raising the question of whose taste is represented in visual generative AI models. In this work, we study an aesthetic evaluation model—LAION Aesthetic Predictor (LAP)—that is widely used to curate datasets to train visual generative image models, like Stable Diffusion, and evaluate the quality of AI-generated images. To understand what LAP measures, we audited the model across three datasets. First, we examined the impact of aesthetic filtering on the LAION-Aesthetics Dataset ( ∼ \sim 1.2B images), which was curated from LAION-5B using LAP. We find that the LAP disproportionally filters in images with captions mentioning women, while filtering out images with captions mentioning men or LGBTQ+ people. Then, we used LAP to score ∼ \sim 330k images across two art datasets, finding the model rates realistic images of landscapes, cityscapes, and portraits from western and Japanese artists most highly. In doing so, the algorithmic gaze of this aesthetic evaluation model reinforces the imperial and male gazes found within western art history. In order to understand where these biases may have originated, we performed a digital ethnography of public materials related to the creation of LAP. We find that the development of LAP reflects the biases we found in our audits, such as the aesthetic scores used to train LAP primarily coming from English-speaking photographers and western AI-enthusiasts. In response, we discuss how aesthetic evaluation can perpetuate representational harms and call on AI developers to shift away from prescriptive measures of "aesthetics" toward more pluralistic evaluation. 1. Introduction Although artistic taste is deeply subjective and highly political ( 10 ) , AI researchers often attempt to quantify the aesthetic quality of media ( 95 ; 39 ) . While aesthetic quality assessment (AQA) has been a topic of research in the computer vision community for decades ( 18 ; 61 ) , this task is particularly important to examine in light of the rapid growth in visual generative AI. Not only are AQA models used to curate the training data for visual generative AI models ( 15 ) , but they are also used to assess the quality of AI-generated images ( 28 ) . As generative AI models are trained on datasets scraped from across the web, researchers rely on computational methods to filter out “low-quality” data. Prior work has called attention to the myriad gender and cultural biases in common dataset filtering techniques—such as blocklists ( 20 ) and image-text similarity metrics ( 38 ) —but the consequences of aesthetic filtering has yet to be explored in detail. In this work, we investigate an AQA model—the LAION-Aesthetics Predictor (LAP)—that has been widely influential in the visual generative AI research and commercial development. For example, LAP was used to curate the open-source LAION-Aesthetics Dataset from which Stability AI pre-trained the earliest versions of Stable Diffusion ( 84 ) . Additionally, LAP is often used to evaluate the quality of AI-generated images ( 19 ; 24 ; 28 ; 35 ) . Despite its far-reaching impact, the LAP itself has yet to be explored in detail. Due to the subjectivity of aesthetic evaluation, we first explored (RQ1) what the LAP model determines to be a “high-quality” image using three datasets. First, we examined the impact of aesthetic filtering on the LAION-Aesthetics Dataset ( \sim 1.2B images), which was curated using LAP. We find that the LAP disproportionally filters in images with captions mentioning women, while filtering out images with captions mentioning men or LGBTQ+ people. Then, we used LAP to score \sim 330k images across two art datasets, finding the model rates realistic images of landscapes, cityscapes, and portraits from western and Japanese artists most highly. In other words, the algorithmic gaze of LAP reproduces the male gaze within western art history ( 7 ) . In order to understand where these biases may have originated, we performed a digital ethnography of public materials to examine (RQ2) how was LAP developed. We find that the development of LAP reflects the biases we found in our audits, such as the aesthetic scores used to train LAP primarily coming from English-speaking photographers and western AI-enthusiasts. In light of our findings, we call on AI researchers to contend with aesthetics as a site of cultural representation. At the same time, we caution that greater “inclusion” in visual generative AI training data risks exacerbating the harms perpetuated by these technologies. In order to create more inclusive visual AI models, we call on developers to shift away from prescriptive, universalist measures of “aesthetics” toward more descriptive, pluralistic evaluations. Finally, we discuss how future FAccT researchers might also combine audits and ethnographies of algorithmic systems. 2. Background & Related Work 2.1. Background on LAION-Aesthetics LAION (Large-Scale Artificial Intelligence Open Network) is a non-profit founded in 2021 by Christoph Schuhmann, a German high school teacher, with the goal of creating open-source datasets and machine learning models ( 62 ) . Despite its humble origins, LAION has highly influenced generative image model development. Stability AI researchers used the LAION-Aesthetics Dataset to pre-train the earliest versions of Stable Diffusion ( 78 ) . Unfortunately, the extent to which contemporary generative image models rely on the LAION datasets can be challenging to asses because developers have become cautious of disclosing their training data. While Stability AI shared that early versions of Stable Diffusion were trained using LAION datasets, the model card for their 2024 model Stable Diffusion 3.5-Large vaguely says it was “trained on a wide variety of data, including synthetic data and filtered publicly available data.” 1 1 1 https://huggingface.co/stabilityai/stable-diffusion-3.5-large This secrecy may be due to the increased scrutiny faced by LAION and generative AI developers ( 75 ; 74 ) , which we will describe below. LAION has faced substantial criticism from both AI researchers and artists. LAION researchers compiled the 5.86 billion image-text pairs of the LAION-5B dtaset by extracting image links and alternative text descriptions from the IMG HTML tags of websites archived by the Common Crawl, a US-based non-profit that has scraped publicly available websites since 2008 ( 76 ; 4 ) . Due to LAION researchers’ indiscriminate data collection practices, researchers found that LAION-5B contained over a thousand images of child sexual abuse material (CSAM) ( 94 ) . In addition to containing CSAM, LAION-5B contains millions of copyrighted images, leading to numerous lawsuits against LAION ( 23 ) and Stability AI ( 96 ) . Furthermore, Birhane et al. found that the image descriptions in LAION-400M and the English language subset of LAION-5B (LAION-2B-en) often contain hate speech ( 9 ) . Although prior research has examined biases in LAION datasets ( 9 ; 38 ) , the LAION-Aesthetics Dataset and the LAION-Aesthetics Predictor (LAP) model used to curate this dataset are underexplored. Not only is the LAION-Aesthetics Dataset widely used to pre-train and fine-tune visual generative AI models, but the LAP model used to

Read the complete text