Optimal classification with endogenous behavior

Elizabeth Maggie Penn · 2025

Read Optimal classification with endogenous behavior by Elizabeth Maggie Penn (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-06127.txt.

Introduction

Elizabeth Maggie Penn studies a classification puzzle where people's behavior reacts to the classifier itself, in a 2025 paper. Algorithms often grade behavior, such as effort or honesty, yet the grading rule changes how people act. That feedback loop, called outcome performativity, can flip what the best rule looks like.

Penn shows that in a simple setting the most accurate classifier is either a threshold rule or its mirror image. A threshold rule rewards those likelier to have behaved well. The mirror rule, oddly, rewards those less likely to have behaved well, and still can win on accuracy when behavior is endogenous. She gives a worked example and extends the result to broader goals. The warning is sharp: a rule tuned only for accuracy can end up pushing group behavior in bad directions.

Catalog record

CREATORElizabeth Maggie Penn
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-06127

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Optimal classification with endogenous behavior Elizabeth Maggie Penn Abstract I consider the problem of classifying individual behavior in a simple setting of outcome performativity where the behavior the algorithm seeks to classify is itself dependent on the algorithm. I show in this context that the most accurate classifier is either a threshold or a negative threshold rule. A threshold rule offers the “good” classification to those individuals more likely to have engaged in a desirable behavior, while a negative threshold rule offers the “good” outcome to those less likely to have engaged in the desirable behavior. While seemingly pathological, I show that a negative threshold rule can maximize classification accuracy when outcomes are performative. I provide an example of such a classifier, and extend the analysis to more general algorithm objectives. A key takeaway is that when behavior is endogenous to classification, optimal classification can negatively correlate with signal information. This may yield negative downstream effects on groups in terms of the aggregate behavior induced by an algorithm. Introduction Algorithms are increasingly used to translate rich data about individual behavior into consequential decisions affecting peoples’ lives. In the process, the prospect of future classification may lead people to change their present behavior in an effort to obtain a better classification outcome. The prospect of a good credit score, for example, may lead someone to undertake activities that make them more credit worthy. The possibility of an audit may reduce someone’s incentives to cheat. Classification algorithms are often designed with these kinds of behavioral goals in mind. Recent work in machine learning has focused on performativity , or situations in which an algorithm affects the data distribution, and in which the optimal algorithm depends on this distribution. The notions of strategic classification ( Hardt et al., 2016 ) and performative prediction ( Perdomo et al., 2020 ) each consider how to classify data that are responsive to an algorithm itself, focusing on the conditions under which it is possible to design an algorithm that properly accounts for performativity. This literature has largely focused on individuals’ efforts to manipulate their data, with individual behavior assumed to be exogenous (a setting termed data performativity ). A smaller literature considers outcome performativity ( Kim and Perdomo, 2023 ) , in which an individual’s true behavioral type may also respond to the algorithm. When outcomes are performative, behavior is endogenous to the algorithm. This latter setting of outcome performativity is the setting I am concerned with. In particular, I assume that the designer of an algorithm knows the data-generating process describing how an individual will respond to algorithmic classification. Anticipating this individual response, an algorithm commits to a classification strategy that will map a signal of the individual’s behavior into a classification outcome for the individual. My question is what an optimal classifier looks like in this context of outcome performativity. In this setting I show that the optimal classifier is either a threshold or a negative threshold rule. A threshold rule offers the “good” classification to those individuals whose outcome likelihoods are greater than some cutpoint, \tau . Threshold rules are well-known in the literature on optimal classification and strategic classification ( Milli et al., 2019 ; Coate and Loury, 1993 ) , and follow from well-known decision-theoretic results ( Brown, Cohen, and Strawderman, 1976 ) . Negative threshold rules are, to my knowledge, less known. A negative threshold rule offers the “good” classification outcome to individuals whose outcome likelihoods are less than some cutpoint, \tau . While seemingly pathological, I show that a negative threshold rule can be the most accurate classifier when outcomes are performative. These results generalize several recent papers on the topic of classification with endogenous behavior, or outcome performativity. Jung et al. (2020) consider the setting of a classification algorithm that is designed to maximize behavioral compliance, showing that the optimal classifier is a (positive) threshold rule with a cutpoint set at the unique crossing point of two behavior-dependent signal distributions. In contrast, my main theorem concerns an accuracy-motivated algorithm, though I provide a corollary extending my result to more general algorithm objectives, and those objectives encompass behavioral compliance-maximization. Penn and Patty (2023) ; Penn and Patty (2024) consider a setting similar to the one considered here but with binary data for the individual (the algorithm simply observes a signal of 0 or a 1 for the individual). In contrast, here I allow the signal space to be the real line. While I model these signals as real numbers arising from two behavior-dependent distributions, I show that we can equivalently model the signal as representing an outcome likelihood for the individual. In this sense, the algorithm translates any outcome likelihood in (0,1) into a classification decision for the individual. A foundational aspect of ethical classification is that a classifier should respond in a positively monotonic way to the information provided to it ( Hardt, Price, and Srebro, 2016 ) . In other words, and all else equal, an individual more likely to be a type “1” should be more likely to be classified “1.” Negative threshold rules violate this property, assigning the more desirable classification outcome to the individuals least likely to deserve it. Such rules can maximize classification accuracy by changing the base rates of behavior in a population. My aim is to pin down the mechanism by which this happens. Contributions 1. I show that for a general setting of outcome performativity, the most accurate classifier is either a threshold rule or a negative threshold rule. 2. I provide an example of optimal accuracy being obtained with a negative threshold rule generating classification outcomes that are negatively correlated with signal information. 3. I show that similar results obtain for a more general objective function allowing the algorithm to differentially weigh true positive, true negative, false positive, and false negative classification. The model and main result Consider two players: an individual i , and an algorithm, D . The individual can take one of two possible actions, \beta_{i}\in\{0,1\} . We term \beta_{i}=1 as compliance and \beta_{i}=0 as noncompliance . If choosing \beta_{i}=1 the individual pays cost \gamma_{i} . \gamma_{i} is private information to the individual, and is drawn from a continuous CDF H:\mathbf{R}\rightarrow[0,1] . After choosing action \beta_{i} , D observes a signal x\in\mathbf{R} that is drawn from an action-conditional distribution f_{\beta_{i}} . Specifically, let f_{1}(x) and f_{0}(x) be two probability density functions that are continuous over the real numbers, with full support. I assume that f_{1}(x) satisfies the strict monotone likelihood ratio property with respect to f_{0}(x) . Letting \pi=\Pr[\beta_{i}=1] , signal x yields outcome likelihood: P(\beta_{i}=1|x)={\pi{f_{1}(x)}\over{\pi f_{1}(x)+(1-\pi)f_{0}(x)}}, and because this likelihood is continuous and strictly increasing, observation of x is equivalent to observation of the outcome likelihood. Note that the assumptions that H is continuous and that f_{1} and f_{0} are continuous with full support are stronger than necessary, but simplify the analysis by allowing us to disregard special cases. Finally, upon observing x the algorithm makes a binary decision for i , d_{i}\in\{0,1\} . D ’s strategy \delta(x) maps each observed signal into a probability that i is classified as a 1 , or: \delta(x)=\Pr[d_{i}=1|x]. I will refer to \delta(x) as a (binary) classification algorithm

Read the complete text