Revisiting the Berkeley Admissions data: Statistical Tests for Causal Hypotheses Sourbh Bhadane, Joris M. Mooij, Philip Boeken, Onno Zoeter Abstract Reasoning about fairness through correlation-based notions is rife with pitfalls. The 1973 University of California, Berkeley graduate school admissions case from Bickel et al. [1975] is a classic example of one such pitfall, namely Simpson’s paradox. The discrepancy in admission rates among male and female applicants, in the aggregate data over all departments, vanishes when admission rates per department are examined. We reason about the Berkeley graduate school admissions case through a causal lens. In the process, we introduce a statistical test for causal hypothesis testing based on Pearl’s instrumental-variable inequalities [ Pearl, 1995 ] . We compare different causal notions of fairness that are based on graphical, counterfactual and interventional queries on the causal model, and develop statistical tests for these notions that use only observational data. We study the logical relations between notions, and show that while notions may not be equivalent, their corresponding statistical tests coincide for the case at hand. We believe that a thorough case-based causal analysis helps develop a more principled understanding of both causal hypothesis testing and fairness. 1]Korteweg-de Vries Institute for Mathematics, University of Amsterdam 2]Booking.com, Amsterdam, The Netherlands 1 Introduction In the fall of 1973 , the Graduate Division of the University of California, Berkeley, made admission decisions for 12763 applicants to its 101 departments. The admission rate for 8442 male applicants was approximately 44.2\% and for 4321 female applicants was approximately 34.6\% . This disparity prompted Bickel et al. [1975] to investigate whether the Graduate Admissions Office discriminated on the basis of sex. The authors found that despite there being a statistically significant disparity in the aggregate data, when each department was examined, the per-department admission rates did not differ significantly between the sexes, thus making this case an instance of Simpson’s paradox. The resolution was that the “proportion of women applicants tends to be high in departments that are hard to get into and low in those that are easy to get into”. The disparity was therefore attributed to societal biases and the authors concluded that there was “no pattern of discrimination on the part of the admissions committee.” In the fairness literature, the Berkeley graduate admissions case is a canonical example of Simpson’s paradox, which illustrates the limitations of correlation-based fairness notions such as demographic parity, and therefore motivates the need for causal reasoning of fairness. Pearl [2009, Section 4.5.4] analyzes the Berkeley example and frames the conclusion of Bickel et al. [1975] as discerning the direct effect of sex on admissions outcome by conditioning on the mediator, namely department choice. Most works in the fairness literature that mention the Berkeley example follow this analysis, which is predicated on the assumption that the causal model includes no latent confounders while the causal graph is akin to a simple mediation graph with sex being the treatment, department choice being the mediator and the admissions decision being the outcome. However, in both Pearl [2009] and Pearl and Mackenzie [2018] , Pearl notes that merely conditioning on department choice might not always be appropriate. In particular, he cites a fascinating exchange between William Kruskal and Peter Bickel in Fairley and Mosteller [1977] where Kruskal objects to the analysis in Bickel et al. [1975] by pointing out that controlling for department leads to erroneous conclusions if there is a confounder that affects department choice and admissions outcome. To the best of our knowledge, subsequent works that mention the Berkeley example, do not address the latent confounder issue, including Pearl [2009] where the analysis assumes that the common causes are observed. Further, while there are multiple causal fairness notions proposed in the literature, 1 1 1 Some directly inspired by the Berkeley admissions case, for example the path-dependent counterfactual fairness notion in Kusner et al. [2017, Appendix S4] . the issue of statistical testing of these fairness notions has received little attention. In this work, we undertake a causal reasoning exercise centered around the Berkeley admissions case. We take the view that a causal analysis is predicated on causal modeling assumptions that define a family of causal models. A fairness notion is either an observational, interventional, counterfactual or a graphical query on a causal model which, as a result, defines a subset of the aforementioned family of causal models, i.e., a fairness notion defines a causal hypothesis . Given that we usually have only observational data at hand the question of fairness boils down to statistical testing of a causal hypothesis. Indeed, the Berkeley admissions data can be thought of as sampled from the joint distribution of the sex, department choice and admissions outcome. 2 2 2 Albeit possibly post-selection, which we don’t address in this work. For the Berkeley admissions case, we consider multiple fairness notions based on graphical, counterfactual and interventional queries to the family of causal models defined by our causal modeling assumptions, which allows latent confounding between department choice and admissions outcome. For these notions, we develop new statistical tests. One of our key insights is that the graphical notion of fairness can be tested by using the instrumental-variable (IV) inequalities [ Pearl, 1995 ] , thus making our proposed statistical test a new test for the IV inequalities. Conversely, any statistical test for the IV inequalities can be used to test for fairness in settings that are analogous to the Berkeley case. In the process, we also prove a result of independent interest, namely the sharpness of the IV inequalities for the case where the instrument and the effect are binary, and the treatment takes any finite number of values. For the Berkeley example, while our proposed fairness notions correspond to different rungs of the causal hierarchy and are in general not equivalent, we show, rather surprisingly, that the tests are equivalent within the IV setting. Although our results are inspired by the Berkeley case, they can also be applied in other analogous settings, e.g. to investigate sex discrimination in awarding distinctions to PhD students [ Bol, 2023 ] . 1.1 Related Work The question of fairness in decision-making and predictive systems has received increased attention since the past few decades. See Hutchinson and Mitchell [2019] , Barocas et al. [2023] for an excellent historical and technical overview, respectively. While attempts at formalizing fairness lead to correlation-based notions such as fairness through unawareness [ Dwork et al., 2012 ] , demographic parity, equality of odds [ Hardt et al., 2016 ] etc., purely observational notions of fairness are at odds with each other Chouldechova [2017] , Kleinberg et al. [2017] and are prone to erroneous conclusions. On the other hand, observational notions of fairness are readily translated to statistical tests. Causal analysis tools such as counterfactuals and interventions provide a framework suitable for fairness. As a result, multiple general fairness notions based on counterfactuals were proposed. Kusner et al. [2017] defined a counterfactual fairness notion that required invariance of the distribution of the decision in a given context, with respect to hypothetical changes in the protected attribute. Nabi and Shpitser [2018] and Zhang et al. [2017] consider path-specific effects. Chiappa [2019] proposes a path-specific counterfactual fairness notion and a related notion appears in the appendix of Kusner et al. [2017] . Another separate line of work seeks to explain observed disparity through causal discrimination mechanisms [ Zhang and Bareinboim, 2018 , Plečko et al., 2024 ] . The Berkeley graduate admissions case makes an appearance in multiple papers to motivate the need for causal fairness notions. Kilbertus et al. [2017] , Plečko et al. [2024] , Kusner et al. [2017] , Chiappa [2019] , Berk et al. [2023] are a few among many works. In addition, the Berkeley example also serves as a motivation to introduce path-specific notions given the assumption that the direct effect of sex on admissions outcome is the only ‘unfair’ path. Also, see Barocas et al. [2023] for a critique of this common assumption. Pearl [2009] considers the Berkeley example at length and illustrates the objection to controlling for the mediator by positing an observed confounder. Despite the fact that most causal fairness works mention the Berkeley example, to the best of our knowledge, no previous work gives a definitive answer to the question of fairness for the Berkeley dataset under unobserved confounding. Kilbertus et al. [2020] discusses the impact of unmeasured confounding under restrictive parametric assumptions. Zhang and Bareinboim [2018] , Plečko et al. [2024] consider fairness models that allow for specific forms of unobserved confounding. Schröder et al. [2024] build on this by providing sensitivity analysis on fairness of prediction models. However, the kinds of unobserved confounding that they allow affects the sensitive attribute which is different from the kind we allow for in the Berkeley dataset. 2 Preliminaries We outline a few definitions that follow the formal setup of Bongers et al. [2021] . Definition 1 (Structural Causal Model (SCM)) . A Structural Causal Model (SCM) is a tuple M={\left({V,W,\mathcal{X},f,P}\right)} where a) V,W are disjoint, finite index sets of endogenous and exogenous random variables respectively, b) \mathcal{X}=\prod_{i\in V\cup W}\mathcal{X}_{i} is the domain which is a product of standard measurable spaces \mathcal{X}_{i} , c) for every v\in V , f_{v}:\mathcal{X}\mapsto\mathcal{X}_{v} is a measurable function, and f=(f_{v})_{v\in V} is called the causal mechanism ; the equations X_{v}=f_{v}(X) for every v\in V are called the structural equations , and d) P{\left({\mathcal{X}_{W}}\right)}=\bigotimes_{w\in W}P(X_{w}) is the exogenous distribution which is a product of probability distributions P(X_{w}) on \mathcal{X}_{w} . Definition 2 (Parent) . Let M={\left({V,W,\mathcal{X},f,P}\right)} be an SCM. k\in V\cup W is a parent of v\in V if and only if it is not the case that for all x_{V\backslash k},f_{v}(x_{V},X_{W}) is constant in x_{k} (if k\in V , resp. X_{k} if k\in W ) P(\mathcal{X}_{W}) -a.s.. Definition 3 (Causal Graph) . Let M={\left({V,W,\mathcal{X},f,P}\right)} be an SCM. The causal graph , G{\left({M}\right)} , is a directed mixed graph with nodes V , directed edges u\longrightarrow v if and only if u\in V is a parent of v\in V , and bidirected edges u\leftrightarrow v if and only if \,\exists w\in W that is a parent of both u,v\in V . For simplicity of exposition, we restrict attention to acyclic SCMs, i.e. SCMs whose causal graph is acyclic (contains no directed cycle X\rightarrow\cdots\rightarrow Y\rightarrow X ). Definition 4 (Observational Distribution) . Given an acyclic SCM, M={\left({V,W,\mathcal{X},f,P}\right)} , the exogenous distribution, P and the causal mechanism f induce a probability distribution over the endogenous variables which is called the observational distribution , P_{M}(X_{V}) . Definition 5 (Hard Intervention) . Given an acyclic SCM, M={\left({V,W,\mathcal{X},f,P}\right)} , an intervention target \,T\subseteq V , and an intervention value x_{T}\in{\cal X}_{T} , the intervened SCM is defined as M_{\text{do}(X_{T}=x_{T})}\triangleq{\left({V,W,\mathcal{X},(f_{V\backslash T},x_{T}),P}\right)}. Further, the observational distribution of the intervened SCM, P_{M_{\text{do}(X_{T}=x_{T})}} , is called an interventional distribution , and denoted by P_{M}{\left({X_{V}\mid\text{do}(X_{T}=x_{T})}\right)} . Definition 6 (Potential Outcome) . Let M={\left({V,W,\mathcal{X},f,P}\right)} be an acyclic SCM, C\subseteq V and x_{C}\in\mathcal{X}_{C} . A random variable X^{\text{do}(x_{C})} taking values in \mathcal{X}_{V}\times\mathcal{X}_{W} is called a potential outcome of M under intervention x_{C} if a) its W -component has the exogenous distribution specified by M , i.e., X^{\text{do}(x_{C})}_{W}\sim P{\left({\mathcal{X}_{W}}\right)} , and b) it satisfies X^{\text{do}(x_{C})}_{C}=x_{C}\ \mathrm{a.s.} and X^{\text{do}(x_{C})}_{V\backslash C}=f_{V\backslash C}{\left({x_{C},X^{\text{do}(x_{C})}_{V\backslash C},X^{\text{do}(x_{C})}_{W}}\right)}\text{ a.s.}. If \,C=\emptyset , we write X instead of X^{\text{do}(x_{\emptyset})} . When dealing with multiple potential outcomes, we assume that they share the same W -component: for any n , for C_{1},C_{2},\cdots,C_{n}\subseteq V and for x_{C_{1}}\in{\cal X}_{C_{1}},x_{C_{2}}\in{\cal X}_{C_{2}},\cdots,x_{C_{n}}\in{\cal X}_{C_{n}} , X_{W}^{\text{do}(x_{C_{1}})}=X_{W}^{\text{do}(x_{C_{2}})}=\cdots=X_{W}^{\text{do}(x_{C_{n}})}=X_{W}\text{ a.s.}. Instrumental variables (IVs) are used to estimate the causal effect of a treatment X on the outcome Y in the presence of latent confounding. Definition 7 (Instrumental Variable (IV) Model Class) . The instrumental variable model class , \mathbb{M}_{\text{IV}} , is a collection of SCMs M such that G{\left({M}\right)} is a subgraph of Figure 1 where Z is the instrument , X is the treatment , and Y is the outcome . We make an explicit positivity assumption and define \mathbb{M}_{\text{IV}+}\triangleq\left\{M\in\mathbb{M}_{\text{IV}}:\forall z,P_{M}(Z=z)>0\right\} . Causal graphs of SCMs in the IV model class rule out the directed edge Z\rightarrow Y and any latent confounding between Z and Y as well. In Section D.4 we show that our results for the IV model class hold even when confounding is allowed between Z and X . Z Y X Figure 1 : Causal graph of M\in\mathbb{M}_{\text{IV}} 3 Berkeley Case: No Latent Confounding S A D (a) M\in\mathbb{M}_{\text{no-cf}} S A D (b) M\in\mathbb{M}_{\text{cf}} S A D (c) M\in H^{0}_{\text{cf-graph}+} and M\in\mathbb{M}_{\text{IV}+} Figure 2 : Causal graphs, G{\left({M}\right)} , assumed in various model classes. Under the semantic framework of SCMs, we first make the same causal modeling assumptions that are commonplace in works that mention the Berkeley admissions case. We compare fairness notions that are tied to these modeling assumptions, with the view that modeling assumptions describe a family of SCMs and fairness notions define a subset of this family. We relate existing general notions of fairness in the literature to this viewpoint. While this is a re-examination of the various existing analyses of the Berkeley admissions case, in the next section, we relax the causal modeling assumptions and consider the more general family of models that allow for confounding between department choice and admissions outcome. The set of endogenous variables consists of the protected attribute, namely sex of the applicant, S , the department they applied to, D , and the decision of the admissions committee, A . We assume that S,A are binary variables and D is a discrete-valued variable taking finite number of values, where S=0,1 corresponds to male, female applicants, respectively, and A=0,1 corresponds to reject and accept, respectively. 3 3 3 The assumption of binary sex is purely for mathematical simplicity. Given that, possibly, societal biases nudge applicants to departments at differing rates depending on their sex, we assume that S affects D . Since departments are the primary decision-making units and have different admission rates, we also assume that D affects A . The question of whether acceptance decisions discriminate against sex centers around the direct causal effect of S on A (w.r.t. the three variables S,D,A ), and therefore we allow such an effect in the model. Indeed, if such a direct causal effect is absent, then the applicant’s sex does not directly affect the admission outcome (only indirectly through department choice). On the other hand, if such a direct causal effect is present, then we would consider this unfair, provided that we know that it is not mediated purely via latent unprotected attributes (e.g., undergraduate department, or undergraduate university chosen by the applicant) that could play a similar ‘fair’ mediating role as D . We will—for the time being—assume no latent unprotected attributes exist that mediate an effect of sex on admissions outcome, and come back to this point in Section 6 . In this section, we also assume the absence of any confounding between the variables (in addition to the absence of any selection bias). The structural equations are given by \displaystyle S \displaystyle=f_{S}(U_{S}), \displaystyle D \displaystyle=f_{D}(S,U_{D}), (1) \displaystyle A \displaystyle=f_{A}(S,D,U_{A}), where U_{S},U_{D} and U_{A} denote independent exogenous random variables. We denote the family of SCMs parameterized by the functions in ( 1 ) and the exogenous distribution as \mathbb{M}_{\text{no-cf}} . For M\in\mathbb{M}_{\text{no-cf}} , the causal graph G(M) is a subgraph of the directed acyclic graph (DAG) in Figure 2(a) . 3.1 Fairness Notions We define a fairness notion to be a certain condition that is required to be satisfied by a causal model to be deemed fair. These conditions can take the form of constraints expressed in terms of observational, interventional, counterfactual or graphical queries on the SCMs in the families of causal models defined by modeling assumptions, in our case, \mathbb{M}_{\text{no-cf}} . While the criteria for the fairness notions in Section 3 are phrased in terms of queries corresponding to different rungs of the causal ladder, in our case, any condition can only be tested using observational data. The investigation of Berkeley’s admission data was initiated on the observation that the well-known fairness notion of demographic parity P_{M}{\left({A=1\mid S=0}\right)}=P_{M}{\left({A=1\mid S=1}\right)} did not hold. This fairness notion is based purely on observational data and we have already noted that it falls prey to Simpson’s paradox. We now present another observational notion of fairness that can be seen as a conditional version of demographic parity. Definition 8 (Observational Notion of Fairness) . M\in\mathbb{M}_{\text{no-cf}} is fair according to the observational notion of fairness if it belongs to \displaystyle H^{0}_{\text{no-cf-obs}}\triangleq\left\{M\in\mathbb{M}_{\text{no-cf}}:\forall d,s,P_{M}{\left({D=d,S=s}\right)}>0\right. \displaystyle\!\!\implies\!\!P_{M}{\left({A=1\,|\,S=s,D=d}\right)}=P_{M}{\left({A=1\,|\,D=d}\right)}\}. Bickel et al. [1975] proposed this notion for the Berkeley data. It is equivalent to the conditional independence A\perp\!\!\!\perp S\mid D . Hence, any valid conditional independence test for A\perp\!\!\!\perp S\mid D provides a statistical test for this notion. Indeed, the analysis of Bickel et al. [1975] shows that the data contain not enough evidence to reject the null hypothesis that this conditional independence holds, and therefore, concludes fairness. From the causal graph of \mathbb{M}_{\text{no-cf}} in Figure 2(a) , a natural subset of fair causal models is those without the edge S\rightarrow A . Definition 9 (Graphical Notion of Fairness) . M\in\mathbb{M}_{\text{no-cf}} is fair according to the graphical notion of fairness if it belongs to the null hypothesis set H^{0}_{\text{no-cf-graph}}\triangleq\left\{M\in\mathbb{M}_{\text{no-cf}}:S\rightarrow A\notin G{\left({M}\right)}\right\} . Pearl [2009, Section 4.5.3] discusses the direct effect in the context of the Berkeley admissions example, where he objects to conditioning on the department and instead proposes intervening on department choice, which corresponds to the controlled direct effect (CDE) [ Pearl, 2001 ] of the ‘treatment’, S , on the outcome, A , for every value of the mediator, i.e., every department choice d . Definition 10 (Interventional Notion of Fairness) . M\in\mathbb{M}_{\text{no-cf}} is fair according to the interventional notion of fairness if it belongs to the null hypothesis set \displaystyle H^{0}_{\text{no-cf-inter}}\triangleq\left\{M\in\mathbb{M}_{\text{no-cf}}:\forall d,s\right. \displaystyle\left.P_{M}{\left({A=1\mid\text{do}(S=s),\text{do}(D=d)}\right)}\right. \displaystyle\left.=P_{M}{\left({A=1\mid\text{do}(D=d)}\right)}\right\}. Recent analyses of the Berkeley example emphasize counterfactual notions of fairness. In Pearl [2009, Section 4.5.4] , Pearl and Mackenzie [2018] , Pearl considers a counterfactual quantity, namely the natural direct effect (NDE) [ Robins and Greenland, 1992 , Pearl, 2001 ] by motivating a hypothetical experiment where “all female candidates retain their department preferences but change their gender [sex] identification (on the application form) from female to male”. Subsequent causal fairness works [ Nabi and Shpitser, 2018 , Chiappa, 2019 ] build on this and propose fairness notions based on known path-specific versions of NDE where the ‘direct path’ from S to A is viewed as ‘unfair’ as opposed to the ‘fair’ path S\rightarrow D\rightarrow A . For the Berkeley example, the NDE (s\rightarrow s^{\prime}) is given by P_{M}{\left({A^{\text{do}(S=s^{\prime},D=D^{\text{do}(S=s)})}=1}\right)}-P_{M}{\left({A^{\text{do}(S=s)}=1}\right)} for s\neq s^{\prime} . Note that by Pearl’s mediation formula [ Pearl, 2001 ] , the above is identified (assuming \forall d,s,P_{M}{\left({D=d,S=s}\right)}>0 ) as \displaystyle\sum\limits_{d} \displaystyle\left(P_{M}{\left({A=1\mid D=d,S=s^{\prime}}\right)}\right. \displaystyle\left.-P_{M}{\left({A=1\mid D=d,S=s}\right)}\right)P_{M}{\left({D=d\mid S=s}\right)}. This implies that if the observational notion of fairness and positivity hold, the NDE is 0 . However, the converse is not necessarily true. For example, if one department favors male applicants and another favors female applicants, then the NDE could be 0 while it is not necessary that the observational notion of fairness holds. Other counterfactual notions of fairness include those by Kusner et al. [2017] . The authors define a counterfactual fairness notion that implies demographic parity (see Section E.1.1 for a proof) for the Berkeley example; we have already seen that this particular fairness notion falls prey to Simpson’s paradox. In the appendix, however, they define a path-dependent notion of counterfactual fairness. 4 4 4 This notion is specifically motivated by the Berkeley example. In Section E.1.2 we show that, in our setting, testing for the path-dependent counterfactual fairness notion is equivalent to testing for the conditional independence A\perp\!\!\!\perp S\mid D . We now propose an alternate counterfactual notion of fairness and later compare testing of the same. Definition 11 (Counterfactual Notion of Fairness) . M\in\mathbb{M}_{\text{no-cf}} is fair according to the counterfactual notion of fairness if it belongs to the null hypothesis set \displaystyle H^{0}_{\text{no-cf-ctrf}} \displaystyle\triangleq\left\{M\in\mathbb{M}_{\text{no-cf}}:\forall d,s,\right. \displaystyle\left.P_{M}(A^{\text{do}(S=s,D=d)}=A^{\text{do}(D=d)})=1\right\} The alternative hypotheses are given by the complement of the null hypotheses w.r.t. \mathbb{M}_{\text{no-cf}} . We consider the interventional and counterfactual fairness notions as analogues of the absence of direct effect that are framed in terms of rung- 2 and rung- 3 notions, namely interventional and counterfactual distributions, respectively. Therefore, the interventional notion of fairness considers comparing interventional distributions that result from intervening on all the variables with intervening on all variables but the protected variable, and the counterfactual fairness notions compare potential outcomes resulting from the same set of interventions. Given that the notions are defined on different rungs of the causal hierarchy, it is perhaps not surprising that they are nested accordingly. The assumption of no confounding simplifies the relations as we can prove equivalence of a few notions under positivity. The proof is deferred to Section C.1 . Lemma 12 . H^{0}_{\text{no-cf-graph}}=H^{0}_{\text{no-cf-ctrf}}\subset H^{0}_{\text{no-cf-inter}}\subset H^{0}_{\text{no-cf-obs}}. If for all s,d , P_{M}(s,d)>0 , then in addition, we have H^{0}_{\text{no-cf-inter}}=H^{0}_{\text{no-cf-obs}} . Despite the nested nature of the fairness notions at different rungs of the causal hierarchy, we prove that the sets of observational distributions that these notions induce are identical. The proof is in Section C.2 . Theorem 13 . Let \displaystyle{\cal P}_{\text{no-cf-graph}} \displaystyle\triangleq\left\{P_{M}{\left({D,A,S}\right)}:M\in H^{0}_{\text{no-cf-graph}}\right\}, \displaystyle{\cal P}_{\text{no-cf-ctrf}} \displaystyle\triangleq\left\{P_{M}{\left({D,A,S}\right)}:M\in H^{0}_{\text{no-cf-ctrf}}\right\}, \displaystyle{\cal P}_{\text{no-cf-inter}} \displaystyle\triangleq\left\{P_{M}{\left({D,A,S}\right)}:M\in H^{0}_{\text{no-cf-inter}}\right\}, \displaystyle{\cal P}_{\text{no-cf-obs}} \displaystyle\triangleq\left\{P_{M}{\left({D,A,S}\right)}:M\in H^{0}_{\text{no-cf-obs}}\right\}. Then {\cal P}_{\text{no-cf-graph}}={\cal P}_{\text{no-cf-ctrf}}={\cal P}_{\text{no-cf-inter}}={\cal P}_{\text{no-cf-obs}}. In summary, despite the fact that we analyze the Berkeley admissions case using multiple fairness notions, under the assumption of no confounding, with observational data, they can all be tested using a conditional independence test. If the data contains enough evidence to reject conditional independence, then the data generating mechanism is unfair w.r.t. the observational notion of fairness (assuming no latent unprotected mediators). On the other hand, if the data does not contain enough evidence to reject conditional independence, then the data generating mechanism is fair w.r.t. the observational notion of fairness. However, this extrapolation of the outcome of the statistical test on the fairness implications does not hold for the interventional, counterfactual and graphical notions. The following example illustrates that for the graphical notion of fairness, an unfaithful causal model, where A is directly affected by S , could satisfy conditional independence. Example 14 . Let M\in\mathbb{M}_{\text{no-cf}} be defined as U_{S}\sim\text{Ber}(\delta),U_{A}\sim\text{Ber}(\varepsilon),U_{D}\sim\text{Ber}(\frac{1}{2}) where \delta,\varepsilon\in[0,1] and S=U_{S},D=S\oplus U_{D},A=S\oplus D\oplus U_{A} . Here, A\perp\!\!\!\perp S\mid D but S is a parent of A , i.e., M\in H^{0}_{\text{no-cf-obs}} , but M\notin H^{0}_{\text{no-cf-graph}} . For the interventional notion of fairness, the following example illustrates that a causal model that violates positivity could satisfy conditional independence but not the interventional notion of fairness. Example 15 . Let M\in\mathbb{M}_{\text{no-cf}} be defined as U_{S}=0,U_{D}=0,U_{A}\sim\text{Ber}{\left({\varepsilon}\right)} where \varepsilon\in[0,\frac{1}{2}) , and S=0,D=0,A=S\oplus U_{A} . Here A\perp\!\!\!\perp S\mid D , but for all d , P_{M}{\left({A=1\mid\text{do}(S=1),\text{do}(D=d)}\right)}=1-\varepsilon\neq P_{M}{\left({A=1\mid\text{do}(D=d)}\right)}=\varepsilon . Therefore, M\in H^{0}_{\text{no-cf-obs}} , but M\notin H^{0}_{\text{no-cf-inter}} . So, if the outcome of the test is that conditional independence cannot be rejected ( M\in H^{0}_{\text{no-cf-obs}} ), then due to the aforementioned observations, we cannot conclude that the underlying causal model belongs to the causal null hypothesis of the interventional or counterfactual or graphical fairness notions, i.e., our conclusion is that fairness is “undecidable” with respect to these notions. However, if the outcome of the statistical test is that there is enough evidence in the data to reject conditional independence ( M\notin H^{0}_{\text{no-cf-obs}} ), then we can conclude that the underlying causal model does not belong to the causal null hypothesis of any of the fairness notions. If we rule out the existence of any latent, unprotected mediator between S and A , we can conclude that the data generating mechanism is unfair. In the next section, we enlarge the class of models to allow for confounding between D and A and perform a similar reasoning exercise. 4 Berkeley Case: With Latent Confounding Between Department And Outcome S A D Figure 3 : Causal graph of a model without assumptions We now take a more careful causal modeling approach. Instead of starting from variables and reasoning about structural equations that we allow, we start with assuming that all structural equations exist. 5 5 5 Since this allows for causal cycles, this would require using e.g. the framework of simple SCMs [ Bongers et al., 2021 ] . For the Berkeley example, Figure 3 shows a causal graph of an SCM that we start with. We now provide rationale for ruling out few structural equations. Based on chronology of events, we rule out those where D directly affects S , where A directly affects D and where A directly affects S . We rule out unobserved common causes of S and D , and S and A since we model S to be sex at birth. While latent selection bias might introduce bidirected edges [ Chen et al., 2024 ] that are incident on S , we assume for now that there is no selection bias in the dataset. The resulting class of SCMs has structural equations of the form S=f_{S}(U_{S}),D=f_{D}(S,U,U_{D}),A=f_{A}(S,D,U,U_{A}) where U,U_{S},U_{D} and U_{A} denote independent exogenous random variables. We define \mathbb{M}_{\text{cf}} to be the family of models parameterized by the above structural equations and the exogenous distribution. Further, we define \mathbb{M}_{\text{cf}+}=\left\{M\in\mathbb{M}_{\text{cf}}:\forall s,P_{M}(S=s)>0\right\} . For M\in\mathbb{M}_{\text{cf}} (and \mathbb{M}_{\text{cf}+} ), the causal graph is a subgraph of the one shown in Figure 2(b) . Although we arrived at allowing confounding between department and outcome through a careful causal modeling approach, this is not a novel consideration. In particular, Kruskal [ Fairley and Mosteller, 1977 , Pg 128-129] demonstrated an example where the existence of a latent confounder, such as state of residence, can render Bickel et al. [1975] ’s analysis incorrect. Other natural latent confounders include, for example, level of department-specific technical skills that influence both the department choice of an applicant and the admissions outcome. Since our modeling assumptions expand the family of SCMs under consideration to \mathbb{M}_{\text{cf}+} , the fairness notions that we discussed in the previous section are modified accordingly to obtain null hypothesis sets H^{0}_{\text{cf-graph}+},H^{0}_{\text{cf-inter}+} and H^{0}_{\text{cf-ctrf}+} . Definition 16 (Fairness Notions with Confounding) . For M\in\mathbb{M}_{\text{cf}+} the null hypothesis set corresponding to the interventional, counterfactual and graphical notion of fairness are \displaystyle H^{0}_{\text{cf-inter}+} \displaystyle\triangleq\left\{M\in\mathbb{M}_{\text{cf}+}:\forall d,s,\right. \displaystyle\left.P_{M}{\left({A=1\mid\text{do}(S=s),\text{do}(D=d)}\right)}\right. \displaystyle\left.=P_{M}{\left({A=1\mid\text{do}(D=d)}\right)}\right\}, \displaystyle H^{0}_{\text{cf-ctrf}+} \displaystyle\triangleq\left\{M\in\mathbb{M}_{\text{cf}+}:\forall d,s,\right. \displaystyle\left.P_{M}(A^{\text{do}(S=s,D=d)}=A^{\text{do}(D=d)})=1\right\}, \displaystyle H^{0}_{\text{cf-graph}+} \displaystyle\triangleq\left\{M\in\mathbb{M}_{\text{cf}+}:S\rightarrow A\notin G{\left({M}\right)}\right\}. While the above notions generalize straightforwardly from the no-confounder setting, this is no longer the case for the observational notion. In addition, while the statistical tests for the no-confounder model are straightforward, this is no longer the case for the aforementioned null hypotheses since A\not\!\perp\!\!\!\perp S\mid D in general. 4.1 Graphical Notion and the Instrumental Variable (IV) Inequalities In the presence of latent confounding, graphical queries, such as absence of edges, impose equality or inequality constraints [ Evans, 2016 , Wolfe et al., 2019 ] in addition to conditional independence constraints which are the only constraints imposed by a DAG. For the Berkeley case with confounding, since the path S\rightarrow D\leftrightarrow A is open when conditioned on D , we have S\not\!\perp\!\!\!\perp A\mid D in general. Our test for the graphical notion of fairness for \mathbb{M}_{\text{cf}+} stems from the observation that a model M\in H^{0}_{\text{cf-graph}+} lies in the instrumental variable (IV) model class \mathbb{M}_{\text{IV}+} where S is considered the instrument, D the treatment, and A the effect. If all modeled endogenous variables are discrete-valued, a necessary condition for the observational distribution 6 6 6 While we express the IV inequalities as a condition satisfied by the observational distribution, in Section B we reason that they are more appropriately expressed as conditions in terms of P_{M}(X,Y\mid\text{do}(Z)) . resulting from M\in\mathbb{M}_{\text{IV}+} is to satisfy the IV inequalities [ Pearl, 1995 ] , which in the context of Figure 1 are given by \max_{x}\sum_{y}\max_{z}P_{M}{\left({X=x,Y=y\mid Z=z}\right)}\leq 1. (2) Since the IV inequalities are only necessary conditions, an arbitrary distribution on X,Y,Z that satisfies the IV inequality does not necessarily imply that it is an entailed distribution of a model from the IV model class. Bonet [2001] showed that for the binary instrument, treatment and effect case, the IV inequalities are also sufficient conditions. In Theorem 17 , we show that for the case where the instrument and outcome are binary and the treatment is discrete-valued with finite support, any distribution that satisfies the IV inequality is also entailed by some causal model from the IV model class. To the best of our knowledge, Theorem 17 is a novel result. We defer the proof to Section D.1 . Theorem 17 . Let X,Y,Z be discrete random variables defined on {\cal X},{\cal Y},{\cal Z} respectively, with |{\cal X}|=n\geq 2,|{\cal Y}|=2,|{\cal Z}|=2 . Define {\cal P}_{\mathbb{M}_{\text{IV}+}}\triangleq\left\{P_{M}(X,Y,Z):M\in\mathbb{M}_{\text{IV}+}\right\} and the set of joint distributions that satisfy the IV inequalities as \displaystyle{\cal P}_{\text{IV}+} \displaystyle\triangleq\left\{P(X,Y,Z):P(X,Y\mid Z)\text{ satisfies }\eqref{eq:iv}\right. \displaystyle\left.\text{ and }\forall z,P(Z=z)>0\right\}. Then {\cal P}_{\text{IV}+}={\cal P}_{\mathbb{M}_{\text{IV}+}}. For the Berkeley admissions case, the observational distribution satisfying the IV inequalities implies that there exists a causal explanation (model) where the directed edge S\rightarrow A is absent, i.e., given that P(A,D,S)\in{\cal P}_{\text{IV}+} , there exists M\in\mathbb{M}_{\text{IV}+} such that P_{M}(A,D,S)=P(A,D,S) . On the other hand, the observational distribution violating the IV inequalities does not necessarily imply that the edge S\rightarrow A is present since the IV model class, \mathbb{M}_{\text{IV}+} , is only a subset of all the models that do not contain the edge S\rightarrow A in the causal graph. For example, the existence of latent confounding between S and A in a model M may result in M\notin\mathbb{M}_{\text{IV}+} , even though G{\left({M}\right)} does not necessarily contain the directed edge S\rightarrow A . However, the causal modeling assumption that defined \mathbb{M}_{\text{cf}+} rules out latent confounding between S and A . Therefore, given our modeling assumptions, H^{0}_{\text{cf-graph}+}=\mathbb{M}_{\text{IV}+} , and in turn, we conclude that violating the IV inequalities implies that M\in\mathbb{M}_{\text{cf}+}\backslash H^{0}_{\text{cf-graph}+} . As in the previous section, it is possible that causal models that lie outside H^{0}_{\text{cf-graph}+} (“unfair” models) induce observational distributions that lie in {\cal P}_{\text{IV}+} , i.e., satisfy the IV inequalities. Therefore, satisfying the IV inequalities is not conclusive evidence that the data-generating mechanism is fair, i.e., our conclusion should be that fairness is undecidable. In Section 5 we introduce a Bayesian test for the IV inequalities. 4.2 Bounds on Interventional Notion of Fairness For M\in\mathbb{M}_{\text{cf}+} , the interventional notion of fairness is equivalent to a vanishing CDE, where the latter is not identifiable in our case. By a response-function parameterization [ Balke, 1995 , Balke and Pearl, 1997 ] of M\in\mathbb{M}_{\text{cf}+} , we can express the interventional distributions in Definition 16 as a linear function of response variables. Further, the observational distribution is also expressed as a linear function of the response variables. Using the symbolic linear programming approach of Balke [1995] , we obtain upper and lower bounds in terms of the observational distribution, specifically, P_{M}{\left({A,D\mid S}\right)} . Indeed, Cai et al. [2008] express the same bounds which we reproduce below. The CDE \displaystyle P_{M}{\left({A=1\mid\text{do}(S=1),\text{do}(D=d)}\right)} \displaystyle-P_{M}{\left({A=1\mid\text{do}(S=0),\text{do}(D=d)}\right)}, lies in the interval \displaystyle\left[\Pr\left(A=1,d\mid S=1\right)+\Pr\left(A=0,d\mid S=0\right)-1,\right. \displaystyle\left.1-\Pr\left(A=0,d\mid S=1\right)-\Pr\left(A=1,d\mid S=0\right)\right]. For the interventional notion of fairness, the CDE must be 0 for all d . By setting the lower bound to be at most 0 and the upper bound to be at least 0 , we recover the IV inequalities in ( 2 ). While Cai et al. [2008] do not point out the connection to the IV inequalities, they find it “remarkable that we [they] get such a simple formula, consisting of only one additive expression in the lower bound and one additive expression in the upper bound”. In the next subsection, we show that the connection to the IV inequalities is not a coincidence. 4.3 A Family of Equivalent Tests The graphical and interventional fairness notions end up imposing identical constraints on the observational distribution. However, note that H^{0}_{\text{cf-inter}+}\supseteq H^{0}_{\text{cf-graph}+} . In fact, we prove in Section D.2 that H^{0}_{\text{cf-graph}+}=H^{0}_{\text{cf-ctrf}+}\subset H^{0}_{\text{cf-inter}+} . Models in H^{0}_{\text{cf-inter}+}\backslash H^{0}_{\text{cf-graph}+} (Example 14 with \varepsilon=\frac{1}{2} ) are such that the edge S\rightarrow A exists in the causal graph and yet, the interventional queries in Definition 16 are equal. Given that the null hypothesis of the interventional fairness notion is a strict superset of that of the graphical fairness notion, we might expect the same relation to hold in the resulting set of observational distributions for these hypotheses, thus giving us potentially different tests. In contrast, like in Section 3 , we show that the corresponding sets of observational distributions resulting from models in H^{0}_{\text{cf-inter}+} , H^{0}_{\text{cf-ctrf}+} , H^{0}_{\text{cf-graph}+} are identical. Section D.3 contains the proof. Theorem 18 . Let \displaystyle{\cal P}_{\text{cf-graph}} \displaystyle\triangleq\left\{P_{M}{\left({D,A,S}\right)}:M\in H^{0}_{\text{cf-graph}+}\right\}, \displaystyle{\cal P}_{\text{cf-inter}} \displaystyle\triangleq\left\{P_{M}{\left({D,A,S}\right)}:M\in H^{0}_{\text{cf-inter}+}\right\}, \displaystyle{\cal P}_{\text{cf-ctrf}} \displaystyle\triangleq\left\{P_{M}{\left({D,A,S}\right)}:M\in H^{0}_{\text{cf-ctrf}+}\right\}. Then {\cal P}_{\text{cf-inter}}={\cal P}_{\text{cf-ctrf}}={\cal P}_{\text{cf-graph}}={\cal P}_{\text{IV}+}, where {\cal P}_{\text{IV}+} is defined in Theorem 17 . In summary, testing for the graphical, interventional and counterfactual notions of fairness, with confounding, all boil down to testing the IV inequalities. 4.4 Comparison With Existing Fairness Notions The utility of considering statistical tests is that we can now compare different fairness notions for a particular case with respect to the same causal modeling assumptions. In this section, we consider the three existing counterfactual fairness notions, namely the NDE [ Nabi and Shpitser, 2018 , Chiappa, 2019 ] , and the counterfactual and path-dependent counterfactual fairness notions in Kusner et al. [2017] . For NDE, Kaufman et al. [2005] obtain bounds for the all-binary setting. Using these bounds, we obtain a strictly weaker test than the IV inequalities. 7 7 7 Intuitively, the reason is the same as in Section 3; the NDE averages over departments, and a positive bias in one department may cancel out against a similarly strong negative bias in another department. Hence, vanishing NDE does not imply that each department takes fair decisions. We show in Section E.2.2 that the counterfactual notion of fairness of Kusner et al. [2017] implies demographic parity even when confounding is allowed. In Section E.2.3 we show that testing the path-dependent counterfactual fairness notion of Kusner et al. [2017] is equivalent to testing the IV inequalities. 5 Bayesian Testing Procedure We start with proposing a Bayesian test for IV inequalities using finite data. Although Ramsahai [2008] proposed a frequentist test and derived the distribution of the likelihood ratio, providing a means to obtain a p-value for the case of a binary treatment, it is unclear what the test would be for our case where the “treatment” is not binary. In addition, Wang et al. [2017] provide a frequentist test that involves multiple one-sided independence tests. In contrast, the Bayesian test has straightforward extensions to other hypotheses that are expressed in terms of the observational distribution, including bounds on unidentifiable causal queries. Consider discrete random variables X,Y,Z where |{\cal X}|=n,|{\cal Y}|=2,|{\cal Z}|=2 . The observational distribution P(X,Y,Z) lies in the 4n-1 dimensional simplex, denoted by \Delta , which is a subset of \mathbb{R}^{4n} . We consider a Bayesian model selection procedure where \displaystyle\mathbb{M}_{0} \displaystyle=\left\{\theta\in\Delta:\theta\text{ satisfies the IV inequalities}\right\}, \displaystyle\mathbb{M}_{1} \displaystyle=\left\{\theta\in\Delta:\theta\text{ does not satisfy the IV inequalities}\right\}. Note that \mathbb{M}_{0}\,\dot{\cup}\,\mathbb{M}_{1}=\Delta . Given a finite dataset of (X,Y,Z) tuples, denoted by R_{1},R_{2},\cdots,R_{m} , and choices of prior distributions for the models, \pi{\left({\theta\mid\mathbb{M}_{0}}\right)},\pi{\left({\theta\mid\mathbb{M}_{1}}\right)} , we report a confidence interval for the posterior probability of satisfying the IV inequalities, i.e., P{\left({\mathbb{M}_{0}\mid R_{1},R_{2},\cdots,R_{m}}\right)}=\int_{\theta\in\mathbb{M}_{0}}P{\left({\theta\mid R_{1},R_{2},\cdots,R_{m}}\right)}d\theta . Given the posterior density P{\left({\theta\mid R_{1},R_{2},\cdots,R_{m}}\right)} , we estimate the posterior probability of \mathbb{M}_{0} by IID sampling from the posterior density n times and counting how often the sample satisfies the IV inequalities, which we denote by N . Since N is a binomial random variable with parameters n and P{\left({\mathbb{M}_{0}\mid R_{1},R_{2},\cdots,R_{m}}\right)} , a confidence interval on P{\left({\mathbb{M}_{0}\mid R_{1},R_{2},\cdots R_{m}}\right)} is readily obtained by the Clopper-Pearson method [ Clopper and Pearson, 1934 ] . Results on Berkeley admission data: We use the UCBAdmissions [ R Core Team, 2023 ] dataset from R that contains counts for each sex-department-admissions outcome tuple for the 6 largest departments. Therefore, |{\cal X}|=6 , |{\cal Y}|=2,|{\cal Z}|=2 . Since the data satisfies the positivity for sex, we can use the Bayesian model selection procedure. For parameters \theta={\left({P(d,a,s):s\in{\cal X}_{S},d\in{\cal X}_{D},a\in{\cal X}_{A}}\right)} ,we choose a flat Dirichlet prior over \Delta giving us \pi(\theta|\mathbb{M}_{i})=c_{i}\text{Dir}{\left({1,1,\cdots,1}\right)}\bm{1}\left[\theta\in\mathbb{M}_{i}\right] where c_{i} is a normalizing constant. The counts from the data are used to obtain the posterior, P(\theta\mid R_{1},R_{2},\cdots,R_{m}) which is also a truncated Dirichlet distribution. Using n=10^{6} samples, we observe no violations of the IV inequalities. Therefore, the confidence interval for the posterior probability of the Berkeley data satisfying the IV inequalities is \left[1-3.69\times 10^{-6},1\right] . As mentioned in Section 4.1 satisfying the IV inequalities implies that fairness is undecidable. In Section F , we carry out a sensitivity analysis by varying the chosen prior. We also report results on a different dataset from Bol [2023] that investigates sex-based discrimination in awarding cum-laude distinctions to graduate students. In addition to the Bayesian test, the maximum likelihood (ML) estimator satisfies the IV inequalities, implying that there isn’t enough evidence to reject the null hypothesis when doing a likelihood ratio test. An implementation of Wang et al. [2017] for the Berkeley dataset also does not reject the null hypothesis (see Section F for details). The code can be found at https://github.com/SourbhBh/BerkeleyCode . 6 Discussion The Berkeley admissions case is a canonical example in the causal fairness literature. Bickel et al. [1975] reached the conclusion of there being no evidence to reject fairness, although under the unrealistic assumption of no unobserved confounding. When allowing for unobserved confounding, we arrived at a different conclusion: since there is very strong evidence that the data satisfies the IV inequalities, it is undecidable from the available data whether the admission procedure was fair or discriminated against sex. While our analysis was centered around the Berkeley case, there are multiple aspects that generalize—a) \mathbb{M}_{\text{cf}} can be thought of as a mediator with a confounder between mediator and outcome, which is common in mediation analysis. b) The approach of fairness notions being causal hypotheses, with respect to the class of models defined by modeling assumptions, that need to be translated into statistical tests to be useful in practice. c) The observation that for the case of inequality constraints on observational data, a straightforward Bayesian testing procedure is available. Generalization: Non-binary variables : While the Berkeley dataset had a binary protected attribute and a binary decision outcome, our approach can be generalized to non-binary variables. The response-function parametrization of \mathbb{M}_{\text{IV}+} can be used to characterize the set of induced observed distributions of X,Y and Z as a convex polyhedral set. Using computer algebra, this polyhedral set can be characterized by linear inequality constraints even when the alphabet sizes are arbitrary. However, the number of linear inequality constraints quickly explodes for non-binary Z (e.g. for binary X and Y , and for |{\cal Z}|=2,3,4 and 5 , we get 12,48,160 and 420 inequalities). Nevertheless, these sets of inequality constraints can be tested using a Bayesian testing procedure akin to the one we outline in Section 5 , thus giving a statistical test for all our fairness notions since the statement of Theorem 18 holds for any alphabet size. Generalization: Relaxing unobserved confounding assumptions In Section 4 we made the assumption of no unobserved confounding between S and any other variable. In Section D.4 we show that our main theorems, Theorem 17 and Theorem 18 hold even in the case of allowing for confounding between S and D . If unobserved confounding is allowed between S and A , the set of observational distributions induced by causal models where the direct effect between sex and admissions outcome is absent is the entire simplex which is the same as the set of observational distributions induced by causal models where the direct effect between sex and admissions outcome is present. Therefore, from observational data, it is not possible to distinguish between the presence/absence of a direct effect from sex to admissions outcome if we allow for confounding between sex and admissions outcome. Unmeasured Mediators : Our conclusions and fairness notions are with respect to the measured variables. The presence of unmeasured mediators could change the interpretation of the results. For example, an unmeasured mediator that is not a ‘protected’ variable, such as choice of undergraduate department, would result in a direct effect of sex on admissions outcome in the marginalized causal model, but might still be considered ‘fair’. Therefore, even if the data does not satisfy the IV inequalities, we can only claim the existence of the direct effect from sex to admissions outcome and not whether there is unfairness. However, if the existence of latent unprotected mediators is ruled out, then we can consider the existence of a direct effect as ‘unfair’. Undecidability : Our analysis can be viewed as a Bayesian model comparison between causal models (i) without a direct effect of S on A , vs. (ii) with a direct effect of S on A . If we only have access to observational data then causal models in (ii) result in a saturated model in the observed distribution space, i.e., the set of induced observational distributions is the entire simplex, whereas for the causal models in (i), the IV inequalities hold. Therefore, satisfying the IV inequalities implies that fairness is ‘undecidable’ since causal models in both (i) and (ii) satisfy the IV inequalities. Violating the IV inequalities, however, would imply that there is a direct effect of sex on admissions outcome. Only with the additional modeling assumption of there being no unprotected mediators between S and A , could it then be concluded that the admissions process was ‘unfair’. Selection Bias: UCBAdmissions dataset only has data from the 6 largest departments as opposed to 85 in Bickel et al. [1975] . Also, the fraction of female students is significantly smaller than the fraction of male students. Hence, it is plausible that (latent) selection mechanisms alter the causal model resulting in violating the assumptions of, for instance, absence of a bidirected edge in the causal graph between S and A [ Chen et al., 2024 ] . Since allowing for selection bias enlarges the model class \mathbb{M}_{\text{cf}+} , and given that the data satisfies the IV inequalities, we conclude that allowing for selection bias will not change our conclusion. Acknowledgements. This work was supported by Booking.com. We thank an anonymous reviewer for their feedback which helped us modulate some of our conclusions. Revisiting the Berkeley Admissions data: Statistical Tests for Causal Hypotheses (Supplementary Material) Appendix A Additional Preliminaries Definition 19 (Twin SCM) . Let M={\left({V,W,{\cal X},P,f}\right)} be an SCM. The twinning operation maps M to the twin SCM M^{\text{twin}}\triangleq{\left({V\cup V^{\prime},W,{\cal X}_{V}\times{\cal X}_{V^{\prime}}\times{\cal X}_{W},P,\tilde{f}}\right)} where V^{\prime}=\left\{v^{\prime}:v\in V\right\} is a disjoint copy of V and the causal mechanism \tilde{f}:{\cal X}_{V}\times{\cal X}_{V^{\prime}}\times{\cal X}_{W}\mapsto{\cal X}_{V}\times{\cal X}_{V^{\prime}} is given by \tilde{f}{\left({x_{V},x_{V^{\prime}},x_{W}}\right)}={\left({f(x_{V},x_{W}),f(x_{V^{\prime}},x_{W})}\right)} . Definition 20 (Solution function) . Let M={\left({V,W,\mathcal{X},f,P}\right)} be an acyclic SCM and C\subseteq V . A solution function of M with respect to C is a measurable mapping g_{C}:{\cal X}_{V\backslash C}\times\mathcal{X}_{W}\mapsto\mathcal{X}_{C} that satisfies the structural equations for C , i.e., for all x_{V\backslash C}\in\mathcal{X}_{V\backslash C} , P(X_{W}) -a.a x_{W}\in\mathcal{X}_{W} , g_{C}{\left({x_{V\backslash C},x_{W}}\right)}=f_{C}{\left({x_{V\backslash C},g_{C}{\left({x_{V\backslash C},x_{W}}\right)},x_{W}}\right)}. Definition 21 (Markov kernels) . Let \mathcal{T} and \mathcal{W} be measurable spaces. A Markov kernel is defined as a measurable map K:\mathcal{T}\mapsto\mathcal{P}{\left({\mathcal{W}}\right)} where \mathcal{P}{\left({\mathcal{W}}\right)} is defined as the space of probability measures on \mathcal{W} . Appendix B IV Inequalities Expressed as Markov Kernels For ( 2 ) to be well defined, we required that P_{M}(Z=z)>0 for all z for any M\in\mathbb{M}_{\text{IV}+} (see Definition ( 7 )). In this section, we relax this requirement by noting that, in fact, IV inequalities are more appropriately expressed in terms of P_{M}{\left({X,Y\mid\text{do}(Z)}\right)} . Lemma 22 . Let \mathbb{M}_{\text{IV}}\triangleq\left\{M:G(M)\text{ is a subgraph of Figure }\ref{fig:iv}\right\} . For any M\in\mathbb{M}_{\text{IV}} , \max_{x}\sum_{y}\max_{z}P_{M}{\left({X=x,Y=y\mid\text{do}(Z=z)}\right)}\leq 1. (3) Proof. Since \displaystyle P_{M}(X=x,Y=y\mid\text{do}(Z=z)) \displaystyle=P_{M}(f_{X}(z,U)=x,f_{Y}(x,U)=y) \displaystyle\leq P(f_{Y}(x,U)=y)=P_{M}(Y=y\mid\text{do}(X=x)), \max_{x}\sum_{y}\max_{z}P_{M}{\left({X=x,Y=y\mid\text{do}(Z=z)}\right)}\leq\max_{x}\sum_{y}P_{M}(Y=y\mid\text{do}(X=x))=1. (4) ∎ Note that ( 3 ) is defined even when \exists z\in{\cal Z} such that P(Z=z)=0 . In contrast, positivity must be assumed in ( 2 ) for the terms to be well-defined. Further, if positivity is assumed, then \mathbb{M}_{\text{IV}}=\mathbb{M}_{\text{IV}+} and ( 3 ) is identical to ( 2 ). Appendix C Proofs for Section 3 C.1 Nested Fairness Notions: Without Confounding See 12 Proof. H^{0}_{\text{no-cf-graph}}=H^{0}_{\text{no-cf-ctrf}} : We first show that H^{0}_{\text{no-cf-graph}}\subseteq H^{0}_{\text{no-cf-ctrf}} . M\in H^{0}_{\text{no-cf-graph}} implies \forall d , f_{A}(s,d,U_{A}) is constant in s P -a.s. Therefore, for all d,s , P_{M}{\left({f_{A}(s,d,U_{A})=f_{A}(S,d,U_{A})}\right)}=1. (5) Therefore, M\in H^{0}_{\text{no-cf-ctrf}} . For the converse, M\in H^{0}_{\text{no-cf-ctrf}} implies ( 5 ). For s\neq s^{\prime} , and all d , P_{M}{\left({f_{A}(s,d,U_{A})=f_{A}(S,d,U_{A})}\right)}=P_{M}{\left({f_{A}(s,d,U_{A})=f_{A}(s^{\prime},d,U_{A})}\right)}P_{M}{\left({S=s^{\prime}}\right)}+P_{M}{\left({S=s}\right)}. From ( 5 ) if P_{M}{\left({S=s^{\prime}}\right)}>0 , we conclude P_{M}{\left({f_{A}(s,d,U_{A})=f_{A}(s^{\prime},d,U_{A})}\right)}=1 . If P_{M}{\left({S=s^{\prime}}\right)}=0 , since ( 5 ) holds for s^{\prime} , i.e., for all d,s^{\prime} , P_{M}{\left({f_{A}(s^{\prime},d,U_{A})=f_{A}(S,d,U_{A})}\right)}=1 , we have P_{M}{\left({f_{A}(s^{\prime},d,U_{A})=f_{A}(s,d,U_{A})}\right)}=1. Therefore, M\in H^{0}_{\text{no-cf-graph}} . H^{0}_{\text{no-cf-ctrf}}\subset H^{0}_{\text{no-cf-inter}} : For M\in H^{0}_{\text{no-cf-ctrf}} , ( 5 ) implies P_{M}{\left({f_{A}(s,d,U_{A})}\right)}=P_{M}{\left({f_{A}(S,d,U_{A})}\right)} for all d,s , implying M\in H^{0}_{\text{no-cf-inter}} . Since Example 14 belongs to H^{0}_{\text{no-cf-inter}}\backslash H^{0}_{\text{no-cf-ctrf}} , the inclusion is strict. H^{0}_{\text{no-cf-inter}}\subset H^{0}_{\text{no-cf-obs}} : For M\in H^{0}_{\text{no-cf-inter}} , P_{M}{\left({A=1\mid\text{do}(D=d),\text{do}(S=s)}\right)} is constant in s . Consider a pair s,d, such that, for M\in H^{0}_{\text{no-cf-inter}} , P_{M}{\left({s,d}\right)}>0 . Then P_{M}{\left({A=1\mid\text{do}(D=d),\text{do}(S=s)}\right)}=P_{M}{\left({A=1\mid D=d,S=s}\right)}. (6) Note that, if, for s^{\prime}\neq s , P_{M}{\left({s^{\prime},d}\right)}=0 , then P_{M}{\left({A=1\mid D=d,S=s}\right)}=P_{M}{\left({A=1\mid D=d}\right)} . If instead, P_{M}{\left({s^{\prime},d}\right)}>0 , then from ( 6 ) for S=s^{\prime} , we have that P_{M}{\left({A=1\mid D=d,S=s}\right)}=P_{M}{\left({A=1\mid D=d,S=s^{\prime}}\right)}=P_{M}{\left({A=1\mid D=d}\right)}. Therefore, we conclude that M\in H^{0}_{\text{no-cf-obs}} implying H^{0}_{\text{no-cf-inter}}\subseteq H^{0}_{\text{no-cf-obs}} . Since the SCM in Example 15 lies in H^{0}_{\text{no-cf-obs}}\backslash H^{0}_{\text{no-cf-inter}} , H^{0}_{\text{no-cf-inter}}\subset H^{0}_{\text{no-cf-obs}} . If for all s,d , P_{M}{\left({s,d}\right)}>0 , then for M\in H^{0}_{\text{no-cf-obs}} , P_{M}{\left({A=1\mid D=d,S=s}\right)}=P_{M}{\left({A=1\mid\text{do}(D=d),\text{do}(S=s)}\right)} is constant in s and equal to P_{M}{\left({A=1\mid\text{do}(D=d)}\right)} . This implies M\in H^{0}_{\text{no-cf-inter}} . Therefore, if for all s,d , P_{M}{\left({s,d}\right)}>0 , then H^{0}_{\text{no-cf-inter}}=H^{0}_{\text{no-cf-obs}} . ∎ C.2 Equivalence of Tests Without Confounding See 13 Proof. From Lemma 12 , {\cal P}_{\text{no-cf-graph}}={\cal P}_{\text{no-cf-ctrf}}\subseteq{\cal P}_{\text{no-cf-inter}}\subseteq{\cal P}_{\text{no-cf-obs}} . Therefore, it suffices to prove that {\cal P}_{\text{no-cf-graph}}={\cal P}_{\text{no-cf-obs}} . For every P_{M}\in{\cal P}_{\text{no-cf-obs}} , P_{M}{\left({A,S,D}\right)}=P_{M}{\left({S}\right)}\otimes P_{M}{\left({D\mid S}\right)}\otimes P_{M}{\left({A\mid D}\right)}. Hence, \exists\tilde{M}\in H^{0}_{\text{no-cf-graph}} such that P_{M}{\left({A,D,S}\right)}=P_{\tilde{M}}{\left({A,D,S}\right)} . ∎ Appendix D Proofs for Section 4 D.1 Sharpness of IV inequalities Z U_{Z} Y X R Figure 4 : Response-function parameterization of M\in\mathbb{M}_{\text{IV}+} See 17 Proof. We prove a more general statement that includes Theorem 17 as a special case. Lemma 23 . Let X,Y,Z be discrete random variables defined on {\cal X},{\cal Y},{\cal Z} respectively, with |{\cal X}|=n\geq 2,|{\cal Y}|=2,|{\cal Z}|=2 . Define \mathcal{K}_{\text{IV}}\triangleq\left\{K(X,Y\mid Z):K(X,Y\mid Z)\text{ satisfies }\eqref{eq:iv_MK}\right\} . Define \mathcal{K}_{\mathbb{M}_{\text{IV}}}\triangleq\left\{P_{M}{\left({X,Y\mid\text{do}(Z)}\right)}:M\in\mathbb{M}_{\text{IV}}\right\}. Then \mathcal{K}_{\text{IV}}=\mathcal{K}_{\mathbb{M}_{\text{IV}}}. Note that {\cal P}_{\text{IV}+}=\left\{P(Z):\forall z,P(Z=z)>0\right\}\otimes\mathcal{K}_{\text{IV}} since assuming positivity, ( 2 ) is identical to ( 3 ). Further, {\cal P}_{\mathbb{M}_{\text{IV}+}}=\left\{P_{M}(Z):M\in\mathbb{M}_{\text{IV}+}\right\}\otimes\mathcal{K}_{\mathbb{M}_{\text{IV}}} since assuming positivity, \mathbb{M}_{\text{IV}}=\mathbb{M}_{\text{IV}+} and P_{M}{\left({X,Y\mid\text{do}(Z)}\right)}=P_{M}{\left({X,Y\mid Z}\right)} for M\in\mathbb{M}_{\text{IV}+} . Since the first factors are identical, Theorem 17 follows from Lemma 23 . ∎ Proof of Lemma 23 . For M\in\mathbb{M}_{\text{IV}} , the response-function parameterization yields a counterfactually equivalent SCM [ Forré and Mooij, 2025 , Section 8.4] \tilde{M}=(V,\tilde{W},\tilde{\mathcal{X}},\tilde{f},\tilde{P}) , where V=\left\{Z,X,Y\right\},\tilde{W}=\left\{R,U_{Z}\right\},\tilde{\mathcal{X}}=\mathcal{X}_{V}\times\mathcal{X}_{\tilde{W}},\tilde{f}={\left({\tilde{f}_{Z},\tilde{f}_{X},\tilde{f}_{Y}}\right)} where we define \mathcal{X}_{R},\tilde{f},\tilde{P} through the function \Phi:\mathcal{X}_{W}\mapsto\mathcal{X}_{\tilde{W}} where \displaystyle\mathcal{X}_{R} \displaystyle\triangleq{\cal X}^{{\cal Z}}\times{\cal Y}^{{\cal X}}, \displaystyle\forall u_{Z},u_{X},u_{Y},u,\Phi{\left({u_{Z},u_{X},u_{Y},u}\right)} \displaystyle\triangleq{\left({{\left({z\mapsto f_{X}(z,u,u_{X}),x\mapsto f_{Y}(x,u,u_{Y})}\right)},u_{Z}}\right)}, \displaystyle\forall u_{Z},\tilde{f}_{Z}(u_{Z}) \displaystyle\triangleq f_{Z}(u_{Z}), \displaystyle\forall r,z,\tilde{f}_{X}{\left({r,z}\right)} \displaystyle\triangleq r_{1}{\left({z}\right)}, \displaystyle\forall r,x,\tilde{f}_{Y}{\left({r,x}\right)} \displaystyle\triangleq r_{2}{\left({x}\right)}, where r={\left({r_{1},r_{2}}\right)} and \tilde{P} is the push-forward distribution \Phi_{*}(P) . Note that \mathcal{X}_{R} is a discrete space, R a discrete random variable, and \tilde{P}(R) a discrete distribution over \mathcal{X}_{R} . Under the response-function parameterization, only \tilde{P}(R) is a parameter. We will consider \tilde{P}(R) to be an element of \mathbb{R}^{n_{X}^{n_{Z}}n_{Y}^{n_{X}}} where \#{\cal X}=n_{X},\#{\cal Z}=n_{Z},\#{\cal Y}=n_{Y} and {\cal K}_{\tilde{\mathbb{M}}_{\text{IV}+}}\triangleq\left\{P_{\tilde{M}}(X,Y\mid\text{do}(Z)):M\in\mathbb{M}_{\text{IV}}\right\} to be a subset of \mathbb{R}^{n_{X}n_{Y}n_{Z}} . Note that because of the counterfactual equivalence of the response-function parameterization, which in turn implies interventional equivalence, {\cal K}_{\tilde{\mathbb{M}}_{\text{IV,r}}}=\mathcal{K}_{\mathbb{M}_{\text{IV}}} . From Lemma 22 , \mathcal{K}_{\mathbb{M}_{\text{IV}}}\subseteq\mathcal{K}_{\text{IV}} . To show the converse, we show that each extreme point of \mathcal{K}_{\text{IV}} is obtained by a point in \mathcal{K}_{\mathbb{M}_{\text{IV}}} . We enumerate all extreme points of \mathcal{K}_{\text{IV}} in Lemma 24 . We show that each such extreme point is obtained by the following response-function. Choose x,x^{\prime}\in{\cal X},y,y^{\prime}\in{\cal Y} with y=y^{\prime} if x=x^{\prime} . Then any response function satisfying \displaystyle r_{1}(z) \displaystyle=\begin{cases}x\quad z=0\\ x^{\prime}\quad z=1\end{cases} \displaystyle r_{2}(\tilde{x}) \displaystyle=\begin{cases}y\quad\tilde{x}=x\\ y^{\prime}\quad\tilde{x}=x^{\prime}\\ \text{arbitrary}\quad\text{otherwise}\end{cases} gives all extreme points of \mathcal{K}_{\text{IV}} . Therefore, \mathcal{K}_{\mathbb{M}_{\text{IV}}}\supseteq\mathcal{K}_{\text{IV}} , implying \mathcal{K}_{\mathbb{M}_{\text{IV}}}=\mathcal{K}_{\text{IV}} . ∎ Lemma 24 . Consider the real vector space \mathbb{R}^{n_{X}n_{Y}n_{Z}} spanned by the canonical basis vectors \left\{\delta_{x,y|z}:x\in{\cal X},y\in{\cal Y},z\in{\cal Z}\right\} where \delta_{x,y|z} denotes a unit vector of length n_{X}n_{Y}n_{Z} where all entries except the one at (x,y,z) are zero. For n_{Y}=n_{Z}=2,n_{X}=n\geq 2 , \mathcal{K}_{\text{IV}} considered as a subset of this vector space is a polyhedral set with extreme points \mathbb{E}=\left\{\delta_{x,y|0}+\delta_{x^{\prime},y^{\prime}|1}:x,x^{\prime}\in{\cal X};y,y^{\prime}\in{\cal Y}:x\neq x^{\prime}\right\}\cup\left\{\delta_{x,y|0}+\delta_{x,y|1}:x\in{\cal X},y\in{\cal Y}\right\}. Proof. Consider \mathcal{K}_{\text{IV}} to be a subset of \mathbb{R}^{n_{X}n_{Y}n_{Z}} where each element of \mathcal{K}_{\text{IV}} is represented as \left\{K{\left({X=x,Y=y\mid Z=z}\right)}\right\}_{x\in{\cal X},y\in{\cal Y},z\in{\cal Z}} and satisfies \displaystyle\forall x\in{\cal X}:K{\left({X=x,Y=0|Z=0}\right)}+K{\left({X=x,Y=1|Z=1}\right)} \displaystyle\leq 1, \displaystyle\forall x\in{\cal X}:K{\left({X=x,Y=0|Z=1}\right)}+K{\left({X=x,Y=1|Z=0}\right)} \displaystyle\leq 1, \displaystyle\forall x\in{\cal X},\forall y\in{\cal Y},\forall z\in{\cal Z}:K{\left({X=x,Y=y|Z=z}\right)} \displaystyle\geq 0, \displaystyle\forall z\in{\cal Z}:\sum_{x,y}K{\left({X=x,Y=y|Z=z}\right)} \displaystyle=1. An elementary result in convex geometry states that a point is an extreme point of a polyhedral set (defined by a set of linear (in)equality constraints) in \mathbb{R}^{m} vector space if and only if it is a feasible point and there exists a subset A of m constraints that are active and linearly independent. For the case at hand, for a point to be an extreme point of \mathcal{K}_{\text{IV}} , it has to satisfy the above 6n+2 constraints (feasibility) and additionally, a subset of n_{X}n_{Y}n_{Z}=4n of them must be active and linearly independent. The two normalization constraints are equality constraints and hence must be active at any feasible point. We first show that all points in \mathbb{E} are extreme points of \mathcal{K}_{\text{IV}} . First, choose x,x^{\prime}\in{\cal X} with x\neq x^{\prime} and choose y,y^{\prime}\in{\cal Y} . It can be verified that \delta_{x,y|0}+\delta_{x^{\prime},y^{\prime}|1} satisfies the IV inequalities with 2 out of the 2n IV inequalities being active. Further, 4n-2 non-negativity constraints and both normalization constraints are active. For the subset A of active constraints we can take e.g. both active IV inequalities together with the 4n_{X}-2 active nonnegativity constraints; one can check that these are linearly independent. Second, choose x\in{\cal X},y\in{\cal Y} . It can be verified that \delta_{x,y|0}+\delta_{x,y|1} satisfies the IV inequalities with 2 out of the 2n IV inequalities being active. Further, 4n-2 non-negativity constraints and both normalization constraints are active. For the subset A of active constraints we can take e.g. both active IV inequalities together with the 4n_{X}-2 active nonnegativity constraints; one can check that these are linearly independent. Finally, we check whether \mathbb{E} exhausts all extreme points. Pick a feasible point b\in\mathbb{R}^{n_{X}n_{Y}n_{Z}} . We will refer to the indices i with b_{i}\neq 0 ( b_{i}=0 ) as the “non-zero (zero) entries of b ”, and to the indices j in active constraint a^{T}b=1 with a_{j}\neq 0 as the “active entries of the constraint”. For a set of entries of b and a set of active entries of one or more active constraints, we refer to their intersection as their “overlap”. We also call the b_{i} corresponding to K(X,Y|Z=z) a “stratum”. Suppose b is an extreme point of \mathcal{K}_{\text{IV}} . Then at least 4n-2 of the 6n inequality constraints must be active. Hence if exactly r IV inequalities are active at b , then we need at least 4n-2-r active nonnegativity constraints at b , or in other words, b must have at least 4n-2-r zero entries. Because both strata of b need to be normalized, b can have at most 4n-2 zero entries. The number of zero entries in b that can overlap with the active entries of the active IV inequalities is upper bounded by r ; indeed, otherwise there would be an active IV inequality for which both its active entries are zero entries of b , contradicting that this IV inequality is active. We will show that the only possibilities for b are the extreme points that we already identified, proceeding case by case. 1. r>2 . This yields a contradiction with the normalization constraints. Indeed, each entry of b appears in exactly one IV inequality, and each IV inequality contains exactly two active entries (one for each stratum). Hence, the sum of those entries of b that correspond with active entries of active IV constraints must be exactly r . However, by the normalization constraints, the sum over all entries of b must be 2. This implies that r\leq 2 . 2. r=2 . Then b must contain at least 4n-4 zero entries, of which at most two can overlap with the active entries of the active IV inequalities. (a) If there are no such overlaps, then all other entries of b must zero entries. This means that we need the 4n-4 nonnegativity constraints corresponding to those other entries in the subset A , as well as the two normalization and inequality constraints; no other active constraints exist that could be added to A . But then the constraints in A would not be linearly independent. Indeed, the following linear dependence between the active constraints is obtained: adding the coefficient vectors of the two active IV inequalities and those of all active non-negativity constraints yields the same result as adding the coefficient vectors of the two normalization constraints. So we arrive at a contradiction. (b) If there is at least one overlap, then b_{j}=1 with j the other active entry in the active IV inequality with the overlap. This implies 2n-1 zeroes in the stratum of j . Consider now the other active IV inequality. Since b_{j}=1 , the active entry corresponding to that stratum must be a zero entry of b . Hence, b_{k}=1 for k the other active entry in this IV inequality. This means that b must be of the form \delta_{x_{1},x_{2}|0}+\delta_{x_{1}^{\prime},x_{2}^{\prime}|1} , with the two non-zero entries corresponding to active entries of two different IV inequalities, and hence we recover the extreme points already identified. 3. r=1 . Then b must contain at least 4n-3 zero entries, of which at most one can overlap with the active entries of active IV inequalities. Because of the normalization constraints, we need at least one non-zero entry in both strata. This means that there must be a stratum x_{3} with exactly one non-zero entry, and then b must be of the form b=\delta_{x_{1},x_{2}|x_{3}}+\gamma\delta_{x_{1}^{\prime},x_{2}^{\prime}|x_{3}^{\prime}}+(1-\gamma)\delta_{x_{1}^{\prime\prime},x_{2}^{\prime\prime}|x_{3}^{\prime}} with \gamma\in(0,1] and x_{3}^{\prime}\neq x_{3} . This means that the active IV inequality must be the one that has active entry (x_{1},x_{2}|x_{3}) . Its other active entry is then (x_{1},1-x_{2}|x_{3}^{\prime}) , and this must be an overlapping zero entry of b . If \gamma=1 then this would also activate the IV inequality that contains active entry (x_{1}^{\prime},x_{2}^{\prime}|x_{3}^{\prime}) . Since the only active IV inequality must also contain active entry (x_{1},x_{2}|x_{3}) , this gives a contradiction, as the two coefficients of b at the active entries sum to 2 instead of 1. Hence, \gamma\in(0,1) , and b contains exactly 4n-3 zero entries. So the active constraints consist of 1 active IV inequality, 2 active normalization constraints and 4n-3 active nonnegativity constraints. However, these are not linearly independent. Indeed: subtracting the coefficient vectors of the 2n-1 nonnegativity constraints in stratum x_{3} from the coefficient vector of the normalization constraint of that stratum, and adding the coefficient vector of the nonnegativity constraint corresponding to (x_{1},1-x_{2}|x_{3}^{\prime}) gives a vector that is identical to the coefficient vector of the active IV inequality. So we have arrived at a contradiction. 4. r=0 . Then b must contain at least 4n-2 zero entries. Because of the normalization constraints, b needs one non-zero entry in both strata, and its value must be 1. This will activate at least one IV inequality. Contradiction. ∎ D.2 Nested Fairness Notions: With Confounding Proposition 25 . H^{0}_{\text{cf-graph}}=H^{0}_{\text{cf-ctrf}}\subset H^{0}_{\text{cf-inter}}, H^{0}_{\text{cf-graph}+}=H^{0}_{\text{cf-ctrf}+}\subset H^{0}_{\text{cf-inter}+}. Proof. If M\in\mathbb{M}_{\text{cf}} , then A^{\text{do}(S=s,D=d)}=f_{A}{\left({s,d,U_{A},U}\right)},\hskip 14.22636ptA^{\text{do}(D=d)}=f_{A}{\left({S,d,U_{A},U}\right)}. (7) For M\in H^{0}_{\text{cf-graph}} , since S is not a parent of A , for all d,f_{A}(s,d,U_{A},U) is constant in s P -a.s. This implies that for all d,s,s^{\prime} , P_{M}{\left({f_{A}{\left({s,d,U_{A},U}\right)}=f_{A}{\left({s^{\prime},d,U_{A},U}\right)}}\right)}=1 Therefore, for all s,d , P_{M}{\left({f_{A}{\left({s,d,U_{A},U}\right)}=f_{A}{\left({S,d,U_{A},U}\right)}}\right)}=1, (8) implying that M\in H^{0}_{\text{cf-ctrf}} . For the converse, M\in H^{0}_{\text{cf-ctrf}} implies ( 8 ). For s\neq s^{\prime} , and all d , \displaystyle P_{M}{\left({f_{A}(s,d,U_{A},U)=f_{A}(S,d,U_{A},U)}\right)} \displaystyle=P_{M}{\left({f_{A}(s,d,U_{A},U)=f_{A}(s^{\prime},d,U_{A},U)}\right)}P_{M}{\left({S=s^{\prime}}\right)}+P_{M}{\left({S=s}\right)}. If P_{M}{\left({S=s^{\prime}}\right)}>0 , we conclude P_{M}{\left({f_{A}(s,d,U_{A},U)=f_{A}(s^{\prime},d,U_{A},U)}\right)}=1 . If P_{M}{\left({S=s^{\prime}}\right)}=0 , since ( 8 ) holds for s^{\prime} , we have P_{M}{\left({f_{A}(s,d,U_{A},U)=f_{A}(s^{\prime},d,U_{A},U)}\right)}=1 . Therefore, M\in H^{0}_{\text{cf-graph}} . Therefore, H^{0}_{\text{cf-graph}}=H^{0}_{\text{cf-ctrf}} . Further, H^{0}_{\text{cf-graph}+}=H^{0}_{\text{cf-ctrf}+} . We now prove that H^{0}_{\text{cf-ctrf}}\subseteq H^{0}_{\text{cf-inter}} . For M\in H^{0}_{\text{cf-ctrf}} , ( 8 ) holds. Therefore, for all s,d , P_{M}{\left({f_{A}{\left({s,d,U_{A},U}\right)}}\right)}=P_{M}{\left({f_{A}{\left({S,d,U_{A},U}\right)}}\right)}. (9) Therefore, H^{0}_{\text{cf-ctrf}}\subseteq H^{0}_{\text{cf-inter}} and subsequently H^{0}_{\text{cf-ctrf}+}\subseteq H^{0}_{\text{cf-inter}+} . Note that the Example 14 lies in \mathbb{M}_{\text{cf}} (and in \mathbb{M}_{\text{cf}+} since P_{M}(S=s)>0 for all s ) for any U that is independent of U_{S},U_{D},U_{A} . Further, P_{M}{\left({A=1\mid\text{do}(S=s),\text{do}(D=d)}\right)}=0.5=P_{M}{\left({A=1\mid\text{do}(D=d)}\right)} ; however, S is a parent of A . Therefore, H^{0}_{\text{cf-ctrf}}\subset H^{0}_{\text{cf-inter}} and H^{0}_{\text{cf-ctrf}+}\subset H^{0}_{\text{cf-inter}+} . ∎ D.3 Equivalence of Statistical Tests S U_{S} A D R Figure 5 : Response-function parameterization of M\in\mathbb{M}_{\text{cf}} See 18 Proof. Like in Section D.1 , we prove a more general statement, Lemma 26 , that includes Theorem 18 as a special case. We define analogues of \mathbb{M}_{\text{cf}+},H^{0}_{\text{cf-notion}+} that remove the positivity assumption, P_{M}(S=s)>0 for all s , as \mathbb{M}_{\text{cf}},H^{0}_{\text{cf-notion}} , respectively (where we use ‘notion’ as a placeholder for ‘graph’, ‘ctrf’ and ‘inter’). Lemma 26 . Let \displaystyle{\cal K}_{\text{cf-graph}} \displaystyle\triangleq\left\{P_{M}{\left({D,A\mid\text{do}(S)}\right)}:M\in H^{0}_{\text{cf-graph}}\right\}, \displaystyle{\cal K}_{\text{cf-inter}} \displaystyle\triangleq\left\{P_{M}{\left({D,A\mid\text{do}(S)}\right)}:M\in H^{0}_{\text{cf-inter}}\right\}, \displaystyle{\cal K}_{\text{cf-ctrf}} \displaystyle\triangleq\left\{P_{M}{\left({D,A\mid\text{do}(S)}\right)}:M\in H^{0}_{\text{cf-ctrf}}\right\}. Then {\cal K}_{\text{cf-inter}}={\cal K}_{\text{cf-ctrf}}={\cal K}_{\text{cf-graph}}=\mathcal{K}_{\text{IV}}, where \mathcal{K}_{\text{IV}} is defined in Lemma 23 . Note that {\cal P}_{\text{IV}+}=\left\{P(Z):\forall z,P(Z=z)>0\right\}\otimes\mathcal{K}_{\text{IV}} since assuming positivity, ( 2 ) is identical to ( 3 ). Further, {\cal P}_{\text{cf-notion}}=\left\{P_{M}(S):M\in H^{0}_{\text{cf-notion}+}\right\}\otimes{\cal K}_{\text{cf-notion}} since assuming positivity, \mathbb{M}_{\text{cf}+}=\mathbb{M}_{\text{cf}} and P_{M}{\left({D,A\mid\text{do}(S)}\right)}=P_{M}{\left({D,A\mid S}\right)} for M\in\mathbb{M}_{\text{cf}+} . Since the first factors are identical, Theorem 18 follows from Lemma 26 . ∎ Proof of Lemma 26 . For M\in\mathbb{M}_{\text{cf}} , the response-function parameterization yields a counterfactually equivalent SCM, \tilde{M} represented by the tuple (V,\tilde{W},\tilde{\mathcal{X}},\tilde{f},\tilde{P}) , where V=\left\{S,D,A\right\},\tilde{W}=\left\{R,U_{S}\right\},\tilde{\mathcal{X}}=\mathcal{X}_{V}\times\mathcal{X}_{\tilde{W}},\tilde{f}={\left({\tilde{f}_{S},\tilde{f}_{D},\tilde{f}_{A}}\right)} where we define \mathcal{X}_{R},\tilde{f},\tilde{P} through the function \Phi:\mathcal{X}_{W}\mapsto\mathcal{X}_{\tilde{W}} where \displaystyle\mathcal{X}_{R} \displaystyle\triangleq\mathcal{X}_{D}^{\mathcal{X}_{S}}\times\mathcal{X}_{A}^{\mathcal{X}_{S}\times\mathcal{X}_{D}}, \displaystyle\forall u_{S},u_{D},u_{A},u,\Phi{\left({u_{S},u_{D},u_{A},u}\right)} \displaystyle\triangleq{\left({{\left({s\mapsto f_{D}(s,u,u_{D}),(s,d)\mapsto f_{A}(s,d,u,u_{A})}\right)},u_{S}}\right)}, \displaystyle\forall u_{S},\tilde{f}_{S}{\left({u_{S}}\right)} \displaystyle\triangleq f_{S}(u_{S}), \displaystyle\forall s,\tilde{f}_{D}{\left({r,s}\right)} \displaystyle\triangleq r_{1}{\left({s}\right)}, \displaystyle\forall s,d,\tilde{f}_{A}{\left({r,s,d}\right)} \displaystyle\triangleq r_{2}{\left({s,d}\right)}, where r={\left({r_{1},r_{2}}\right)} and \tilde{P} is the push-forward distribution \Phi_{*}(P) . Note that \mathcal{X}_{R} is a discrete space, R a discrete random variable, and \tilde{P}(R) a discrete distribution over \mathcal{X}_{R} . Under the response-function parameterization, only \tilde{P}(R) is a parameter and we will abuse notation and denote it as \tilde{P} henceforth. Therefore, we can represent H^{0}_{\text{cf-graph}} in the parameter space as \bar{H}^{0}_{\text{graph}}\triangleq\left\{\tilde{P}\in\triangle{\left({{\cal X}_{R}}\right)}:\tilde{P}{\left({r_{1},r_{2}}\right)}\neq 0\text{ implies }\forall d,r_{2}(0,d)=r_{2}{\left({1,d}\right)}\right\}. (10) To express H^{0}_{\text{cf-inter}} , we express the interventional Markov kernels P_{\tilde{M}}{\left({A\mid\text{do}(S),\text{do}(D)}\right)} in terms of \tilde{P} . Since counterfactual equivalence implies interventional equivalence, for all s,d , P_{M}{\left({A=1\mid\text{do}(S=s),\text{do}(D=d)}\right)}=P_{\tilde{M}}{\left({A=1\mid\text{do}(S=s),\text{do}(D=d)}\right)} , where \displaystyle P_{\tilde{M}}{\left({A=1\mid\text{do}(S=s),\text{do}(D=d)}\right)} \displaystyle=\sum\limits_{{\left({r_{1},r_{2}}\right)}\in{\cal X}_{R}}\bm{1}{\left[{r_{2}{\left({s,d}\right)}=1}\right]}\tilde{P}{\left({r_{1},r_{2}}\right)}, (11) \displaystyle P_{\tilde{M}}{\left({A=1\mid\text{do}(D=d)}\right)} \displaystyle=\sum\limits_{s^{*}}\sum\limits_{{\left({r_{1},r_{2}}\right)}\in{\cal X}_{R}}\bm{1}{\left[{r_{2}{\left({s^{*},d}\right)}=1}\right]}\tilde{P}{\left({r_{1},r_{2}}\right)}P_{\tilde{M}}{\left({s^{*}}\right)}, (12) Subtracting ( 11 ) from ( 12 ) we get \displaystyle P_{\tilde{M}}{\left({A=1\mid\text{do}(S=s),\text{do}(D=d)}\right)}-P_{\tilde{M}}{\left({A=1\mid\text{do}(D=d)}\right)} \displaystyle={\left({\sum\limits_{{\left({r_{1},r_{2}}\right)}\in{\cal X}_{R}}{\left({\bm{1}{\left[{r_{2}{\left({0,d}\right)}=1}\right]}-\bm{1}{\left[{r_{2}{\left({1,d}\right)}=1}\right]}}\right)}\tilde{P}{\left({r_{1},r_{2}}\right)}}\right)}P_{\tilde{M}}{\left({s^{\prime}}\right)}=0 (13) for M\in H^{0}_{\text{cf-inter}} , where s^{\prime}\neq s . Similarly, \displaystyle P_{\tilde{M}}{\left({A=1\mid\text{do}(S=s^{\prime}),\text{do}(D=d)}\right)}-P_{\tilde{M}}{\left({A=1\mid\text{do}(D=d)}\right)} \displaystyle={\left({\sum\limits_{{\left({r_{1},r_{2}}\right)}\in{\cal X}_{R}}{\left({\bm{1}{\left[{r_{2}{\left({0,d}\right)}=1}\right]}-\bm{1}{\left[{r_{2}{\left({1,d}\right)}=1}\right]}}\right)}\tilde{P}{\left({r_{1},r_{2}}\right)}}\right)}P_{\tilde{M}}{\left({s}\right)}=0. (14) Since both ( 13 ) and ( 14 ) hold, the response-function parameterized analogue of H^{0}_{\text{cf-inter}} is \bar{H}^{0}_{\text{inter}}\triangleq\left\{\tilde{P}\in\triangle{\left({{\cal X}_{R}}\right)}:\forall d,\sum\limits_{{\left({r_{1},r_{2}}\right)}\in{\cal X}_{R}}{\left({\bm{1}{\left[{r_{2}{\left({0,d}\right)}=1}\right]}-\bm{1}{\left[{r_{2}{\left({1,d}\right)}=1}\right]}}\right)}\tilde{P}{\left({r_{1},r_{2}}\right)}=0\right\}. (15) Note that both \bar{H}^{0}_{\text{graph}} and \bar{H}^{0}_{\text{inter}} are polyhedra in \triangle{\left({{\cal X}_{R}}\right)} . Further, \bar{H}^{0}_{\text{graph}}\subseteq\bar{H}^{0}_{\text{inter}} . While, \bar{H}^{0}_{\text{graph}},\bar{H}^{0}_{\text{inter}} are collections of distributions, we will also refer to them as collection of response-function-parameterized SCMs. From interventional equivalence (which follows as a result of counterfactual equivalence) of the response-function-parameterization, we have \displaystyle{\cal K}_{\text{cf-graph}} \displaystyle=\left\{P_{\tilde{M}}{\left({D,A\mid\text{do}(S)}\right)}:\tilde{M}\in\bar{H}^{0}_{\text{graph}}\right\} \displaystyle{\cal K}_{\text{cf-inter}} \displaystyle=\left\{P_{\tilde{M}}{\left({D,A\mid\text{do}(S)}\right)}:\tilde{M}\in\bar{H}^{0}_{\text{inter}}\right\}. We now show that {\cal K}_{\text{cf-inter}}={\cal K}_{\text{cf-graph}}=\mathcal{K}_{\text{IV}} . First, notice that {\cal K}_{\text{cf-inter}}\supseteq{\cal K}_{\text{cf-graph}} since \bar{H}^{0}_{\text{inter}}\supseteq\bar{H}^{0}_{\text{graph}} . We first show that {\cal K}_{\text{cf-inter}}\subseteq\mathcal{K}_{\text{IV}} and then {\cal K}_{\text{cf-graph}}=\mathcal{K}_{\text{IV}} which concludes the argument. {\cal K}_{\text{cf-inter}}\subseteq\mathcal{K}_{\text{IV}} : The solution function of the response-function parameterized SCM, g_{A,D}:{\cal X}_{S}\times{\cal X}_{R}\mapsto{\cal X}_{A}\times{\cal X}_{D} induces a mapping from \triangle{\left({{\cal X}_{R}}\right)} which can be considered as a subset of \mathbb{R}^{\#{\cal X}_{R}} to the set of Markov kernels P_{\tilde{M}}{\left({D,A\mid\text{do}(S)}\right)} which can be considered to be a subset of \mathbb{R}^{\#{\left({{\cal X}_{A}}\right)}\times\#{\left({{\cal X}_{D}}\right)}\times\#{\left({{\cal X}_{S}}\right)}} . The condition in ( 15 ) implies that for all d , \sum\limits_{r:r_{2}{\left({0,d}\right)}=1}\tilde{P}(r)=\sum\limits_{r:r_{2}{\left({1,d}\right)}=1}\tilde{P}(r). (16) Since, \sum\limits_{r}\tilde{P}{\left({r}\right)}=1 , \sum\limits_{r:r_{2}{\left({0,d}\right)}=0}\tilde{P}(r)=\sum\limits_{r:r_{2}{\left({1,d}\right)}=0}\tilde{P}(r). (17) Denote P_{\tilde{M}}{\left({D=d,A=a\mid\text{do}(S=s)}\right)} by P_{\tilde{M}}{\left({d,a||s}\right)} . For P_{\tilde{M}}{\left({d,a||s}\right)}\in{\cal K}_{\text{cf-inter}} , P_{\tilde{M}}{\left({d,a||s}\right)}=\sum\limits_{r:r_{1}{\left({s}\right)}=d,r_{2}{\left({s,d}\right)}=a}\tilde{P}(r). Therefore, from ( 16 ), \displaystyle\sum\limits_{r:r_{2}{\left({0,d}\right)}=1}\tilde{P}(r) \displaystyle=P_{\tilde{M}}(1,d||0)+\sum\limits_{r:r_{1}(0)\neq d,r_{2}{\left({0,d}\right)}=1}\tilde{P}(r) (18) \displaystyle=\sum\limits_{r:r_{2}{\left({1,d}\right)}=1}\tilde{P}(r) \displaystyle=P_{\tilde{M}}(1,d||1)+\sum\limits_{r:r_{1}(1)\neq d,r_{2}{\left({1,d}\right)}=1}\tilde{P}(r). (19) From ( 17 ), \displaystyle\sum\limits_{r:r_{2}{\left({0,d}\right)}=0}\tilde{P}(r) \displaystyle=P_{\tilde{M}}(0,d||0)+\sum\limits_{r:r_{1}(0)\neq d,r_{2}{\left({0,d}\right)}=0}\tilde{P}(r) (20) \displaystyle=\sum\limits_{r:r_{2}{\left({1,d}\right)}=0}\tilde{P}(r) \displaystyle=P_{\tilde{M}}(0,d||1)+\sum\limits_{r:r_{1}(1)\neq d,r_{2}{\left({1,d}\right)}=0}\tilde{P}(r). (21) Since from ( 16 ), \sum\limits_{r}\tilde{P}{\left({r}\right)}=\sum\limits_{r:r_{2}{\left({0,d}\right)}=0}\tilde{P}(r)+\sum\limits_{r:r_{2}{\left({0,d}\right)}=1}\tilde{P}(r)=\sum\limits_{r:r_{2}{\left({0,d}\right)}=0}\tilde{P}(r)+\sum\limits_{r:r_{2}{\left({1,d}\right)}=1}\tilde{P}(r)=1. Substituting from ( 20 ) and ( 19 ), P_{\tilde{M}}(0,d||0)+\sum\limits_{r:r_{1}(0)\neq d,r_{2}{\left({0,d}\right)}=0}\tilde{P}(r)+P_{\tilde{M}}(1,d||1)+\sum\limits_{r:r_{1}(1)\neq d,r_{2}{\left({1,d}\right)}=1}\tilde{P}(r)=1. Similarly, substituting from ( 21 ) and ( 18 ), P_{\tilde{M}}(0,d||1)+\sum\limits_{r:r_{1}(1)\neq d,r_{2}{\left({1,d}\right)}=0}\tilde{P}(r)+P_{\tilde{M}}(1,d||0)+\sum\limits_{r:r_{1}(0)\neq d,r_{2}{\left({0,d}\right)}=1}\tilde{P}(r)=1. This implies P_{\tilde{M}}(0,d||0)+P_{\tilde{M}}(1,d||1)\leq 1,P_{\tilde{M}}(0,d||1)+P_{\tilde{M}}(1,d||0)\leq 1 . These are precisely the IV inequalities and they are satisfied. Therefore, {\cal K}_{\text{cf-inter}}\subseteq\mathcal{K}_{\text{IV}} . {\cal K}_{\text{cf-graph}}=\mathcal{K}_{\text{IV}} follows from Theorem 17 since M\in H^{0}_{\text{cf-graph}} implies M\in\mathbb{M}_{\text{IV}} . By Proposition 25 , the lemma follows. ∎ D.4 Relaxing the assumption of no confounding between S and D In this section, we prove that Theorem 17 and Theorem 18 hold when \mathbb{M}_{\text{IV}+} and \mathbb{M}_{\text{cf}+} are expanded by allowing for confounding between S and D . Denote the corresponding expanded models by \mathbb{M}_{\text{IV}+ZX} and \mathbb{M}_{\text{cf}+SD} , respectively, where the former has structural equations of the form Z=f_{Z}(U_{Z},U_{ZX}),X=f_{X}(Z,U_{X},U_{ZX},U_{XY}),Y=f_{Y}(X,U_{Y},U_{XY}) where U_{Z},U_{X},U_{Y},U_{ZX},U_{XY} are independent exogenous random variables, and the latter has structural equations of the form S=f_{S}(U_{S},U_{SD}),D=f_{D}(S,U_{D},U_{SD},U_{DA}),A=f_{A}(S,D,U_{A},U_{DA}) where U_{S},U_{D},U_{A},U_{SD},U_{DA} are independent exogenous random variables. The corresponding expanded causal null hypothesis corresponding to the fairness notions are denoted by H^{0}_{\text{cf-notion}+SD} where we use ‘notion’ as a placeholder for ‘graph’,‘ctrf’, ‘inter’. For M\in\mathbb{M}_{\text{IV}+ZX} , pick U\sim\text{Unif}\left[0,1\right] and a deterministic map h:\mathcal{Z}\times\left[0,1\right]\mapsto\mathcal{U}_{ZX} such that U_{ZX}=h(Z,U) a.s.. Note that such an h always exists for any random variable U_{ZX} taking values in a standard measurable space (see e.g. [ Forré and Mooij, 2025 , Corollary 2.7.7] ). Define \tilde{M}=(V=\{\tilde{Z},\tilde{X},\tilde{Y}\},W=\{U_{\tilde{Z}},U_{\tilde{X}},U,U_{\tilde{Y}},U_{\tilde{XY}}\},{\cal X}={\cal X}_{V}\times{\cal X}_{W},\tilde{f}=(f_{\tilde{Z}},f_{\tilde{X}},f_{\tilde{Y}}),\tilde{P}) where a) \forall u_{\tilde{Z}},f_{\tilde{Z}}(u_{\tilde{Z}})\triangleq u_{\tilde{Z}} , b) \forall u_{\tilde{X}},u,u_{\tilde{XY}},\tilde{z},f_{\tilde{X}}(u_{\tilde{X}},u,u_{\tilde{XY}},\tilde{z})\triangleq f_{X}(\tilde{z},h(\tilde{z},u),u_{\tilde{X}},u_{\tilde{XY}}) , c) \forall u_{\tilde{Y}},u_{\tilde{XY}},\tilde{x},f_{\tilde{Y}}(u_{\tilde{Y}},u_{\tilde{XY}},\tilde{x})\triangleq f_{Y}(u_{\tilde{Y}},u_{\tilde{XY}},\tilde{x}) , and \tilde{P}=P_{M}(Z)\otimes P_{X}\otimes\text{Unif}\left[0,1\right]\otimes P_{Y}\otimes P_{XY} where P_{X},P_{Y},P_{XY},P_{Z},P_{ZX} are marginals of P over U_{X},U_{Y},U_{XY},U_{Z},U_{ZX} respectively. Note that P_{M}(Z) is the pushforward of P_{Z}\otimes P_{ZX} through f_{Z} thus making \tilde{P} a product distribution over the exogenous random variables. By the above construction, \tilde{M}\in\mathbb{M}_{\text{IV}+} . For every M\in\mathbb{M}_{\text{IV}+ZX} , P_{M}(X,Y,Z)=P_{M}(Z)\times P_{M}(X,Y\mid Z)=P_{\tilde{M}}(\tilde{Z})\times P_{\tilde{M}}(\tilde{X},\tilde{Y}\mid\text{do}(\tilde{Z}))\in{\cal P}_{\mathbb{M}_{\text{IV}+}} . Therefore, \{P_{M}(Z,X,Y):M\in\mathbb{M}_{\text{IV}+ZX}\}={\cal P}_{\mathbb{M}_{\text{IV}+}} . Using a similar argument that replaces the labels Z,X,Y by S,D,A , \{P_{M}(S,D,A):M\in H^{0}_{\text{cf-notion}+SD}\}={\cal P}_{\text{cf-notion}} . This implies Theorem 17 and Theorem 18 hold for \mathbb{M}_{\text{IV}+ZX} and \mathbb{M}_{\text{cf}+SD} respectively. Appendix E Comparison With Existing Notions E.1 Without Confounding E.1.1 Counterfactual Fairness and Demographic Parity We restate the counterfactual notion of fairness from Kusner et al. [2017] for the Berkeley example below. Definition 27 (Counterfactual Fairness [ Kusner et al., 2017 ] ) . M\in\mathbb{M}_{\text{no-cf}} is fair if for all s,d , P_{M}{\left({s,d}\right)}>0 implies P_{M}{\left({A^{\text{do}(S=s^{\prime})}\mid D=d,S=s}\right)}=P_{M}{\left({A^{\text{do}(S=s)}\mid D=d,S=s}\right)} for s^{\prime}\neq s . The counterfactual fairness notion of Kusner et al. [2017] implies demographic parity for the Berkeley example without allowing for confounding. Proposition 28 . If M\in\mathbb{M}_{\text{no-cf}} is counterfactually fair according to Definition 27 , then P_{M} satisfies demographic parity, i.e., for all s,s^{\prime} such that P_{M}(s),P_{M}(s^{\prime})>0 , P_{M}{\left({A=1|S=s}\right)}=P_{M}{\left({A=1|S=s^{\prime}}\right)}. Proof. S U_{S} S^{\prime} A A^{\prime} D D^{\prime} U_{D} U_{A} Figure 6 : Causal graph of twin network (M^{\text{twin }})^{\text{do}(S^{\prime}=s^{\prime})} The right-hand side in Definition 27 is P_{M}{\left({A\mid D=d,S=s}\right)} . Therefore, for s,d such that P_{M}(s,d)>0 , counterfactual fairness implies that P_{M}{\left({A^{\text{do}(S=s^{\prime})},D=d\mid S=s}\right)}=P_{M}{\left({A,D=d\mid S=s}\right)}. Marginalizing D , P_{M}{\left({A^{\text{do}(S=s^{\prime})}\mid S=s}\right)}=P_{M}{\left({A\mid S=s}\right)}. (22) By Rule 2 of do-calculus, if P_{M}(S=s)>0 , \displaystyle P_{M}{\left({A\mid S=s}\right)} \displaystyle=P_{M}{\left({A\mid\text{do}(S=s)}\right)}, \displaystyle\stackrel{{\scriptstyle(a)}}{{=}}P_{M}{\left({A^{\text{do}(S=s^{\prime})}\mid S=s}\right)}, \displaystyle\stackrel{{\scriptstyle(b)}}{{=}}P_{M^{\text{twin}}}{\left({A^{\prime}\mid\text{do}(S^{\prime}=s^{\prime}),S=s}\right)}, \displaystyle\stackrel{{\scriptstyle(c)}}{{=}}P_{M}{\left({A\mid\text{do}(S=s^{\prime})}\right)}, \displaystyle\stackrel{{\scriptstyle(d)}}{{=}}P_{M}{\left({A\mid S=s^{\prime}}\right)}. where (a) follows from ( 22 ), (b) follows from expressing the counterfactual P_{M}{\left({A^{\text{do}(S=s^{\prime})}\mid S=s}\right)} in the twin network model, (c) follows from the twin network, and (d) follows from Rule 2 of do-calculus since P_{M}(s^{\prime})>0 . Therefore, counterfactual fairness implies demographic parity. ∎ Note that demographic parity falls prey to Simpson’s paradox in the Berkeley example. The above result shows that a valid test for demographic parity is a valid test for Kusner et al. [2017] ’s counterfactual fairness notion for the assumed model class, \mathbb{M}_{\text{no-cf}} . E.1.2 Path-dependent Counterfactual Fairness We next show that testing the path-dependent counterfactual fairness notion given in the appendix of Kusner et al. [2017] coincides with a conditional independence test A\perp\!\!\!\perp S\mid D . Definition 29 (Path-dependent Counterfactual Fairness [ Kusner et al., 2017 ] ) . M\in\mathbb{M}_{\text{no-cf}} is fair if for all s,d, P_{M}{\left({s,d}\right)}>0 implies P_{M}{\left({A^{\text{do}(S=s^{\prime},D=d)}=1\mid D=d,S=s}\right)}=P_{M}{\left({A^{\text{do}(S=s,D=d)}=1\mid D=d,S=s}\right)} for s^{\prime}\neq s . Proposition 30 . \left\{P_{M}(A,D,S):M\in\mathbb{M}_{\text{no-cf}}\text{ satisfies path-dependent counterfactual fairness}\right\}={\cal P}_{\text{no-cf-obs}}. Proof. S U_{S} S^{\prime} A A^{\prime} D D^{\prime} U_{D} U_{A} Figure 7 : Causal graph of twin network (M^{\text{twin }})^{\text{do}(S^{\prime}=s^{\prime},D^{\prime}=d)} for M\in\mathbb{M}_{\text{no-cf}} We first show that if M satisfies the path-dependent counterfactual fairness notion then M\in H^{0}_{\text{no-cf-obs}} . The path-dependent counterfactual fairness notion implies that for all s,d such that P_{M}(s,d)>0 , P_{M}{\left({A^{\text{do}(S=s^{\prime},D=d)}=1\mid D=d,S=s}\right)}=P_{M}{\left({A=1\mid D=d,S=s}\right)}. If for s^{\prime}\neq s , P_{M}(s^{\prime},d)>0 , then we simplify P_{M}{\left({A^{\text{do}(S=s^{\prime},D=d)}=1\mid D=d,S=s}\right)} using the twin network in Figure 7 . \displaystyle P_{M}{\left({A^{\text{do}(S=s^{\prime},D=d)}=1\mid D=d,S=s}\right)} \displaystyle=P_{M^{\text{twin}}}{\left({A^{\prime}=1\mid\text{do}(D^{\prime}=d,S^{\prime}=s^{\prime}),D=d,S=s}\right)} \displaystyle\stackrel{{\scriptstyle(a)}}{{=}}P_{M^{\text{twin}}}{\left({A^{\prime}=1\mid\text{do}(D^{\prime}=d,S^{\prime}=s^{\prime})}\right)}, \displaystyle\stackrel{{\scriptstyle(b)}}{{=}}P_{M}{\left({A=1\mid D=d,S=s^{\prime}}\right)}, where (a) follows since S,D\perp\!\!\!\perp A^{\prime} in the intervened twinned SCM and (b) follows from the twinned SCM. This implies that P_{M}{\left({A=1\mid D=d,S=s}\right)}=P_{M}{\left({A=1\mid D=d,S=s^{\prime}}\right)}=P_{M}{\left({A=1\mid D=d}\right)}. If instead, P_{M}(s^{\prime},d)=0 , then still P_{M}{\left({A=1\mid D=d,S=s}\right)}=P_{M}{\left({A=1\mid D=d}\right)} . Therefore, M\in H^{0}_{\text{no-cf-obs}} . Clearly, if M\in H^{0}_{\text{no-cf-graph}} , path-dependent counterfactual fairness is satisfied. However, note that, Example 14 satisfies path-dependent counterfactual fairness but does not belong to H^{0}_{\text{no-cf-graph}} . The conclusion follows from Theorem 13 . ∎ E.2 With Confounding In this section, we compare the statistical tests that result from the NDE that the notions of Nabi and Shpitser [2018] and Chiappa [2019] are based on, and Kusner et al. [2017] ’s counterfactual fairness and path-dependent counterfactual fairness notions, when confounding is allowed. E.2.1 NDE With confounding between the mediator and the outcome, Kaufman et al. [2005] obtain bounds on the NDE for the all-binary variable case by the linear programming approach of Balke and Pearl [1997] . The resulting bounds are implied by the IV inequalities but not equivalent to them, which gives us a strictly weaker test than the IV inequalities. For completeness, we present the bounds below. The lower and upper bounds for NDE(A;0\rightarrow 1) are \displaystyle\max\left.\begin{cases}P(A=0\mid S=0)-1,\\ P(A=0\mid D=0\mid S=0)-P(A=1\mid D=1\mid S=0)+P(A=1\mid D=0\mid S=1)-1\\ P(A=0\mid D=1\mid S=0)-P(A=1\mid D=0\mid S=0)+P(A=1\mid D=1\mid S=1)-1\end{cases}\right\}, \displaystyle\min\left.\begin{cases}1-P(A=1\mid S=0),\\ 1+P(A=0\mid D=1\mid S=0)-P(A=1\mid D=0\mid S=0)-P(A=0\mid D=0\mid S=1)\\ 1+P(A=0\mid D=0\mid S=0)-P(A=1\mid D=1\mid S=0)-P(A=0\mid D=1\mid S=1)\end{cases}\right\}. The lower and upper bounds for NDE(A;1\rightarrow 0) are \displaystyle\max\left.\begin{cases}P(A=0\mid S=1)-1,\\ P(A=1\mid D=0\mid S=0)-P(A=1\mid D=1\mid S=1)+P(A=0\mid D=0\mid S=1)-1\\ P(A=1\mid D=1\mid S=0)-P(A=1\mid D=0\mid S=1)+P(A=0\mid D=1\mid S=1)-1\end{cases}\right\}, \displaystyle\min\left.\begin{cases}1-P(A=1\mid S=1),\\ 1+P(A=0\mid D=0\mid S=1)-P(A=0\mid D=1\mid S=0)-P(A=1\mid D=1\mid S=1)\\ 1+P(A=0\mid D=1\mid S=1)-P(A=0\mid D=0\mid S=0)-P(A=1\mid D=0\mid S=1)\end{cases}\right\}. Equating the NDE to 0 , gives us a strictly larger null hypothesis compared to the one obtained based on the IV inequalities. This implies that the resulting statistical test is strictly weaker. E.2.2 Counterfactual Fairness [ Kusner et al., 2017 ] The proof of Proposition 28 also holds when confounding is allowed since the implications of the do-calculus rules in the proof hold even in the twin network with confounding. E.2.3 Path-dependent Counterfactual Fairness [ Kusner et al., 2017 ] Proposition 31 . If M\in\mathbb{M}_{\text{cf}+} satisfies path-dependent counterfactual fairness then P_{M}(D,A,S)\in{\cal P}_{\text{cf-graph}}={\cal P}_{\text{IV}+} . If M\in H^{0}_{\text{cf-graph}+} , then M satisfies path-dependent counterfactual fairness. Proof. We show that a model M\in\mathbb{M}_{\text{cf}+} that satisfies the path-dependent counterfactual fairness notion is observationally equivalent to a model in H^{0}_{\text{cf-graph}+} . This implies that the set of observational distributions of models that satisfy the path-dependent counterfactual notion of fairness, are described by {\cal P}_{\text{cf-graph}} . If M\in\mathbb{M}_{\text{cf}+} satisfies path-dependent counterfactual fairness, then for all s,d such that P_{M}{\left({s,d}\right)}>0 , P_{M}{\left({A^{\text{do}(S=s^{\prime},D=d)}=1\mid D=d,S=s}\right)}=P_{M}{\left({A^{\text{do}(S=s,D=d)}=1\mid D=d,S=s}\right)} for s^{\prime}\neq s . Note that the right-hand side above is P_{M}{\left({A=1\mid D=d,S=s}\right)} . The counterfactual P_{M}{\left({A^{\text{do}(S=s^{\prime},D=d)}=1\mid D=d,S=s}\right)} is given by the push-forward of P(U_{A},U|D=d,S=s) through f_{A}(s^{\prime},d,U_{A},U) . Because of the independence on the value of s^{\prime} , the same holds for the function \bar{f}_{A}{\left({d,U_{A},U}\right)}=\frac{1}{2}{\left({f_{A}(0,d,U_{A},U)+f_{A}(1,d,U_{A},U)}\right)}. Consider an SCM, \bar{M} , that is identical to M except for the causal mechanism of A being \bar{f}_{A} . Clearly, \bar{M}\in H^{0}_{\text{cf-graph}+} and P_{M}(S,D)=P_{\bar{M}}(S,D) . By the above argument, for all s,d such that P_{M}(s,d)>0 , we have P_{M}{\left({A=1\mid D=d,S=s}\right)}=P_{\bar{M}}{\left({A=1\mid D=d,S=s}\right)} . This implies that P_{M}(D,A,S)\in{\cal P}_{\text{cf-graph}}={\cal P}_{\text{IV}+} . S U_{S} S^{\prime} A A^{\prime} D D^{\prime} U_{D} U_{A} U Figure 8 : Causal graph of twin network (M^{\text{twin }})^{\text{do}(S^{\prime}=s^{\prime},D^{\prime}=d)} for M\in H^{0}_{\text{cf-graph}+} Conversely, if M\in H^{0}_{\text{cf-graph}+} , then from the twin network of M in Figure 8 , clearly A^{\prime}\perp\!\!\!\perp S^{\prime}\mid S,D and therefore, path-dependent counterfactual fairness is satisfied. ∎ Therefore, for the assumed model class, a valid statistical test for path-dependent counterfactual fairness is also a valid test for the IV inequalities from Theorem 17 and vice versa. Appendix F Additional Results for Bayesian Testing Procedure: Cum-laude Dataset and Prior Sensitivity We consider the dataset from Bol [2023] that contains data from 5239 PhD students in the Netherlands studying at a large Dutch university from 2011-2021. Bol [2023] observed a bias in the percentage of ‘cum-laude’ distinctions awarded to male PhD students ( 6.57\% ) versus female PhD students ( 3.68\% ). As in the Berkeley example, there is data on the sex of the student, their academic field and whether they were awarded cum-laude. Unlike the Berkeley example, there are more covariates that measure additional information, including the sex composition of the dissertation committee, the sex composition of the supervisory team that includes the promoters and co-promoters. For the current analyses we don’t take into account these covariates and only analyze the dataset with respect to sex, academic field and award outcome. As reported by Bol [2023] , unlike the Berkeley dataset, the bias among female and male cum-laude award rates does not vanish when conditioned on department. Therefore, with the assumption of no confounding between the academic field choice and the cum-laude award outcome, the conclusion of the conditional independence test implies that the data generating mechanism is unfair when assuming that no latent unprotected mediators between sex and cum-laude award rates exist. Allowing for confounding requires us to use the Bayesian testing procedure proposed in Section 5 for the IV inequalities. We have |{\cal X}|=|{\cal X}_{D}|=6 , |{\cal Y}|=|{\cal X}_{A}|=2,|{\cal Z}|=|{\cal X}_{S}|=2 . We choose a flat Dirichlet prior over parameters ( \theta=\left\{P(d,a,s):d\in{\cal X}_{D},a\in{\cal X}_{A},s\in{\cal X}_{S}\right\} ) in both models \mathbb{M}_{0},\mathbb{M}_{1} , i.e., for i=0,1,\pi(\theta|\mathbb{M}_{i})=c_{i}\text{Dir}{\left({1,1,\cdots,1}\right)} where c_{i} is a normalizing constant. The counts from the data R_{1},R_{2}\cdots R_{m} are used to obtain the posterior, P(\theta\mid R_{1},R_{2},\cdots R_{m}) which is also a Dirichlet distribution. Using n=10^{6} samples, we observe no violations of the IV inequality. Therefore, the confidence interval for the posterior probability of the cum-laude data satisfying the IV inequalities is \left[1-3.69\times 10^{-6},1\right] . Hence, when allowing for confounding, on arrives at the conclusion that, given the available data and restricting the analysis to only three variables, the fairness of the data-generating mechanism is undecidable. Prior Sensitivity: For both the Berkeley dataset and the Bol dataset, the final confidence interval is dependent on the choice of the prior. We presented the analysis with a flat Dirichlet prior, for both datasets. We find that the lower limit of the confidence interval does not change as we vary the parameter \alpha over the interval \left[10^{-2},10^{5}\right] for a Dirichlet \text{Dir}{\left({\alpha,\alpha,\cdots,\alpha}\right)} prior. Frequentist Test of Wang et al. [2017] : The frequentist test of Wang et al. [2017] converts every IV inequality into a one-sided association test for a 2\times 2 contingency table. Specifically, for fixed d,a , an IV inequality of the form \Pr{\left({D=d,A=a\mid S=1}\right)}+\Pr{\left({D=d,A=1-a\mid S=0}\right)}\leq 1 is transformed into \gamma^{d,a}\leq 0 where \gamma^{d,a}\triangleq\Pr{\left({Q^{d,a}=1\mid S=1}\right)}-\Pr{\left({Q^{d,a}=1\mid S=0}\right)} where Q^{d,a}=\begin{cases}\bm{1}\left[D=d,A=a\right]&\text{if }S=1,\\ 1-\bm{1}\left[D=d,A=1-a\right]&\text{if }S=0.\\ \end{cases} Note that Q^{d,a} and S are binary random variables and \gamma^{d,a}=0 if and only if Q^{d,a}\perp\!\!\!\perp S . Further \gamma^{d,a}\in[-1,1] for all d,a . Since the direction of the one-sided test matters, we check the sign of the difference of conditional probabilities using maximum likelihood (ML) estimates and then conduct a Pearson’s chi-square test for independence. For the Berkeley data, the ML estimates of \gamma^{d,a} were negative (less than -0.6 ) for all d,a , and the independence tests rejected the null hypothesis of independence with p-value 0.0 (i.e., less than the smallest positive number representable using double precision floating point format, i.e., <5\times 10^{-324} ). We take this to be significant evidence that the null hypothesis of \gamma^{d,a}\leq 0 is not rejected. As noted in Wang et al. [2017] , although we test for multiple IV inequalities, the Bonferroni correction is 1/2 and does not scale as the number of IV inequalities.