Persuasive Privacy Joshua J Bon, James Bailie, Judith Rousseau, Christian P Robert Abstract We propose a novel framework for measuring privacy from a Bayesian game-theoretic perspective. This framework enables the creation of new, purpose-driven privacy definitions that are rigorously justified, while also allowing for the assessment of existing privacy guarantees through game theory. We show that pure and probabilistic differential privacy are special cases of our framework, and provide new interpretations of the post-processing inequality in this setting. Further, we demonstrate that privacy guarantees can be established for deterministic algorithms, which are overlooked by current privacy standards. 1 Introduction The scientific and economic value of data grows alongside technological advances. New hardware and software developments enable, but often require, larger and more complex datasets to function effectively. As the importance of input data to these systems becomes increasingly recognized, so too does the loss of privacy for data providers. In this context, data privacy emerges as a critical issue for fields such as statistics and machine learning, as well as for scientific and industrial endeavours that rely on sensitive data. In the last two decades, differential privacy ( 20 , DP,) and its variants ( 14 , see) have become the de facto standard for data privacy. However, DP continues to face conceptual and practical challenges, including difficulties in interpreting and communicating its parameters ( 8 ) ; gaps between its implementation and legal or social notions of privacy ( 49 ) ; and the large, potentially vacuous, privacy loss budgets often seen in its real-world deployments ( 19 ; 48 ) . With these concerns in mind, we develop a framework to (i) generate privacy definitions that are fit for purpose yet rigorously justified; (ii) assess existing privacy guarantees using a Bayesian game-theoretic approach; and (iii) evaluate the privacy of deterministic algorithms. This latter capability is incompatible with DP and its variants, and is particularly important to the problem of assessing the privacy leakage of invariant statistics—deterministic summaries of the sensitive data which are integral to many types of statistical dissemination, including the US Decennial Census ( 1 ; 4 ) . Our new framework for privacy is developed rigorously and systematically using game theory. We make explicit assumptions to establish our framework and discuss their merits and necessity. Our approach allows for a semantics-first understanding of data privacy, where each assumption can be tested against real-world considerations, and provides privacy definitions which are easy to interpret, communicate and tailor because the framework is constructed from an agent-based game. In contrast, semantic interpretations of DP have been constructed post-hoc ( 33 ; 52 ) and can be difficult to understand ( 44 ; 7 ) . We address this limitation whilst facilitating future work to relax or modify our assumptions when they are not well suited to a particular use case. We show that privacy definitions generated by our framework satisfy a composition property and a weaker version of the standard post-processing inequality. We discuss an agent-based interpretation of the standard post-processing inequality, and show how it is trivial to ensure that it holds in practice for our definitions. We also show how our framework encompasses pure \varepsilon -DP and probabilistic (\varepsilon,\delta) -DP. 1.1 Related Work Statistical data privacy ( 50 ) has an extensive literature which stretches back to at least the 1970’s ( 9 ) and includes both modern, formal theories (e.g., DP) as well as traditional statistical disclosure control ( 30 ; 53 , SDC,) . Although the problem of data privacy is typically not expressed in terms of game theory, this field nevertheless shares with our work the concepts of Sender (the agency releasing statistics) and Receiver (the adversary seeking to exploit the published data). The concern in statistical data privacy is to limit Receiver’s ability to learn about individual data points from the published statistics ( 16 ; 21 ) ; we generalise this with a “privacy function” which represents Sender’s loss (in an abstract sense) due to Receiver’s decision. This is similar to 6 which frames privacy in terms of Receiver’s ability to inflict harm through the use of the published data. As with our work, Bayesian formulations of Receiver are common in both DP ( 20 ; 37 ; 36 ; 33 ; 34 , see e.g.,) and traditional SDC ( 22 ; 15 , see e.g.,) . However, we are, to the best of our knowledge, the first to consider Sender as a player in the game, similar to the set-up of Bayesian Persuasion ( 31 ) . Several lines of work connect DP and game theory, ranging from game-theoretic DP parameter settings ( 38 ; 29 ) to the use of DP for mechanism design ( 42 ; 45 ; 46 ) as well as the pricing personal data with DP ( 24 ; 10 ; 47 ; 23 ; 40 ; 39 ) . In contrast, we provide a fully game-theoretic foundation and justification of DP, amongst other privacy definitions. 1.2 Notation We use the terms privacy definition and guarantee interchangeably, and refer to the set of all data release mechanisms satisfying a certain privacy definition as a privacy class . A given privacy definition will generate a privacy class. For example, the set of all mechanisms satisfying \epsilon -DP is a privacy class. The sensitive dataset is denoted by x . We consider \mathsf{X} to be some universe of possible datasets, such that x\in\mathsf{X} . Probability distributions P,Q:\mathcal{X}\rightarrow[0,1] expressing uncertainty about the value of x\in\mathsf{X} are defined on a measurable space (\mathsf{X},\mathcal{X}) , and a family of such distributions is denoted by \mathcal{P} . For example, P=\mathcal{N}(\mu,\Sigma) may express the uncertainty held by a player about the data x\in\mathbb{R}^{n} . Expectations of a function f:\mathsf{X}\rightarrow\mathbb{R} under P will be written as \mathbb{E}_{X\sim P}[f(X)] . We denote Markov kernels by M:(\mathsf{Z},\mathcal{T})\rightarrow[0,1] for input and output measurable spaces 1 1 1 When multiple kernels are required we will also denote measurable spaces by (\mathsf{T}_{k},\mathcal{T}_{k}) for positive integers k . (\mathsf{Z},\mathcal{Z}) and (\mathsf{T},\mathcal{T}) , respectively. That is, M(z,\cdot) is a probability distribution on (\mathsf{T},\mathcal{T}) for a given z\in\mathsf{Z} . We use Markov kernels to represent data release mechanisms. For example, releasing \bar{x} with additive Gaussian noise can be written as M(x,\cdot)=\mathcal{N}(\bar{x},\sigma^{2}) . Deterministic mechanisms can be expressed as M(x,\cdot)=\delta_{f(x)} for a function f , where \delta_{t} is a degenerate probability distribution with point mass at t . The identity kernel will be denoted by \mathrm{Id} . The probability of an event E\in\mathcal{T} under a mechanism M(x,\cdot) is denoted by \mathbb{P}_{x}(E) , conditional on x\in\mathsf{X} . The composition (resp. tensor product) of two Markov kernels, say M and K , is denoted by MK (resp. M\otimes K ). For M:(\mathsf{Z},\mathcal{T}_{1})\rightarrow[0,1] and K:(\mathsf{Z}\times\mathsf{T}_{1},\mathcal{T}_{2})\rightarrow[0,1] , the composition and tensor product are defined as \displaystyle MK(z,\cdot) \displaystyle=\int M(z,\mathrm{d}t_{1})K((z,t_{1}),\cdot)~\text{and} \displaystyle(M\otimes K)(z,\mathrm{d}(t_{1},t_{2})) \displaystyle=M(z,\mathrm{d}t_{1})K((z,t_{1}),\mathrm{d}t_{2}), respectively, for fixed z\in\mathsf{Z} . We say that a Markov kernel M , defined by M((x,y),\cdot) for all (x,y)\in\mathsf{X}\times\mathsf{Y} , is independent of x if M((x_{1},y),\cdot)=M((x_{2},y),\cdot) for all x_{1},x_{2}\in\mathsf{X} . We denote regular conditional probability distributions ( 17 , see e.g.,) using conditioning notation. For example, if P is a prior distribution for x then P(\;\cdot\mid T) denotes the posterior distribution, after observing T\sim M(x,\cdot) . We use of proper scoring rules ( 25 , for an overview, see) . Let S:(\mathcal{P},\mathsf{X})\rightarrow\mathbb{R}\cup\{-\infty,\infty\} be a (negatively-orientated) scoring rule. Scoring rules measure the success of predictive distribution P to estimate the truth x by S(P,x) , where S(P,x)\rho(d_{2},x) . Privacy is positively-orientated (for Sender) and hence higher values of \rho(d,x) indicate higher privacy under value x , for an adversarial decision d . Sender may assess privacy with one or more privacy functions, an extension we consider in Section 3 . We focus on the case of one privacy function for ease of exposition in this section. 1 . Let \mathcal{D}=\{[a,b]:a,b\in\mathbb{R},a\leq b\} and \mathsf{X}=\mathbb{R} . Given s>0 , the interval privacy function is defined as \rho(d,x)=\begin{cases}0&\text{if}~x\in d~\text{and}~|d|\leq s,\\ 1&\text{otherwise}.\end{cases} In this example, privacy is a binary outcome. Privacy is achieved by Sender if the true data x is not contained in Receiver’s decision, an interval, or if the interval is sufficiently large. 2 . Let \mathcal{D} be the set of probability density functions on \mathsf{X}=\mathbb{R} with respect to some given measure \mu . For d\in\mathcal{D} and x\in\mathsf{X} , the negative log-probability privacy function is \rho(d,x)=-\log d(x) . In this example, Receiver’s decision is a density function. If this density function has a relatively high value at the true data x , then Sender has less privacy. Sender has no explicit control over Receiver’s decision d , and Receiver’s agenda is unknown to Sender. In Section 2.2 , we use assumptions to derive Receiver’s optimal decision, so that Sender can assess privacy with such a response from Receiver. Before this, we comment on the need for transparent privacy guarantees. 2.1.2 Transparent guarantees Transparency is important in the context of privacy as Sender is typically required to convince external parties (e.g., regulators or the public) that the mechanism in question satisfies a privacy guarantee. As such, Sender will share the mechanism M and their chosen privacy definition so that the privacy status of M can be verified. Sharing the privacy definition is equivalent to sharing \mathfrak{C} , the privacy class generated by the definition (for which M\in\mathfrak{C} ). We describe adherence to the transparency principle by the following assumption. 1 . Sender shares the mechanism M and privacy class \mathfrak{C} , for which M\in\mathfrak{C} , with Receiver. Further, the definitions of M and \mathfrak{C} do not depend on the data. The additional caveat that the data does not determine M and \mathfrak{C} ensures no information about x is leaked to Receiver when Sender shares M and \mathfrak{C} . For example, the constant mechanism that releases the data, M_{x}(z,\cdot)=\delta_{x} , would completely reveal the data, but may satisfy privacy guarantees without Assumption 1 ( 3 ) . Note that the latter condition also dictates that the privacy definition being developed here will not depend on the true value of the data. This is discussed further in the next section. 2.2 Receiver’s Decision To derive Receiver’s optimal decision, or best response, we make the following assumption. 2 . Receiver makes Bayesian decisions, i.e., they are Bayes rational ( 2 ) . Receiver’s uncertainty about the sensitive data is represented by their belief , a distribution over x . In particular, Receiver holds a prior on the values of the data, a data-prior Q\in\mathcal{P} , and knows the mechanism M generating the output by Assumption 1 . Receiver updates their belief after observing the realised output T from M(x,\cdot) . With this information Receiver constructs their data-posterior , Q_{T}=Q(\;\cdot\mid T), (1) the Bayes update after observing T\sim M(x,\cdot) . Note that the effect of the chosen mechanism M is implicit in Q_{T} . For instance, if, for all x\in\mathsf{X} , the probabilities M(x,\cdot) have a common dominating measure \mu and densities m(x,\cdot) with respect to \mu , then Q_{T}(\mathrm{d}x)\propto Q(\mathrm{d}x)m(x,T) where m(\cdot,T) is the likelihood function conditional on T . Before continuing to Receiver’s optimal decision, we make the following remark about transparency. . If the privacy class was dependent on the data then knowledge of such a class, say \mathfrak{C}_{x} , would restrict the support of the data-posterior to \mathsf{X}^{\prime}=\{x\in\mathsf{X}:M\in\mathfrak{C}_{x}\} . Therefore, excluding this possibility by Assumption 1 ensures that the data-posterior is specified as in ( 1 ), and is not restricted 2 2 2 Conditioning the data-posterior on some additional information is not an inherent problem. Rather, \mathsf{X}^{\prime}=\{x\in\mathsf{X}:M\in\mathfrak{C}_{x}\} depends on the privacy definition generating \mathfrak{C}_{x} . The same privacy definition we will come to define using the data-posterior, now conditioned on \mathsf{X}^{\prime} depending on \mathfrak{C}_{x} . Allowing this possibility would result in a self-referential definition of privacy, hence the caveat in Assumption 1 . to the set \mathsf{X}^{\prime} . If Receiver has a loss function \ell:(\mathcal{D},\mathsf{X})\rightarrow\mathbb{R} , then by Assumption 2 Receiver makes Bayes decisions according to \ell and their belief P\in\mathcal{P} about the data. Their optimal decision under P and \ell is therefore d^{P}_{\ell}\in\arginf_{d\in\mathcal{D}}\mathbb{E}_{X\sim P}[\ell(d,X)] , noting that the infimum may not be unique. Specifically, before observing the output T , Receiver’s belief is their data-prior Q , whilst afterward it is their data-posterior Q_{T} . Interestingly, the worst-case data-averaged loss function (from Sender’s point of view) for privacy is \ell=\rho , as shown in the proposition below. 1 . If \ell=\rho , then Sender attains the worst-case data-averaged privacy value. That is, \mathbb{E}_{X\sim P}[\rho(d^{P}_{\rho},X)]\leq\mathbb{E}_{X\sim P}[\rho(d^{P}_{\ell},X)] for any \ell:(\mathcal{D},\mathsf{X})\rightarrow\mathbb{R} . All proofs are deferred to Appendix C . In light of Proposition 1 , we now restrict our attention to the case where Receiver has loss function \ell=\rho , where \rho is Sender’s privacy function. 3 . Receiver’s loss function satisfies \ell(d,x)=\rho(d,x) for all d\in\mathcal{D} and x\in\mathsf{X} . For simplicity’s sake, we denote Receiver’s decision under Assumption 3 as d^{P}\in\arginf_{d\in\mathcal{D}}\mathbb{E}_{X\sim P}[\rho(d,X)]. (2) 2.3 Sender’s Attained Privacy Sender’s attained privacy value under Receiver’s optimal decision is \rho(d^{P},x) , where P is Receiver’s belief. After Receiver has observed the output, Sender’s privacy value is \rho(d^{Q_{T}},x) , which will depend on the mechanism chosen. As such, Sender wishes to choose a mechanism which persuades Receiver to make decisions which have a limited impact on privacy. The attained privacy value also has an interpretation as a proper scoring rule. 2 . Let S:(\mathcal{P},\mathsf{X})\rightarrow\mathbb{R} be defined as S(P,x)=\rho(d^{P},x), then S is a negatively-orientated proper scoring rule. Proposition 2 is proved in 27 under mild technical conditions ( 11 ; 13 , see also,) . We call S a privacy score , which measures how well Receiver’s belief (the distribution P ) predicts the true value of the data x . Privacy scores can be derived from the privacy functions stated in Examples 1 and 2 as follows. 3 continues=ex:interval. The interval privacy score with length s>0 is S(P,x)=\begin{cases}0&\text{if}~x\in[m^{P}-\frac{s}{2},m^{P}+\frac{s}{2}]\\ 1&\text{otherwise}.\end{cases} where m^{P}=\arg\max_{m\in\mathbb{R}}P([m-\frac{s}{2},m+\frac{s}{2}]) , assuming the maximiser is unique for all P\in\mathcal{P} . When P is a symmetric unimodal distribution, m^{P} will be its median. In this example, Receiver’s optimal decision is the interval of length s with maximum probability under P . If this interval does not contain x then Sender has retained privacy. 4 continues=ex:neglogprobprivfunc. The negative log-probability privacy score is S(P,x)=-\log p(x), where p is the density of P with respect to a given measure \mu , in the case where \mathcal{P} consists of distributions which are absolutely continuous with respect to \mu . This example leads to the well-known log-probability score (which is related to the Kullback–Leibler divergence). Appendix A contains a technical note defining the negative log-probability score more generally, required for Section 4 . Proper scoring rules can be constructed from loss functions under Bayes acts (decisions) with mild conditions ( 27 ) . As such, working with (proper) privacy scores is equivalent to the privacy-function approach discussed thus far. That is, the specification of \rho and \mathcal{D} generates a privacy score S , whilst a privacy score S implies the existence of an equivalent 3 3 3 We prove the latter statement in Appendix B for completeness. Proof of the former statement is in 27 with further discussion in 5 . \rho and \mathcal{D} . This allows us to measure privacy using either privacy functions or proper scoring rules with our framework. Proper scoring rules 4 4 4 It is also the case that a scoring rule lacking propriety (i.e., not proper) can often be adjusted to gain propriety ( 5 ) . are convenient and natural to consider, so we will focus on these for the remainder of the paper. 2.4 Sender’s Decision In this section, we specify how Sender distinguishes between mechanisms, based on Receiver’s optimal decision. Our forthcoming assumptions govern Sender’s approach to assessing privacy, but we note that the framework established thus far is also amenable to contexts which may necessitate an alternative approach. We assume Sender assesses mechanisms based on their relative effect on privacy. That is, Sender chooses a mechanism based on its relative privacy score , \Delta(Q,T,x)=S(Q,x)-S(Q_{T},x). The relative privacy score is negatively-orientated for Sender. That is, lower values of \Delta(Q,T,x) represent greater privacy. Typically, S(Q,x)>S(Q_{T},x) as privacy decreases after information is released, and \Delta(Q,T,x) will be positive, but given stochasticity in T , the relative privacy score can take negative values. An alternative to a relative assessment is for Sender to assess the absolute privacy using S(Q_{T},x) . However, this would not capture the change in Receiver’s belief, which would lead to very strict assessments of privacy. As an extreme example, if Receiver already has full knowledge of the dataset, i.e., Q=\delta_{x} , then every mechanism would have equal absolute privacy. The relative privacy score is stochastic since it depends on T\sim M(x,\cdot) . But Sender must assess privacy before T is generated. The stochasticity can be accounted for in a number of ways, including by expectation or tail probability. We choose the latter to favour robustness in our definition of privacy. 4 . For a given data-prior Q and dataset x , Sender considers a mechanism M private only if \mathbb{P}_{x}\left[\Delta(Q,T,x)\leq\kappa\right]\geq 1-\delta, (3) for some maximum acceptable privacy loss \kappa\geq 0 and small probability of failure 0\leq\delta\ll 1 . Assumption 4 dictates that Sender wishes to ensure privacy for all events except (possibly) some with low probability. 5 5 5 Considering the expectation, rather than tail probabilities, may be useful in other contexts. From the game-theoretic perspective, ( 3 ) encodes a binary value function (relative privacy under Receiver’s optimal decision) for Sender. Further discussion of the binary value function is given in Section 2.5 . So far, we have conditioned on the existence of a known data-prior Q . In most cases, it will be difficult for Sender to know Receiver’s data-prior, even approximately. For a robust definition of privacy we consider the worst case over a reasonable class of priors \mathcal{Q}_{x}\subset\mathcal{P} that Receiver may hold. This class may depend on the data value x , since this allows us to construct classes where bounds on Receiver’s adversarial strength are constant as x varies. We will drop the subscript when \mathcal{Q}_{x} does not depend on x . Sender’s consideration of a set of data-priors is formalised as follows. 5 . Sender considers M private if ( 3 ) holds uniformly for all data-priors Q\in\mathcal{Q}_{x} and datasets x\in\mathsf{X} . That ( 3 ) holds uniformly over all datasets x\in\mathsf{X} is sufficient to ensure that the privacy class (to be defined) does not depend on the data, as required by Assumption 1 . Assumptions 4 and 5 imply that Sender will assess the privacy of a mechanism M by validating \inf_{x\in\mathsf{X}}\inf_{Q\in\mathcal{Q}_{x}}\mathbb{P}_{x}\left[\Delta(Q,T,x)\leq\kappa\right]\geq 1-\delta. (4) In particular, mechanisms satisfying ( 4 ) are robust to (i) the value of the dataset, (ii) the data-prior held by Receiver, and (iii) the (possibly) stochastic outcome of the mechanism. 2.5 Game Interpretations To provide a complete game-theoretic interpretation of our framework, we can take the following view. Suppose a third player, Nature, has a dataset x\in\mathsf{X} , where x is unknown to Sender and Receiver. Nature will reveal the dataset to Sender, who will then share information about x to Receiver through a mechanism. Before this, Sender publicly commits to using a mechanism M(x,\cdot) to share information with Receiver. Once x is revealed, Sender shares the output of the mechanism with Receiver. In this scenario, if Sender wishes to be robust to the data value, data-prior, and randomness of M , they can use ( 4 ) to assess the mechanism. As such, the inclusion of Nature provides a complete game-theoretic interpretation for protecting all x\in\mathsf{X} (in Assumption 5 ), whilst Assumption 4 and consideration of the worst Q\in\mathcal{Q}_{x} (in Assumption 5 ) can be attributed to Sender’s choice to be robust. In this game, if Sender chooses the mechanism from some set \mathcal{M} , then their objective function will be v(M)=1\{M\in\mathfrak{C}\}, (5) where \mathfrak{C} is the privacy class defined by ( 4 ). Therefore, private mechanisms attain unit utility and non-private mechanisms attain zero utility. Typically, ( 5 ) will not have a unique maximum, and the resulting set of ‘‘optimal’’ private mechanisms will be indistinguishable. 6 6 6 A natural tie-breaking strategy is to introduce a secondary utility function which selects the “best” private mechanism, in some manner. For example, a function measuring statistical efficiency could be used to select a mechanism from \mathfrak{C} . This characterisation aligns with contemporary privacy definitions, which typically assess if privacy is attained by a given mechanism or not. In contrast to this game-theoretic interpretation, we make the following comments on real-world assessment of data privacy. Firstly, when Sender is the custodian of the data, we believe that the transparency justification for protecting all x\in\mathsf{X} is more compelling than the use of Nature as a third player. In other words, protecting all x\in\mathsf{X} is better motivated as a sufficient condition for Assumption 1 . Secondly, the privacy of a mechanism is typically assessed individually, rather than as a set or utility function, i.e., comparing ( 4 ) to ( 5 ). Indeed, this is how most, if not all, privacy guarantees are introduced and discussed. Hence, without loss of generality, we focus on privacy assessment of one mechanism for the remainder of the paper. 3 Persuasive Privacy We extend our definition of privacy to include multiple privacy scores (equivalently privacy functions) if required. We denote the (non-empty) set of privacy scores under consideration by \mathcal{S} . The class of Receiver data-priors is \mathcal{Q}_{x}\subset\mathcal{P} . The privacy parameters are \kappa>0 which is the maximal allowable change in privacy, and 0\leq\delta\ll 1 , representing a small probability of failing to meet this maximal change. 3 Persuasive Privacy. A mechanism M is said to be (\mathcal{S},\mathcal{Q}_{x},\kappa,\delta) -PP if \inf_{S\in\mathcal{S}}\inf_{x\in\mathsf{X}}\inf_{Q\in\mathcal{Q}_{x}}\mathbb{P}_{x}\left[\Delta_{S}(Q,T,x)\leq\kappa\right]\geq 1-\delta, (6) where \Delta_{S}(Q,T,x)=S(Q,x)-S(Q_{T},x) . When \mathcal{S} is a singleton, say \mathcal{S}=\{S\} , we use (S,\mathcal{Q}_{x},\kappa,\delta) as shorthand for (\{S\},\mathcal{Q}_{x},\kappa,\delta) . Though the dependence is not explicitly stated, the choice of mechanism defines the probability \mathbb{P}_{x}(\cdot) and affects the construction of the data-posterior Q_{T} in \Delta_{S}(Q,T,x) . We now consider some properties of (\mathcal{S},\mathcal{Q}_{x},\kappa,\delta) -PP mechanisms. 3.1 Composition We can attain a composition rule for persuasive privacy when the family of posterior distributions \mathcal{Q}_{x} is closed under Bayes updating by the mechanisms considered. First we define this conjugacy condition, before stating the composition property. 4 Conjugacy. A family of distributions \mathcal{Q}_{x} is conjugate to a mechanism M:(\mathsf{X},\mathcal{T})\rightarrow[0,1] if for every Q\in\mathcal{Q}_{x} the posterior distribution Q(\;\cdot\mid M,T)\in\mathcal{Q}_{x} almost surely where T\sim M(z,\cdot) , for all z\in\mathsf{Z} . A composition property for a guarantee specifies the privacy of a composed mechanism, M_{1}\otimes M_{2} , where the second mechanism M_{2}((x,T),\cdot) is possibly dependent on the output of the first T\sim M_{1}(x,\cdot) . We reference measurable spaces (\mathsf{T}_{1},\mathcal{T}_{1}) and (\mathsf{T}_{2},\mathcal{T}_{2}) where \mathsf{T}_{k} is the output space of M_{k} . 3 . Let M_{k}:(\mathsf{Y}_{k},\mathcal{T}_{k})\rightarrow[0,1] for k\in\{1,2\} where \mathsf{Y}_{1}=\mathsf{X} and \mathsf{Y}_{2}=\mathsf{X}\times\mathsf{T}_{1} . Assume M_{1} is (\mathcal{S},\mathcal{Q}_{x},\kappa_{1},\delta_{1}) -PP, and M_{2}((\cdot,t),\cdot) is (\mathcal{S},\mathcal{Q}_{x},\kappa_{2},\delta_{2}) -PP for all t\in\mathsf{T}_{1} . If \mathcal{Q}_{x} is conjugate to M_{1} and M_{2} almost surely, then M_{1}\otimes M_{2} is (\mathcal{S},\mathcal{Q}_{x},\kappa_{1}+\kappa_{2},\delta_{1}+\delta_{2}) -PP. 3.2 Post-Processing In this section we distinguish between two types of post-processing properties that are desirable privacy guarantees. To assist with the exposition, we use privacy classes generated by privacy definitions. In particular, let \mathrm{D} be a specific persuasive privacy guarantee from Definition 3 and denote the privacy class generated by \mathrm{D} as \mathfrak{C}(\mathrm{D}) . Using this description we can describe two types of post-processing properties. We will use a Markov kernel K:(\mathsf{T}_{1},\mathcal{T}_{2})\rightarrow[0,1] . Unlike the case of composition, the kernel K is independent of the data x . 5 Receiver Post-Processing. A guarantee \mathrm{D} satisfies the receiver post-processing property if M\in\mathfrak{C}(\mathrm{D}) implies that M\otimes K\in\mathfrak{C}(\mathrm{D}) for all Markov kernels K independent of the data x . The receiver post-processing property dictates that after observing the mechanism output, Receiver cannot gain additional information about the data by post-processing the output. That is, the privacy guarantee remains no matter what transformation is applied to the output by Receiver. The above interpretation of this property is a cornerstone justification for the adversarial robustness of differential privacy. Here, we establish it for persuasive privacy. 4 . All persuasive privacy guarantees satisfy the receiver post-processing property. Next we consider post-processing by Sender. This following alternative is what is known in the literature as the “post-processing inequality”. 6 Sender Post-Processing. A guarantee \mathrm{D} satisfies the sender post-processing property if M\in\mathfrak{C}(\mathrm{D}) implies that MK\in\mathfrak{C}(\mathrm{D}) for all Markov kernels K independent of the data x . The mechanism MK is often referred to as “chaining” M and K ( 28 ) . The sender post-processing property dictates that further (random) transformations of a private output by Sender preserves the privacy guarantee, when only the final transformed output is shared with Receiver. This property is useful as a tool to establish privacy for complex mechanisms by transformations of simpler mechanisms for which a guarantee can be established. This is the typical use of the “post-processing inequality” in the literature. We can also state that receiver post-processing is a special case of sender post-processing by taking K=\mathrm{Id}\otimes K^{\prime} and observing that MK=M\otimes K^{\prime} , for a Markov kernel K^{\prime} independent of the data. As such, sender post-processing is a stronger requirement than receiver post-processing. Whilst persuasive privacy satisfies receiver post-processing, it does not satisfy the sender post-processing property. 5 . There exists a persuasive privacy guarantee that does not satisfy the sender post-processing property. Despite this negative result, it is still trivial for Sender to release output from MK with the same guarantee as M . Specifically, releasing output from M\otimes K , instead of just from the marginal MK , will satisfy the receiver post-processing property but on Sender’s side. From the view of Bayesian Persuasion, this result indicates that Sender can control the privacy loss by releasing output from M in addition to MK (jointly), when the influence on decisions exerted by output from MK is worse than M (or not established). 4 Differential Privacy We can interpret some variants of differential privacy in the persuasive privacy framework. First we state the definition for probabilistic differential privacy ( 41 ; 26 , PDP,) . To define PDP, we denote the set of neighbours for some x\in\mathsf{X} by \mathfrak{N}_{x}=\{x^{\prime}\in\mathsf{X}:x^{\prime}\sim x\} for a binary neighbour relation 7 7 7 We also use “ \sim ” to relate random variables to their distribution, but the intended meaning will be clear from context. “ \sim ”, and the neighbourhood by \mathfrak{N}=\{(x,x^{\prime})\in\mathsf{X}^{2}:x^{\prime}\in\mathfrak{N}_{x}\} . For example, \mathfrak{N}=\{(x,x^{\prime})\in\mathsf{X}^{2}:H(x,x^{\prime})\leq 1\} where H is the Hamming distance, is typical. We denote the neighbourhood with shorthand x\sim x^{\prime} for simplicity. 7 Probabilistic Differential Privacy. A mechanism M is said to be (\epsilon,\delta) -PDP if \inf_{x\sim x^{\prime}}\mathbb{P}_{x}\left[m(T\mid x)\leq\exp\{\epsilon\}m(T\mid x^{\prime})\right]\geq 1-\delta, where T\sim M(x,\cdot) and m(x,\cdot) is the probability density (mass) function of M conditional on x . Having defined PDP, we now prove an equivalence to PP mechanisms. Let L be the negative log-probability score for discrete distributions 8 8 8 The negative log-probability score for discrete distributions is formalised in Appendix A . , and consider a class of neighbouring alternative hypothesis data-priors \mathcal{H}=\{Q\in\mathcal{P}_{2}:\exists(z,z^{\prime})\in\mathfrak{N},Q(\{z,z^{\prime}\})=1\} where \mathcal{P}_{2} is the class of two-component discrete probability distributions. Note that \mathcal{H} does not depend on the value of x . 6 . A mechanism M is (\epsilon,\delta) -PDP if and only if M is (L,\mathcal{H},\epsilon,\delta) -PP. Since pure DP is a special case of PDP, when \delta=0 we recover \epsilon -DP. The proof reveals that the infimum in the definition of persuasive privacy occurs in the limit of Receiver’s prior probability on the true data Q(\{x\})\rightarrow 0 . We can use this to interpret probabilistic differential privacy as protecting against a worst case where Receiver has vanishingly small probability on the truth. That is, under the negative log-probability score and class of neighbouring alternative hypothesis priors, the gain in information about x is largest when Receiver has close to zero probability on this outcome. Proposition 6 can also be proved for a smaller class of data-priors \mathcal{H}_{x}=\{Q\in\mathcal{P}_{2}:Q(\{x\})=1-Q(\{x^{\prime}\})>0,x^{\prime}\in\mathfrak{N}_{x}\} , limited to the priors which has positive probability on the truth. This indicates that using the larger class gives no additional privacy guarantee. Essentially, we ignore data-priors that have no mass on the truth as there will be no change in the privacy score in this case. Considering the properties discussed in Section 3.2 , it is well known that PDP does not satisfy the post-processing inequality ( 35 ; 43 ) . However, our discussion reveals that this is not a drawback for the privacy properties of PDP, rather a restriction of the tools that can be used to establish a PDP guarantee. Furthermore, it is trivial to gain a PDP guarantee for a post-processed mechanism using Proposition 4 . One can simply augment the output of the transformed mechanism with the output of the original mechanism. Establishing PDP as a special case of our framework contributes new semantics to the understanding of differential privacy. Sender wishes to limit the relative privacy score, the difference of negative log-probability scores under the data-posterior and data-prior, and assess the worst case under the class of neighbouring alternative hypothesis data-priors. Our explicit construction illuminates potential areas of weakness in the differential privacy setup. 5 Privacy for Deterministic Mechanisms In this section we provide an illustrative and important example where is it possible to assess the persuasive privacy of a deterministic function of the data. This contrasts differential privacy and its variants, where non-vacuous guarantees cannot be constructed for deterministic mechanisms. The simplest mechanism considered in data privacy is the empirical average \bar{x}=\frac{1}{n}\sum_{i=1}^{n}x_{i} for n\geq 2 . Adding noise to the average can yield a differentially private mechanism. Assuming \mathsf{X} is bounded, Laplace or Gaussian noise yield a pure or approximate DP guarantee, respectively ( 18 ) . However, in traditional SDC it is often assumed that for large n no noise is actually required for the average to be private ( 51 , assuming that differencing attacks and group disclosures are not possible, see e.g.,) , whilst for DP, the scale of the additive noise of a mean goes to zero as n\rightarrow\infty . We construct a persuasive privacy guarantee to formalise the intuition that releasing the empirical average of the data with no noise is private, under reasonable assumptions. To begin we set out the constituent elements of this guarantee. For simplicity we assume \mathsf{X}=\mathbb{R}^{n} . The Dawid–Sebastiani Score ( 12 , DSS,) is a scoring rule depending only on the first two moments of a distribution P . The DSS is a proper scoring rule when the variance of P is finite for all P\in\mathcal{P} , and can be seen as a special case of the negative log-probability score for Gaussian distributions. For our purposes, we define a marginal version of the DSS. 8 Marginal Dawid–Sebastiani Score. If Q has support on \mathsf{X}\subset\mathbb{R}^{n} , the marginal DSS is defined as D_{i}(Q,x)=\log\sigma^{2}_{i}(Q)+\frac{[x_{i}-\mu_{i}(Q)]^{2}}{\sigma^{2}_{i}(Q)}, where \mu_{i}(Q) and \sigma^{2}_{i}(Q) are the marginal mean and variance, respectively, for the i th dimension of Q . The marginal DSS is a proper scoring rule, recovering the i th marginal mean and variance. That is, if D_{i}(Q,P)=D_{i}(Q,Q) then \mu_{i}(Q)=\mu_{i}(P) and \sigma^{2}_{i}(Q)=\sigma^{2}_{i}(P) . From the marginal DSS, we construct a set of scoring rules \mathcal{I}=\{D_{i}\}_{i=1}^{n} to measure the privacy of each marginal in the dataset. One interpretation of this choice is that there are n individuals in the dataset each with x_{i}\in\mathbb{R} , and we wish to protect the worst-case outcome for all individuals. Next we define the class of data-priors held by Receiver. Let \mathcal{N}(\mu,\Sigma) denote a multivariate Gaussian distribution on \mathbb{R}^{n} for \mu\in\mathbb{R}^{n} and positive-definite \Sigma\in\mathbb{R}^{n\times n} . Let \Sigma=\sigma^{\top}\Phi\sigma where \sigma is the vector of marginal standard deviations, and \Phi is the correlation matrix of \Sigma . Finally, let c_{\Phi}\in[1,\infty) denote the condition number of \Phi and define the averages \bar{\mu}=\frac{1}{n}\sum_{i=1}^{n}\mu_{i} and \overline{\Sigma}=\frac{1}{n^{2}}\sum_{i=1}^{n}\sum_{j=1}^{n}[\Sigma]_{ij} . Note that \bar{\mu} and \overline{\Sigma} are the mean and variance of \overline{X} , respectively, where X\sim\mathcal{N}(\mu,\Sigma) . Define the class of data-priors \mathcal{G}_{x}^{r} as the set \left\{\mathcal{N}(\mu,\Sigma):\frac{(\bar{x}-\bar{\mu})^{2}}{\overline{\Sigma}}\leq r_{1},c_{\Phi}\leq r_{2}\left(1-\frac{\sigma_{i}^{2}}{\|\sigma\|_{2}^{2}}\right)~\forall i\right\}, for r_{1}>0 and r_{2}>1 . The first condition defining \mathcal{G}_{x}^{r} states that Receiver’s prior guess for \bar{x} cannot be too poor, relative to the variance of the average under Receiver’s prior. In contrast, the second condition limits how strong Receiver’s prior is, by controlling degeneracy in the data-prior. 7 . The average mechanism M(x,\cdot)=\delta_{\bar{x}} satisfies (\mathcal{I},\mathcal{G}_{x}^{r},\kappa^{r},0) -PP with \kappa^{r}=r_{1}+\log r_{2} . Proposition 7 establishes a persuasive privacy guarantee for a deterministic mechanism. The parameters (\kappa^{r},0) of the privacy guarantee are determined by the relative strength of Receiver, which is specified by the values r_{1} and r_{2} defining the class of data-priors \mathcal{G}_{x}^{r} . The data-posterior that is considered in this example is a degenerate Gaussian distribution constrained to the manifold determined by \bar{x} . In some sense, we avoid the degeneracy and complexity of the multivariate data-posterior by using the marginal DSS to assess privacy. This is an interesting contrast to the persuasive privacy interpretation of PDP, where the complexity of the multivariate distribution is handled by using a restrictive class of discrete data-priors. In light of Proposition 7 , the data-prior restrictions in \mathcal{G}_{x}^{r} have the following interpretation. Firstly, Receiver gains too much information when they hold a data-prior where the average of the means, \bar{\mu} , is far away from the truth with strong conviction (i.e., a small \overline{\Sigma} value). Secondly, no single marginal variance can account for too much of the total marginal variance, relative to the condition number of the correlation matrix, which measures the overall strength of correlations. Otherwise, revealing \bar{x} , which constrains the prior to an (n-1) -dimensional manifold, will indirectly reveal too much about the i th component (assuming r_{1} is small). This can occur when \sigma_{i}\gg\sigma_{j} for all j\neq i , since knowledge of the manifold and the remaining components ( j\neq i ) with high certainty determines x_{i} . Clearly, (\mathcal{I},\mathcal{G}_{x}^{r},\kappa^{r},0) -PP is not robust to all Receivers with Gaussian priors, as we restrict the class of priors considered. However, it may be possible for robust versions of the mean (or indeed other deterministic mechanisms) to have such a property. Considering the interactions between stochasticity and robustness under our general framework is left for future work. 6 Conclusion We have considered a class of robust asymmetric Stackelberg games to construct a framework for defining privacy guarantees. In this game, Sender can be thought to model Receiver as a Bayesian agent, whilst making decisions that are robust to stochasticity and the data-prior Receiver may hold. This setup has several benefits, including the possibility of generating fit-for-purpose privacy guarantees using privacy functions and scores. We demonstrated how to recover PDP using our framework, and elucidated how privacy guarantees can be established for deterministic mechanisms. The ability to construct persuasive privacy guarantees for deterministic mechanisms suggests the potential for new privacy definitions with improved privacy-utility trade-offs. However, care must be taken to ensure that guarantees instantiated from the persuasive privacy framework are adequate for their specific context. For example, the privacy guarantee for the empirical average in Section 5 achieves perfect statistical efficiency, at the cost of a restricted class of data-priors and the specific use of the marginal DSS as a privacy score. We did not consider statistical utility explicitly in our work, but general methods for determining the utility-optimal mechanisms in a given privacy class is an interesting future research direction. Further, work applying our framework to specific contexts whilst reasoning about appropriate data-prior classes and privacy scores is ongoing. Impact Statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here. Appendix A Negative log-probability score for discrete distributions To accommodate the set of all discrete distributions (which does not have a common dominating measure) we define the negative log-probability score for discrete distributions as follows. Let \mathsf{X}=\mathbb{R}^{n} and \mathcal{X} be the Borel \sigma -algebra for \mathsf{X} . Consider probability measures of the form Q=\sum_{z\in\mathsf{Z}}w_{z}\delta_{z} for some countable set \mathsf{Z}\subset\mathsf{X} , and where w_{z}>0 for all z\in\mathsf{Z} with \sum_{z\in\mathsf{Z}}w_{z}=1 . Denote the set of probability measures of this form by \mathcal{P}_{2} , is the class of two-component discrete probability distributions, and note that the singleton \{x\}\in\mathcal{X} for all x\in\mathsf{X} . 9 . Let the discrete negative log-probability score L:(\mathcal{P}_{2},\mathsf{X})\rightarrow\mathbb{R}\cup\{-\infty,\infty\} be defined as L(Q,x)=\log Q(\{x\}), with convention that \log 0=-\infty . Hence, for some Q=\sum_{z\in\mathsf{Z}}w_{z}\delta_{z} , the score is L(Q,z)=-\log w_{z} for z\in\mathsf{Z} and L(Q,z)=\infty for z\notin\mathsf{Z} . The scoring rule L has the following propriety property. 8 . The scoring rule L is strictly proper relative to the class of distributions \mathcal{P}_{2} . Proof. Consider P,Q\in\mathcal{P}_{2} . We have P=\sum_{y\in\mathsf{Y}}v_{y}\delta_{y} and Q=\sum_{z\in\mathsf{Z}}w_{z}\delta_{z} for some \mathsf{Y},\mathsf{Z}\subset\mathsf{X} and positive probabilities v_{y} and w_{z} defined over y\in\mathsf{Y} and z\in\mathsf{Z} , respectively. The expected score L(P,Q) is L(P,Q)=\begin{cases}-\sum_{z\in\mathsf{Z}}w_{z}\log v_{z}&\text{if}~\mathsf{Z}\subset\mathsf{Y}\\ \infty&\text{otherwise,}\end{cases} whilst L(Q,Q)=-\sum_{z\in\mathsf{Z}}w_{z}\log w_{z} . Case 1. If \mathsf{Z}\not\subset\mathsf{Y} then L(Q,Q)0 and ( 8 ) is equivalent to \inf_{x\sim x^{\prime}}\inf_{w\in(0,1)}\mathbb{P}_{x}\left[\log\left(\frac{m(T\mid x)}{m(T\mid x)w+m(T\mid x^{\prime})(1-w)}\right)\leq\epsilon\right]\geq 1-\delta. (9) Case 1. Assume that for all x^{\prime}\sim x , almost surely under \mathbb{P}_{x} , m(T\mid x^{\prime})>0 . Then the inequality in ( 9 ) simplifies to \frac{m(T\mid x^{\prime})}{m(T\mid x)}\geq\frac{\exp\{-\epsilon\}-w}{1-w}, noting that m(T|x)>0 almost surely under \mathbb{P}_{x} . We can see that \mathbb{P}_{x}\left[\frac{m(T\mid x^{\prime})}{m(T\mid x)}\geq\frac{\exp\{-\epsilon\}-w}{1-w}\right]=1 when \exp\{-\epsilon\}\leq w<1 , and is an increasing function in w for 00 (by assumption of positive definiteness of \Sigma ), we can state that 1-\frac{v_{i}^{2}}{v}=\frac{n^{2}(v-v_{i}^{2})}{n^{2}v}=\frac{\gamma_{i}^{\top}\Phi\gamma_{i}}{\sigma^{\top}\Phi\sigma}\geq\frac{\lambda_{n}}{\lambda_{1}}\frac{\sum_{j\neq i}\sigma_{j}^{2}}{\|\sigma\|_{2}^{2}}\geq\frac{\lambda_{n}}{\lambda_{1}}\left(1-\frac{\sigma_{i}^{2}}{\|\sigma\|_{2}^{2}}\right). (11) Considering this data-posterior with the marginal DSS, the i th relative privacy score is \Delta_{i}(Q,\bar{x},x)=\frac{(\mu_{i}-x_{i})^{2}}{\sigma_{i}^{2}}-\frac{(\mu_{i}(Q_{\bar{x}})-x_{i})^{2}}{\sigma_{i}^{2}(Q_{\bar{x}})}-\log\left(1-\frac{v_{i}^{2}}{v}\right). This can be expressed as \displaystyle\Delta_{i}(Q,\bar{x},x) \displaystyle=\frac{\left(1-\frac{v_{i}^{2}}{v}\right)(\mu_{i}-x_{i})^{2}-(\sigma_{i}\frac{v_{i}}{v}(\bar{x}-\bar{\mu})+\mu_{i}-x_{i})^{2}}{\left(1-\frac{v_{i}^{2}}{v}\right)\sigma_{i}^{2}}-\log\left(1-\frac{v_{i}^{2}}{v}\right) \displaystyle=\frac{-\frac{v_{i}^{2}}{v}(\mu_{i}-x_{i})^{2}-\sigma_{i}^{2}\frac{v_{i}^{2}}{v^{2}}(\bar{x}-\bar{\mu})^{2}-\frac{2v_{i}\sigma_{i}}{v}(\mu_{i}-x_{i})(\bar{x}-\bar{\mu})}{\left(1-\frac{v_{i}^{2}}{v}\right)\sigma_{i}^{2}}-\log\left(1-\frac{v_{i}^{2}}{v}\right) \displaystyle=\frac{-v_{i}^{2}(\mu_{i}-x_{i})^{2}-\sigma_{i}^{2}\frac{v_{i}^{2}}{v}(\bar{x}-\bar{\mu})^{2}-2v_{i}\sigma_{i}(\mu_{i}-x_{i})(\bar{x}-\bar{\mu})}{\left(v-v_{i}^{2}\right)\sigma_{i}^{2}}-\log\left(1-\frac{v_{i}^{2}}{v}\right) \displaystyle=\frac{-\left[v_{i}(\mu_{i}-x_{i})+\sigma_{i}(\bar{x}-\bar{\mu})\right]^{2}+\sigma_{i}^{2}(1-\frac{v_{i}^{2}}{v})(\bar{x}-\bar{\mu})^{2}}{\left(v-v_{i}^{2}\right)\sigma_{i}^{2}}-\log\left(1-\frac{v_{i}^{2}}{v}\right) \displaystyle\leq\frac{(\bar{x}-\bar{\mu})^{2}}{v}-\log\left(1-\frac{v_{i}^{2}}{v}\right) \displaystyle\leq\frac{(\bar{x}-\bar{\mu})^{2}}{v}-\log\frac{\lambda_{n}}{\lambda_{1}}\left(1-\frac{\sigma_{i}^{2}}{\|\sigma\|_{2}^{2}}\right), noting v-v_{i}^{2}\geq 0 as established by ( 10 ) and using ( 11 ). Since v=\overline{\Sigma} and c_{\Phi}=\lambda_{1}/\lambda_{n} , we find \displaystyle\Delta_{i}(Q,\bar{x},x) \displaystyle\leq r_{1}-\log r_{2}, as \frac{(\bar{x}-\bar{\mu})^{2}}{\overline{\Sigma}}\leq r_{1} and c_{\Phi}\left(1-\frac{\sigma_{i}^{2}}{\|\sigma\|_{2}^{2}}\right)^{-1}\leq r_{2} by assumption on the class of data-priors. ∎