Faithful Group Shapley Value

Kiljae LeeOhio State University [email protected] Ziqi LiuCarnegie Mellon University [email protected] Weijing TangCarnegie Mellon University [email protected] Yuan ZhangOhio State University [email protected] · 2025

Read Faithful Group Shapley Value by Kiljae LeeOhio State University [email protected] Ziqi LiuCarnegie Mellon University [email protected] Weijing TangCarnegie Mellon University [email protected] Yuan ZhangOhio State University [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-19013.txt.

Introduction

Kiljae Lee, Ziqi Liu, Weijing Tang, and Yuan Zhang fix group level data pricing against gaming, in work from 2025. Data Shapley scores how much each data point helps a learning model. Buyers often need group scores instead, since providers supply data in batches rather than one point at a time.

Current group scores can be gamed by shell groups that split to inflate their worth. Their faithful group score is built to block that split trick by design. A new method computes the score fast with proven accuracy bounds for large groups. Tests beat leading methods on both speed and closeness to true values. Fair batch pricing for data markets is the clear target use.

Catalog record

CREATORKiljae LeeOhio State University [email protected] Ziqi LiuCarnegie Mellon University [email protected] Weijing TangCarnegie Mellon University [email protected] Yuan ZhangOhio State University [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-19013

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Faithful Group Shapley Value Kiljae LeeOhio State University [email protected] Ziqi LiuCarnegie Mellon University [email protected] Weijing TangCarnegie Mellon University [email protected] Yuan ZhangOhio State University [email protected] Abstract Data Shapley is an important tool for data valuation, which quantifies the contribution of individual data points to machine learning models. In practice, group-level data valuation is desirable when data providers contribute data in batch. However, we identify that existing group-level extensions of Data Shapley are vulnerable to shell company attacks , where strategic group splitting can unfairly inflate valuations. We propose Faithful Group Shapley Value (FGSV) that uniquely defends against such attacks. Building on original mathematical insights, we develop a provably fast and accurate approximation algorithm for computing FGSV. Empirical experiments demonstrate that our algorithm significantly outperforms state-of-the-art methods in computational efficiency and approximation accuracy, while ensuring faithful group-level valuation. 1 Introduction As data become increasingly crucial in modern machine learning, quantifying its value has significant implications for fair compensation and data market design [ 29 , 39 ] . The Shapley value, a foundational concept from cooperative game theory [ 30 ] , stands out as the unique valuation method satisfying four desirable axioms for fair data valuation [ 6 , 13 ] . Consequently, it has been applied to diverse applications, including collaborative intelligence [ 22 , 25 ] and copyright compensation [ 35 ] , and quantifying feature importance in explainable AI [ 26 ] . Figure 1: Left: GSV; right: FGSV (our method). Vertical span: valuation. Group A is fixed; group B engages increasing degrees of shell company attack (left \to right). Detailed experimental set-up in Appendix A Many real-world scenarios demand evaluation of data sets instead of individual data points. In applications like data marketplaces [ 39 ] or compensation allocation for generative AI [ 4 , 35 ] , data are contributed by owners who possess entire datasets, making group-level data valuation a natural choice. A more technical motivation is that individual-level Shapley values are often computationally challenging to approximate for big data; while group-level valuation is much faster and can provide useful insights [ 16 ] . In explainable AI, group-level feature importance reportedly provides more robust and interpretable valuation [ 32 , 3 ] . Prior works extend the Shapley framework to group data valuation by treating each pre-defined group as an atomic unit and then applying the standard Shapley value formulation [ 14 , 20 , 35 , 32 ] , which we refer to as Group Shapley Value (GSV). However, we demonstrate—both theoretically and empirically—that this approach suffers from susceptibility to strategic manipulation through partitioning. Consider the valuation of a fixed data group A . As shown in Figure 1 , partitioning the remaining data points into smaller subgroups reduces the GSV for A , despite no change in data content. In other words, unfair players may exploit the loophole in GSV by splitting their data among puppy subsidiaries, a vulnerability we coin as shell company attack . To address this issue, we propose a faithfulness axiom for group data valuation, such that the total valuation of the same set of data remains unchanged, regardless of how others are subdivided. Based on this, we introduce the Faithful Group Shapley Value (FGSV) for a group as the sum of individual Data Shapley values of its members, and prove that it uniquely satisfies the set of axioms that are desirable for faithful group data valuation. Figure 1 demonstrates that FGSV effectively defends against the shell company attack. While FGSV safeguards fairness, its exact computation requires combinatorial computation of individual Shapley values. Although numerous approximation algorithms for individual Shapley values have been developed [ 2 , 6 , 12 , 34 , 26 , 3 , 38 , 24 , 27 ] , simply applying them and summing up results in each group is slow for large groups and may compound approximation errors. Importantly, we proved that a small subset of terms dominate in FGSV’s formula. As a result, we developed an original algorithm that directly approximates FGSV fast and accurately. Our numerical experiments demonstrate its computational efficiency and approximation accuracy against summing up individual Shapley values computed by state-of-the-art (SOTA) methods. We also applied FGSV to fair copyright compensation on Stable Diffusion models for image generation. 2 Preliminaries: individual and group Shapley values Individual Shapley value (SV). Let \mathcal{D}=\{z_{1},\ldots,z_{n}\}\in\mathcal{Z}^{n} denote the training data, where n:=|\mathcal{D}| . Let S\subseteq[n] be an index set of cardinality s:=|S| , and write {\cal S}:=\{z_{i}:i\in S\} . A utility function U({\cal S}) assigns a performance score (such as classification accuracy) to method trained based on data {\cal S} . To simplify notation, we may refer to “ U({\cal S}) ” by index as “ U(S) ” and write them interchangeably. The individual Shapley value of data point i is defined as \operatorname{SV}(i):=\sum_{S\subseteq[n]\setminus\{i\}}\frac{|S|!(n-|S|-1)!}{n!}\{U(S\cup\{i\})-U(S)\}. (1) The widespread adoption of Shapley value stems from its strong theoretical foundation that it uniquely satisfies four desired axioms, called null player , symmetry , linearity , and efficiency . We will review these principles as part of our axiomatization for group Shapley value in Definition 2 . Group Shapley value (GSV). As mentioned in Section 2 , most group-level Shapley values [ 14 , 35 , 32 ] adopt the so-called group-as-individual (GaI) approach. Suppose the entire data set is partitioned into (K+1) disjoint groups: \mathcal{D}=S_{0}\cup S_{1}\cup\cdots\cup S_{K} , in which, S_{0}\subseteq[n] is the group we aim to evaluate. The group Shapley value (GSV) of S_{0} , as in [ 14 , 35 , 32 ] , is defined as \displaystyle\operatorname{GSV}(S_{0})= \displaystyle~\sum_{I\subseteq\{1,\dots,K\}}\frac{|I|!(K-|I|)!}{(K+1)!}\Big\{U\Big(\big\{\cup_{k\in I}S_{k}\big\}\cup S_{0}\Big)-U\Big(\cup_{k\in I}S_{k}\Big)\Big\}. (2) Clearly, ( 2 ) is a “set version” of ( 1 ). Thus GSV also satisfies the four axioms for SV at the group level. However, it is questionable whether these axioms, originally formulated for individual-level valuation, are appropriate for groups. Fairness issue with GSV. The GSV valuation of S_{0} as in ( 2 ) can be impacted by how the rest of the data are grouped. A serious consequence is that GSV is prone to the shell company attack : splitting a group into smaller subgroups will increase the total valuation earned by the same set of data points. We have seen an illustration in Figure 1 in Section 1 . Now we theoretically formalize this observation. Proposition 1 (Shell company attack) . Suppose in \mathcal{D}=\{z_{1},\dots,z_{n}\} are an i.i.d. sample. Let \bar{U}(s)=\mathbb{E}_{S\sim\mathcal{P}^{s}}[U(S)] denote the expected utility for data S (which only depends on s=|S| ). Now we split a group S_{k} into two non-empty subgroups S_{k}^{\prime} and S_{k}^{\prime\prime} (i.e., S_{k}^{\prime}\cup S_{k}^{\prime\prime}=S_{k} , S_{k}^{\prime}\cap S_{k}^{\prime\prime}=\emptyset , and S_{k}^{\prime},S_{k}^{\prime\prime}\neq\emptyset ), If \bar{U}(s) satisfies a prudence condition: \displaystyle\Delta_{s}^{3}\bar{U}(s):=\bar{U}(s+3)-3\bar{U}(s+2)+3\bar{U}(s+1)-\bar{U}(s)>0, (3) then \mathbb{E}[\operatorname{GSV}(S_{k})]<\mathbb{E}[\operatorname{GSV}(S_{k}^{\prime})]+\mathbb{E}[\operatorname{GSV}(S_{k}^{\prime\prime})]. (4) Condition ( 3 ) is a familiar concept in economics, characterizing risk-aversion behaviors under uncertainty [ 7 ] . It is also observed in machine learning, where the performance of a learning method saturates as the same type of data repeatedly come in [ 17 , 10 ] . 3 Our method 3.1 Faithfulness axiom and Faithful Group Shapley Value (FGSV) Motivated by the faithfulness issue, we introduce a set of axioms that formalize desirable principles for faithful group data valuation. We first axiomatize group-level data valuation that generalizes the Shapley’s

Read the complete text