Can large language models be privacy preserving and fair medical coders?
Ali Dadsetan, Dorsa Soleymani, Xijie Zeng, Frank Rudzicz · 2024
Read Can large language models be privacy preserving and fair medical coders? by Ali Dadsetan, Dorsa Soleymani, Xijie Zeng, Frank Rudzicz (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-05533.txt.
Introduction
Ali Dadsetan, Dorsa Soleymani, Xijie Zeng, and Frank Rudzicz test whether privacy tools hurt language models that code medical records, in a 2024 paper. Medical coding assigns diagnosis codes to patient text, and hospitals guard that text closely. A method called differential privacy can shield the data during training. The team measured two costs of using it on a large hospital data set.
Accuracy fell hard: scores on the top fifty codes dropped by more than 40 percent in the private models. A fairness gap also widened, with recall for male and female patients differing by over three points more than before. The authors say both trade-offs need study before such models can be deployed. Their results map where the losses land so later work can attack them.
Catalog record
| CREATOR | Ali Dadsetan, Dorsa Soleymani, Xijie Zeng, Frank Rudzicz |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-05533
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can large language models be privacy preserving and fair medical coders? Ali Dadsetan, Dorsa Soleymani, Xijie Zeng, Frank Rudzicz Abstract Protecting patient data privacy is a critical concern when deploying machine learning algorithms in healthcare. Differential privacy (DP) is a common method for preserving privacy in such settings and, in this work, we examine two key trade-offs in applying DP to the NLP task of medical coding (ICD classification). Regarding the privacy-utility trade-off, we observe a significant performance drop in the privacy preserving models, with more than a 40% reduction in micro F1 scores on the top 50 labels in the MIMIC-III dataset. From the perspective of the privacy-fairness trade-off, we also observe an increase of over 3% in the recall gap between male and female patients in the DP models. Further understanding these trade-offs will help towards the challenges of real-world deployment. † † volume: LEAVE UNSET † † year: 2024 † † submitted: LEAVE UNSET † † published: LEAVE UNSET † † workshop: Machine Learning for Health (ML4H) 2024 keywords Differential privacy, ICD classification, Fairness Data and Code Availability We use the MIMIC-III dataset, which is publicly available, under a specific training course and a data use agreement. We are making our code publicly available. Institutional Review Board (IRB) This work is exempt from IRB review, as it exclusively uses data that is publicly available. 1 Introduction Applying pre-trained, transformer-based models has been a subject of recent interest in the medical domain ( Huang et al. 2022 ; Yang et al. 2022b ; Wang et al. 2024 ) . While these applications offer many benefits, some privacy concerns may exist including leaking sensitive information from training datasets ( Carlini et al. 2021 ) . Moreover, if a curious actor gets access to the parameters of a model trained on sensitive data, they may be able to use methods to check if a specific piece of information has been part of the training dataset for that model ( Carlini et al. 2022 ; Mireshghallah et al. 2022 ) . Privacy-preserving learning has been applied effectively in NLP tasks outside of healthcare. Gradient-perturbation-based differential privacy ( Abadi et al. 2016 ; Li et al. 2022 ) is a popular set of methods that have been applied successfully to language generation and sentence classification using LLMs ( Li et al. 2022 ) . In parallel, differential privacy has been studied in healthcare, which is considered to be challenging. Suriyakumar et al. 2021 showed that the performance gap between private and non-private algorithms is bigger in healthcare datasets, and hypothesized that the cause is long-tail distributions common in healthcare. They also showed a privacy/fairness trade-off in which privacy-preserving versions of algorithms diminished the influence of minority groups on inference. However, in the non-private versions, this was not the case. One shortcoming of that study was that the authors only considered time-series and imaging data, and not text. While there is existing literature on privacy in text data, there remains a need for further exploration of privacy trade-offs specifically in NLP applications within the healthcare domain. In this work, we partially fill this gap by studying various trade-offs of applying differentially private learning algorithm to the task of assiging International Classification of Diseases (ICD) codes to discharge notes, which is a common application of NLP in healthcare. 2 Background A popular mathematical formalism for the definition of private algorithms in machine learning is differential privacy (DP). In a differentially private algorithm, inclusion or exclusion of a particular sample in the training dataset has a limited effect on any expected outcomes ( Dwork et al. 2014 ) . More formally, we say that a randomized algorithm \mathcal{M} is (\epsilon,\delta) approximately differentially private, if given any two datasets differing only by one data point X,X^{\prime} , the probability that the output of the algorithm on these datasets would occur in any specific event, such as Y , would not differ more than a multiplicative factor of \exp(\epsilon) plus a constant \delta : \mathbb{P}(\mathcal{M}(X)\in Y)\leq\exp(\epsilon)\mathbb{P}(\mathcal{M}(X^{\prime})\in Y)+\delta. In gradient-based learning, differential privacy of an algorithm can be guaranteed by modifying the gradient in a specific way before applying the optimization method of choice. Given a batch of B samples \set{B} and the corresponding gradients \nabla\mathcal{L}_{i} , i\in\set{B} , the DP-SGD algorithm ( Abadi et al. 2016 ) works by clipping each \nabla\mathcal{L}_{i} to ensure its length does not exceed C , then obtaining the average of the clipped gradient for the batch, and finally adding a isotropic normal noise with covariance matrix \sigma^{2}\mathbb{I} : \tilde{\vec{g}}=\frac{1}{B}\sum_{i\in\set{B}}\text{Clip}(\nabla\mathcal{L}_{i},C)+\mathcal{N}(0,\sigma^{2}\mathbb{I}). Instead of using the usual \vec{g}=\frac{1}{B}\sum_{i\in\set{B}}\nabla\mathcal{L}_{i} , \tilde{\vec{g}} is used by the optimization algorithm. The final “privacy loss”, \epsilon , depends on the magnitude of the added noise, \sigma ,the noise multiplier, the number of gradient updates applied to the model, the sampling rate, q=\frac{B}{N} where N is the dataset size, and the chosen approximation constant, \delta . This relationship has been studied extensively ( Abadi et al. 2016 ; Mironov et al. 2019 ; Wang et al. 2019 ; Koskela et al. 2020 ; Gopi et al. 2021 ) , and tight bounds exist ( Gopi et al. 2021 ) . 2.1 Differential Privacy for LLMs Recent work explored applying differential privacy techniques to LLMs to enable private fine-tuning while maintaining both utility and privacy. In fact, large pre-trained language models (PLMs) can be effective differentially private learners when fine-tuned with appropriate hyperparameters and objectives ( Li et al. 2022 ) . Authors in ( Li et al. 2022 ) showed that training PLMs such as RoBERTa for sentence classification tasks (e.g., MNLI) and GPT-2 for language generation tasks (e.g., E2E) with strong privacy guarantees \epsilon\in\{3,8\} is achievable with only marginally reduced performance compared to their non-private counterparts. Another challenge of applying gradient-perturbation-based DP algorithms to NLP tasks is their increased memory and computational complexity. From this perspective, the most demanding aspect of the algorithm is gradient clipping ( Li et al. 2022 ; Bu et al. 2023b ) . The clipping operation with clip factor C has the role of controlling the sensitivity of the gradient, and is defined as \mathrm{Clip}(\nabla\mathcal{L}_{i})=\min(1,\frac{C}{\|\nabla\mathcal{L}_{i}\|_{2}})\nabla\mathcal{L}_{i}. Clipping in DP-SGD occurs on a per-example basis. The naïve implementation may fail to leverage the parallel capabilities of GPUs, significantly increasing execution time by disregarding batch computations and calculating gradients for each sample. Other approaches involve implicit clipping, as seen in methods such as ghost clipping ( Li et al. 2022 ) and Book Keeping ( Bu et al. 2023b ) . A third option is to perform clipping on a per-group basis, where a group may be defined as either a layer of the neural network or a set of parameters for a layer (e.g., weights of a linear layer) ( McMahan et al. 2017 ; Bu et al. 2023a ) . Here, to ensure that the gradient for each sample does not exceed a maximum length of C , we assign a clipping threshold to each group of parameters. Specifically, the parameter set \theta is partitioned into k groups, each representing parameters of a layer, denoted as \theta=(\theta_{1},\dots,\theta_{k}) . Correspondingly, the gradients are partitioned as \nabla_{\theta}\mathcal{L}=(\nabla_{\theta_{1}}\mathcal{L},\dots,\nabla_{\theta_{k}}\mathcal{L}) . To ensure that the overall gradient \nabla_{\theta}\mathcal{L} does not