Improving LLM Group Fairness on Tabular Data via In-Context Learning

Valeriia Cherepanova, Chia-Jung Lee, Nil-Jana Akpinar, Riccardo Fogliato, Martin Andres Bertran, Michael Kearns, James Zou · 2024

Read Improving LLM Group Fairness on Tabular Data via In-Context Learning by Valeriia Cherepanova, Chia-Jung Lee, Nil-Jana Akpinar, Riccardo Fogliato, Martin Andres Bertran, Michael Kearns, James Zou (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04642.txt.

Introduction

Valeriia Cherepanova and co-authors seek fairer group outcomes when large language models predict from table data, in work from twenty twenty-four. Models can do well on tables with few rows, thanks to built in knowledge. But their calls may favor one group over another, and text based fixes do not map well to tables.

The team tried four routes: tuned prompts, soft prompt training, smart choice of few shot examples, and self review through step by step thought. Tests ran on four data sets with both open and closed models. Each route lifted group fairness while keeping overall scores high. The study gives plain advice on which route fits which limits. Fairness here means near equal positive rates across groups.

Catalog record

CREATORValeriia Cherepanova, Chia-Jung Lee, Nil-Jana Akpinar, Riccardo Fogliato, Martin Andres Bertran, Michael Kearns, James Zou
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-04642

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Improving LLM Group Fairness on Tabular Data via In-Context Learning Valeriia Cherepanova, Chia-Jung Lee, Nil-Jana Akpinar, Riccardo Fogliato, Martin Andres Bertran, Michael Kearns, James Zou Abstract Large language models (LLMs) have been shown to be effective on tabular prediction tasks in the low-data regime, leveraging their internal knowledge and ability to learn from instructions and examples. However, LLMs can fail to generate predictions that satisfy group fairness, that is, produce equitable outcomes across groups. Critically, conventional debiasing approaches for natural language tasks do not directly translate to mitigating group unfairness in tabular settings. In this work, we systematically investigate four empirical approaches to improve group fairness of LLM predictions on tabular datasets, including fair prompt optimization, soft prompt tuning, strategic selection of few-shot examples, and self-refining predictions via chain-of-thought reasoning. Through experiments on four tabular datasets using both open-source and proprietary LLMs, we show the effectiveness of these methods in enhancing demographic parity while maintaining high overall performance. Our analysis provides actionable insights for practitioners in selecting the most suitable approach based on their specific requirements and constraints. 1 Introduction In recent years, the scope of large language models (LLMs) has broadened significantly beyond traditional natural language processing tasks, with recent research demonstrating their effectiveness in tackling challenges on tabular data, including predictive tasks ( Hegselmann et al., 2023 ; Yin et al., 2020 ) . Typically, structured data is converted into textual format and provided to the language model along with a concise task description and key features. Notably, it has been shown that language models are particularly beneficial in scenarios with limited training data, as they can utilize internal knowledge about world from pre-training combined with textual instructions and few-shot examples to make predictions Slack and Singh (2023) . Although considerable research has been devoted to exploring and addressing issues of stereotypical bias and fairness in language models applied to natural language tasks, tabular datasets present distinct challenges, particularly in group fairness. It is important to differentiate group fairness in the context of tabular data from conventional notions of fairness in NLP tasks: group fairness in tabular problems hinges on class labels and the representation of various demographic groups within these labels, while stereotypical fairness in NLP has primarily focused on bias in model representations. Notably, achieving fairness in the typical NLP sense does not automatically ensure group-fair predictions in tabular tasks due to potential disparities in class distributions. Recent studies have started exploring how language models handle group fairness when applied to tabular data, revealing noticeable fairness discrepancies among different demographic groups. Liu et al. (2024) and Li et al. (2024) evaluate a few baseline methods for improving group fairness in tabular tasks, including resampled fine-tuning, and few-shot learning with label flipping and find these methods to have limited effectiveness. A recent survey paper ( Fang et al., 2024 ) recognizes the challenge of mitigating inherent biases in large language models through conventional fine-tuning and few-shot learning and highlights the need for more effective strategies to address group unfairness in tabular tasks. In this work we examine four approaches for empirically improving demographic parity of LLMs when applied to making predictions on tabular data. These approaches include in-context methods such as prompt optimization, soft prompt tuning, few-shot in-context learning, and self-refining predictions to promote fairness. We empirically evaluate these methods using both open-source and proprietary models across four tabular datasets, demonstrating their effectiveness. Based on our analysis, we provide actionable recommendations to practitioners on the most suitable method for different scenarios, and discuss how these approaches may be adapted to other notions of fairness. 2 Related Work 2.1 Large Language Models on Tabular Data A growing body of work has applied deep learning algorithms to tabular data ( Yin et al., 2020 ; Herzig et al., 2020 ; Huang et al., 2020 ; Levin et al., 2022 ; Zhu et al., 2023 ) . Relevant to our setting, some of these studies have employed LLMs to analyze tabular data that is serialized into formatted text. They show that descriptive feature names, well-defined instructions, in-context examples, and chain-of-thought reasoning enhances LLM performance ( Zhao et al., 2023 ; Marvin et al., 2023 ; Chen, 2022 ) . Some specifically focus on classification tasks ( Hegselmann et al., 2023 ; Wang et al., 2024 ; Liu et al., 2022 ; Fang et al., 2024 ; Jaitly et al., 2023 ) , which is also the focus of our work. The prior knowledge of LLMs allows them to perform better than traditional algorithms such as XGBoost in low-data regimes ( Slack and Singh, 2023 ; Hegselmann et al., 2023 ) . However, LLM predictions can reflect inherent biases, affecting the fairness of their outcomes ( Hu and Du, ; Liu et al., 2023 ) . Liu et al. (2023) ’s work is closely related to ours: they analyze the accuracy and fairness of LLM predictions, concluding that traditional ML models exhibit fewer disparities. Although in-context learning and finetuning do not fully close the fairness gap, label-flipping in in-context examples significantly reduces biases, albeit at the cost of prediction performance. Our work contributes to this literature by introducing four in-context learning approaches for mitigating the demographic parity gap in tabular data predictions, demonstrating their effectiveness across widely-used fairness datasets. 2.2 Bias and Stereotypes in LLMs Despite their promising capabilities, language models also exhibit biases and stereotypes ( Bolukbasi et al., 2016 ; Bender et al., 2021 ; Chu et al., 2024 ) . These biases mostly originate from the training data, which often contain historical and societal prejudices embedded within the text. Biases have been reported with respect to several demographic groups, e.g., gender, race, ethnicity, and socioeconomic status ( Wan et al., 2023 ; Haim et al., 2024 ; Santurkar et al., 2023 ) . With the use of these models becoming more widespread, these biases have the risk to substantially reinforce harmful stereotypes and perpetuate existing inequalities, especially when deployed in high-stakes settings ( Zou and Schiebinger, 2018 ) . Addressing these biases is essential, and several mitigation strategies have been proposed for this purpose, including data augmentation, prompt tuning and few-shot learning ( Zhao et al., 2017 ; Wang et al., 2021 ; Sun et al., 2019 ; Zmigrod et al., 2019 ; Mattern et al., 2022 ; Fatemi et al., 2021 ; Aguirre et al., 2023 ) . However, effectively applying these strategies to the large-scale pretraining corpora remains challenging. Finally, biases can be hard to detect and several datasets and methods have been proposed to help identify them ( Caliskan et al., 2017 ; May et al., 2019 ; Webster et al., 2020 ; Kurita et al., 2019 ; Nangia et al., 2020 ; Blodgett et al., 2021 ) . 2.3 Fairness on Tabular Data Much of the work on classification and algorithmic fairness has focused on tabular datasets ( Mehrabi et al., 2021 ; Caton and Haas, 2024 ; Pessach and Shmueli, 2023 ; Fabris et al., 2022 ; Barocas et al., 2023 ; Chouldechova and Roth, 2018 ; Fogliato et al., 2020 ) . Consequently, there is a wide range of research describing the properties and trade-offs of predictive algorithms on this type of

Read the complete text