Enforcing Consistency and Fairness in Multi-level Hierarchical Classification with a Mask-based Output Layer

Shijing Chen, Shoaib Jameel, Mohamed Reda Bouadjenek, Feilong Tang, Usman Naseem, Basem Suleiman, Hakim Hacid, Flora D. Salim, Imran Razzak · 2025

Read Enforcing Consistency and Fairness in Multi-level Hierarchical Classification with a Mask-based Output Layer by Shijing Chen, Shoaib Jameel, Mohamed Reda Bouadjenek, Feilong Tang, Usman Naseem, Basem Suleiman, Hakim Hacid, Flora D. Salim, Imran Razzak (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-15566.txt.

Introduction

Shijing Chen and eight co-authors published this machine learning paper in 2025. It deals with sorting items into a tree of labels, where each label sits inside a broader one above it. Standard tools often break the tree, for example by placing an item in a narrow class but not in its parent class.

The authors add a mask-based output layer that can sit on top of many existing models. The layer blocks label choices that break the tree, so every output stays consistent. It can also be tuned to guard fairness for sensitive traits. In their tests, the layer improved both fairness and accuracy at once. It beat language models that were debiased during training, which makes it useful for shopping, health, and school settings.

Catalog record

CREATORShijing Chen, Shoaib Jameel, Mohamed Reda Bouadjenek, Feilong Tang, Usman Naseem, Basem Suleiman, Hakim Hacid, Flora D. Salim, Imran Razzak
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-15566

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Enforcing Consistency and Fairness in Multi-level Hierarchical Classification with a Mask-based Output Layer Shijing Chen, Shoaib Jameel, Mohamed Reda Bouadjenek, Feilong Tang, Usman Naseem, Basem Suleiman, Hakim Hacid, Flora D. Salim, Imran Razzak Abstract Traditional Multi-level Hierarchical Classification (MLHC) classifiers often rely on backbone models with n n independent output layers. This structure tends to overlook the hierarchical relationships between classes, leading to inconsistent predictions that violate the underlying taxonomy. Additionally, once a backbone architecture for an MLHC classifier is selected, adapting the model to accommodate new tasks can be challenging. For example, incorporating fairness to protect sensitive attributes within a hierarchical classifier necessitates complex adjustments to maintain the class hierarchy while enforcing fairness constraints. In this paper, we extend this concept to hierarchical classification by introducing a fair, model-agnostic layer designed to enforce taxonomy and optimize specific objectives, including consistency, fairness, and exact match. Our evaluations demonstrate that the proposed layer not only improves the fairness of predictions but also enforces the taxonomy, resulting in consistent predictions and superior performance. Compared to Large Language Models (LLMs) employing in-processing de-biasing techniques and models without any bias correction, our approach achieves better outcomes in both fairness and accuracy, making it particularly valuable in sectors like e-commerce, healthcare, and education, where predictive reliability is crucial. 1 Introduction The growing complexity of real-world datasets has led to the widespread use of multi-level hierarchical structures, making Multi-level Hierarchical Classification (MLHC) essential in modern data analysis. In domains such as e-commerce, where large-scale product datasets need effective categorization, MLHC plays a pivotal role Silla and Freitas (2011) ; Tieppo et al. (2022) . For example, in an online store for beauty products, items are organized into a hierarchical taxonomy. At the top level ( \ell_{1} ) might be a broad category such as Beauty , which branches into subcategories like Hair Care at level ( \ell_{2} ), and even more specific classes like Hair Color or Shampoo at level ( \ell_{3} ). MLHC leverages these taxonomies to accurately classify items so that the results can be used to aid the recommendation of products based on their hierarchical relationships, which in turn enhances user experience, improves personalization, and drives sales. This hierarchical structure enables MLHC to capture semantic relationships between categories, making it indispensable in sectors like e-commerce where effective data organization and classification are crucial for scaling user interaction Dumais and Chen (2000) ; Agrawal et al. (2013) ; Li et al. (2020) ; Shen et al. (2012) . (a) Amazon product review taxonomy. (b) Product review taxonomy performance. (c) Gender performance difference. Figure 1: (a) Example of an Amazon product review classified across multiple levels of the taxonomy. (b) Proportion of correctly classified product reviews for each level of our taxonomy of the Amazon product review dataset, and the proportion of reviews incorrectly classified but for which other levels in the taxonomy were correctly identified. (c) Performance difference between male and female predictions using the BERT + Flat classifier model on the Amazon product review dataset. The percentages highlighted are actual accuracy differences between different genders. Despite the advantages of MLHC, conventional methods still face significant challenges, particularly in ensuring both consistency and fairness across multiple levels of the hierarchy. Flat classifiers, which ignore the hierarchical relationships between categories, often result in inconsistent predictions, as shown in Figure 1a . Figure 1b demonstrates the potential accuracy gains that can be achieved by employing consistent hierarchical classifiers. For each level’s accuracy shown in Figure 1b , a portion of misclassified instances (4.29% for \ell_{1} , 6.45% for \ell_{2} , and 15.57% for \ell_{3} ) can be attributed to incorrect predictions while other levels in the hierarchy were correctly classified. By leveraging information from higher levels ( \ell_{1} and \ell_{2} ) that are correctly classified, the classifier can make more informed predictions at the lower levels ( \ell_{3} ), thus reducing misclassification. This suggests that accuracy at each level could improve if classifiers leverage the correct classifications from other levels in the taxonomy. Hence, enforcing consistency across hierarchical levels becomes a crucial and intriguing challenge in this domain. Additionally, existing models often inherit biases from the underlying data De-Arteaga et al. (2019) ; Guo and Caliskan (2021) ; Nangia et al. (2020) . Especially, Large Language Models (LLMs), which are trained on vast amounts of textual data, can amplify these biases, resulting in outputs that reinforce harmful stereotypes or exclude minority perspectives Bender et al. (2021) ; Bommasani et al. (2021) . This highlights significant limitations of LLMs related to unfairness and bias, emphasizing the need for strategies to detect and mitigate these issues before utilizing their representational power. In large-scale datasets, unfairness and bias (e.g., age and gender) can significantly impact the fairness of predictions, resulting in unequal experiences for different users. Such biases undermine the reliability of machine learning systems and create ethical concerns, especially in applications where fairness is vital. Ensuring consistency in hierarchical classification is not only essential for improving overall accuracy but also plays a crucial role in addressing fairness. As shown in Figure 1c , male predictions tend to be more accurate than female predictions across all levels of the hierarchy. This disparity highlights the need for a classifier that not only enforces consistency but also promotes fairness by ensuring that correct information at higher levels of the taxonomy is propagated downwards, improving performance for all demographic groups. Different methods have been proposed for MLHC, which can be classified based on how they utilize the hierarchical structure. Specifically, we distinguish between three primary approaches: (i) the flat classification approach, where the class hierarchy is completely ignored. In this approach, predictions are made solely for the bottom levels, with the assumption that all ancestor classes are implicitly attributed to the instance as well; (ii) the local classification approach, which involves training a separate multi-class classifier at each parent node in the hierarchy to distinguish between its child nodes; and (iii) the global classification approach Zhang et al. (2024) ; Bettouche et al. (2024) ; Liu et al. (2024) ; Chen et al. (2025) , where a single classifier is responsible for handling the entire class hierarchy. In this paper, we argue that flat classifiers , by ignoring the hierarchical relationships between class levels, often results in inconsistent classifications. For instance, as shown in Figure 1 , the data entry of a Hair Color product is correctly classified as Beauty and Hair Care , but incorrectly as Men’s Fragrance at the leaf node. Furthermore, we argue that it is impractical to train and maintain n separate networks for local classification approaches , which can be redundant and costly in real-world applications. As a result, we favor global classification approaches , which addresses the limitations of flat and local methods. However, existing methods still face several key challenges: (i) they do not inherently embed the taxonomy structure, (ii) they often rely on complex neural network architectures with n independent output layers that do not interact, (iii) they frequently produce predictions that are inconsistent with the taxonomy, and (iv) they typically operate with a fixed n , limiting flexibility and requiring extensive hyperparameter tuning to optimize n for different scenarios. Like traditional classifiers, MLHC

Read the complete text