Matryoshka Concept Bottleneck Models

Ziye Chen ∗,1,2 Hongbin Lin ∗,2 Jie Li 1 Lijie Hu 1 1 Mohamed bin Zayed University of Artificial Intelligence 2 The Hong Kong University of Science and Technology (Guangzhou) 3 The Hong Kong University of Science and Technology · 2026

Read Matryoshka Concept Bottleneck Models by Ziye Chen ∗,1,2 Hongbin Lin ∗,2 Jie Li 1 Lijie Hu 1 1 Mohamed bin Zayed University of Artificial Intelligence 2 The Hong Kong University of Science and Technology (Guangzhou) 3 The Hong Kong University of Science and Technology (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-20612.txt.

Introduction

Ziye Chen and three colleagues propose Matryoshka Concept Bottleneck Models, a single model that can use more or fewer human concepts as needed. Concept bottleneck models ground predictions in ideas people can grasp, but fixing errors at test time costs expert time. One fixed concept set forces exhaustive checks, while many tailored models cost too much to build and keep. Their nested design lets one model adapt its concept use to the budget at hand.

Catalog record

CREATORZiye Chen ∗,1,2 Hongbin Lin ∗,2 Jie Li 1 Lijie Hu 1 1 Mohamed bin Zayed University of Artificial Intelligence 2 The Hong Kong University of Science and Technology (Guangzhou) 3 The Hong Kong University of Science and Technology
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE78990
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.20612

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Matryoshka Concept Bottleneck Models Ziye Chen ∗,1,2 Hongbin Lin ∗,2 Jie Li 1 Lijie Hu 1 1 Mohamed bin Zayed University of Artificial Intelligence 2 The Hong Kong University of Science and Technology (Guangzhou) 3 The Hong Kong University of Science and Technology Abstract Concept Bottleneck Models (CBMs) have emerged as a prominent paradigm for interpretable deep learning, learning by grounding predictions in human-understandable concepts. However, their practical deployment is hindered by the high cost of test-time intervention, as correcting model errors typically requires human experts to manually inspect and verify a large set of predicted concepts. Existing approaches suffer from a fundamental structural limitation: they either adopt a single static concept set, forcing experts to exhaustively annotate concepts and incurring prohibitive intervention costs, or train multiple models tailored to different concept budgets, resulting in substantial computational and maintenance overhead. To address this challenge, we propose the Matryoshka Concept Bottleneck Model (MCBM) , a unified architecture that enables adaptive concept utilization within a single model. Inspired by Matryoshka Representation Learning, MCBM organizes concepts into a nested hierarchy based on maximum relevance and minimum redundancy, allowing inference at multiple levels of conceptual granularity without retraining. Theoretically, we show that MCBM reduces the expected intervention costs from linear to logarithmic order, O ⁡ ( log ⁡ K ) O(\log K) , while guaranteeing monotonic performance improvement. Empirically, extensive experiments demonstrate that MCBM matches the performance of independently trained models while enabling dynamic and efficient expert interaction. 1 Introduction As deep learning models are increasingly deployed in high-stakes domains such as healthcare and finance, transparency and interpretability have become central concerns 22 ; 16 . Concept Bottleneck Models (CBMs) 18 address this challenge by decomposing predictions into two interpretable stages: mapping inputs to human-understandable concepts and making predictions solely based on these concepts. Beyond providing explanations, CBMs enable human experts to intervene by inspecting and correcting erroneous concept predictions, motivating growing interest in effective and interactive CBM design 34 ; 14 ; 33 . However, the practical utility of CBMs is hindered by the high cost of test-time intervention. Existing CBM approaches treat concepts as a flat and unordered set, requiring experts to manually inspect and correct many concepts to fix a single prediction 5 , rendering large-scale deployment impractical. While reducing the number of concepts can lower this burden, it often leads to a substantial drop in predictive performance 12 ; 15 ; 13 ; 6 . As a result, practitioners face a rigid design choice: either deploy a single high-dimensional CBM that is costly to intervene on, or train and maintain multiple models with different concept budgets, incurring significant computational overhead and lacking flexibility for real-world deployment 39 ; 21 . This rigidity arises from how concepts are structured and utilized in existing CBMs. In fine-grained tasks, many concepts are redundant, yet standard CBM approaches treat them as independent and equally important, leading to inefficient intervention. A natural solution is to exploit the information-theoretic structure of concept sets. Organizing concepts into a nested hierarchy based on Maximum Relevance and Minimum Redundancy (mRMR) concentrates discriminative information in early concept dimensions. This organization aligns with Matryoshka Representation Learning, where representations are ordered by decreasing importance within a single model, offering a principled route to efficient and adaptive intervention. To operationalize this insight, we introduce the Matryoshka Concept Bottleneck Model (MCBM) . As illustrated in Figure 1 , MCBM is a unified architecture designed for flexible and cost-efficient deployment. Unlike existing methods that require training separate models for different concept budgets, our framework is trained once and supports adaptive inference across a wide range of intervention constraints. Specifically, we employ the mRMR criterion to construct a hierarchical ordering of concepts, ensuring that the most informative and least redundant concepts are prioritized. The model is then optimized using a nested objective function that enforces predictive accuracy at multiple levels of conceptual granularity simultaneously. Consequently, MCBM functions as a plug-and-play module, allowing practitioners to dynamically trade off annotation cost and predictive performance without retraining. Our main contributions are summarized as follows: • Methodology: We introduce MCBM, a unified architecture that integrates mRMR-based concept ordering with Matryoshka Representation Learning, enabling elastic deployment and efficient intervention from a single training run. • Theoretical Analysis: We show that the proposed nested, importance-based structure reduces the expected intervention cost from linear O(K) to logarithmic O(\log K) under standard information decay assumptions. • Experiments: We empirically validate MCBM on the CUB, LAD, and CelebA datasets, demonstrating performance comparable to independently trained baselines with substantially reduced intervention effort. Figure 1: Matryoshka Concept Bottleneck Models Architecture. The input image is encoded into raw logits, which are then permuted based on the pre-computed mRMR ranking. This yields an ordered concept vector where information density is concentrated at the beginning. Multiple parallel heads (Matryoshka Heads) then perform classification using nested prefixes of this ordered vector. 2 Related Work Concept Bottleneck Models and the Rigid Cost Paradigm. Concept Bottleneck Models (CBM) 18 have established themselves as a cornerstone of interpretable deep learning, particularly in high-stakes domains like healthcare 1 ; 36 and dermatology 8 , where distinguishing specific attributes is critical for trust. The core appeal of CBM lies in their ability to support test-time intervention 29 , allowing human experts to rectify mispredictions. Recent work improves CBM along orthogonal axes such as robustness, label-efficiency, faithfulness, and concept-set completeness 12 ; 24 ; 37 ; 38 ; 21 ; 3 . However, despite these advances in model quality, existing approaches fundamentally neglect the inference efficiency of the bottleneck itself. Whether supervised or label-free 24 ; 35 , these models structure concepts as a flat, unordered bottleneck , imposing a rigid O(K) verification cost on the user regardless of the input’s difficulty. Our Matryoshka CBM (MCBM) addresses this overlooked dimension by introducing a nested hierarchy, enabling efficient, logarithmic-cost interventions without sacrificing the robustness or faithfulness gains of prior arts. Sparse, Hybrid, and Adaptive CBM. To mitigate the computational and cognitive overhead of CBM, a second line of work introduces sparsity or hybrid mechanisms. Hybrid CBM combine concept bottlenecks with standard black-box branches to recover performance 2 ; 12 , but often at the cost of full interpretability. Sparse CBM utilize regularization techniques (e.g., L_{1} penalties, Gumbel-Softmax) to dynamically select a small subset of active concepts per input 28 ; 32 . Similarly, Adaptive CBM attempt to adjust the bottleneck capacity to handle distribution shifts 7 . However, these methods typically result in unstructured sparsity where the model activates a disparate set of concepts for each individual sample. This lack of a globally consistent ordering prevents systematic resource truncation (e.g., for deployment on edge devices with fixed memory budgets). Furthermore, they lack a theoretical guarantee for intervention efficiency. In contrast, MCBM enforces a globally consistent, importance-based ordering via mRMR, enabling predictable plug-and-play deployment and theoretically bounded intervention costs. Causal and Stochastic CBM. Causal and stochastic CBMs model dependencies among concepts to improve intervention propagation and uncertainty-aware reasoning 30 ; 9 . This direction is complementary to ours. Such models answer how a correction should propagate through a concept graph, while MCBM answers which concepts should be queried first under a strict human or hardware budget 27 ; 11

Read the complete text