Mixtures of Transparent Local Models

Niffa Cheick Oumar Diaby [email protected], Thierry Duchesne [email protected], Mario Marchand [email protected] · 2026

Read Mixtures of Transparent Local Models by Niffa Cheick Oumar Diaby [email protected], Thierry Duchesne [email protected], Mario Marchand [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10541.txt.

Introduction

Niffa Cheick Oumar Diaby, Thierry Duchesne, and Mario Marchand tackle a trust problem in machine learning. Models now guide choices in many parts of daily life. Users want to see why a model gives the answer it gives. Clear models also help check for safety and for unfair bias.

The team proposes a blend of simple local models, each one easy to read on its own. Near any given case, the blend acts like one small, clear model. Far away, other local parts take over. The design aims to keep the whole system open to inspection. It offers a path to predictions that people can audit step by step.

Catalog record

CREATORNiffa Cheick Oumar Diaby [email protected], Thierry Duchesne [email protected], Mario Marchand [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE119 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-10541

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Mixtures of Transparent Local Models Niffa Cheick Oumar Diaby [email protected], Thierry Duchesne [email protected], Mario Marchand [email protected] Abstract The predominance of machine learning models in many spheres of human activity has led to a growing demand for their transparency. The transparency of models makes it possible to discern some factors, such as security or non-discrimination. In this paper, we propose a mixture of transparent local models as an alternative solution for designing interpretable (or transparent) models. Our approach is designed for the situations where a simple and transparent function is suitable for modeling the label of instances in some localities/regions of the input space, but may change abruptly as we move from one locality to another. Consequently, the proposed algorithm is to learn both the transparent labeling function and the locality of the input space where the labeling function achieves a small risk in its assigned locality. By using a new multi-predictor (and multi-locality) loss function, we established rigorous PAC-Bayesian risk bounds for the case of binary linear classification problem and that of linear regression. In both cases, synthetic data sets were used to illustrate how the learning algorithms work. The results obtained from real data sets highlight the competitiveness of our approach compared to other existing methods as well as certain opaque models. Author One and Author Two † † heading: 23 2022 1- 1/21; Revised 5/22 9/22 21-0000 † † shortheadings: Mixtures of Transparent Local Models / Diaby and Duchesne and Marchand † † firstpage: 1 † † editor: My editor keywords PAC-Bayes, risk bounds, local models, transparent models, mixtures of local transparent models. 1 Introduction Interpretable machine learning models are important for fairness testing, causality assessment and reliability or robustness screening. This is especially true in high-stakes sectors such as finance, transportation and health care. Consequently, several state laws have emerged to regulate the use of machine learning models and enforce some level of interpretability and transparancy such as Law 25 of the Province of Quebec ( 30 ) , Article 22 of the General Data Protection Regulation (GDPR) of the European Union ( 32 ) , and other similar laws in the United States ( 36 ) . Following these growing attentions, two main strategies have been developed to address transparency and explainability issues. The first is the use of intrinsic interpretable model design techniques. Intrinsic interpretable models are prediction models whose decision-making process is understandable by humans without the need of any additional tool or algorithm. Example of models that belong to this family are generalized linear models (GLMs) (linear regression, log-linear model, etc.) ( 28 ; 27 ) , generalised additive models (GAMs) ( 18 ) , decision trees ( 7 ; 35 ) , case-based reasoning ( 20 ; 40 ) , rule sets ( 26 ; 48 ) and rule lists ( 5 ; 34 ) . However, while the way in which transparent models make decisions is clear and fully understood by humans, the complexity of these types of models is usually restricted to improve their interpretability. The second solution is to resort to methods for explaining complex models, also called black box models. These methods usually rely on transparent models to explain locally or globally non-transparent models. They include, in particular, instruments such as TREPAN ( 9 ) , LIME ( 37 ) , SHAP ( 23 ) , BETA ( 21 ) , Anchors ( 38 ) or MUSE ( 22 ) . Unfortunately, the explanations of these models in post-hoc fashion have proved to be unreliable and highly manipulatable ( 1 ; 11 ; 2 ) . In this paper, we therefore propose to focus on alternative solutions for designing interpretable models. Our solution is based on the use of multiple model techniques to design a mixture of transparent local models . Succinctly, a mixture of transparent local models divides the problem space into several subspaces in order to learn interpretable predictors (transparent local models) in the different regions. Instead of designing a single complex model, which is potentially difficult to interpret, it might be more interesting to build simple and interpretable local models around certain points of interest. A point of interest can be an instance for which we want to predict the label based on similar instances present in the training samples. Before moving into the theory of mixture of transparent local models , let us highlight why this approach could be better than learning a complex global model. Let {\mathcal{D}} be some unknown data-generating distribution over {\mathcal{X}}\times{\mathcal{Y}} , where {\mathcal{X}} denotes the input space and {\mathcal{Y}} the label space. Similarly, {\mathcal{D}}^{{}^{\prime}} is a data-generating distribution over {\mathcal{X}}^{{}^{\prime}}\times{\mathcal{Y}}^{{}^{\prime}} , where {\mathcal{X}}^{{}^{\prime}} and {\mathcal{Y}}^{{}^{\prime}} designate, respectively, the input space and the label space. Consider two sets of training samples S\overset{{\rm\mbox{\tiny def}}}{=}\{({\mathbf{x}}_{i},y_{i})\}_{i=1}^{m} and S^{{}^{\prime}}\overset{{\rm\mbox{\tiny def}}}{=}\{({\mathbf{x}}_{i}^{{}^{\prime}},y_{i}^{{}^{\prime}})\}_{i=1}^{m} of size m , where each example ({\mathbf{x}}_{i},y_{i})\in{\mathcal{X}}\times{\mathcal{Y}} and ({\mathbf{x}}_{i}^{{}^{\prime}},y_{i}^{{}^{\prime}})\in{\mathcal{X}}^{{}^{\prime}}\times{\mathcal{Y}}^{{}^{\prime}} . Suppose we would like to build a predictive model on each data set S and S^{{}^{\prime}} , having access to a class of polynomial functions \mathcal{H}_{k} of order k . (a) Simple global model. (b) Complex global model. Figure 1 : Example of global models. The global model learned from the data set S , is simpler and has an acceptable level of performance. However, the global model learned from the data set S^{{}^{\prime}} is much more complex, requiring a polynomial of high order k . Thus, Figure 1 illustrates that, on certain tasks, learning a global model might force it to be very complex and therefore difficult to interpret . Now, let us reformulate the predictive modelling problem on the data set S{{}^{\prime}} as seven sub-problems. Each sub-problem consists of learning a simple transparent local model on a part of the data set situated in a specific locality, particularly around the vicinity of a point of interest, as shown in Figure 2 . Figure 2 : Example of mixture of transparent local models. In this case, we find for each sub problem, that a reasonable level of performance could be achieved with a polynomial of degree 0 or 1 . Therefore, by combining these seven simple local models, we could gain an additional gain on the performance level assessed across all examples in the data set S{{}^{\prime}} . In other words, a low risk could be achieved with the use of local models that can be interpreted . Related Work: Multiple model techniques in interpretable machine learning involve combining two or more models to increase performance gain or interpretability. Here are some works based on this approach. 45 proposed Oblique Treed Sparse Additive Models (OT-SpAMs) which divide the input space into regions with sparse oblique tree splitting and assign local sparse additive experts to individual regions. Other works are Hybrid Interpretable Models ( 31 ; 46 ; 15 ) , which are systems that involve the cooperation of a transparent model and a complex black box. At inference time, any input of the hybrid model is assigned to either its interpretable or complex component based on a gating mechanism. 43 proposed The League of Experts. They adapt a particular case of ensemble learning, in which each ensemble member is associated with a specific part of the input space. More specifically, they introduce

Read the complete text