A Path to Simpler Models Starts With Noise

Abstract

The Rashomon set is the set of models that perform approximately equally well on a given dataset, and the Rashomon ratio is the fraction of all models in a given hypothesis space that are in the Rashomon set. Rashomon ratios are often large for tabular datasets in criminal justice, healthcare, lending, education, and in other areas, which has practical implications about whether simpler models can attain the same level of accuracy as more complex models. An open question is why Rashomon ratios often tend to be large. In this work, we propose and study a mechanism of the data generation process, coupled with choices usually made by the analyst during the learning process, that determines the size of the Rashomon ratio. Specifically, we demonstrate that noisier datasets lead to larger Rashomon ratios through the way that practitioners train models. Additionally, we introduce a measure called pattern diversity, which captures the average difference in predictions between distinct classification patterns in the Rashomon set, and motivate why it tends to increase with label noise. Our results explain a key aspect of why simpler models often tend to perform as well as black box models on complex, noisier datasets.

1. Introduction

It is possible that for many datasets, a simple predictive model can perform as well as the best black box model – we simply have not found the simple model yet. Interestingly, however, we may be able to infer whether such a simple model exists without finding it first, and we may be able to determine conditions under which such simple models are likely to exist.

We already know that many datasets exhibit the “Rashomon effect” [7], which is that for many real-world tabular datasets, many models can describe the data equally well. If there are many good models, it is more likely that at least one of them is simple (e.g., sparse) [39]. Thus, a key question in determining the existence of simpler models is to understand why and when the Rashomon effect happens. This is a difficult question, and there has been little study of it. The literature on the Rashomon effect has generally been more practical, showing either that the Rashomon effect often exists in practice [39, 13, 45], showing how to compute or visualize the set of good models for a given dataset [52, 14, 15, 1, 32, 53, 54, 48], or trying to reduce underspecification by learning a diverse ensemble of models [25, 37]. However, no prior works have focused on understanding what causes this phenomenon in the first place.

Our thesis is that noise is both a theoretical and practical motivator for the adoption of simpler models. Specifically, in this work, we refer to noise in the generation process that determines the labels. In noisy problems, the label is more difficult to predict. Data about humans, such as medical data or criminal justice data, are often noisy because many things worth predicting (such as whether someone will commit a crime within 2 years of release from prison, or whether someone will experience a medical condition within the next year) have inherent randomness that is tied to random processes in the world (Will the person get a new job? How will their genetics interact with their diet?). It might sound intuitive that noisy data would lead to simpler models being useful, but this is not something most machine learning practitioners have internalized – even on noisy datasets, they often use complicated, black box models, to which post-hoc explanations are added. Our work shows how practitioners who understand the bias-variance trade-off naturally gravitate towards more interpretable modes in the presence of noise.

We propose a path which begins with noise, is followed by decisions made by human analysts to compensate for that noise, and that ultimately leads to simpler models. In more detail, our path follows these steps: 1) Noise in the world leads to increased variance of the labels. 2) Higher label variance leads to worse generalization (larger differences between training and test/validation performance). 3) Poor generalization from the training set to the validation set is detected by analysts on the dataset using techniques such as cross-validation. As a result, the analyst compensates for anticipated poor test performance in a way that follows statistical learning theory. Specifically, they choose a simpler hypothesis space, either through soft constraints (i.e., increasing regulation), hard constraints (explicit limits on model complexity, or model sparsification), or by switching to a simpler function class. Here, the analyst may lose performance on the training set but gain validation and test performance. 4) After reducing the complexity of the hypothesis space, the analyst’s new hypothesis space has a larger Rashomon ratio than their original hypothesis space. The Rashomon ratio is the fraction of models in the function class that perform close to the empirical risk minimizer. It is the fraction of functions that performs approximately-equally-well to the best one. This set of “good” functions is called the Rashomon set, and the Rashomon ratio measures the size of the Rashomon set relative to the function class. This argument (that lower complexity function classes leads to larger Rashomon ratios) is not necessarily intuitive, but we show it empirically for 19 datasets. Additionally, we prove this holds for decision trees of various depths under natural assumptions. The argument boils down to showing that the set of non-Rashomon set models grows exponentially faster than the set of models inside the Rashomon set. As a result, since the analyst’s hypothesis space now has a large Rashomon ratio, a relatively large fraction of models that are left in the simpler hypothesis are good, meaning they perform approximately as well as the best models in that hypothesis space. From that large set, the analyst may be able to find even a simpler model from a smaller space that also performs well, following the argument of Semenova et al. [39]. As a reminder, in Step 3 the analysts discovered that using a simpler model class improves test performance. This means that these simple models attain test performance that is at least that of the more complex (often black box) models from the larger function class they used initially.

In this work, we provide the mathematics and empirical evidence needed to establish this path, focusing on Steps 1, 2, and 4 because Step 3 follows directly (however, we provide empirical evidence for Step 3 as well). Moreover, for the case of ridge regression with additive attribute noise, we prove directly that adding noise to the dataset results in an increased Rashomon ratio. Specifically, the additive noise acts as ${\ell }_2$-regularization, thus it reduces the complexity of the hypothesis space (Step 3) and causes the Rashomon ratio to grow (Step 4).

Even if the analyst does not reduce the hypothesis space in Step 3, noise still gives us larger Rashomon sets. We show this by introducing pattern diversity, the average Hamming distance between all classification patterns produced by models in the Rashomon set. We show that under increased label noise, the pattern diversity tends to increase, which implies that when there is more noise, there are more differences in model predictions, and thus, there could be more models in the Rashomon set. Hence, a much shorter version of the path also works: Noise in the world causes an increase in pattern diversity, which means there are more diverse models in the Rashomon set, including simple ones.

It is becoming increasingly common to demand interpretable models for high-stakes decision domains (criminal justice, healthcare, etc.) for policy reasons such as fairness or transparency. Our work is possibly the first to show that the noise inherent in many such domains leads to technical justifications for demanding such models.

2. Related Work

Rashomon set.

The Rashomon set, named after the Rashomon effect coined by Leo Breiman [7], is based on the observation that often there are many equally good explanations of the data. When these are contradictory, the Rashomon effect gives rise to predictive multiplicity [30, 6, 18]. Rashomon sets have been used to study variable importance [15, 14, 42], for characterizing fairness [40, 9, 2], to improve robustness and generalization, especially under distributional shifts [37, 25], to study connections between multiplicity and counterfactual explanations [36, 53, 8], and to help in robust decision making [46]. Some works focused on trying to compute the Rashomon set for specific hypothesis spaces, such as sparse decision trees [52], generalized additive models [54], and decision lists [32]. Other works focus on near-optimality to find a diverse set of solutions to mixed integer problems [1], a set of targeted predictions under a Bayesian model [23], or estimate the Rashomon volume via approximating model in Reproducing Kernel Hilbert Space [31]. Black et al. [6] shows that the predictive multiplicity metric defined as expected pairwise disagreement increases with expected variance over the models in the Rashomon set. On the contrary, we focus on probabilistic variance in labels in the presence of noise.

Metrics of the Rashomon set.

To characterize the Rashomon set, multiple metrics have been proposed [39, 38, 30, 49, 18, 6]. The Rashomon ratio [39], and the pattern Rashomon ratio [38] measure the Rashomon set as a fraction of models or predictions within the hypothesis space; ambiguity and discrepancy [30, 49] indicate the number of samples that received conflicting estimates from models in the Rashomon set; Rashomon capacity [18] measures the Rashomon set for probabilistic outputs. Here, we focus on the Rashomon ratio and pattern Rashomon ratio. We also introduce pattern diversity. Pattern diversity is close to expected pairwise disagreement (as in Black et al. [6]), however, it uses unique classification patterns (see Appendix G).

Learning with noise.

Learning with noisy labels has been extensively studied [34], especially for linear regression [5] and, more recently, for neural networks [44] to understand and model effects of noise. Stochastic gradient descent with label noise acts as an implicit regularizer [12] and noise has been added to hidden units [35], labels [41], or covariances [50] to prevent overfitting in deep learning. When the labels are noisy, constructing robust loss [16], adding a slack variable for each training sample [19], or early stopping [27] also helps to improve generalization. In this work, we study why simpler models are often suitable for noisier datasets from the perspective of the Rashomon effect.

3. Notation and Definitions

Consider a training set of $n$ data points $S=\left\{z_1,z_2,\dots ,z_n\right\}$, such that each $z_i=\left(x_i,y_i\right)$ is drawn i.i.d. from an unknown distribution $𝒟$, where $𝒳\subset {\mathbb{R}}^m$, and we have binary labels $𝒴\in \{-1,1\}$. Denote $ℱ$ as a hypothesis space, where $f\in ℱ$ obeys $f:𝒳\to 𝒴$. Let $\varphi :𝒴\times 𝒴\to {\mathbb{R}}^{+}$ be a 0–1 loss function, where for point $z=(x,y)$ and hypothesis $f$, the loss function is $\varphi \left(f\right(x),y)=1_{\left[f\right(x)\ne y]}$. Finally, let $\hat{L}\left(f\right)$ be an empirical risk $\hat{L}\left(f\right)=\frac{1}{n}{\sum }_{i=1}^n \varphi \left(f\left(x_i\right),y_i\right)$, and let $\hat{f}$ be an empirical risk minimizer: $\hat{f}\in \mathrm{a}\mathrm{r}\mathrm{g}{\mathrm{m}\mathrm{i}\mathrm{n}}_{f\in ℱ} \hat{L}\left(f\right)$. If we want to specify the dataset on which $\hat{f}$ was computed, we will indicate it by an index, ${\hat{f}}_S$.

The Rashomon set contains all models that achieve near-optimal performance and can be defined as:

Definition 1 (Rashomon set). For dataset $S$, a hypothesis space $ℱ$, and a loss function $\varphi$, given $\theta \ge 0$, the Rashomon set ${\hat{R}}_{\text{set}}(ℱ,\theta )$ is:

$${\hat{R}}_{\mathit{\text{set}}}\left(ℱ,\theta \right):=\left\{f\in ℱ:\hat{L}\left(f\right)\le \hat{L}\left(\hat{f}\right)+\theta \right\},$$

where $\hat{f}$ is an empirical risk minimizer for the training data $S$ with respect to loss function $\varphi :\hat{f}\in \mathrm{a}\mathrm{r}\mathrm{g}{\mathrm{m}\mathrm{i}\mathrm{n}}_{f\in ℱ} \hat{L}\left(f\right)$, and $\theta$ is the Rashomon parameter.

Rashomon parameter $\theta$ determines the risk threshold $\left(\hat{L}\right(\hat{f})+\theta )$, such that all models with risk lower than this threshold are inside the set. For instance, if we stay within 1% of the accuracy of the best model, then $\theta =0.01$. Given parameter $\gamma >0$, we extend the definition to the true risk by defining the true Rashomon set, containing all models with a bound on true risk $R_{\mathit{\text{set}}}(ℱ,\gamma )=\{f\in ℱ:\left.L\left(f\right)\le L\left(f^{*}\right)+\gamma \right\}$, where $f^{*}$ is optimal model, $f^{*}=\mathrm{a}\mathrm{r}\mathrm{g}{\mathrm{m}\mathrm{i}\mathrm{n}}_{f\in ℱ} L\left(f\right)$.

In this work, we study how noise influences the Rashomon set and choices that practitioners make in the presence of noise. We measure the Rashomon set in different ways, including the Rashomon ratio, pattern Rashomon ratio, and pattern diversity (defined in Section 6). For a discrete hypothesis space, the Rashomon ratio [39] is the ratio of the number of models in the Rashomon set compared to the hypothesis space, ${\hat{R}}_{\mathit{\text{ratio}}}(ℱ,\theta )=\frac{\left|{\hat{R}}_{\mathit{\text{set}}}(ℱ,\theta )\right|}{\left|ℱ\right|}$, where |·| denotes cardinality. It is possible to weight the hypothesis space by a prior to define a weighted Rashomon ratio if desired.

Given a hypothesis $f$ and a dataset $S$, a predictive pattern (or pattern) $p$ is the collection of outcomes from applying $f$ to each sample from $S:p^f=\left[f\left(x_1\right),\dots ,f\left(x_i\right),\dots ,f\left(x_n\right)\right]$. We say that pattern $p$ is achievable on the Rashomon set if there exists $f\in {\hat{R}}_{\mathit{\text{set}}}(ℱ,\theta )$ such that $p^f=p$. Let pattern Rashomon set $\pi (ℱ,\theta )=\left\{p^f:f\in {\hat{R}}_{\mathit{\text{set}}}(ℱ,\theta ),p^f={\left[f\left(x_i\right)\right]}_{i=1}^n\right\}$ be all unique patterns achievable by functions from ${\hat{R}}_{\mathit{\text{set}}}(ℱ,\theta )$ on dataset $S$. Finally, let $\mathrm{\Psi }\left(ℱ\right)$ be the pattern hypothesis set, meaning that it contains all patterns achievable by models in the hypothesis space, $\mathrm{\Psi }\left(ℱ\right)=\left\{p^f:f\in ℱ,p^f=\right.\left.{\left[f\left(x_i\right)\right]}_{i=1}^n\right\}$. The pattern Rashomon ratio is the ratio of patterns in the pattern Rashomon set to the pattern hypothesis set: ${\hat{R}}_{\mathit{\text{ratio}}}^{\mathit{\text{pat}}}(ℱ,\theta )=\frac{\left|\pi \right(ℱ,\theta \left)\right|}{\left|\mathrm{\Psi }\right(ℱ\left)\right|}$.

In the following sections, we walk along the steps of our proposed path. Rather than trying to prove these points for every possible situation (which would be volumes beyond what we can handle here), we aim to find at least some way to illustrate that each step is reasonable in a natural setting.

4. Increase in Variance due to Noise Leads to Larger Rashomon Ratios

4.1. Step 1. Noise Increases Variance

One would think that something as simple as uniform label noise would not really affect anything in the learning process. In fact, we would expect that adding such noise would just uniformly increase the losses of all functions, and the Rashomon set would stay the same. However, this conclusion is (surprisingly) not true. Instead, noise adds variance to the loss, which, in turn, prevents us from generalizing.

For infinite data distribution $𝒟$ consider uniform label noise, where each label is flipped independently with probability $\rho <\frac{1}{2}$. If $\tilde{y}$ is a flipped label, $P(\tilde{y}\ne y)=\rho$. If the empirical risk of $f$ is over $\frac{1}{2}$ after adding noise, we transform $f$ to $-f$. For a given model $f\in ℱ$ let ${\sigma }^2(f,𝒟)$ be the variance of the loss, meaning that ${\sigma }^2(f,𝒟)={\mathrm{V}\mathrm{a}\mathrm{r}}_{z~𝒟}l(f,z)$. We show in the following theorem that, for a given $f\in ℱ$, label noise increases the variance of the loss.

Theorem 2 (Variance increases with label noise). Consider infinite true data distribution $𝒟$, and uniform label noise, where each label is flipped independently with probability $\rho$. Let ${𝒟}_{\rho }$ denote the noisy version of $𝒟$. Consider 0–1 loss $l$, and assume that there exists at least one function $\bar{f}\in ℱ$ such that $L_{𝒟}\left(\bar{f}\right)<\frac{1}{2}-\gamma$. For a fixed $f\in ℱ$, let ${\sigma }^2\left(f,{𝒟}_{\rho }\right)$ be the variance of the loss, ${\sigma }^2\left(f,{𝒟}_{\rho }\right)={Var}_{z~{𝒟}_{\rho }}l(f,z)$ on data distribution ${𝒟}_{\rho }$. For any $0<{\rho }_1<{\rho }_2<\frac{1}{2}$,

$${\sigma }^2\left(f,{𝒟}_{{\rho }_1}\right)<{\sigma }^2\left(f,{𝒟}_{{\rho }_2}\right).$$

The proof of Theorem 2 is in Appendix A This covers the uniform noise case, but variance increases more generally, and we prove this for several other common cases in Appendix A. More specifically, we show that the variance increases with other types of label noise, such as non-uniform label noise (see Theorem 12 in Appendix A) and margin noise (see Theorem 15 in Appendix A). For non-uniform label noise, for a sample $z=(x,y)$, each label $y$ is flipped independently with probability ${\rho }_x$, meaning that noise can depend on $x$. This noise model is more realistic than uniform label noise and allows modeling of cases when one sub-population has much more noise than another. We model margin noise such as that which arises from high-dimensional Gaussians. Because of the central limit theorem, data often follow Gaussian distributions, therefore this noise is realistic and models mistakes near decision boundary. Label noise in datasets is common. In fact, real-world datasets reportedly have between 8.0% and 38.5% label noise [44, 43, 24, 28, 51]. We hypothesize that a significant amount of label noise in real-world datasets is a combination of Gaussian (due to the central limit theorem) and random noise (for example, because of clerical errors causing label noise).

For the true Rashomon set $R_{\mathit{\text{set}}}(ℱ,\gamma )$, we consider the maximum variance for all models in the true set: ${\sigma }^2={\mathrm{s}\mathrm{u}\mathrm{p}}_{f\in R_{set}(ℱ,\gamma )} {\mathrm{V}\mathrm{a}\mathrm{r}}_{z~𝒟}l(f,z)$. Then, from Theorem 2 we have that maximum expected variance over the Rashomon set increases with noise.

Corollary 3 (Maximum variance increases with label noise). Under the same assumptions as in Theorem 2, we have that

$$\underset{f\in R_{{\mathit{\text{set}}}_{{𝒟}_{{\rho }_1}}}(ℱ,\gamma )}{\mathrm{s}\mathrm{u}\mathrm{p}} {\sigma }^2\left(f,{𝒟}_{{\rho }_1}\right)<\underset{f\in R_{{\mathit{\text{set}}}_{{𝒟}_{{\rho }_2}}}(ℱ,\gamma )}{\mathrm{s}\mathrm{u}\mathrm{p}} {\sigma }^2\left(f,{𝒟}_{{\rho }_2}\right).$$

The next step is to show that this increased maximum variance leads to worse generalization.

4.2. Step 2. Higher Variance Leads to Worse Generalization

Here we use an argument based on generalization bounds. Generalization bounds have been the key theoretical motivation for much of machine learning, including support vector machines (SVMs), because the margin that SVMs optimize appears in a bound. While bounds themselves are not directly used in practice, the terms in the bounds tend to be important quantities in practice. Our bound cannot be calculated in practice because it uses population information on the right side, but it still provides insight and motivation.

Unlike standard bounds, we will use the fact that the user is using empirical risk minimization, and cross-validation to assess overfitting. Thus, for $\hat{f}$, there are two possibilities: $\hat{f}$ is in the true Rashomon set, or it is not. If it is not, then for the empirical risk minimizer $\hat{f}$, the difference between the true and the empirical risk must be at least $\gamma$, which will be detected with high probability in cross-validation [20, 33]. In that case, the user will reduce their hypothesis space and we move to Step 3. If $\hat{f}$ is in the true Rashomon set, it obeys the following bound.

Theorem 4 (Variance-based “generalization bound”). Consider dataset $S$, 0–1 loss $l$, and finite hypothesis space $ℱ$. With probability at least $1-\delta$, we have that for every $f\in R_{\mathit{\text{set}}}(ℱ,\gamma )$:

$$L\left(f\right)-\hat{L}\left(f\right)\le \frac{2}{3n}\text{log}\left(\frac{\left|R_{\mathit{\text{set}}}\left(ℱ,\gamma \right)\right|}{\delta }\right)+\sqrt{\frac{2{\sigma }^2}{n}\text{log}\left(\frac{\left|R_{\mathit{\text{set}}}\left(ℱ,\gamma \right)\right|}{\delta }\right)},$$ (1)

where ${\sigma }^2={\mathrm{s}\mathrm{u}\mathrm{p}}_{f\in R_{set}(ℱ,\gamma )} {\mathrm{V}\mathrm{a}\mathrm{r}}_{z~𝒟}l(f,z)$, and $n$ is number of samples in $S={\left\{z_i\right\}}_{i=1}^n~𝒟$.

The proof of Theorem 4 is in Appendix C. Note that generalization bounds are usually based on Hoeffding’s inequality (see Lemma 17), which is a special case of Bernstein’s inequality (see Lemma 16). In fact, we show in Appendix B that Bernstein’s inequality, which we used to prove Theorem 4, can be sharper than Hoeffding’s when the variance is less than ${\sigma }_f^2<\frac{1}{12}$ for a given $f$. Theorem 4 is easily generalized to continuous hypothesis spaces through a covering argument over the true Rashomon set (as an example, see Theorem 20), where the complexity is measured as the size of the cover over the true Rashomon set instead of the number of models in the true Rashomon set.

Let $c(ℱ,n)=\frac{2}{3n}\text{log}\left(\frac{\left|R_{\mathit{\text{set}}}(ℱ,\gamma )\right|}{\delta }\right)$, which is the first term in the bound (1) in Theorem 4. According to Theorem 8 in Semenova et al. [39] under random label noise, the true Rashomon set does not decrease in size. Therefore, $c(ℱ,n)$ at least does not decrease with more noise as it depends only on complexity. However, the second term $\sqrt{3{\sigma }^{2}c(ℱ,n)}$ depends on the maximum loss variance, which increases with label noise, as motivated in the previous section. This means with more noise in the labels, we would expect worse generalization, which would generally lead practitioners who are using a validation set to reduce the complexity of the hypothesis space.

As discussed earlier, we use cross-validation to assess whether $\hat{f}$ overfits. From Proposition 5 (proved in Appendix D) we infer that if the empirical risk minimizer (ERM) does not highly overfit, it has a high chance to be in the true Rashomon set and thus Theorem 4 applies.

Proposition 5 (ERM can be close to the true Rashomon set). Assume that through the cross-validation process, we can assess $\xi$ such that $L\left(\hat{f}\right)-\hat{L}\left(\hat{f}\right)\le \xi$ with high probability (at least $1-{ϵ}_{\xi }$) with respect to the random draw of data. Then, for any $ϵ>0$, with probability at least $1-e^{-2n{ϵ}^2}-{ϵ}_{\xi }$ with respect to the random draw of training data, when $\xi +ϵ\le \gamma$, then $\hat{f}\in R_{\mathit{\text{set}}}(ℱ,\gamma )$.

4.3. Step 3. Practitioner Chooses a Simpler Hypothesis Space

We have shown earlier that noisier datasets lead to higher variance and worse generalization. The question we consider here is whether one can see the results of these bounds in practice and would actually reduce the hypothesis space. For example, consider four real-world datasets and the hypothesis space of decision trees of various depths. In Figure 1 (a) we show that as label noise increases, so does the gap between risks (1– accuracy) evaluated on the training set and on a hold out dataset for a fixed depth tree, and thus, as a result, during the validation process, the smaller depth would be chosen by a reasonable analyst. We simulate this by using cross validation to select the optimal tree depth in CART, as shown in Figure 1 (b). As predicted, the optimal tree depth decreases as noise increases. We also show that a similar trend happens for the gradient boosted trees in Figure 7 in Appendix K.2. More specifically, with more noise, the best number of estimators, as chosen based on cross-validation, decreases. We describe the setup in detail in Appendix K.2.

Figure 1:

Practitioner’s validation process in the presence of noise for CART. For a fixed tree depth, as we add noise, the gap between training and validation accuracy increases (Subfigure a). As we use cross validation to select tree depth, the best tree depth decreases with noise (Subfigure b).

4.4. Step 4. Rashomon Ratio is Larger for Simpler Spaces

For simpler hypothesis spaces, it may not be immediately obvious that the Rashomon ratio is larger, i.e., a larger fraction of a simpler model class consists of “good” models. Intuitively, this is because the denominator of the ratio (the total number of models) increases faster than the numerator (the number of good models) as the complexity of the model class increases. As we will see, this is because the good models in the simpler model class tend to give rise to more bad models in the more complex class, and the bad models do not tend to give rise to good models as much. We explore two popular model classes: decision trees and linear models. The results thus extend to forests (collections of trees) and generalized additive models (which are linear models in enhanced feature space).

Consider data that live on a hypercube (e.g., the data have been binarized, which is a common pre-processing step [29, 3, 52, 47]) and a hypothesis space of fully grown trees (complete trees with the last level also filled) of a given depth $d$. Denote this hypothesis space as ${ℱ}_d$. For example, a depth 1 tree has 1 node and 2 leaves. Under natural assumptions on the quality of features of classifiers and the purity of the leaves of trees in the Rashomon set, we show that the Rashomon ratio is larger for hypothesis spaces of smaller-depth trees.

Proposition 6 (Rashomon ratio is larger for decision trees of smaller depth). For a dataset $S=X\times Y$ with binary feature matrix $X\in \{0,1{\}}^{n\times m}$, consider a hypothesis space ${ℱ}_d$ of fully grown trees of depth $d$. Let the number of dimensions $m<2^{2^d}$. Assume: (Leaves are correct) all leaves in all trees in the Rashomon set have at least $\lceil \theta n\rceil$ more correctly classified points than incorrectly classified points; (Bad features) there is a set of $m_{bad}\ge d$ “bad” features such that the empirical risk minimizer of models using only the bad features is not in the Rashomon set. Then ${\hat{R}}_{\mathit{\text{ratio}}}\left({ℱ}_{d+1},\theta \right)<{\hat{R}}_{\mathit{\text{ratio}}}\left({ℱ}_d,\theta \right)$.

The proof of Proposition 6 is in Appendix E. Both assumptions are typically satisfied in practice.

To demonstrate our point, we computed the Rashomon ratio and pattern Rashomon ratio for 19 different datasets for hypothesis spaces of decision trees and linear models of different complexity (see Figure 2). As the complexity of the hypothesis space increases, we see an obvious decrease in the Rashomon ratio and pattern Rashomon ratio.

Figure 2:

Calculation showing that the Rashomon ratio (a) and pattern Rashomon ratio (b) decrease for the hypothesis space of decision trees of fixed depth from 1 to 7 for 14 different datasets (a) and for the hypothesis space of linear models of sparsity from 1 to 4 for 6 different datasets (b). Each line represents a different dataset, each dot represents the log of the Rashomon ratio or pattern Rashomon ratio. Both ratios decrease as we move to a more complex hypothesis space.

To compute the Rashomon ratio, for trees, we use TreeFARMS [52], which allows us to enumerate the whole Rashomon set for sparse trees. For linear models, we design a two-step approach that allows us to compute all patterns in the Rashomon set. First, for every sample $z_i$ we solve a simple optimization problem that checks if there exists a model in the Rashomon set that misclassifies this sample. If there is no such model, changing the label of $z_i$ will not add more patterns to the Rashomon set, therefore, we can ignore $z_i$. In the second step, we grow a search tree and bound all paths that will lead to patterns outside of the Rashomon set. We describe our approach for pattern computation in detail in Appendix K.3 and discuss the experimental setup in Appendix K.4.

Completion of the Path.

After reducing the complexity of the hypothesis space in Step 3, the practitioner finds a larger Rashomon ratio for the newly chosen lower complexity hypothesis space according to Step 4. As a reminder of the thesis of Semenova et al. [39], with large Rashomon ratios, there are many good models, among which may exist even simpler models that perform well. Thus, the path, starting from noise, is a powerful way to explain what we see in practice, which is that simple models often perform well [17].

Note that to follow the identified path, the machine learning practitioner does not need to know the exact amount of noise. As long as they suspect some noise present in the dataset, the results of this paper apply, and the practitioner would expect a good performance from simpler models.

5. Rashomon Ratio for Ridge Regression Increases under Additive Attribute Noise

For linear regression, adding multiplicative or additive noise to the training data is known to be equivalent to regularizing the model parameters [5]. Moreover, the more noise is added, the stronger this regularization is. Thus, noise leads directly to Step 3 and a choice of a smaller (simpler) hypothesis space. Also, for ridge regression, the Rashomon volume (the numerator of the Rashomon ratio) can be computed directly [39] and depends on the regularization parameter. Building upon these results, we prove that noise leads to an increase of the Rashomon ratio (as in Step 4 of the path).

Given dataset $S$, the ridge regression model is learned by minimizing the penalized sum of squared errors: $\hat{L}\left(\omega \right)={\hat{L}}_{LS}\left(\omega \right)+C{\omega }^T\omega$, where ${\hat{L}}_{LS}\left(\omega \right)=\frac{1}{n}{\sum }_{i=1}^n {\left(x_i^T\omega -y_i\right)}^2$ is the least squares loss, $C$ is a regularization parameter, and $\omega \in {\mathbb{R}}^m$ is a parameter vector for a linear model $f={\omega }^{T}x$.

We will assume that there is a maximum loss value ${\hat{L}}_{\text{max}}$, such that any linear model that has higher regularized loss than ${\hat{L}}_{\text{max}}$ is not being considered within the hypothesis space. For instance, an upper bound for ${\hat{L}}_{\text{max}}$ is the value of the loss at the model that is identically $\stackrel{-}{0}$, namely $\hat{L}\left(\stackrel{-}{0}\right)=\frac{1}{n}{\sum }_i y_i^2$. Thus, for every reasonable model $f={\omega }^{T}x,f\in ℱ\equiv \hat{L}\left(\omega \right)\le {\hat{L}}_{\text{max}}$. On the other hand, the best possible value of the least squares loss is ${\hat{L}}_{LS}=0$, and therefore we get that $C{w}^{T}w\le {\hat{L}}_{\text{max}}$, or alternatively, $w^{T}w\le {\hat{L}}_{\text{max}}/C$. This defines the hypothesis space as an ${\ell }_2$-norm ball in $m$-dimensional space, the volume of which we can compute.

To measure the numerator of the Rashomon ratio we will use the Rashomon volume $𝒱\left({\hat{R}}_{\mathit{\text{set}}}(ℱ,\theta )\right)$, as defined in Semenova et al. [39]. In the case of ridge regression, the Rashomon set is an ellipsoid in $m$-dimensions, thus the Rashomon volume can be computed directly. Therefore, we have the Rashomon ratio as the ratio of the Rashomon volume to the volume of the ${\ell }_2$-norm ball that defines the hypothesis space.

Next, we show that under additive attribute noise, the Rashomon ratio increases:

Theorem 7 (Rashomon ratio increases with noise for ridge regression). Consider dataset $S=X\times Y,X$ is a non-zero matrix, and a hypothesis space of linear models $ℱ=\left\{f={\omega }^{T}x,\omega \in {\mathbb{R}}^m,{\omega }^T\omega \le \right.\left.{\hat{L}}_{\text{max}}/C\right\}$. Let ${ϵ}_i$, such that ${ϵ}_i~𝒩(\stackrel{-}{0},\lambda I)$ ($\lambda >0$, I is identity matrix), be i.i.d. noise vectors added to every sample: $x_i^{\prime }=x_i+{ϵ}_i$. Consider options ${\lambda }_1>0$ and ${\lambda }_2>0$ that control how much noise we add to the dataset. For ridge regression, if ${\lambda }_1<{\lambda }_2$, then the Rashomon ratios obey $\left.\left.{\hat{R}}_{{\mathit{\text{ratio}}}_{{\lambda }_1}}(ℱ,\theta )\right)<{\hat{R}}_{{\mathit{\text{ratio}}}_{{\lambda }_2}}(ℱ,\theta )\right)$.

The proof of Theorem 7 is in Appendix F. Note that while in Theorem 7 we directly show that adding noise to the training data is equivalent to stronger regularization leading us directly to Step 3, Steps 1 and 2 of the path identified in the previous section are automatically satisfied. We formally prove that additive noise still leads to an increase of the variance of losses for least squares loss (similar to Theorems 2, 12, and 15) in Theorem 19 in Appendix F, and we show that an increase in the maximum variance of losses leads to worse generalization bound (similar to Theorem 4) for the squared loss in Theorem 20 in Appendix F.

Returning to the Path.

For ridge regression, we have now built a direct noise-to-Rashomon-ratio argument showing that, in the presence of noise, the Rashomon ratios are larger. As before, for larger Rashomon ratios, there are multiple good models, including simpler ones that are easier to find.

6. Rashomon Set Characteristics in the Presence of Noise

Now we discuss a different mechanism for obtaining larger Rashomon sets. Suppose the practitioner knows the data are noisy. They would then expect a large Rashomon set, which we speculate in this section is explained by noise in the data. To show this, we define pattern diversity as a characteristic of the Rashomon set and show that it is likely to increase with label noise. Pattern diversity is an empirical measure of differences in patterns on the dataset. It computes the average distance between the patterns, which allows us not only to assess how large the Rashomon set is but also how diverse it is. Once the practitioner knows they have a large Rashomon set, they could hypothesize from the reasoning in Semenova et al. [39] that a simple model might perform well for their dataset.

6.1. Pattern Diversity: Definition, Properties and Upper Bound

Recall that $\pi (ℱ,\theta )$ is the set of unique classification patterns produced by the Rashomon set of $ℱ$ with the Rashomon parameter $\theta$.

Definition 8 (Pattern diversity). For Rashomon set ${\hat{R}}_{\mathit{\text{set}}}(ℱ,\theta )$ the pattern diversity div $\left({\hat{R}}_{\mathit{\text{set}}}(ℱ,\theta )\right)$ is defined as:

$$\mathit{\text{div}}\left({\hat{R}}_{set}\left(ℱ,\theta \right)\right)=\frac{1}{n\mathrm{\Pi }\mathrm{\Pi }}\sum _{\begin{array}{c}j\le \mathrm{\Pi }\\ p_j~\pi \left(ℱ,\theta \right)\end{array}} \sum _{\begin{array}{c}k\le \mathrm{\Pi }\\ p_k~\pi \left(ℱ,\theta \right)\end{array}} H\left(p_j,p_k\right),$$

where $n=\left|S\right|$, $\mathrm{\Pi }=\left|\pi \right(ℱ,\theta \left)\right|$, and $H\left(p_j,p_k\right)={\sum }_{i=1}^n {\mathbb{1}}_{\left[p_j^i\ne p_k^i\right]}$ is the Hamming distance (in our case it computes the number of samples at which predictions are different), and |·| denotes cardinality.

Pattern diversity measures pairwise differences between patterns of functions in the pattern Rashomon set. Pattern diversity is in the range [0, 1), where it is 0 if the pattern set contains one pattern or no patterns. Among different measures of the Rashomon set, the pattern diversity is the closest to the pattern Rashomon ratio [38] and expected pairwise disagreement (as in [6]). In Appendix G, we discuss similarities and differences between pattern diversity and these measures.

Given a sample $z_i$, let $a_i=\frac{1}{\mathrm{\Pi }}{\sum }_{k=1}^{\mathrm{\Pi }} {\mathbb{1}}_{\left[p_k^i=y_i\right]}$, where $p_k^i$ is $i^{\text{th}}$ index of the $k^{\text{th}}$ pattern, denote the probability with which patterns from the pattern Rashomon set classify $z_i$ correctly. We will call $a_i$ sample agreement over the pattern Rashomon set. When $a_i=1$, then all patterns agreed and correctly classified $z_i$. If $a_i=\frac{1}{2}$, only half of the models were able to correctly predict the label. As we will show, when more samples have sample agreement near $\frac{1}{2}$, we have higher pattern diversity. We can compute pattern diversity using average sample agreements instead of the Hamming distance according to the theorem below.

Theorem 9 (Pattern diversity via sample agreement). For 0–1 loss, dataset $S$, and pattern Rashomon set $\pi (ℱ,\theta )$, pattern diversity can be computed as $\mathit{\text{div}}\left({\hat{R}}_{\mathit{\text{set}}}(ℱ,\theta )\right)=\frac{2}{n}{\sum }_{i=1}^n a_i\left(1-a_i\right)$, where $a_i=\frac{1}{\mathrm{\Pi }}{\sum }_{k=1}^{\mathrm{\Pi }} {\mathbb{1}}_{\left[p_k^i=y_i\right]}$ is sample agreement over the pattern Rashomon set.

The proof of Theorem 9 is in Appendix H. In Theorem 23 in Appendix I, we show that average sample agreement (over all samples $z_i$) is inversely proportional to the average loss of the patterns from the pattern Rashomon set. Using this intuition, we can upper bound the pattern diversity by the empirical risk of the empirical risk minimizer and the Rashomon parameter $\theta$.

Theorem 10 (Upper bound on pattern diversity). Consider hypothesis space $ℱ$, 0–1 loss, and empirical risk minimizer $\hat{f}$. For any $\theta \ge 0$, pattern diversity can be upper bounded by

$$div\left({\hat{R}}_{\mathit{\text{set}}}\left(ℱ,\theta \right)\right)\le 2\left(\hat{L}\left(\hat{f}\right)+\theta \right)\left(1-\left(\hat{L}\left(\hat{f}\right)+\theta \right)\right)+2\theta .$$ (2)

The proof of Theorem 10 is in Appendix I. The bound (2) emphasizes how important the performance of the empirical risk minimizer is for understanding pattern diversity. If dataset is well separated so that the empirical risk is small, then pattern diversity will also be small, as there are not many different ways to misclassify points and stay within the Rashomon set. As dataset becomes noisier, in expectation, we expect the empirical risk to increase and thus pattern diversity as well. We will show theoretically and experimentally that pattern diversity is likely to increase under label noise.

6.2. Label Noise is Likely to Increase Pattern Diversity

Let $S_{\rho }$ be a version of $S$ with uniformly random label noise, creating randomly perturbed labels $\tilde{y}$ with probability $0<\rho <\frac{1}{2}:P\left({\tilde{y}}_i\ne y_i\right)=\rho$. Let $\mathrm{\Omega }\left(S_{\rho }\right)$ be the uniform distribution over all $S_{\rho }$. From Theorem 10 denote the upper bound on pattern diversity as $U_{\text{div}}\left({\hat{R}}_{\mathit{\text{set}}}(ℱ,\theta )\right)=2\left(\hat{L}\right(\hat{f})+\theta )(1-(\hat{L}\left(\hat{f}\right)+\theta \left)\right)+2\theta$. We show that it increases with uniform label noise.

Theorem 11 (Upper bound on pattern diversity increases with label noise). Consider a hypothesis space $ℱ$, 0–1 loss, and a dataset $S$. Let $\rho \in \left(0,\frac{1}{2}\right)$ be the probability with which each label $y_i$ is flipped independently, and let $S_{\rho }~\mathrm{\Omega }\left(S_{\rho }\right)$ denote a noisy version of $S$. For the Rashomon parameter $\theta \ge 0$, if ${\mathrm{i}\mathrm{n}\mathrm{f}}_{f\in ℱ} {\mathbb{E}}_{S_{\rho }}{\hat{L}}_{S_{\rho }}\left(f\right)<\frac{1}{2}-\theta$ and ${\hat{L}}_S\left({\hat{f}}_S\right)<\frac{1}{2}$, then adding noise to the dataset increases the upper bound on pattern diversity of the expected Rashomon set:

$$U_{div}\left({\hat{R}}_{{set}_S}(ℱ,\theta )\right)<U_{div}\left({\hat{R}}_{{\mathit{\text{set}}}_{{\mathbb{E}}_{S_{\rho }}~\mathrm{\Omega }\left(S_{\rho }\right)}{S}_{\rho }}(ℱ,\theta )\right).$$

The proof of Theorem 11 is in Appendix J. In the general case, it is challenging to find a closed-form formula for pattern diversity or design a lower bound without strong assumptions about the data distribution or the hypothesis space. Therefore, we empirically examine the behavior of pattern diversity alongside other characteristics of the Rashomon set for different datasets and show these characteristics tend to increase with more noise.

6.3. Experiment for Pattern Diversity and Label Noise

We expect many different datasets to have larger Rashomon set measurements as data become more noisy. As before, we consider uniform label noise, where each label is flipped independently with probability $\rho$. For different noise levels, we compute diversity, and the number of patterns in the Rashomon set for 12 different datasets for the hypothesis space of sparse decision trees of depth 4 (see Figure 3 (a)–(b)). For every dataset, we introduce up to 25% label noise $(\rho \in [0,0.25\left]\right)$, or Accuracy $\left(\hat{f}\right)-50\mathrm{\%}$, whichever is smaller. This means that data with lower accuracy (before adding noise) will have shorter plots, since noise is along the horizontal axis of our plots in Figure 3. For each noise level $\rho$, we do 25 draws of $S_{\rho }$, and for each draw $S_{\rho }$ we recompute the Rashomon set. For decision trees we use TreeFARMS [52], which allows us to compute the number of trees in the Rashomon set. We discuss results for the hypothesis space of linear models in Appendix K.5.

Figure 3:

Rashomon set characteristics such as the number of trees in the Rashomon set (Subfigure a), the number of patterns in the Rashomon set (Subfigure b), and pattern diversity (Subfigure c) tend to increase with uniform label noise for hypothesis spaces of sparse decision trees. The top row of the figure shows datasets with lower empirical risk (“cleaner” datasets) and the bottom row shows datasets with higher empirical risk (“noisier” datasets).

Some key observations from Figure 3: First, the number of trees and patterns in the Rashomon set on average increases with noise. This means that the Rashomon ratio and pattern Rashomon ratio increase with noise as well since the hypothesis space stays the same. Second, noisier datasets, that initially had lower empirical risk (e.g., Compas, Coffee House, FICO) have more models in the Rashomon set (and thus higher Rashomon ratios) as compared to datasets with lower empirical risk (Car Evaluation, Monks1, Monks3). Finally, pattern diversity, on average, tends to increase with noise for the majority of the datasets, except FICO and Bar, where it stays about the same (most likely due to larger inherent noise in data before we added more noise).

Returning to the Path.

If the practitioner observes more noise in the data, it could be already the case that the Rashomon set is large. Then, there are many good models, among which simpler or interpretable model are likely to exist [39].

7. Limitations

While we believe our results have illuminated that the Rashomon effect often exists when data are noisy, the connection between noise and increased Rashomon metrics is mostly supported by experiments, rather than a tight set of theoretical bounds. Specifically, we do not have a lower bound on diversity nor a correlation between diversity and the pattern Rashomon ratio (or Rashomon ratio) yet. This connection deserves further exploration and strengthening.

In previous sections, we showed that we expect an increase in pattern diversity while adding more label noise. In Section 6.3, we also observe an increase in the number of trees and patterns in the Rashomon set under label noise. In both cases, we used 0–1 loss. While in Section 5 we studied ridge regression (and thus used squared loss), ways to extend our results (both experimental and theoretical) to different distance-based classification loss functions are not yet fully clear. In the case of classification with distance-based losses, such as exponential loss, the effect of label noise can be difficult to model without taking properties of the data distribution into account. For example, regardless of the amount of noise in the dataset, a single misclassified outlier could essentially define the Rashomon set. The exponential loss could be very sensitive to the position of this outlier, leading to a small Rashomon set. This does not apply to the analysis in this paper, since the 0–1 loss we used is robust to outliers. Perhaps techniques that visualize the loss landscape [26] can be helpful in characterizing the Rashomon set for distance-based classification losses.

Conclusion

Our results have profound policy implications, as they underscore the critical need to prioritize interpretable models in high-stakes decision-making. In a world where black box models are often used for high-stakes decisions, and yet the data generation processes are known to be noisy, our work sheds light on the false premise of this dangerous practice – that black box models are likely to be more accurate. Our findings have particular relevance for critical domains such as criminal justice, healthcare, and loan decisions, where individuals are subjected to the outputs of these models. The use of interpretable models in these areas can safeguard the rights and well-being of these individuals and ensure that decision-making processes are transparent, fair, and accountable.