Fair admission risk prediction with proportional multicalibration

Abstract

Fair calibration is a widely desirable fairness criteria in risk prediction contexts. One way to measure and achieve fair calibration is with multicalibration. Multicalibration constrains calibration error among flexibly-defined subpopulations while maintaining overall calibration. However, multicalibrated models can exhibit a higher percent calibration error among groups with lower base rates than groups with higher base rates. As a result, it is possible for a decision-maker to learn to trust or distrust model predictions for specific groups. To alleviate this, we propose proportional multicalibration, a criteria that constrains the percent calibration error among groups and within prediction bins. We prove that satisfying proportional multicalibration bounds a model’s multicalibration as well its differential calibration, a fairness criteria that directly measures how closely a model approximates sufficiency. Therefore, proportionally calibrated models limit the ability of decision makers to distinguish between model performance on different patient groups, which may make the models more trustworthy in practice. We provide an efficient algorithm for post-processing risk prediction models for proportional multicalibration and evaluate it empirically. We conduct simulation studies and investigate a real-world application of PMC-postprocessing to prediction of emergency department patient admissions. We observe that proportional multicalibration is a promising criteria for controlling simultaneous measures of calibration fairness of a model over intersectional groups with virtually no cost in terms of classification performance.

1. Introduction

Today, machine learning (ML) models have an impact on outcome disparities across sectors due to their wide-spread use in decision-making. When applied in clinical decision support (CDS), ML models help care providers decide whom to prioritize to receive finite and timesensitive resources among a population of potentially very ill patients. These resources include hospital beds (Barak-Corren et al., 2021a; Dinh and Berendsen Russell, 2021), organ transplants (Schnellinger et al., 2021), specialty treatment programs (Henry et al., 2015; Obermeyer et al., 2019), and, recently, ventilator and other breathing support tools to manage the COVID-19 pandemic (Riviello et al., 2022).

In scenarios like these, decision makers typically rely on risk prediction models to be calibrated. Calibration measures the extent to which a model’s risk scores, $R$, match the observed probability of the event, $P\left(y\right)$ (Brier et al., 1950). Perfect calibration implies that $P\left(y\mid R=r\right)=r$, for all values of $r$. Calibration allows the risk scores to be used to rank patients in order of priority and informs care providers about the urgency of treatment. However, models that are not equally calibrated among subgroups defined by different sensitive attributes (race, ethnicity, gender, income, etc.) may lead to systematic denial of resources to marginalized groups (e.g. (Obermeyer et al., 2019; Ashana et al., 2021; Roberts, 2011; Zelnick et al., 2021; Ku et al., 2021)). For example, Obermeyer et al. (2019) analyzed a large health system algorithm used to enroll high-risk patients into care management programs and showed that, at a given risk score, Black patients exhibited significantly poorer health than white patients.

When evidence of algorithmic bias is observed, for example in estimates of kidney function (Diao et al., 2021), it is unclear how care should be adjusted for affected patient groups. Variance in clinical judgements that are not evidence-based are themselves a source of unfairness, referred to as unwarranted clinical variation (Harrison et al., 2019; Sutherland and Levesque, 2020; Newhouse and Garber, 2013). Ideally, patients with the same evidence, e.g. model risk prediction, would receive the same standard of care. In this work, we propose measures and methods designed to reduce unwarranted variation in care that may arise from biased prediction models.

To address equity in calibration, Hebert-Johnson et al. (2018) proposed a fairness measure called multicalibration (MC), which asks that calibration be satisfied simultaneously over many flexibly-defined subgroups. Remarkably, MC can be satisfied efficiently by post-processing risk scores without negatively impacting the generalization error of a model, unlike other fairness concepts like demographic parity (Foulds and Pan, 2020) and equalized odds (Hardt et al., 2016). This has motivated the use of MC in practical settings (e.g. Barda et al. (2021)) and has spurred several extensions (Kim et al., 2019; Jung et al., 2021; Gupta et al., 2021; Gopalan et al., 2022). If we bin our risk predictions, the MC criteria specifies that, for every group within each bin, the absolute difference between the mean observed outcome and the mean of the predictions should be small.

As Barocas et al. (2019) note, fair calibration reduces to a more general fairness notion dubbed sufficiency. Under sufficiency, the expected outcome should be independent of group membership, conditioned on the risk prediction. We see this criteria as highly desirable in CDS contexts. Sufficiency eliminates a source of uncertainty from decision-making: how to interpret a model recommendation from a model that has been observed to perform more or less well on certain subpopulations. Given a risk score from a model satisfying sufficiency, a decision maker cannot distinguish between patient outcomes on the basis of their group membership. Thus, they cannot justify variations from the model’s recommendations on the basis of patient identity alone.

In this work, we start by assessing the conditions under which MC satisfies sufficiency. To do so, we derive a fairness criteria directly from sufficiency, differential calibration (DC). DC is an extension of differential fairness (Foulds et al., 2019b), both named due to their relation to differential privacy (Dwork and Roth, 2013). DC constrains ratios of population risk between groups within risk prediction bins. We show that DC measures the extent to which one satisfies sufficiency in an interpretable way. In short, among patients assigned the same risk score from a model satisfying $\epsilon$-DC, the outcome is at most $e^{\epsilon }$ more likely in one group compared to any another. A low $\epsilon$-DC thereby constrains the amount by which a decision maker may learn to unequally trust the model for different groups.

By relating sufficiency to MC, we describe a shortcoming of MC that can occur when the outcome probabilities are strongly tied to group membership. Under this condition, the amount of calibration error relative to the expected outcome can be unequal between groups. This inequality hampers the ability of MC to (approximately) guarantee sufficiency except by setting extremely low error thresholds, resulting in the need for large numbers of updates.

We propose a simple variant of MC called proportional multicalibration (PMC) that instead requires the proportion of calibration error within each bin and group to be small. We prove that PMC bounds both multicalibration and differential calibration. We show that PMC can be satisfied with an efficient post-processing method, similarly to MC (Pfisterer et al., 2021). Proportionally multicalibrated models thereby obtain robust fairness guarantees that are less dependent on population risk categories. It does so in fewer steps than MC by prioritizing updates to groups with high proportional calibration error.

Finally, we investigate the application of these methods to predicting patient admissions in the emergency department, a real-world resource allocation task that is targeted by current CDS models (Barak-Corren et al., 2021a). We create a benchmark dataset for this task using the recently released MIMIC-IV emergency department dataset Johnson et al. (2021), and benchmark PMC- and MC-based postprocessing approaches. We show that post-processing for PMC results in models that are accurate, multicalibrated, and differentially calibrated.

2. Reconciling Multicalibration and Sufficiency

2.1. Preliminaries

We consider the task of training a risk prediction model for a population of individuals with outcomes, $y\in \left\{0,1\right\}$, and features, $x\in 𝒳$. Let $D$ be the joint distribution from which individual samples $\left(y,x\right)$ are drawn. We assume the outcomes $y$ are random samples from underlying independent Bernoulli distributions, denoted as $p^{*}\left(x\right)\in \left[0,1\right]$. Individuals can be further grouped into collections of subsets, $𝒞\subseteq 2^{𝒳}$, such that $S\in 𝒞$ is the subset of individuals belonging to $S$, and $x\in S$ indicates that individual $x$ belongs to group $S$. We denote our risk prediction model as $R\left(x\right):𝒳\to \left[0,1\right]$.

In order to consider calibration in practice, the risk predictions are typically discretized into bins. We represent discretization by a parameter, $\lambda$, that specifies the width of the bins. As an example, $\lambda =0.1$ corresponds to decile bins. For brevity, proofs and some formal definitions in the following sections are given in Appendices A.2 and A.3.2.

2.2. Multicalibration

Hebert-Johnson et al. (2018) define multicalibration by first defining calibration with respect to a subset (i.e., group) of individuals:

Definition 1 ($\alpha$-calibration) Let $S\subseteq 𝒳$. For $\alpha \in \left[0,1\right],R$ is $\alpha$-calibrated with respect to $S$ if there exists some $S^{\prime }\subseteq S$ with $\left|S^{\prime }\right|\ge \left(1-\alpha \right)\left|S\right|$ such that for all $r\in \left[0,1\right]$,

$$\left|\underset{D}{\mathbb{E}}\left[y\mid R=r,x\in S^{\prime }\right]-r\right|\le \alpha .$$

MC then guarantees that $\alpha$-calibration holds over every subset from a collection of subsets:

Definition 2 ($\alpha$-Multicalibration) Let $𝒞\subseteq 2^{𝒳}$ be a collection of subsets of $𝒳,\alpha \in \left[0,1\right].A$ predictor $R$ is $\alpha$-multicalibrated on $𝒞$ if for all $S\in 𝒞,R$ is $\alpha$-calibrated with respect to $S$.

We note that, according to Definition 1, a model need only be calibrated over a sufficiently large subset of each group $\left(S^{\prime }\right)$ in order to satisfy the definition. This relaxation is used to maintain a satisfactory definition of MC when working with discretized predictions. For simplicity, we conduct most of our analysis using the continuous versions of fairness definitions like Definition 2 (see Appendix A.3.1 for an extended discussion).

MC is one of few approaches to achieving fairness that does not require a significant trade-off to be made between a model’s generalization error and the improvement in fairness it provides. As Hébert-Johnson et al. (2018) show, this is because achieving multicalibration is not at odds with achieving accuracy in expectation for the population as a whole. This separates calibration fairness from other fairness constraints like demographic parity and equalized odds (Hardt et al., 2016), both of which may denigrate the performance of the model on specific groups (Chouldechova, 2017; Pleiss et al., 2017). In clinical settings, such trade-offs may be difficult or impossible to justify. In addition to its alignment with accuracy in expectation, Hébert-Johnson et al. (2018) propose an efficient post-processing algorithm for MC similar on boosting. We discuss additional extensions to MC in Appendix A.1.

2.3. Measuring Sufficiency via Differential Calibration

MC provides a sense of fairness by approximating calibration by group, which is perfectly satisfied when $P_D\left(y\mid R=r,x\in S\right)=r$ for all $S\in C$ and $r\in \left[0,1\right]$. Calibration by group is closely related to the sufficiency fairness criterion (Barocas et al., 2019). Sufficiency states that the outcome probability is independent from $𝒞$, conditioned on the risk score. In the binary group setting $\left(𝒞=\left\{S_i,S_j\right\}\right)$, we can express sufficiency as

$$P_D\left(y\mid R,x\in S_i\right)=P_D\left(y\mid R,x\in S_j\right)$$ (1)

or equivalently,

$$P_D\left(y\mid R,x\in S_i\right)/P_D\left(y\mid R,x\in S_j\right)=1.$$

Unlike calibration by group, sufficiency does not stipulate that the risk scores be calibrated, yet from a fairness perspective, sufficiency and calibration by group are equivalent (Barocas et al., 2019). In both cases, the sense of fairness stems from the desire for $R$ to capture everything about group membership that is relevant to predicting $y$.

Under sufficiency, the risk score is equally informative of the outcome, regardless of group membership. Because of this, given a risk score from a model satisfying sufficiency, a decision maker cannot distinguish between patient outcomes on the basis of their group membership. In this sense, a model satisfying sufficiency provides an added level of trust in deployment: we know that a decision maker cannot justify different decisions on the basis of the patient’s identity for the same risk prediction. If risk prediction models satisfied sufficiency, it would eliminate the need for group-specific decision protocol given recommendations from the same model. Consider, for example, the decision-making uncertainty related to estimates of kidney function (Diao et al., 2021).

Below, we define an approximate measure of sufficiency that constrains pairwise differentials between groups, and accomodates binned predictions:

Definition 3 ($\epsilon$-Differential Calibration) Let $𝒞\subseteq 2^{𝒳}$ be a collection of subsets of $𝒳$. A model $R\left(x\right)$ is $\epsilon$-differentially calibrated with respect to $𝒞$ if, for all pairs $\left(S_i,S_j\right)\in 𝒞\times 𝒞$ for which $P_D\left(S_i\right),P_D\left(S_j\right)>0$, for any $r\in \left[0,1\right]$,

$$e^{-\epsilon }\le \frac{{\mathbb{E}}_D\left[y\mid R=r,x\in S_i\right]}{{\mathbb{E}}_D\left[y\mid R=r,x\in S_j\right]}\le e^{\epsilon }.$$ (2)

By inspection we see that $\epsilon$ in $\epsilon$-DC measures the extent to which $R$ satisifies sufficiency. That is, when $P\left(y\mid R=r,x\in S_i\right)\approx P\left(y\mid R=r,x\in S_j\right)$ for all pairs $\left(S_i,S_j\right),\epsilon \approx 0$. $\epsilon$-DC requires that, for any risk score, the outcome $y$ is at most $e^{\epsilon }$ times more likely among one group than another, and a minimum of $e^{-\epsilon }$ less likely.

Definition 3 fits the general definition of a differential fairness measure proposed by Foulds et al. (2019a) and previously used to study demographic parity criteria (Foulds and Pan, 2020). We describe the relation in more detail in Appendix A.1.1, including Eq. (2)’s connection to differential privacy (Dwork and Lei, 2009) and pufferfish privacy (Kifer and Machanavajjhala, 2014).

Taken alone, DC does not prevent a decision-maker from equally distrusting a model for all patients. That is because it does not guarantee the model is globally calibrated. Rather, it makes it harder to distinguish the calibration quality of the model between groups.

2.4. The differential calibration of multicalibrated models is limited by low-risk groups

At a basic level, the form of MC and sufficiency differ: MC constrains absolute differences between groups across prediction bins, whereas sufficiency constrains pairwise differentials between groups. To reconcile MC and DC/sufficiency more formally, we pose the following question: if a model satisfies $\alpha$-MC, what, if anything does this imply about the $\epsilon$-DC of the model? (In Appendix A.4, Theorem 22, we answer the inverse question). We now show that multicalibrated models have a bounded DC, but that this bound is limited by small values of $R$.

Theorem 4 Let $R\left(x\right)$ be a model satisfying $\alpha$-$MC$ on a collection of subsets $𝒞\in 2^{𝒳}$. Let $r_{\text{min}}={\text{min}}_{S\in 𝒞}{\mathbb{E}}_D\left[R\mid R=r,x\in S\right]$ be the minimum expected risk prediction among $S\in 𝒞$ and $r\in [0,1]$. Then $R\left(x\right)$ is $\left(\text{ln}\frac{r_{min}+\alpha }{r_{min}-\alpha }\right)$-differentially calibrated.

Theorem 4 illustrates the important point that, in terms of percentage error, MC does not provide equal protection to groups with different risk profiles. Imagine a model satisfying (0.05)-MC for groups $S\in 𝒞$. Consider individuals receiving model predictions $R\left(x\right)=\mathrm{0.9.}$ MC guarantees that, for any category $\left\{x:x\in S,R\left(x\right)=0.9\right\}$, the expected outcome probability is at least $0.9-\alpha =0.85$ and at most $0.9+\alpha =0.95$. This bounds the percent error among groups with this prediction to about 6%. In contrast, consider individuals for whom $R\left(x\right)=0.3$; each group may have a true outcome probability as low as 0.25, which is an error of 20% - about 3.4x higher than the percent error in the higher-risk group.

In Appendix Theorem 22, we show that differentially calibrated models can bound multicalibration only if they are also $\alpha$-calibrated (Definition 1).

3. Proportional Multicalibration

We are motivated to define a measure that is efficiently learnable like MC (Definition 2) but better aligned with the multiplicative interpretation of sufficiency, like DC (Definition 3). To do so, we define PMC, a variant of MC that constrains the proportional calibration error of a model among subgroups and risk strata. In this section, we show that bounding a model’s PMC is enough to meaningfully bound DC and MC. Furthermore, we provide an efficient algorithm for satisfying PMC based on a simple extension of MC/Multiaccuracy boosting (Kim et al., 2019). We begin by defining proportional calibration, which expresses calibration error as a percentage of the outcome probability among a group.

Definition 5 ($\alpha$-Proportional Calibration) Let $S\subseteq 𝒳$. For $\alpha >0,R$ is $\alpha$-proportionally calibrated with respect to $S$ if there exists some $S^{\prime }\subseteq S$ with $\left|S^{\prime }\right|\ge \left(1-\alpha \right)\left|S\right|$ such that for all $r\in \left[0,1\right]$,

$$\left|\underset{D}{\mathbb{E}}\left[y\mid R=r,x\in S^{\prime }\right]-r\right|\le \alpha \underset{D}{\mathbb{E}}\left[y\mid R=r,x\in S^{\prime }\right].$$

Proportional multicalibration is then defined by requiring Definition 5 be satisified among a collection of groups:

Definition 6 ($\alpha$-Proportional Multicalibration) Let $𝒞\subseteq 2^{𝒳}$ be a collection of subsets of $𝒳,\alpha >0$. A predictor $R$ is $\alpha$-proportionally multicalibrated on $𝒞$ if for all $S\in 𝒞,R$ is $\alpha$-proportionally calibrated with respect to $S$.

We also define a discretized version of PMC in Appendix A.3.2 that is useful for implementing the measure in Algorithm 1 and measuring PMC in our experiments. In Appendix A.3.1, we show $\left(\alpha ,\lambda \right)$-PMC meaningfully bounds $\alpha$-PMC under different discretizations, such that we can minimize $\left(\alpha ,\lambda \right)$-PMC to achieve low $\alpha$-PMC.

In practice, we must ensure that the outcome probability for any (group, prediction bin) category is greater than zero for PMC to be meaningful. We later introduce a lower bound, $\rho$, to prevent the outcome probability from being too small. It is common in clinical settings to only show a risk prediction to a decision maker if it exceeds some threshold; in those settings, $\rho$ can be set to match this threshold.

Comparison to Differential Calibration

Rather than constraining the differentials of prediction- and group- specific outcomes among all pairs of subgroups in $𝒞\times 𝒞$ as in DC (Definition 3), PMC constrains the relative error of each group in $𝒞$. In practical terms, this makes it more efficient to calculate PMC by a factor of $O\left(\left|𝒞\right|\right)$ steps compared to DC. In addition, PMC constrains each group’s calibration with respect to ground truth, whereas DC only constrains the differentials between groups. We formalize the relationship between these two measures below.

Theorem 7 Let $R\left(x\right)$ be a model satisfying $\left(\alpha \right)$-PMC on a collection $𝒞$. Then $R\left(x\right)$ is $\left(ln\frac{1+\alpha }{1-\alpha }\right)$-differentially calibrated.

Theorem 7 demonstrates that $\alpha$-proportionally multicalibrated models satisfy a straightforward notion of differential fairness that depends monotonically only on $\alpha$. Because PMC bounds DC, proportionally calibrated models inherit desirable properties of differentially calibrated models, especially the following:

Corollary 8 Let $R\left(x\right)$ be a model satisfying $\left(\alpha \right)$-PMC on a collection $𝒞$. Then $R\left(x\right)$ satisfies $\left(ln\frac{1+\alpha }{1-\alpha }\right)$-pufferfish privacy with respect to $𝒞$.

Corollary 8 follows from the definitions of $\epsilon$-DC and pufferfish privacy (Appendix Definition 13, Kifer and Machanavajjhala (2014)). This property is perhaps best understood as a guarantee of trust rather than privacy. Namely, it guarantees that the patient’s group identity does not provide useful information to a decision-maker in deciding how much to trust model predictions. The patient’s group identity is in this sense “private” from the model’s calibration. Unlike many privacy-preserving algorithms, we do not need to add add any noise to the model to satisfy DC; conversely, it is achieved by making the model better calibrated for some groups.

Comparison to Multicalibration

Rather than constraining the absolute difference between risk predictions and the outcome as in MC, PMC requires that the calibration error be a small fraction of the expected risk in each category $\left(S,I\right)$. In this sense, it provides a stronger protection than MC by requiring calibration error to be a small fraction regardless of the risk group. In many contexts, we would argue that this is also more aligned with the notion of fairness in risk prediction contexts. Under MC, the underlying probability of an outcome within a group affects the fairness protection that is received (i.e., the percentage error that Definition 2 allows). Because underlying probabilities of many clinically relevant outcomes vary significantly among subpopulations, multicalibrated models may systematically permit higher percentage error to specific groups. The difference in relative calibration error among populations with different risk profiles also translates in weaker sufficiency guarantees, as demonstrated in Theorem 4. In contrast, PMC provides a fairness guarantee that is less dependent on subpopulation risks.

In the following theorem, we show that MC is also constrained when a model satisfies PMC.

Theorem 9 Let $R\left(x\right)$ be a model satisfying $\alpha$-PMC on a collection $𝒞$. Then $R\left(x\right)$ is $\left(\frac{\alpha }{1-\alpha }\right)$-multicalibrated on $𝒞$.

The proof of Theorem 9 is given in Appendix A.2. This theorem implies that a proportionally calibrated model with sufficiently low $\alpha$ will satisfy a similarly low value of MC. We further discuss and illustrate the bounds given by Theorems 4, 7 and 9 in Appendix A.3.

Because PMC bounds a model’s multicalibration, proportionally multicalibrated models also bound the generalization error of a predictor such that it is close to the “best in class” predictions.

Corollary 10 Let $R\left(x\right)$ be a model satisfying $\left(\alpha \right)\text{-}PMC$ on a collection $𝒞$. Let $\mathscr{H}$ be a set of predictors, $p^{*}$ the outcome-generating distribution, and $h^{*}={\text{arg min}}_{h\in \mathscr{H}}{\Vert h-p^{*}\Vert }^2$. Then

$${\Vert R-p^{*}\Vert }^2-{\Vert h^{*}-p^{*}\Vert }^2<\frac{6\alpha }{1-\alpha }.$$

This guarantee follows directly from Theorem 5 in Hebert-Johnson et al. (2018). In a nutshell, it means that proportionally multicalibrated models are bound to be close to the best predictor of their hypothesis class $\left(\mathscr{H}\right)$. This contrasts fair calibration with other approaches to fairness that typically form a trade-off with a model’s overall error. We further illustrate relationship between PMC, DC, and MC in Appendix Fig. 6.

3.1. Learning proportionally multicalibrated predictors

In Algorithm 1, we propose an extension of MCBoost (Pfisterer et al., 2021) to efficiently update risk predictors to satisfy PMC. Algorithm 1 works by checking for calibration errors among groups and prediction intervals that violate the user threshold, and adjusting these predictions towards the target. PMCBoost differs in two main ways: first, it updates whenever calibration error is not within $\alpha \overline{y}$ for all categories, as opposed to simply within $\alpha$. Second, it ignores updates for categories with low outcome probability (less than $\rho$). Next, we prove that PMCBoost learns an $\left(\alpha ,\lambda \right)$-PMC model in a polynomial number of steps.

Proposition 11 Define $\alpha ,\lambda ,\gamma ,\rho >0$. Let $𝒞\subseteq 2^{𝒳}$ be a collection of subsets of $𝒳$ such that, for all $S\in 𝒞,P_D\left(S\right)>\gamma$. Let $R\left(x\right)$ be a risk prediction model to be post-processed. For all $\left(S,I\right)\in 𝒞\times {\Lambda }_{\lambda }$, let $E\left[y\mid R\in I,x\in S\right]>\rho$. There exists an algorithm that satisfies $\left(\alpha ,\lambda \right)-PMC$ with respect to $𝒞$ in $O\left(\frac{\left|C\right|}{{\alpha }^3{\lambda }^2{\rho }^2\gamma }\right)$ steps.

We analyze Algorithm 1 and show it satisfies Proposition 11 in Appendix A.2. This more stringent threshold requires an additional $O\left(\frac{1}{{\rho }^2}\right)$ steps compared to the algorithm for MC, where $\rho >0$ is a lower bound on the expected outcome within a category $\left(S,I\right)$.

Achieving proportional multicalibration with MCBoost

Another way to achieve ${\alpha }_{P^{-}}$ PMC is to use the existing MCBoost algorithm, but setting ${\alpha }_M=\rho {\alpha }_P$. In other words, setting a very low value for ${\alpha }_M$ should also satisfy Theorem 6 because, if ${\alpha }_M$ can be made small enough, the calibration error on all categories will be small compared to the outcome prevalence, ${\mathbb{E}}_D\left[y\mid R\in I,x\in S\right]$. However, to achieve PMC guarantees by MCBoost requires a large number of unnecessary updates for high risk groups, since the DC and PMC of multicalibrated models are limited by low-risk groups (Theorem 4). Furthermore, the number of steps in MCBoost (and PMCBoost scales as an inverse high-order polynomial of $\alpha$ (cf. Thm. 2 (Hebert-Johnson et al., 2018)).

We can compare the algorithm complexity of MCBoost and PMCBoost to get a better comparison. Say we would like to satisfy $\alpha$-PMC using MCBoost. Hebert-Johnson et al. (2018) show that MCBoost takes $O\left(\frac{\left|C\right|}{{\alpha }^3{\lambda }^2\gamma }\right)$ steps to complete. We must reduce $\alpha$-MC by a factor of $\rho$ to satisfy $\alpha$-PMC; this increases the number of MCBoost steps by a factor of $\left(\frac{1}{{\rho }^3}\right)$. For a reasonable value of $\rho \left(\text{say},\rho =0.1\right)$, this takes 1000x more steps, and corresponds to 10x more steps than it would take to satisfy $\alpha$-PMC with PMCBoost.

Now consider the reverse: satisfying $\alpha$-MC using PMCBoost. Because of Theorem 9, we need only reduce $\alpha$-PMC by a factor of $\frac{1}{1+\alpha \text{-MC }}$, which increases the number of PMCBoost steps by ${(1+\alpha \text{-MC })}^3$. For example, to achieve $\alpha$-MC $=0.1$, PMCBoost takes about 1.3x more steps than MCBoost.

4. Experiments

In our first set of experiments (Section 4), we study MC and PMC in simulated population data to understand and validate the analysis in previous sections. In the second section, we compare the performance of varied model treatments on a real world hospital admission task, using an implementation of Algorithm 1. We make use of empirical versions of our fairness definitions which we refer to as MC loss, PMC loss, and DC loss. In short, these measures calculate the maximum (proportional) calibration error or pairwise calibration differential among subgroups and risk categories in the data sample. Due to space constraints the formal definitions are given in Appendix A.3.2 (Definitions 19, 20 and 21).

Simulation study

We simulate data from $\alpha$ multicalibrated models. For simplicity, we specify a data structure with a one-to-one correspondence between subset and model estimated risk, such that for all $x$ in $S,R\left(x\right)=R\left(x\mid x\in S\right)=R\left(S\right)$. Therefore all information for predicting the outcome based on the features in $x$ is contained in the attributes $𝒜$ that define subgroup $S$. Outcome probability is specified as $p_i^{*}=P_D\left(y\mid x\in S_i\right)=0.2+0.01\left(i-1\right)$ and $i=1,\cdots ,N_s$, where $N_s$ is the number of subsets $S$, defined by $𝒜$ and indexed by $i$ with increasing $p^{*}$. For each group, $R_i=R\left(S_i\right)=R\left(x\mid x\in S_i\right)=p_i^{*}-{\Delta }_i$. We randomly select ${\Delta }_i$ for one group to be $\pm \alpha$ and for the remaining groups, ${\Delta }_i=\pm \delta$, where $\delta ~\text{Uniform}\left(\text{min}=0,\text{max}=\alpha \right)$. In all cases, the sign of ${\Delta }_i$ is determined by a random draw from a Bernoulli distribution. For these simulations we set $N_S=61$ and $\alpha =0.1$, such that $p_i^{*}\in \left[0.2,0.8\right]$ and $R_i\in \left[0.1,0.9\right]$. We generate $N_{\text{sim }}=1000$ simulated datasets, with $n=1000$ observations per group, and for each $S_i$, we calculate the ratio of the absolute mean error to $p_i^{*}$, i.e. the PMC loss function for this data generating mechanism.

We also simulate three specific scenarios where: 1) $\left|{\Delta }_i\right|$ is equivalent for all groups (Fixed); 2) $\left|{\Delta }_i\right|$ increases with increasing $p_i^{*}$; and 3) $\left|{\Delta }_i\right|$ decreases with increasing $p_i^{*}$, with $\alpha =0.1$ in each case. These scenarios compare when $\alpha$ is determined by all groups, the group with the lowest outcome probability, and the group with the highest outcome probability, respectively.

Hospital admission

Next, we test PMC alongside other methods in application to prediction of inpatient hospital admission for patients visiting the emergency department (ED). Overcrowding and long wait times in EDs have been shown to increase odds of inpatient death (5%, CI 2–8%), length of stay (0.8%, CI 0.5–1%), and costs per admission (1%, CI 0.7–2%) (Sun et al., 2013). The burden of overcrowding and long wait times in EDs is significantly higher among non-white, non-Hispanic patients and socio-economically marginalized patients (James et al., 2005; McDonald et al., 2020).

Recent work has demonstrated risk prediction models that can expedite patient visits by predicting patient admission at an early stage of a visit with a high degree of certainty (AUC ≥ 0.9 across three large care centers) (Barak-Corren et al., 2017b,a, 2021b,a). Our goal is to ensure no group of patients will be over- or underprioritized over another by these models, which could exacerbate the treatment and outcome disparities that currently exist.

We construct a prediction task similar to previous studies but using a new data resource: the MIMIC-IV-ED repository (Johnson et al., 2021). After data preparation (see Appendix A.5), the cohort consists of 173,561 visits with demographics and admission statistics given in Table 1. Table 2 shows the list of features used for prediction, modelled off prior work (Barak-Corren et al., 2021a). In Table 1 we observe stark differences in admission rates by demographic group and gender, suggesting that the use of a proportional measure of calibration could be appropriate for this task. We trained and evaluated ${\ell }_1$-penalized logistic regression (LR), random forest (RF), and deep neural network (DNN) models of patient admission, with and without post-processing with MCBoost (Pfisterer et al., 2021) or PMCBoost. We varied $\alpha ,\gamma$, and $\rho$ to characterize parameter sensitivity among the methods (Table 3). For each of the parameter settings, we conducted 100 repeat experiments with different shuffles of the data. Comparisons are reported on a test set of 20% of the data for each trial. Additional experiment details are available in Appendix A.5.

Table 1:

Admission prevalence (Admissions/Total (%)) among patients in the MIMIC-IV-ED data repository, stratified by the intersection of ethnoracial group and gender.

Gender	F	M	Overall
Ethnoracial Group
American Indian/Alaska Native	70/257 (27%)	82/170 (48%)	152/427 (36%)
Asian	1043/3595 (29%)	1032/2384 (43%)	2075/5979 (35%)
Black/African American	3124/27486 (11%)	2603/14458 (18%)	5727/41944 (14%)
Hispanic/Latino	1063/10262 (10%)	1168/5795 (20%)	2231/16057 (14%)
Other	1232/5163 (24%)	1479/3849 (38%)	2711/9012 (30%)
Unknown/Unable to Obtain	1521/2156 (71%)	2074/2377 (87%)	3595/4533 (79%)
White	18147/50174 (36%)	18951/45435 (42%)	37098/95609 (39%)
Overall	26200/99093 (26%)	27389/74468 (37%)	53589/173561 (31%)

Table 2:

Features used in the hospital admission task.

Description	Features
Vitals	temperature, heartrate, resprate, o2sat, systolic blood pressure, diastolic blood pressure
Triage Acuity	Emergency Severity Index (Tanabe et al., 2004)
Check-in Data	chief complaint, self-reported pain score
Health Record Data	no. previous visits, no. previous admissions
Demographic Data	ethnoracial group, gender, age, marital status, insurance, primary language

Table 3:

Parameters for the hospital admission prediction experiment.

Parameter	Values
tolerance ($\alpha$)	(0.001, 0.01, 0.1, 0.2)
min group probability ($\gamma$)	(0.05, 0.1)
binning ($\lambda$)	0.1
min outcome probability ($\rho$)	(0.01, 0.1)
Base Model	LR, RF, DNN
Groups	(race/ethnicity, gender), (race/ethnicity, gender, insurance product)

5. Results

Fig. 2 shows the results of our simulation study. The results indicate that, without the proportionality factor, $\alpha$-multicalibrated models exhibit a dependence between the group prevalence and the amount of proportional calibration loss. The results demonstrate why $\alpha$-MC alone is not sufficient to achieve sufficiency, particularly when outcome probabilities vary by group.

Figure 2:

The relationship between MC, PMC, and outcome prevalence as illustrated via a simulation study in which the rates of the outcome are associated with group membership. Gray points denote the PMC loss of a (0.1)-MC model on 1000 simulated datasets, and colored lines denote three scenarios in which each group’s calibration error $\left(\left|\Delta \right|\right)$ varies. Although MC is identical in all scenarios, PMC loss is higher among groups with lower positivity rates in most scenarios unless the groupwise calibration error increases with positivity rate.

Results on the hospital admission prediction task are summarized in Figs. 3 to 5 and Table 4. As Table 4 shows, PMCBoost has a small effect on predictive performance $(\Delta \text{AUROC}<0.6\text{\%})$ while improving DC loss and PMC loss by 10–81% across LR, RF, and DNN models. Somewhat surprisingly, PMCBoost improves MC loss more effectively than MCBoost for both DNN (19% versus 11%) and RF models (18% versus 6%). We observe that PMCBoost and MCBoost do not improve MC loss of LR models, although PMCBoost improves other metrics of fairness for LR (PMC loss and DC loss) without changing AUROC. In Fig. 4, we illustrate the calibration improvement of RF models using PMCBoost for three patient subgroups defined by the intersection of race/ethnicity, gender, and insurance type.

Figure 3:

A comparison of LR, RF, and DNN models, with and without MCBoost and PMCBoost, on the hospital admission task. From left to right, trained models are compared in terms of test set AUROC, MC loss, PMC loss, and DC loss. Points represent the median performance over 100 shuffled train/test splits with bootstrapped 99% confidence intervals. We test for significant differences between post-processing methods using two-sided Wilcoxon rank-sum tests with Bonferroni correction. $\text{ns}:p<=1;{}^{**}:1\text{e-}03<p<=1\text{e-}02;{}^{***}:1\text{e-}04<p<=1\text{e-}03;{}^{****}:p<=1\text{e-}04$.

Figure 5:

Number of iterations to convergence for PMCBoost and MCBoost when optimizing for the same value of $\alpha$-PMC $\left({\alpha }_{\mathit{\text{PMCe}}}\right)$.

Table 4:

Median percent change in loss from the base model using MC- and PMCBoost.

		AUROC	MC	PMC	DC
	ML	Δ%	Δ%	Δ%	Δ%
MCBoost	DNN	−0.58	−10.83	−16.27	−2.41
	LR	−0.01	10.95	−9.50	−6.46
	RF	−0.09	−5.91	−55.44	−13.88
PMCBoost	DNN	−0.56	−19.49	−28.83	−9.91
	LR	0.01	14.18	−37.90	−14.90
	RF	−0.27	−17.53	−80.82	−23.52

Figure 4:

Calibration curves for RF models with and without PMCBoost postprocessing.

Computational Cost

As mentioned in Section 3.1, MCBoost may be configured to satisfy $\alpha$-PMC by setting $\alpha$ to a much smaller value, although this may take an excessive number of steps. To better understand this trade-off, we empirically compared MC- and PMCBoost by the number of steps required for each to reach their best performance in Fig. 5. On average, MCBoost requires approximately 5x more updates to achieve similar performance on PMC loss as PMCBoost, due to its dependence on very small values of $\alpha$. Wall clock times scale similarly, as detailed in Table 5.

Table 5:

For MCBoost and PMCBoost, we compare the average number of updates and wall clock time (s) taken to train for the equivalent values of $\alpha$-PMC.

			Iterations	Time (s)
ML	${\alpha }_{\mathit{\text{PMCe}}}$	Postprocessing
LR	0.001	MC	2094.0	678.0
		PMC	263.0	320.0
	0.010	MC	631.0	223.0
		PMC	153.0	202.0
RF	0.001	MC	1996.0	556.0
		PMC	323.0	234.0
	0.010	MC	713.0	197.0
		PMC	256.0	167.0

Sensitivity Analysis

In Appendix A.6, we look at detailed performance comparisons of MCBoost and PMCBoost over values of $\alpha$ and group definitions in Figs. 8 to 10. We observe that, while low values of $\alpha$ for MCBoost improve its PMC loss performance, PMCBoost typically performs as well or better, particularly for larger values of $\alpha$, and does so in fewer iterations.

6. Discussion and Conclusion

In this paper we have analyzed multicalibration through the lens of sufficiency and differential calibration to reveal the sensitivity of this metric to correlations between outcome rates and group membership. We have proposed a measure, PMC, that alleviates this sensitivity and attempts to capture the “best of both worlds” of MC and sufficiency. PMC provides equivalent percentage calibration protections to groups regardless of their risk profiles, and in so doing, bounds a model’s differential calibration. We provide an efficient algorithm for learning PMC predictors by postprocessing a given risk prediction model. On a real-world and clinically relevant task (admission prediction), we have shown that post-processing three types of models with PMC leads to better performance across all three fairness metrics, with a small impact on predictive performance.

Our preliminary analysis suggests PMC can be a valuable metric for training fair algorithms in resource allocation contexts. Future work could extend this analysis on both the theoretical and practical side. On the theoretical side, the generalization properties of the PMC measure should be established and its sample complexity quantified, as Rose (2018) did with MC. Additional extensions of PMC could establish a bound on the accuracy of PMC-postprocessed models in a similar vein to work by Kim et al. (2019) and Hebert-Johnson et al. (2018). On the empirical side, future works should benchmark PMC on a larger set of real-world problems, and explore use cases in more depth. In addition, user studies could be employed to validate whether proportionally multicalibrated models do in fact instill more trust in decision-makers compared to baseline models.

Figure 1:

Visualization of the constraint differences between multicalibration and proportional multicalibration (PMC). The filled area represents the maximum the predicted risk can deviate from the fraction of positives in the population, for any group. Below $\rho$, the constraint is constant for PMC.

Appendix A. Appendix

In this section, we include additional comparisons to related work, additional definitions, proofs to the theorems in the main text, and additional experimental details. The code to reproduce the figures and experiments is available here: https://github.com/cavalab/proportional-multicalibration.

A.1. Related Work

Definitions of Fairness

There are myriad ways to measure fairness that are covered in more detail in other works (Barocas et al., 2019; Chouldechova and Roth, 2018; Castelnovo et al., 2021). We briefly review three notions here. The first, demographic parity, requires the model’s predictions to be independent of patient demographics $\left(A\right)$. Although a model satisfying demographic parity can be desirable when the outcome should be unrelated to sensitive attributes (Foulds and Pan, 2020), it can be unfair if important risk factors for the outcome are associated with those attributes (Hardt et al., 2016). For example, it may be more fair to admit socially marginalized patients to a hospital at a higher rate if they are assessed less able to manage their care at home. Furthermore, if the underlying rates of illness vary demographically, requiring demographic parity can result in a healthier patients from one group being admitted more often than patients who urgently need care.

When the base rates of admission are expected to differ demographically, we can instead ask that the model’s errors be balanced across groups. One such notion is equalized odds, which states that for a given $Y$, the model’s predictions should be independent of $A$. Satisfying equalized odds is equivalent to having equal FPR and FNR for every group in $A$.

When the model is used for patient risk stratification, as in the target use case in this paper, it is important to consider a model’s calibration for each demographic group in the data. Because risk prediction models influence who is prioritized for care, an unfairly calibrated model can systematically under-predict risk for certain demographic groups and result in under-allocation of patient care to those groups. Thus, guaranteeing group-wise calibration via an approach such as multicalibration also guarantees fair patient prioritization for health care provision. In some contexts, risk predictions are not directly interpreted, but only used to rank patients, which in some contexts is sufficient for resource allocation. Authors have proposed various ways of measuring the fairness of model rankings, for example by comparing AUROC between groups (Kallus et al., 2020).

Approaches to Fairness

Many approaches to achieving fairness guarantees according to demographic parity, equalized odds and its relaxations have been proposed (Dwork et al., 2012; Hardt et al., 2016; Berk et al., 2017; Jiang and Nachum, 2019; Kearns et al., 2018). When choosing an approach, is important to carefully weigh the relative impact of false positives, false negatives, and miscalibration on patient outcomes, which differ by use case. When group base rates differ (i.e., group-specific positivity rates), equalized odds and calibration by group cannot both be satisfied (Kleinberg et al., 2016). Instead, one can often equalized multicalibration while satisfying relaxations of equalized odds such as equalized accuracy, where $\mathit{\text{Accuracy}}=\mu TPR+\left(1-\mu \right)\left(1-FPR\right)$ for a group with base rate $\mu$. However, to do so may require denigrating the performance of the model on specific groups (Chouldechova, 2017; Pleiss et al., 2017), which is unethical in our context.

As mentioned in the introduction, we are also motivated to utilize approaches to fairness that 1) dovetail well with intersectionality theory, and 2) provide privacy guarantees. Most work in the computer science/machine learning space does not engage with the broader literature on socio-cultural concepts like intersectionality, which we see as a gap that makes adoption in real-world settings difficult (Hanna et al., 2020). One exception to this statement is differential fairness (Foulds et al., 2019a), a measure designed with intersectionality in mind. In addition to being a definition of fairness that provides equal protection to groups defined by intersections of protected attributes, models satisfying $\epsilon$-differential fairness also satisfy $\epsilon$-pufferfish privacy. This privacy guarantee is very desirable in risk prediction contexts, because it limits the extent to which the model reveals sensitive information to a decision maker that has the potential to influence their interpretation of the model’s recommendation. However, prior work on differential fairness has been limited to using it to control for demographic parity, which is not an appropriate fairness measure for our use case (Foulds and Pan, 2020).

Multicalibration has inspired several extensions, including relaxations such as multiaccuracy (Kim et al., 2019), low-degree multicalibration (Gopalan et al., 2022), and extensions to conformal prediction and online learning (Jung et al., 2021; Gupta et al., 2021). Noting that multicalibration is a guarantee over mean predictions on a collection of groups $𝒞$, Jung et al. (2021) propose to extend multicalibration to higher-order moments (e.g., variances), which allows one to estimate a confidence interval for the calibration error for each category. Gupta et al. (2021) extend this idea and generalize it to the online learning context, in which an adversary chooses a sequence of examples for which one wishes to quantify the uncertainty of different statistics of the predictions. Recent work has also utilized higher order moments to “interpolate” between the guarantees provided by multiaccuracy, which only requires accuracy in expectation for groups in $𝒞$, and multicalibration, which requires accuracy in expectation at each prediction interval (Kim et al., 2019). Like proportional multicalibration (Definition 6), definitions of multicalibration for higher order moments provide additional criteria for quantifying model performance over many groups; in general, however, much of the focus in other work is on statistics for uncertainty estimation. Like these works, one may view our proposal for proportional multicalibration as alternative definition of what it means to be multicalibrated. The key difference is that proportional multicalibration measures the degree to which multicalibration depends on differences in outcome prevalence between groups, and in doing so provides guarantees of pufferfish privacy and differential calibration.

Dwork et al. (2019) study the relation of fair rankings to multicalibration, and, in a similar vein to differential fairness measures, formulate a fairness measure for group rankings using the relations between pairs of groups. However, these definitions are specific to the ranking relation between the groups, whereas differential calibration cares only about the outcome differential (conditioned on model predictions) between pairs of groups.

A.1.1. Differential Fairness

DF was explicitly defined to be consistent with the social theoretical framework of intersectionality. This framework dates back as early as the social movements of the ‘60s and ‘70s (Collins and Bilge, 2020) and was brought into the academic mainstream by pioneering work from legal scholar Kimberlé Crenshaw (Crenshaw, 1989, 1991) and sociologist Patricia Hill Collins (Collins, 1990). Central to intersectionality is that hierarchies of power and oppression are structural elements that are fundamental to our society. Through an intersectional lens, these power structures are viewed as interacting and co-constituted, inextricably related to one another. To capture this viewpoint, DF (Foulds et al., 2019a) constrains the differential of a general data mechanism among all pairs of groups, where groups are explicitly defined as the intersections of protected attributes in $𝒜$.

Definition 12 ($\epsilon$-differential fairness) (Foulds et al., 2019a) Let $\Theta$ denote a set of distributions and let $x~\theta$ for $\theta \in \Theta$. A mechanism $M\left(x\right)$ is $\epsilon$-differentially fair with respect to $\left(𝒞,\Theta \right)$ for all $\theta \in \Theta$ with $x~\theta$, and $m\in$ Range $\left(M\right)$ if, for all $\left(S_i,S_j\right)\in 𝒞\times 𝒞$ where $P\left(S_i\mid \theta \right)>0,P\left(S_j\mid \theta \right)>0$

$$e^{-\epsilon }\le \frac{P_{M,\theta }\left(M\left(x\right)=m\mid S_i,\theta \right)}{P_{M,\theta }\left(M\left(x\right)=m\mid S_j,\theta \right)}\le e^{\epsilon }$$ (3)

Definition 13 (Pufferfish Privacy) Let the collection of subsets $𝒞$ represent sets of secrets. A mechanism $M\left(x\right)$ is $\epsilon$-pufferfish private (Kifer and Machanavajjhala, 2014) with respect to $\left(𝒞,\Theta \right)$ if for all $\theta \in \Theta$ with $x~\theta$, for all secret pairs $\left(S_i,S_j\right)\in 𝒞\times 𝒞$ and $y\in \mathit{\text{Range}}\left(M\right)$,

$$e^{-\epsilon }\le \frac{P_{M,\theta }\left(M\left(x\right)=y\mid S_i,\theta \right)}{P_{M,\theta }\left(M\left(x\right)=y\mid S_j,\theta \right)}\le e^{\epsilon },$$ (4)

when $S_i$ and $S_j$ are such that $P\left(S_i\mid \theta \right)>0,P\left(S_j\mid \theta \right)>0$

Note on pufferfish and differential privacy

Although Eq. (3) is notable in its similarity to differential privacy (Dwork and Lei, 2009), they differ in important ways. Differential privacy aims to limit the amount of information learned about any one individual in a database by computations performed on the data (e.g. $M\left(x\right))$. Pufferfish privacy only limits information learned about the group membership of individuals as defined by $𝒞$. Kifer and Machanavajjhala (2014) describe in detail the conditions under which these privacy frameworks are equivalent.

Efficiency Property

Foulds et al. (2019a) also define an interesting property of $\epsilon$ differential fairness that allows guarantees of higher order (i.e., marginal) groups to be met for free; the property is given in Appendix A.3.2.

Definition 14 (Efficiency Property) (Foulds et al., 2019a) Let $M\left(x\right)$ be an $\epsilon$-differentially fair mechanism with respect to $\left(𝒞,\Theta \right)$. Let the collection of subsets $𝒞$ group individuals according to the Cartesian product of attributes $A\subseteq 𝒜$. Let $𝒢$ be any collection of subsets that groups individuals by the Cartesian product of attributes in $A^{\prime }$, where $A^{\prime }\subset A$ and $A^{\prime }\ne \varnothing$. Then $M\left(x\right)$ is $\epsilon$-differentially fair in $\left(𝒢,\Theta \right)$.

The authors call this the “intersectionality property”, although in practice it guarantees the reverse: if a model satisfies $\epsilon$-DF for the low level (i.e. intersectional) groups in $𝒞$, then it satisfies $\epsilon$-DF for every higher-level (i.e. marginal) group. For example, if a model is $\left(\epsilon \right)$-differentially fair for intersectional groupings of individuals by race and sex, then it is $\epsilon$-DF for the higher-level race and sex groupings as well. Whereas the number of intersections grows exponentially as additional attributes are protected (Kearns et al., 2018), the number of total possible subgroupings grows at a larger combinatorial rate: for $p$ protected attributes, we have ${\sum }_{k=1}^p\left(\begin{array}{l}p\hfill \\ k\hfill \end{array}\right)m_a^k$ groups, where $m_a$ is the number of levels of attribute $a$.

Limitations

To date, analysis of DF for predictive modeling has been limited to defining $R\left(x\right)$ as the mechanism, which is akin to asking for demographic parity. Under demographic parity, one requires that model predictions be independent from group membership entirely, and this limits the utility of it as a fairness notion. Although a model satisfying demographic parity can be desirable when the outcome should be unrelated to $𝒞$ (Foulds and Pan, 2020), it can be unfair if important risk factors for the outcome are associated with demographics (Hardt et al., 2016). For example, if the underlying rates of an illness vary demographically, requiring demographic parity can result in a healthier patients from one group being admitted more often than patients who urgently need care.

A.2. Proofs for Theorems in the Main Text

Theorem 4 Let $R\left(x\right)$ be a model satisfying $\alpha -MC$ on a collection of subsets $𝒞\in 2^{𝒳}$. Let $r_{\text{min }}=mi{n}_{S\in 𝒞}{\mathbb{E}}_D\left[R\mid R=r,x\in S\right]$ be the minimum expected risk prediction among $S\in 𝒞$ and $r\in \left[0,1\right]$. Then $R\left(x\right)$ is $\left(ln\frac{r_{min}+\alpha }{r_{min}-\alpha }\right)$. differentially calibrated.

Proof Let $r={\mathbb{E}}_D\left[R\mid R=r,x\in S\right]$ and $p^{*}={\mathbb{E}}_D\left[y\mid R=r,x\in S\right].\alpha$-MC guarantees that $r-\alpha \le p^{*}\le r+\alpha$ for all groups $S\in 𝒞$ and predictions $r\in$[0, 1]. Plugging these lower and upper bounds into Eq. (2), we observe that the lower bound on $\epsilon$-DC for $R\left(x\right)$ is given by $\frac{r+\alpha }{r-\alpha }\le e^{\epsilon }$. The maximum of the left-hand side for a fixed $\alpha$ occurs at the smallest value of $r$; therefore $R\left(x\right)$ satisfies $\text{ln}\frac{r_{\text{min}}+\alpha }{r_{\text{min}}-\alpha }\le \epsilon$. By switching the numeratator and denominator we obtain the minimum differential and the left-hand side constraint from Definition 3, i.e. $e^{-\epsilon }\le \frac{r_{\text{min}}-\alpha }{r_{\text{min}}+\alpha }$. Thus $R\left(x\right)$ is $\left(\text{ln}\frac{r_{\text{min}}+\alpha }{r_{\text{min}}-\alpha }\right)$-differentially calibrated. ■

Theorem 7 Let $R\left(x\right)$ be a model satisfying $\left(\alpha \right)-PMC$ on a collection $𝒞$. Then $R\left(x\right)$ is $\left(ln\frac{1+\alpha }{1-\alpha }\right)$-differentially calibrated.

Proof Let $r={\mathbb{E}}_D\left[R\mid R=r,x\in S\right]$ and $p^{*}={\mathbb{E}}_D\left[y\mid R=r,x\in S\right]$. If $R\left(x\right)$ satisfies $\alpha$-PMC (Definition 6), then $r/\left(1+\alpha \right)\le p^{*}\le r/\left(1-\alpha \right)$. Solving for the upper bound on $\epsilon$-DC, we immediately have $\epsilon \ge \text{ln}\frac{r\left(1+\alpha \right)}{r\left(1-\alpha \right)}\ge \text{ln}\frac{1+\alpha }{1-\alpha }$. ■

Theorem 9 Let $R\left(x\right)$ be a model satisfying $\alpha$-PMC on a collection $𝒞$. Then $R\left(x\right)$ is $\left(\frac{\alpha }{1-\alpha }\right)$-multicalibrated on $𝒞$.

Proof To distinguish the parameters, let $R\left(x\right)$ be a model satisfying $\delta$-PMC. Let $r={\mathbb{E}}_D\left[R\mid R=r,x\in S\right]$ and $p^{*}={\mathbb{E}}_D\left[y\mid R=r,x\in S\right]$. Then $r/\left(1+\delta \right)\le p^{*}\le r/\left(1-\delta \right)$. We solve for the upper bound on $\alpha$-MC from Definition 2 for the case when $p^{*}>r$. This yields

$$\begin{gathered} \alpha \le p^{*}-r \\ \le \frac{r}{1-\delta }-r \\ =r\frac{\delta }{1-\delta } \\ \le \frac{\delta }{1-\delta }. \end{gathered}$$

We can also solve for the lower bound on $\alpha$-MC from Definition 2 for the case when $p^{*}<r$. This yields

$$\begin{gathered} \alpha \le r-p^{*} \\ \le r-\frac{r}{1+\delta } \\ =r\frac{\delta }{1+\delta } \\ \le \frac{\delta }{1+\delta }. \end{gathered}$$

For any $\delta >0,\frac{\delta }{1-\delta }>\frac{\delta }{1+\delta }$. Therefore the first case $\left(p^{*}>r\right)$ limits the multicalibration of $R\left(x\right)$.■

Proposition 11 Define $\alpha ,\lambda ,\gamma ,\rho >0$. Let $𝒞\subseteq 2^{𝒳}$ be a collection of subsets of $𝒳$ such that, for all $S\in 𝒞,P_D\left(S\right)>\gamma$. Let $R\left(x\right)$ be a risk prediction model to be post-processed. For all $\left(S,I\right)\in 𝒞\times {\Lambda }_{\lambda }$, let $E\left[y\mid R\in I,x\in S\right]>\rho$. There exists an algorithm that satisfies $\left(\alpha ,\lambda \right)-PMC$ with respect to $𝒞$ in $O\left(\frac{\left|C\right|}{{\alpha }^3{\lambda }^2{\rho }^2\gamma }\right)$ steps.

Proof

We show that Algorithm 1 converges using a potential function argument (Bansal and Gupta, 2019), similar to the proof techniques for the MC boosting algorithms in Hebert-Johnson et al. (2018); Kim et al. (2019). Let $p_i^{*}$ be the underlying risk, $R_i$ be our initial model, and $R_i^{\text{'}}$ be our updated prediction model for individual $i\in S_r$, where $S_r=\left\{x\mid x\in S,R\left(x\right)\in I\right\}$ and $\left(S,I\right)\in 𝒞\times {\Lambda }_{\lambda }$. We use $p^{*},R$, and $R^{\prime }$ without subscripts to denote these values over $S_r$. We cannot easily construct a potential argument using progress towards $\left(\alpha ,\lambda \right)$-PMC, since its derivative is undefined at ${\mathbb{E}}_D\left[y\mid R\in I,x\in S\right]=0$. Instead, we analyze progress towards the difference in the ${\ell }_2$ norm at each step.

$$\begin{gathered} \Vert p^{*}-R\Vert -\Vert p^{*}-R^{\prime }\Vert \\ =\sum _{i\in S_r}{\left(p_i^{*}-R_i\right)}^2-\sum _{i\in S_r}{\left(p_i^{*}-\text{squash}\left(R_i+\Delta r\right)\right)}^2 \\ \ge \sum _{i\in S_r}\left({\left(p_i^{*}-R\right)}^2-{\left(p_i^{*}-\left(R_i+\Delta r\right)\right)}^2\right) \\ =\sum _{i\in S_r}\left(2p_i^{*}\Delta r-2R_i\Delta r-\Delta r^2\right) \\ =2\Delta r\sum _{i\in S_r}\left(p_i^{*}-R_i\right)-\left|S_r\right|\Delta r^2 \end{gathered}$$ (5)

From Algorithm 1 we have

$$\Delta r=\frac{1}{\left|S_r\right|}\sum _{i\in S_r}\left(p_i^{*}-R_i\right)$$

Substituting into Eq. (5) gives

$$\Vert p^{*}-R\Vert -\Vert p^{*}-R^{\prime }\Vert \ge \left|S_r\right|\Delta r^2$$

We know that $\left|S_r\right|\ge \alpha \lambda \gamma N$, and that the smallest update $\Delta r$ is $\alpha \rho$. Thus,

$$\Vert p^{*}-R\Vert -\Vert p^{*}-R^{\prime }\Vert \ge {\alpha }^3{\rho }^2\lambda \gamma N$$

Since our initial loss, $\Vert p^{*}-R\Vert$, is at most $N$, Algorithm 1 converges in at most $O\left(\frac{1}{{\alpha }^3{\rho }^2\lambda \gamma }\right)$ updates for category $S_r$.

To understand the total number of steps, including those without updates, we consider the worst case, in which only a single category $S_r$ is updated in a cycle of the for loop (if no updates are made, the algorithm exits). Since each repeat consists of at most $\left|C\right|/\lambda$ loop iterations, this results in $O\left(\frac{\left|C\right|}{{\alpha }^3{\lambda }^2{\rho }^2\gamma }\right)$ total steps. ■

A.3. Extended Theoretical Analysis

Illustrating Relationships between Definitions

Fig. 6 shows how the definitions of MC, DC, and PMC are related. In each subplot, the x and y coordinates map the guarantee from one metric (x axis) to the implied guarantee in the other metric (y axis).

The right panel of Fig. 6 illustrates this relation in comparison to the DC-MC relationship described in Appendix A.4, Theorem 22. At small values of $\epsilon$ and $\alpha$ and when the model is perfectly calibrated overall, $\alpha$-PMC and $\epsilon$-DC behave similarly. However, given $\delta >0$, $\epsilon$-differentially calibrated models suffer from higher MC error than proportionally calibrated models when $\alpha$-PMC < 0.3. The right graph also illustrates the feasible range of $\alpha$ for $\alpha$-PMC is $0<\alpha <0.5$, past which it does not provide meaningful $\alpha$-MC. The steeper relation between $\alpha$-PMC and MC may have advantages or disadvantages, depending on context. It suggests that, by optimizing for $\alpha$-PMC, small improvements to this measure can result in relatively large improvements to MC; conversely, $\epsilon$-DC models that are well calibrated may satisfy a lower value of $\alpha$-MC over a larger range of $\epsilon$.

A.3.1. Discretization

To clarify and simplify our analysis, we work mainly with the continuous versions of multicalibration and proportional multicalibration, under the assumption that minimizing the discretized versions (i.e., binning $\left(x\right)$) will translate to low values of the continous version. In this section we provide detailed bounds on the continuous versions of PMC and DC that are implied by the discretized versions.

Figure 6:

A comparison of $\epsilon$-DC, $\alpha$-MC, and $\alpha$-PMC in terms of their parameters $\alpha$ and $\epsilon$. In both panes, the x value is a given value of one metric for a model, and the y axis is the implied value of the other metric, according to Theorem 22-Theorem 9. The left filled area denotes the dependence of the privacy/DC of $\alpha$-multicalibrated models on the minimum risk interval, $r_{\text{min}}\in \left[0.01,1.0\right]$. The right filled area denotes the dependence of the MC of $\epsilon$-differentially calibrated models on their overall calibration, $\delta \in \left[0.0,0.5\right].\alpha$-PMC does not have these sensitivities.

First, we will formally define two different discretization schemes. The first, $\lambda$-discretization, defines equally spaced bins on the interval [0, 1], as follows.

For ensuring multiplicative closeness under PMC, it can be useful to instead discretize the prediction bins so that the bins are equally spaced on a log scale. We define such a discretization below.

Definition 15 ($\left(\lambda ,\rho \right)$-geometric discretization.) Let $\lambda \in \left[0,1\left],\rho \in \right[0,1\right]$. The $\left(\lambda ,\rho \right)$-geometric discretization of [0, 1] is denoted by a set of intervals, ${\Lambda }_{\lambda }^{\rho }=\left\{{\left\{I_j\right\}}_{j=0}^{1/\lambda -1}\right\}$, where $I_j=\left[{\rho }^{\left(1-j\lambda \right)},{\rho }^{\left(1-j\lambda -\lambda \right)}\right)$.

(Hebert-Johnson et al., 2018) define a discretized version of MC in which $R\left(x\right)$ is binned according to a discretization parameter, $\lambda$:

Definition 16 $\left(\alpha ,\lambda \right)$-multicalibration) Let $𝒞\subseteq 2^{𝒳}$ be a collection of subsets of $𝒳$. For any $\alpha ,\lambda >0$, a predictor $R$ is $\left(\alpha ,\lambda \right)$-multicalibrated on $𝒞$ if, for all $I\in {\Lambda }_{\lambda }$ and $S\in 𝒞$ where $P_D\left(R\in I\mid x\in S\right)\ge \alpha \lambda$

$$\left|\underset{D}{\mathbb{E}}\left[y\mid R\in I,x\in S\right]-\underset{D}{\mathbb{E}}\left[R\mid R\in I,x\in S\right]\right|\le \alpha .$$

(Hébert-Johnson et al., 2018) establish that $\left(\alpha ,\lambda \right)$-multicalibrated models are at most $\left(\alpha +\lambda \right)$-multicalibrated. In an analagous fashion, we show below that $\left(\alpha ,\lambda \right)$-PMC implies $\left(\alpha +\lambda /\rho \right)$-PMC for bins defined by a $\lambda$-discretization. When using a $\left(\lambda ,\rho \right)$-geometric discretization, $\left(\alpha ,\lambda \right)$-PMC implies $\left(\alpha {\rho }^{-\lambda }+{\rho }^{-\lambda }-1\right)$-PMC, which can be a tighter bound than the former.

Claim 1 Define $\rho ,\alpha ,\lambda >0$ and let $𝒞\subseteq 2^{𝒳}$ be a collection of subsets of $𝒳$. Let ${\mathbb{E}}_D[y\mid R\in I,x\in S]\ge \rho$ for all $S\in 𝒞$ and $I\in {\Lambda }_{\lambda }$. Let $R\left(x\right)$ be a model satisfying $\left(\alpha ,\lambda \right)$-proportional multicalibration. Then $R\left(x\right)$ is at most $\left(\alpha +\frac{\lambda }{\rho }\right)$-proportionally multicalibrated.

Proof

By Definition 18, $R\left(x\right)$ satisfies

$$\frac{\left|{\mathbb{E}}_D\left[y\mid R\in I,x\in S\right]-{\mathbb{E}}_D\left[R\mid R\in I,x\in S\right]\right|}{{\mathbb{E}}_D\left[y\mid R\in I,x\in S\right]}\le \alpha$$

for categories $\left(S,I\right)\in 𝒞\times {\Lambda }_{\lambda }$ satisfying $P_D\left(R\left(x\right)\in I\mid x\in S\right)\ge \alpha \lambda$. Given $1/\lambda$ bins, the subset where $P_D\left(R\left(x\right)\in I\mid x\in S\right)<\alpha \lambda$ has a size of at most $\alpha \left|S\right|$. Therefore there is a subset $\left|S^{\prime }\right|\ge \left(1-\alpha \right)\left|S\right|$ where for all $r\in {\Lambda }_{\lambda },\alpha$-PMC (Definition 6) is satisfied.

Let $\delta$ be the constaint on $\delta$-PMC. Let $p^{*}={\mathbb{E}}_D\left[y\mid R=r,x\in S\right]$ and $r={\mathbb{E}}_D\left[R\mid R=r,x\in S\right]$. Consider the case $r>p^{*}$ and let $\alpha =\left(r-p^{*}\right)/p^{*}$. $\lambda$-discretization shifts $r$ by at most $\lambda$. Let

$$\delta \le \left(r+\lambda -p^{*}\right)/p^{*}$$

Substituting $r\le \alpha p^{*}+p^{*}$ yields

$$\delta \le \alpha +\frac{\lambda }{p^{*}}$$

Plugging in $\rho$ as the minimum of $p^{*}$, we complete the proof.■

The term $\frac{\lambda }{\rho }$ can be potentially large when $\rho <\lambda$. One way to avoid this issue is to make the change in $R\left(x\right)$ between bins scale with $R\left(x\right)$ using Definition 15. What makes Definition 15 different from $\lambda$-discretization is that the intervals are a multiplicative, rather than additive, distance apart. Hence, for a given $r\in \left[0,1\right]$, a model satisfying $\left(\alpha ,\lambda \right)$-PMC can have its prediction shift by at most a factor of ${\rho }^{-\lambda }$. This leads us to the following claim.

Claim 2 Define $\rho ,\alpha ,\lambda >0$ and let $𝒞\subseteq 2^{𝒳}$ be a collection of subsets of $𝒳$. Let ${\mathbb{E}}_D\left[y\mid R\in I,x\in S\right]\ge \rho$ for all $S\in 𝒞$ and $I\in {\Lambda }_{\lambda }$. Let $R\left(x\right)$ be a model satisfying $\left(\alpha ,\lambda \right)$-proportional multicalibration. Given a $\left(\lambda ,\rho \right)$-geometric discretization, $R\left(x\right)$ is at most $\left(\alpha {\rho }^{-\lambda }+{\rho }^{-\lambda }-1\right)$-proportionally multicalibrated.

Proof

By Definition 18, $R\left(x\right)$ satisfies

$$\frac{\left|{\mathbb{E}}_D\left[y\mid R\in I,x\in S\right]-{\mathbb{E}}_D\left[R\mid R\in I,x\in S\right]\right|}{{\mathbb{E}}_D\left[y\mid R\in I,x\in S\right]}\le \alpha$$

for categories $\left(S,I\right)\in 𝒞\times {\Lambda }_{\lambda }$ satisfying $P_D\left(R\left(x\right)\in I\mid x\in S\right)\ge \alpha \lambda$. Given $1/\lambda$ bins, the subset where $P_D\left(R\left(x\right)\in I\mid x\in S\right)<\alpha \lambda$ has a size of at most $\alpha \left|S\right|$. Therefore there is a subset $\left|S^{\prime }\right|\ge \left(1-\alpha \right)\left|S\right|$ where for all $r\in {\Lambda }_{\lambda },\alpha$-PMC (Definition 6) is satisfied.

Let $\delta$ be the constaint on $\delta$-PMC. Let $p^{*}={\mathbb{E}}_D\left[y\mid R=r,x\in S\right]$ and $r={\mathbb{E}}_D\left[R\mid R=r,x\in S\right]$. Consider the case $r>p^{*}$ and let $\alpha =(r-p^{*})/p^{*}$, i.e. the tight bound. $\left(\lambda ,\rho \right)$-geometric discretization shifts $r$ by at most a factor of ${\rho }^{-\lambda }$. This implies

$$\delta \le \left(r{\rho }^{-\lambda }-p^{*}\right)/p^{*}$$

Substituting $r=\alpha p^{*}+p^{*}$ yields

$$\delta \le \alpha {\rho }^{-\lambda }+{\rho }^{-\lambda }-1.$$

■

Figure 7:

Relationship between $\left(\alpha ,\lambda \right)$-PMC and $\alpha$-PMC given a geometric discretization. Illustrated for $\left(\alpha ,\lambda \right)-\text{PMC}=0.1$, for various values of $\rho$ and $\lambda$.

We illustrate the relationship between $\left(\alpha ,\lambda \right)$-PMC and $\alpha$-PMC given a geometric discretization in Fig. 7, which quantifies the relationship for different settings of $\lambda$ and $\rho$.

A.3.2. Additional Definitions

Definition 17 ($\lambda$-discretization.) Given $\lambda \in [0,1]$, the $\lambda$-discretization of $\left[0,1\right]$ is denoted by a set of intervals, ${\Lambda }_{\lambda }=\left\{{\left\{I_j\right\}}_{j=0}^{1/\lambda -1}\right\}$, where $I_j=\left[j\lambda ,\left(j+1\right)\lambda \right)]$.

Definition 18 $\left(\alpha ,\lambda \right)$-PMC) $A$ model $R\left(x\right)$ is $\left(\alpha ,\lambda \right)$-proportionally multicalibrated with respect to a collection of subsets $𝒞$ if, for all $S\in 𝒞$ and $I\in {\Lambda }_{\lambda }$ satisfying $P_D\left(R\in I\mid x\in S\right)\ge \alpha \lambda$,

$$\frac{\left|{\mathbb{E}}_D\left[y\mid R\in I,x\in S\right]-{\mathbb{E}}_D\left[R\mid R\in I,x\in S\right]\right|}{{\mathbb{E}}_D\left[y\mid R\in I,x\in S\right]}\le \alpha .$$ (6)

The following loss functions are empirical analogs of the definitions of MC, PMC, and DC, and are used in the experiment section to measure performance.

Definition 19 (MC loss) Let $D={\left\{{(y,x)}_i\right\}}_{i=0}^N~D$, and let $\alpha ,\lambda ,\gamma >0$. Define a collection of subsets $𝒞\in 2^{𝒳}$ such that for all $S\in 𝒞,\left|S\right|\ge \gamma N$. Let $S_I=\left\{x:R\left(x\right)\in I,x\in S\right\}$ for $\left(S,I\right)\in 𝒞\times {\Lambda }_{\lambda }$. Define the collection $𝒮$ containing all $S_I$ satisfying $S_I\ge \alpha \lambda N$. The $MC$ loss of a model $R\left(x\right)$ on $𝒟$ is

$$\underset{S_I\in 𝒮}{\text{max}}\frac{1}{\left|S_I\right|}\left|\sum _{i\in S_I}{y}_i-\sum _{i\in S_I}{R}_i\right|$$

Definition 20 (PMC loss) Let $𝒟={\left\{{(y,x)}_i\right\}}_{i=0}^N~D$, and let $\alpha ,\lambda ,\gamma ,\rho >0$. Define a collection of subsets $𝒞\in 2^{𝒳}$ such that for all $S\in 𝒞,\left|S\right|\ge \gamma N$. Let $S_I=\left\{x:R\left(x\right)\in I,x\in S\right\}$ for $\left(S,I\right)\in 𝒞\times {\Lambda }_{\lambda }$. Define the collection $𝒮$ containing all $S_I$ satisfying $S_I\ge \alpha \lambda N$. Let $\frac{1}{\left|S_I\right|}{\sum }_{i\in S_I}{y}_i\ge \rho$. The PMC loss of a model $R\left(x\right)$ on $𝒟$ is

$$\underset{S_I\in 𝒮}{\text{max}}\frac{\left|{\sum }_{i\in S_I}{y}_i-{\sum }_{i\in S_I}{R}_i\right|}{{\sum }_{i\in S_I}{y}_i}$$

Definition 21 (DC loss) Let $𝒟={\left\{{(y,x)}_i\right\}}_{i=0}^N~D$, and let $\alpha ,\lambda ,\gamma >0.$ Define a collection of subsets $𝒞\in 2^{𝒳}$ such that for all $S\in 𝒞,\left|S\right|\ge \gamma N$. Given a risk model $R\left(x\right)$ and prediction intervals $I$, Let $S_I=\left\{x:R\left(x\right)\in I,x\in S\right\}$ for $\left(S,I\right)\in 𝒞\times {\Lambda }_{\lambda }$. Define the collection $𝒮$ containing all $S_I$ satisfying $S_I\ge \alpha \lambda N$. The DC loss of a model $R\left(x\right)$ on $𝒟$ is

$$\underset{\left(S_I^a,S_I^b\right)\in 𝒮\times 𝒮}{\text{max}}\text{ log}\left|\frac{1}{\left|S_I^a\right|}\sum _{i\in S_I^a}{y}_i-\frac{1}{\left|S_I^b\right|}\sum _{j\in S_I^b}{y}_j\right|$$

A.4. Additional Theorems

A.4.1. Differentially Calibrated models with global calibration are multicalibrated

Here we show that, under the assumption that a model is globally calibrated (satisfies $\delta$-calibration), models satisfying $\epsilon$-DC are also multicalibrated.

Theorem 22 Let $R\left(x\right)$ be a model satisfying $\left(\epsilon ,\lambda \right)-DC$ and $\delta$-calibration. Then $R\left(x\right)$ is $(1-e^{-\epsilon }+\delta ,\lambda )$-multicalibrated.

Proof

From Eq. (2) we observe that $\epsilon$ is bounded by the two groups with the largest and smallest group- and prediction- specific probabilities of the outcome. Let $I_M$ be the risk stratum maximizing $\left(\epsilon ,\lambda \right)$-DC, and let $p_n={\text{max}}_{S\in 𝒞}{P}_D(y\mid R\in I_M,x\in S)$ and $p_d={\text{min}}_{S\in 𝒞}{P}_D(y\mid R\in I_M,x\in S)$. These groups determine the upper and lower bounds of $\epsilon$ as $e^{-\epsilon }\le p_d/p_n$ and $p_n/p_d\le e^{\epsilon }$.

We note that $p_d\le P_D\left(y\mid R\in I_M\right)\le p_n$, since $P\left(y\mid R\in I_M\right)=\frac{1}{N}{\sum }_{S\in 𝒞}\left|S\right|P_D(y\mid R\in I_M,x\in S)$, and $p_n$ and $p_d$ are the extreme values of $P\left(y\mid R\in I_M,x\in S\right)$ among $S$. So, $\alpha$-MC is bound by the group outcome that most deviates from the predicted value, which is either $p_n$ or $p_d$. Let $r=P_D\left(R\mid R\in I_M\right)$. There are then two scenarios to consider:

1. $\alpha \le \left|p_n-r\right|=p_n-r$ when $r\le \frac{1}{2}\left(p_n+p_d\right)$; and

2. $\alpha \le \left|p_d-r\right|=r-p_d$ when $r\ge \frac{1}{2}\left(p_n+p_d\right)$.

We will look at the first case. Let $p_r^{*}=P_D\left(y\mid R\in I_M\right)$. Due to $\delta$-calibration, $p_r^{*}-\delta \le r\le p_r^{*}+\delta$. Then

$$\begin{gathered} \alpha \le p_n-r \\ \le p_n-\left(p_r^{*}-\delta \right) \\ \le p_n-p_d+\delta \\ =p_n\left(1-e^{-\epsilon }\right)+\delta \\ \alpha \le 1-e^{-\epsilon }+\delta . \end{gathered}$$

Above we have used the facts that $r\le p_r^{*}-\delta ,p_r^{*}\ge p_d,p_d\le e^{-\epsilon }{p}_n$, and $p_n\le 1$. The second scenario is complementary and produces the identical bound. ■

Theorem 22 formally describes how $\delta$-calibration controls the baseline calibration error contribution to $\alpha$-MC, while $\epsilon$-DC limits the deviation around this value by constraining the (log) maximum and minimum risk within each category.

A.4.2. Multicalibrated Models Satisfy Intersectional Guarantees

In contrast to DF, MC (Hebert-Johnson et al., 2018) was not designed to explicitly incorporate the principles of intersectionality. However, we show that it provides an identical efficiency property to DF in the theorem below. Given an individual’s attributes $x=\left(x_1,\dots ,x_d\right)$, it will be useful to refer to subsets we wish to protect, e.g. demographic identifiers. To do so, we define $𝒜=\left\{A_1,\dots ,A_p\right\},p\le d$, such that $A_1$ is the set of values taken by attribute $x_1$.

Theorem 23 Let the collection of subsets $𝒞\subseteq 2^{𝒳}$ define groups of individuals according to the Cartesian product of attributes $A\subseteq 𝒜$. Let $𝒢\in 2^{𝒳}$ be any collection of subsets that groups individuals by the Cartesian product of attributes in $A^{\prime }$, where $A^{\prime }\subset A$ and $A^{\prime }\ne \varnothing$. If $R\left(x\right)$ satisfies $\alpha -MC$ on $𝒞$, then $R\left(x\right)$ is $\alpha$-multicalibrated on $𝒢$.

In proving Theorem 23, we will make use of the following lemma.

Lemma 24 The $\alpha$-MC criteria can be rewritten as: for a collection of subsets $𝒞\subseteq 𝒳,\alpha \in \left[0,1\right]$, and $r\in \left[0,1\right]$,

$$\underset{c\in 𝒞}{\text{max}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in c\right]\le r+\alpha$$

and

$$\underset{c\in 𝒞}{\text{min}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in c\right]\ge r-\alpha$$

Proof The lemma follows from Definition 2, and simply restates it as a constraint on the maximum and minimum expected risk among groups at each prediction level. ■

Proof [Proof of Theorem 23] We use the same argument as Foulds et al. (2019a) in proving this property for DF. Define $Q$ as the Cartesian product of the protected attributes included in $𝒜$, but not ${𝒜}^{\prime }$. Then for any $\left(y,x\right)~D$

$$\begin{gathered} \underset{g\in 𝒢}{\text{max}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in g\right] \\ =\underset{g\in 𝒢}{\text{max}}\sum _{q\in Q}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in g\cap q\left]P\right[x\in q\mid x\in g\right] \end{gathered}$$ (7)

$$\le \underset{g\in 𝒢}{\text{max}}\sum _{q\in Q}\underset{q^{\prime }\in Q}{\text{max}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in g\cap q^{\prime }\right]P\left[x\in q\mid x\in g\right]$$ (8)

$$=\underset{g\in 𝒢}{\text{max}}\underset{q^{\prime }\in Q}{\text{max}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in g\cap q^{\prime }\right]$$ (9)

$$=\underset{c\in 𝒞}{\text{max}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in c\right].$$ (10)

Moving from Eq. (7) to Eq. (8) follows from substituting the maximum value of ${\mathbb{E}}_D\left[y\mid R\left(x\right)=r,x\right]$ for observations in the intersection of subsets in $𝒢$ and $Q$ which is the upper limit of the expression in Eq. (7). Moving from Eq. (8) to Eq. (9) follows from recognizing that the sum $P\left[x\in q\mid x\in g\right]$ for all subsets in $𝒬$ is 1. Finally, moving from Eq. (9) to Eq. (10) follows from recognizing that the intersections of subsets in $𝒢$ and $𝒬$ that satisfy Eq. (9), must define a subset of $𝒞$. Applying the same argument, we can show that

$$\underset{g\in 𝒢}{\text{min}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in g\right]\ge \underset{c\in 𝒞}{\text{min}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in c\right].$$

Substituting into Theorem 24,

$$\underset{g\in 𝒢}{\text{max}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in g\right]\le \alpha +r$$

and

$$\underset{g\in 𝒢}{\text{min}}\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in g\right]\ge r-\alpha$$

or

$$\left|\underset{D}{\mathbb{E}}\left[y\mid R\left(x\right)=r,x\in g\right]-r\right|\le \alpha$$

for all $g\in 𝒢$. Therefore $R\left(x\right)$ is $\alpha$-multicalibrated with respect to $𝒢$. ■

As a concrete example, imagine we have the protected attributes $A=\{\mathit{\text{race}}\in \left\{B,W\right\}$, gender $\in \left\{M,F\right\}\}$. According to Theorem 23, $𝒞$ would contain four sets: $\left\{\left(B,M\right),\left(B,F\right),\left(W,M\right),\left(W,F\right)\right\}.$ In contrast, there are eight possible sets in $𝒢:\left\{\left(B,M\right),\left(B,F\right),\left(W,M\right),\left(W,F\right),\left(B,*\right),\left(W,*\right),\left(*,M\right),\left(*,F\right)\right\}$, where the wildcard indicates a match to either attribute. As noted in Appendix A.1.1, the efficiency property is useful because the number of possible sets in $𝒢$ grows at a large combinatorial rate, rate as additional attributes are added; meanwhile $𝒞$ grows at a slower, yet exponential, rate. For an intuition for why this property holds, consider that the maximum calibration error of two subgroups is at least as large as the maximum expected error of those groups combined; e.g., the maximum calibration error in a higher order groups such as $\left(B,*\right)$ will be covered by the maximum calibration error in either $(B,M)$ or $(B,F)$.

A.5. Additional Experiment Details

Models

The deep neural network (DNN) was a five layer feed-forward NN with 100 units per layer and ReLU activations. We trained using an adam solver with a learning rate of 0.001, a batch size of 200 and used early stopping to terminate when a 10% validation set did not improve for 10 epochs.

Both LR and DNN models used median imputation and feature normalization as preprocessing steps. For the RF, we used the XGBoost implementation Chen and Guestrin (2016) which handles missing data natively.

Training

Models were trained on a heterogenous computing cluster. Each training instance was limited to a single core and 4 GB of RAM. We conducted a full parameter sweep of the parameters specified in Table 3. A single trial consisted of a method, a parameter setting from Table 3, and a random seed. Over 100 random seeds, the data was shuffled and split 75%/25% into train/test sets. Results in the manuscript are summarized over these test sets.

Code

Code for the experiments is available here: https://github.com/cavalab/proportional-multicalibration. Code is licensed under GNU Public License v3.0.

Data

We make use of data from the MIMICIV-ED repository, version 1.0, to train admission risk prediction models (Johnson et al., 2021). This resource contains more than 440,000 ED admissions from Beth Isreal Deaconness Medical Center between 2011 and 2019. We preprocessed these data to construct an admission prediction task in which our model delivers a risk of admission estimate for each ED visitor after their first visit to triage, during which vitals are taken. Additional historical data for the patient was also included (e.g., number of previous visits and admissions). A list of features is given in Table 2.

A.6. Additional Experimental Results

Table 3 lists a few parameters that may affect the performance of post-processing for both MC and PMC. Of particular interest when comparing MC versus PMC post-processing is the parameter $\alpha$, which controls how stringent the calibration error must be across categories to terminate, and the group definition $\left(A\right)$, which selects which features of the data will be used to asses and optimize fairness. We look at the performance of MC and PMC postprocessing over values of $\alpha$ and group definitions in Figs. 8 to 10. Finally, we empirically compare MC- and PMC-postprocessing by the number of steps required for each to reach their best performance in Fig. 11 and Table 5.

From Fig. 8, it is clear that post-processing has a minimal effect on AUROC in all cases; note the differences disappear if we round to two decimal places. When post processing with RF, we do note a relationship between lower values of $\alpha$ and a very slight decrease in performance, particularly for MC-postprocessing.

Figs. 9 and 10 show performance between methods on MC loss and PMC loss, respectively. In terms of MC loss, PMC-postprocessing tends to produce models with the lowest loss, at $\alpha$ values greater than 0.01. Lower values of $\alpha$ do not help MC-postprocessing in most cases, suggesting that these smaller updates may be overfitting to the post-processing data. In terms of PMC loss (Fig. 10), we observe that performance by MC-postprocessing is highly sensitive to the value of $\alpha$. For smaller values of $\alpha$, MC-postprocessing is able to achieve decent performance by these metrics, although in all cases, PMC-postprocessing generates a model with a better median loss value at some configuration of $\alpha$.

We assess how many steps/updates MC and PMC take for different values of $\alpha$ in Fig. 11, and summarize empirical measures of running time in Table 5. On the figure, we annotate the point for which each post-processing algorithm achieves the lowest median value of PMC loss across trials. Fig. 11 validates that PMC-postprocessing is more efficient than MC-postprocessing at producing models with low PMC loss, on average requiring 4.0x fewer updates to achieve its lowest loss on test. From Table 5 we observe that PMC typically requires a larger number of updates to achieve its best performance on MC loss (about 2x wall clock time and number of updates), whereas MC-postprocessing requires a larger number of updates to achieves its best performance on PMC loss and DC loss, due to its dependence on very small values of $\alpha$. We accompany these results with the caveat that they are based on performance on one real-world task, and wall clock time measurements are influenced by the heterogenous cluster environment; future work could focus on a larger empirical comparison.

Table 6:

The number of times each postprocessing method achieved the best score among all methods, out of 100 trials.

postprocessing metric	Base Model	MC	PMC
AUROC	5	88	6
MC loss	8	21	70
PMC loss	0	27	72
DC loss	0	36	63

We quantify how often the use of each postprocessing algorithm gives the best loss for each metric and trial in Table 6. PMCBoost (Algorithm 1) achieves the best fairness the highest percent of the time, according to DC loss (63%), MC loss (70%), and PMC loss (72%), while MC-postprocessed models achieve the best AUROC in 88% of cases. This provides strong evidence that, over a large range of $\alpha$ values, PMCBoost is beneficial compared to MCBoost.

Figure 8:

AUROC test performance versus $\alpha$ across experiment settings. Rows are different ML base models, and columns are different attributes used to define $𝒞$. The color denotes the post-processing method.

Figure 9:

MC loss test performance versus $\alpha$ across experiment settings. Rows are different ML base models, and columns are different attributes used to define $𝒞$. The color denotes the post-processing method.

Figure 10:

PMC loss test performance versus $\alpha$ across experiment settings. Rows are different ML base models, and columns are different attributes used to define $𝒞$. The color denotes the post-processing method

Figure 11:

Number of post-processing updates by MC and PMC versus $\alpha$ across experiment settings. Rows are different ML base models, and columns are different attributes used to define $𝒞$. The color denotes the post-processing method. Each result is annotated with the median PMC loss for that method and parameter combination.

Data and Code Availability

This paper uses the MIMIC-IV-ED dataset (Johnson et al., 2021), which is available on the PhysioNet repository (Goldberger et al., 2000). Code for the experiments is available here: https://github.com/cavalab/proportional-multicalibration.