Regression with race-modifiers: towards equity and interpretability

Abstract

The pervasive effects of structural racism and racial discrimination are well-established and offer strong evidence that the effects of many important variables on health and life outcomes vary by race. Alarmingly, standard practices for statistical regression analysis introduce racial biases into the estimation and presentation of these race-modified effects. We advocate abundance-based constraints (ABCs) to eliminate these racial biases. ABCs offer a remarkable invariance property: estimates and inference for main effects are nearly unchanged by the inclusion of race-modifiers. Thus, quantitative researchers can estimate race-specific effects “for free”—without sacrificing parameter interpretability, equitability, or statistical efficiency. The benefits extend to prominent statistical learning techniques, especially regularization and selection. We leverage these tools to estimate the joint effects of environmental, social, and other factors on 4th end-of-grade readings scores for students in North Carolina (n = 27,638) and identify race-modified effects for racial (residential) isolation, PM_2.5 exposure, and mother’s age at birth.

Health and life outcomes are inextricably linked to race (1, 2). Racial disparities exist in birth outcomes, mortality, disease onset and progression, socioeconomic status, and police-involved deaths, along with many other health and life outcomes (2–4). These disparities persist even after adjusting for socioeconomic status and occur through multiple pathways (1). Structural racism contributes to significant differences in the quality of education, housing, employment opportunities, accumulation of wealth, access to medical care, and treatment in the criminal justice system (1, 2, 5–7). Perceived racial discrimination impacts both mental and physical health through heightened stress responses, health behaviors, and traumatic experiences (8, 9). Thus, rigorous studies of health and life outcomes must carefully consider race as a primary factor.

That race permeates so many aspects of an individual’s life course is a strong indicator that the effects of important factors $\left(X\right)$ on health and life outcomes $\left(Y\right)$ may be race-specific (10). Regression analysis—the primary statistical tool to quantify how these covariates $X$ determine, predict, or associate with an outcome $Y$—must therefore consider race-modifiers for $X$. Indeed, there is abundant and growing evidence for race-specific effects, including the effects of red-lining, PM_2.5 exposure, and cigarette use on mortality risks (11–13); maternal age, poverty, education, and hypertension on infant birthweight, infant mortality, and maternal stroke risk (14–16); education level on multiple health outcomes (17); mood/anxiety disorder on chronic physical health conditions (18); perceived racism on mental health (9); age on allostatic load scores, known as “racial weathering” (19); and the timing of hypertension, insulin resistance, or diabetes onset (20, 21), among many others (22–26). The identification and quantification of race-modified effects are essential to understand and eliminate harmful race disparities in health and life outcomes (27).

To provide context for race-modified effects, we present a regression of 4th end-of-grade reading scores on racial (residential) isolation (RI) and race (Figure 1). The dataset, detailed and reanalyzed subsequently, includes n = 27,638 students in North Carolina (58% non-Hispanic (NH) White, 36% NH Black, 6% Hispanic). RI measures the geographic separation of NH Black individuals and communities from other race groups, and thus is an important measure of structural racism (5, 6, 28–30). The main-only (or ANCOVA) model (Figure 1a) includes race only as an additive effect, which restricts the RI effect to be common across race groups. This widely-used model reports the same adverse effect of RI on reading scores for all students. However, the race-modified model (Figure 1b) provides the essential context: the RI effect is significantly negative for NH Black students, but not for other race groups. Thus, a race-modified model is necessary to uncover and quantify these racial discrepancies in the effects of structural racism on educational outcomes.

Fig. 1.

Linear regression models for an outcome variable $Y$ with a continuous covariate $X$ and a categorical (or nominal) covariate race. The models parameterize the expected outcome, $E(Y|X=x,\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{e}=r)=\mu (x,r),$ with corresponding x-effect (or slope) ${\mu }_x^{\prime }(r):=\mu (x+1,r)-\mu (x,r)$. (a) The main-only model assumes a global (race-invariant) $x$-effect. (b) The race-modified model allows for race-specific $x$-effects. Fit to 4th end-of-grade reading scores in North Carolina, the main-only model obscures important race-specific differences in the effects of racial isolation (RI) that are uncovered by the race-modified model. The negative RI effect observed globally in (a) is driven by the negative RI effect for non-Hispanic Black (NHB) students, which does not persist for Hispanic (Hisp) or non-Hispanic White (NHW) students in (b).

Despite these benefits, there are significant racial biases that occur in commonplace estimation, inference, and presentation of results for regression analysis with race as a covariate. It is well-known that both the main-only and race-modified models (Figure 1) are overparametrized: neither ${\left\{{\alpha }_0,{\beta }_r\right\}}_r$ nor ${\left\{{\alpha }_1,{\gamma }_r\right\}}_r$ are identifiable without further constraints. Any constant could be added to ${\alpha }_0$ and subtracted from each $\left\{{\beta }_r\right\}$, and similarly for ${\alpha }_1$ and $\left\{{\gamma }_r\right\}$, which alters each parameter but leaves the model unchanged. This nonidentifiability is often called the “dummy variable trap” in reference to the use of “dummy variables” to encode categorical variables (Figure 1). Critically, neither the main nor the race-specific parameters can be estimated or interpreted without additional constraints.

Undoubtedly, the most common approach is reference group encoding (RGE): a reference group is selected, typically NH White, and removed $\left({\beta }_{\mathrm{N}\mathrm{H}\mathrm{W}}={\gamma }_{\mathrm{N}\mathrm{H}\mathrm{W}}=0\right)$. This is the default for all major statistical software implementations of (generalized) linear regression, including R, SAS, Python, MATLAB, and Stata, as well as textbook treatments of linear regression (31–33). However, RGE output is racially biased (34), difficult to interpret, and obscures important main and race-modified effects. We categorize these significant limitations into presentation bias and statistical bias.

Presentation bias.

Table 1 (left) displays standard output for a race-modified model. Under RGE, the RI effect (red) actually refers to the RI effect only for NH White individuals, ${\alpha }_1={\alpha }_1+{\gamma }_{\mathrm{N}\mathrm{H}\mathrm{W}}={\mu }_x^{\prime }\left(\mathrm{N}\mathrm{H}\mathrm{W}\right)$. Similarly, Intercept refers to the NH White intercept, ${\alpha }_0={\alpha }_0+{\beta }_{\mathrm{N}\mathrm{H}\mathrm{W}}=\mu (0,\mathrm{N}\mathrm{H}\mathrm{W})$. We emphasize that the presentation format in Table 1 (left) is predominant in scientific journals. Among recent publications in social science journals, it was found that 92% of such tables used NH White as the reference group, while less than half explicitly stated the reference group (34).

Table 1.

Linear regression output under default reference group encoding (RGE; left) and abundance-based constraints (ABCs; right): race-modified effects of racial isolation (RI) on 4th end-of-grade reading scores for students in North Carolina (y ~ 1 + RI + race + RI:race).

Default: reference group encoding (RGE)
Variable	Estimate (SE)	p-value
Intercept	0.267 (0.009)	<0.001
Racial isolation (RI)	−0.013 (0.011)	0.220
Mother’s race
NH Black	−0.670 (0.014)	<0.001
Hispanic	−0.191 (0.025)	<0.001
RI × Mother’s race
RI:NH Black	−0.057 (0.014)	<0.001
RI:Hispanic	0.034 (0.027)	0.210
Proposed: abundance-based constraints (ABCs)
Variable	Estimate (SE)	p-value
Intercept	0.014 (0.007)	0.038
Racial isolation (RI)	−0.032 (0.007)	<0.001
Mother’s race
NH White	0.253 (0.006)	<0.001
NH Black	−0.417 (0.009)	<0.001
Hispanic	0.062 (0.023)	0.006
RI × Mother’s race
RI:NH White	0.018 (0.006)	0.001
RI:NH Black	−0.038 (0.009)	<0.001
RI:Hispanic	0.052 (0.024)	0.031

The default regression output (RGE, left) induces presentation bias: the estimated Intercept and RI effect (red) refer to the NH White group. This is inequitable, unclear, and misleading: it obfuscates the highly significant and detrimental effects of RI on reading scores (for the main-only model, ${\hat{\alpha }}_1^M=-0.042,\text{SE}({\hat{\alpha }}_1^M)=0.007,p<0.001)$. The regression output under ABCs (right) eliminates presentation bias, confirms the estimated RI effect and standard errors from the main-only model (blue), and clearly highlights and quantifies the critical result that the adverse RI effect is more than doubled for NH Black students $({\hat{\mu }}_{\text{RI}}^{\prime }\left(\text{NHB}\right)={\hat{\alpha }}_{\text{RI}}+{\hat{\gamma }}_{\text{RI}:\text{NHB}}=-0.032+-0.038=-0.070$; Figure 1b).

First, this output is inequitable: it elevates a single race group above others. Further, all other race-specific effects are presented relative to the NH White group. For instance, RI:NH Black refers to the difference between the RI effects for NH Black students and NH White students: ${\gamma }_{\mathrm{N}\mathrm{H}\mathrm{B}}={\mu }_x^{\prime }\left(\mathrm{N}\mathrm{H}\mathrm{B}\right)-{\alpha }_1={\mu }_x^{\prime }\left(\mathrm{N}\mathrm{H}\mathrm{B}\right)-{\mu }_x^{\prime }\left(\mathrm{N}\mathrm{H}\mathrm{W}\right)$. This framing presents NH White as “normal” and other race groups as “deviations from normal”, which is known to bias interpretations of results (35). Second, this output is unclear: it is nowhere indicated that the intercept and RI effects are specific to NH White students. A cursory inspection of this output might result in a mistaken interpretation of the RI effect as a global effect, rather than a NH White effect. Finally, this output is misleading: the RI effect is reported to be small and insignificant, despite clear evidence to the contrary (Figure 1). Under RGE, the addition of the race-modifier substantially alters the estimates and reduces the statistical power for the RI main effect $\left({\alpha }_1\right)$.

Statistical bias.

The racial inequity in RGE also permeates statistical estimation and inference. Modern statistical learning commonly features penalized regression, variable selection, and Bayesian inference (36). Broadly, these regularization strategies seek to stabilize (i.e., reduce the variance of) estimators, typically by “shrinking” coefficients toward zero. This approach is particularly useful in the presence of a moderate to large number of covariates that may be correlated. However, under RGE, shrinking or setting coefficients to zero introduces racial bias to the estimation. Critically, shrinkage or selection of the race-specific terms, ${\gamma }_r\to 0$, does not innocuously shrink toward a global slope; rather, it implies that the coefficient on $x$ for race $r$ is pulled toward that of the NH White group, ${\mu }_x^{\prime }\left(r\right)={\alpha }_1+{\gamma }_r\to {\alpha }_1$, and ${\alpha }_1={\mu }_x^{\prime }\left(\mathrm{N}\mathrm{H}\mathrm{W}\right)$. Not only is this estimator racially biased, but also it attenuates the estimated differences between the $x$-effects for each race and NH White individuals. Identification and quantification of such race-modified effects are precisely the goals of race-modified models. Furthermore, RGE cannot distinguish between shrinkage toward a global, race-invariant $x$-effect and shrinkage toward the NH White $x$-effect: both require ${\gamma }_r\to 0$ for all $r$. A fundamental goal of penalized estimation and selection in this context is to remove unnecessary race-modifiers. However, with RGE, the cost is racial bias in the shrinkage and selection. Thus, default RGE cannot fully and equitably leverage the state-of-the-art in statistical learning.

Although RGE is used in the overwhelming majority of regression analyses, there are several alternatives. Subgroup analysis subsets the data by (race) groups and fits separate regression models (12, 25, 30, 37). This approach produces race-specific intercepts and slopes, and thus implicitly acknowledges the importance of race-modifiers. However, subgroup analysis does not estimate global (race-invariant) x-effect estimates or inference, cannot incorporate information-sharing or regularization across race groups (often leading to variance inflation and reduced power), and cannot test for race-modifier effects. Sum-to-zero (STZ) constraints address the inequities in RGE, but the resulting model parameters are difficult to interpret and the estimators do not offer any of the appealing statistical properties provided by our preferred approach. Overparametrized estimation omits any identifying constraints and relies on regularized regression to produce unique estimators. But the model parameters remain nonidentified, so the estimates remain extremely difficult to interpret. These estimates also fail to offer the useful statistical properties discussed subsequently. Finally, marginal means (38) use post-processing to provide useful model summaries, and will be identical for RGE, STZ, and ABCs under ordinary least squares estimation. Nonetheless, it remains imperative to choose an identifiable model parametrization that delivers interpretable and equitable model parameters, appealing properties for estimation and inference, and suitable behavior for regularized regression and variable selection. Notably, marginal means that are estimated by regularized regression will depend on the model parametrization (RGE, STZ, ABCs, etc.).

The primary goal of this paper is to describe and validate alternative statistical methods that eliminate these racial biases. Our preferred approach ensures equitable and interpretable parameters with accompanying estimators that offer unique and appealing statistical properties. We apply these tools to identify and quantify the race-modified effects of multiple environmental, social, and other factors on 4th end-of-grade readings scores for students in North Carolina. Although we focus on race, the proposed methods remain applicable for other categorical covariates including sex, national origin, religion, and other protected groups. This work is accompanied by a companion paper on statistical theory (39), an R package lmabc that implements our estimation and inference methods, and an online vignette that provides accessible examples and documentation: https://drkowal.github.io/lmabc/.

Results

Abundance-Based Constraints (ABCs) for Linear Regression.

We update the race-modified model (Figure 1b) for multivariable regression with $p$ covariates $\mathit{X}={\left(X_1,\dots ,X_p\right)}^{\mathrm{\top }}$, where the effect of each variable may be modified by race:

$$\begin{gathered} \mu \left(\mathit{x},r\right)={\alpha }_0+\sum _{j=1}^p{x}_j{\alpha }_j+{\beta }_r+\sum _{j=1}^p{x}_j{\gamma }_{r,j} \\ ={\alpha }_0+{\mathit{x}}^{\top }\mathit{\alpha }+{\beta }_r+{\mathit{x}}^{\top }{\gamma }_r \end{gathered}$$ [1]

where $\mathit{\alpha }={\left({\alpha }_1,\dots ,{\alpha }_p\right)}^{\mathrm{\top }}$ are the main $x$-effects and ${\mathit{\gamma }}_r={\left({\gamma }_{r,1},\dots ,{\gamma }_{r,p}\right)}^{\mathrm{\top }}$ are the race-modifier effects. The main-only version omits all interactions $\left({\gamma }_{r,j}=0\right)$. The intercepts are race-specific, $\mu (0,r)={\alpha }_0+{\beta }_r$, while the race-modified model yields race-specific slopes for each variable $j=1,...,p$:

$$\begin{array}{l}{\mu }_{x_j}^{\mathrm{\text{'}}}(r):=\mu \left(x_j+1,{\mathit{x}}_{-j},r\right)-\mu \left(x_j,{\mathit{x}}_{-j},r\right)\\ & ={\alpha }_j+{\gamma }_{r,j}.\end{array}$$

The parameters ${\left\{{\alpha }_0,{\beta }_r\right\}}_r$ and ${\left\{{\alpha }_j,{\gamma }_{r,j}\right\}}_{r,j}$ must be further constrained to enable unique estimation and meaningful inference. Linear constraints of the form ${\sum }_r{c}_r{\beta }_r=0$ and ${\sum }_r{c}_r{\gamma }_{r,j}=0$ are most common: RGE sets $c_1=1$ and $c_r=0$ for $r>1$, while STZ uses $c_r=1$ for all $r$. However, the equitability, interpretability, and statistical properties of the parameters and estimators depend critically on the choice of $\left\{c_r\right\}$.

We advocate abundance-based constraints (ABCs) that use the race group abundances:

$$\begin{gathered} \sum _r{\hat{\pi }}_r{\beta }_r=0,\sum _r{\hat{\pi }}_r{\gamma }_{r,j}=0\text{for}j=1,\dots ,p \\ {\hat{\pi }}_r=\text{proportion}\text{in}(\text{race})\text{group}r \end{gathered}$$

or equivalently, ${\mathbb{E}}_{\hat{\pi }}\left({\beta }_R\right)=0$ and ${\mathbb{E}}_{\hat{\pi }}\left({\gamma }_{R,j}\right)=0$ for all $j$, where the expectation is taken over a categorical random variable $R$ with $\mathbb{P}(R=r)={\hat{\pi }}_r$. If known, the population proportions may be used for $\left\{{\hat{\pi }}_r\right\}$; otherwise, we use the sample proportions. ABCs, under various names, have appeared previously, but only for main-only models (40–42). Critically, motivation for this approach is severely lacking; even among previous work that mentions ABCs, they are routinely dismissed in favor of RGE or STZ. Here, we promote ABCs for race-modified models based on new arguments for equitability, interpretability, and special statistical properties.

To evaluate equitability and interpretability, we consider the meaning of each parameter in the race-modified model. Under ABCs, the race-modified model satisfies ${\mathbb{E}}_{\hat{\pi }}\{\mu (\mathit{x},R)\}={\alpha }_0+{\mathit{x}}^{\mathrm{\top }}\mathit{\alpha }$, which produces a global (race-invariant) linear regression. As a consequence, each main x-effect may be expressed as the race-averaged slope for the jth variable:

$${\alpha }_j=\sum _r{\hat{\pi }}_r{\mu }_{x_j}^{\prime }\left(r\right)={\mathbb{E}}_{\hat{\pi }}\left\{{\mu }_{x_j}^{\prime }\left(R\right)\right\}.$$

Unlike with RGE, where ${\alpha }_j={\mu }_{x_j}^{\prime }$ (NHW), ABCs do not anchor each main $x$-effect to the NH White group and instead provide a global interpretation for these key parameters. The benefits cascade down to the other parameters:

$${\gamma }_{r,j}={\mu }_{x_j}^{\prime }(r)-{\alpha }_j={\mu }_{x_j}^{\prime }(r)-{\mathbb{E}}_{\hat{\pi }}\left\{{\mu }_{x_j}^{\prime }(R)\right\}$$

which is the difference between the race-specific slope and the race-averaged slope for variable $j$. The intercept also retains a convenient, more equitable interpretation. Suppose that each continuous covariate is centered, ${\bar{x}}_j=0$. Then the intercept parameter is a marginal expectation:

$${\alpha }_0=\frac{1}{n}\sum _{i=1}^n{\mathbb{E}}_{\hat{\pi }}\left\{\mu \left({\mathit{x}}_i,R\right)\right\}={\mathbb{E}}_{{\hat{p}}_{\mathit{x}}}{\mathbb{E}}_{\hat{\pi }}\{\mu (\mathit{X},R)\}$$

where the expectation is taken (separately) over $\mathit{X}~{\hat{p}}_{\mathit{x}}$ for ${\hat{p}}_{\mathit{x}}$ the empirical distribution of ${\left\{{\mathit{x}}_i\right\}}_{i=1}^n$ and $R~\hat{\pi }$. The race-specific intercept coefficients proceed similarly:

$${\beta }_r=\mu \left(0,r\right)-{\alpha }_0={\mathbb{E}}_{{\hat{p}}_{\mathit{x}}}\left\{\mu \left(\mathit{X},r\right)\right\}-{\mathbb{E}}_{{\hat{p}}_{\mathit{x}}}{\mathbb{E}}_{\hat{\pi }}\left\{\mu \left(\mathit{X},R\right)\right\}.$$

Again, unlike for RGE, the parameters ${\alpha }_0$ and ${\beta }_r$ no longer elevate the NH White group. Instead, ABCs define all parameters as 1) global, race-averaged main effects or 2) race-specific deviations.

Estimation and Inference.

Given data ${\left\{y_i,{\mathit{x}}_i,r_i\right\}}_{i=1}^n$, the race-modified model with ABCs is estimated by applying linearly-constrained ordinary least squares (OLS) estimation. Standard errors, confidence intervals, and hypothesis testing are available as in traditional OLS estimation. Options for regularized (ridge, lasso, etc.) regression are provided (see Methods). Because the estimators satisfy ABCs, they retain the same properties and interpretations as the parameters above.

Statistical Properties.

A central obstacle with race-modified models is that, for default approaches (RGE, STZ, etc.), the inclusion of these interaction terms fundamentally alters the interpretations, estimates, and standard errors for the main $x$-effects. We observe this empirically (Table 1, left): compared to the main-only model, the race-modified model under RGE attenuates the RI main effect $\left({\hat{\alpha }}_1^M=-0.042\right.$ vs. $\left.{\hat{\alpha }}_1=-0.013\right)$ and inflates the standard error $\left(\mathrm{S}\mathrm{E}\left({\hat{\alpha }}_1^M\right)=0.007\right.$ vs. $\mathrm{S}\mathrm{E}\left({\hat{\alpha }}_1\right)=0.011$. These results are not contradictory: the RI effect is weaker for the NH White group (Figure 1b) than for the aggregate (Figure 1a), while NH White students represent a subset (58%) of the full sample. The broader implication is that analysts may be reluctant to include race-modifiers. However, omitting race-modifiers can produce misleading results (Figure 1).

ABCs resolve these problems. The first key property of ABCs is estimation invariance: the OLS estimates of the main $x$-effects are nearly identical between the main-only model and the race-modified model, under appropriate conditions. For $p=1$ (Figure 1), ABCs uniquely yield the remarkable result

$${\hat{\alpha }}_1^M\approx {\hat{\alpha }}_1\text{whenever}{\hat{\sigma }}_{x\left[r\right]}^2\approx {\hat{\sigma }}_{x\left[1\right]}^2\text{for}\text{all}\text{race}\text{groups}r$$ [2]

where ${\hat{\sigma }}_{x\left[r\right]}^2=n_r^{-1}{s}_r^2-{\bar{x}}_r^2$ is the (scaled) sample variance of ${\left\{x_i\right\}}_{i=1}^n$ within each race group $r$, with $n_r=n{\hat{\pi }}_r$, $s_r^2={\sum }_{r_i=r}{x}_i^2$ and ${\bar{x}}_r=n_r^{-1}{\sum }_{r_i=r}{x}_i$. Similar results are available for general $p>1$ under suitable modifications of the equal-variance condition (39).

The equal-variance condition in Eq. (2) requires that the scale of x is approximately the same for each race group. Otherwise, a one-unit change in $x$ is not comparable across race groups. In that case, race-specific slopes are necessary, and the global slope from the main-only model (${\alpha }_1^M$) is not a suitable summary. However, the estimation invariance of ABCs is empirically robust to violations of the equal-variance condition. This condition is strongly violated for RI in Table 1 $\left({\hat{\sigma }}_{\mathrm{R}\mathrm{I}\left[\mathrm{N}\mathrm{H}\mathrm{W}\right]}=0.688,{\hat{\sigma }}_{\mathrm{R}\mathrm{I}\left[\mathrm{N}\mathrm{H}\mathrm{B}\right]}=1.063,{\hat{\sigma }}_{\mathrm{R}\mathrm{I}\left[\mathrm{H}\mathrm{i}\mathrm{s}\mathrm{p}\right]}=0.936\right)$, yet the main effect estimates remain similar $\left({\hat{\alpha }}_1^M=-0.042\right.$ vs. $\left.{\hat{\alpha }}_1=-0.032\right)$ and the standard errors (and $p$-values) are identical ( $\mathrm{S}\mathrm{E}\left({\hat{\alpha }}_1^M\right)=0.007$ vs. $\mathrm{S}\mathrm{E}\left({\hat{\alpha }}_1\right)=0.007$ ). Similar results are observed for simulated data (Figure A.1), which further show that estimation invariance does not hold for RGE or STZ.

The second key property of ABCs is related to inference: the main $x$-effect standard errors are equal or smaller under the race-modified model, $\mathrm{S}\mathrm{E}\left({\hat{\alpha }}_1\right)\le \mathrm{S}\mathrm{E}\left({\hat{\alpha }}_1^M\right)$, whenever 1) the equal-variance condition in Eq. (2) holds and 2) the estimated residual variance is equal or smaller under the race-modified model than the main-only model (39). When the race-modifier term (x:race) has small or moderate impact, then the standard errors for the main $x$-effect are approximately the same between the main-only and race-modified models (see Table 1 and Figure 2). However, if the race-modifier term explains substantial variability in $Y$, then the race-modified model can actually increase statistical power for the main $x$-effect compared to the main-only model. Thus, contrary to intuition, the race-modified model—with its greater complexity and additional parameters to estimate—provides superior inference for the main $x$-effect than in the simpler, main-only model.

Fig. 2.

Estimates and 95% confidence intervals for the main effects in the multivariable regression without race-modifiers (black) and the multivariable regression with race-modifiers under ABCs (blue) and RGE (red). Results are presented for blood lead level (BLL), PM_2.5 exposure (PM_2.5), racial isolation (RI), mother’s age (mAge), and birthweight percentile for gestational age (BWTpct), each of which is interacted with race in the expanded model (blue, red); additional covariates include sex, mother’s education level, mother’s marital status, mother’s smoking status, and economically disadvantaged (Table 2). ABCs exhibit invariance: despite the additional race-modifier parameters, the point and interval estimates for the main effects (blue) are nearly indistinguishable from those in the main effects-only model (black), thus effectively allowing the inclusion of race-modifiers “for free”. In contrast, the RGE terms (red) correspond to the x-effects for the NH White group and deviate substantially for PM_2.5, RI, and mAge, including shifts in location and much wider intervals.

With ABCs, the analyst may include race-modifiers “for free”: the estimates and inference for the main $x$-effects are nearly unchanged by the addition of race-modifiers (x:race). This result is unique to ABCs and makes no assumptions about the true relationship between $Y$, $X$, and race. Notably, arbitrary dependencies are permitted between $X$ and race—including varying means and distributions of $X$ by race group—as long as the equal-variance condition holds. Thus, this result is distinct from classical estimation invariance results with OLS that require uncorrelatedness (43).

Sparsity.

Sparsity is often prioritized to remove extraneous parameters, reduce estimation variability, and simplify interpretations. Regularized regression can produce sparse estimates, but depends critically on the parametrization. Importantly, sparsity of the race-modifiers, ${\gamma }_{r,j}=0$, is meaningful under ABCs: it implies that the race-specific slope equals the race-averaged slope, ${\mu }_{x_j}^{\prime }(r)={\mathbb{E}}_{\hat{\pi }}\left\{{\mu }_{x_j}^{\prime }(R)\right\}+{\gamma }_{r,j}={\mathbb{E}}_{\hat{\pi }}\left\{{\mu }_{x_j}^{\prime }(R)\right\}$. This eliminates the racial bias and inequity under RGE, where the same sparsity instead implies that the race-specific slope equals the NH White slope, ${\mu }_{x_j}^{\prime }(r)={\mu }_{x_j}^{\prime }(\mathrm{N}\mathrm{H}\mathrm{W})+{\gamma }_{r,j}={\mu }_{x_j}^{\prime }(\mathrm{N}\mathrm{H}\mathrm{W})$.

An especially concerning case arises when the race-modifier is nonzero $\left({\gamma }_{r,j}\ne 0\right)$, but the main $x$-effect is zero $\left({\alpha }_j=0\right)$. Statistical approaches often eschew this scenario, and instead require that interactions are nonzero only if a main effect is nonzero (44, 45). Such restrictions are not necessary for ABCs: it is plausible that some race-specific x-effects are nonzero, ${\mu }_{x_j}^{\prime }(r)={\gamma }_{r,j}+{\alpha }_j={\gamma }_{r,j}\ne 0$, while the race-averaged $x$-effect is zero, ${\alpha }_j={\mathbb{E}}_{\hat{\pi }}\left\{{\mu }_{x_j}^{\prime }(R)\right\}=0$. Alarmingly, fitting a main-only model would produce misleading results. Applying Eq. (2), the estimated x-effect would be near zero, ${\hat{\alpha }}_j^M\approx 0$, when in fact the $x$-effect is both significant and race-specific. Thus, it is possible that existing quantitative analyses based on main-only models (i.e., without race-modifiers) obscure both important and race-specific effects of certain variables (Figures 1 and 2).

NC Education Data Analysis.

We apply the proposed methods to study the effects of multiple environmental, social, and other factors on educational outcomes—and assess whether, and how, these effects vary by race. Using ABCs, we fit equitable and interpretable race-modified models, empirically evaluate estimation and inference invariance properties, and study regularized (lasso) regression solution paths under competing parametrizations.

Data overview.

We construct a cohort of n = 27,638 students in North Carolina (NC) by linking three administrative datasets:

NC Detailed Birth Records include maternal and infant characteristics for all documented live births in NC. We compute maternal covariates—mother’s race, age (mAge), education level, marital status, and smoking status—and child covariates, sex and birthweight percentile for gestational age (BWTpct). RI is computed using residential addresses at birth.

NC Blood Lead Surveillance includes blood lead level (BLL) measurements for each child. Lead is an adverse environmental exposure with well-known effects on cognitive development and educational outcomes (46, 47).

NC Standardized Testing Data contains 4th end-of-grade standardized reading scores, economic disadvantage status (determined by participation in the National Lunch Program), and residential address at time-of-test. The reading scores, standardized by the year of test (2010, 2011, or 2012), serve as the outcome variable . The residential information is used to estimate the average PM_2.5 exposure (PM_2.5) over the year prior to the test, which is an adverse environmental exposure linked to educational outcomes (48).

Data characteristics are in Table A.1; additional details are provided elsewhere (30, 49, 50). Data management, access, and analysis are governed by data use agreements and an Institutional Review Board–approved research protocol at the University of Illinois Chicago.

Race-modified regression with ABCs.

We estimate a multivariable linear regression for 4th end-of-grade reading scores that includes these environmental, social, and other factors, as well as race-modifiers (Table 2). Each continuous covariate (BLL, PM_2.5, RI, mAge, and BWTpct) is centered and scaled and each categorical variable (mother’s race, child’s sex, mother’s education level, mother’s marital status, mother’s smoking status, and economically disadvantaged) is identified using ABCs (see Methods). Race-modifiers are included for BLL, PM_2.5, RI, mAge, and BWTpct. Standard model diagnostics confirm linearity, homoskedasticity, and Gaussian error assumptions.

Table 2.

Linear regression output (under ABCs) for the race-modified effects of environmental, social, and other factors on 4th end-of-grade reading scores for students in North Carolina.

Variable	Estimate (SE)	p-value
Intercept	0.000 (0.007)	0.976
Blood lead level (BLL)	−0.016 (0.006)	0.003
PM2.5 exposure (PM2.5)	−0.005 (0.005)	0.403
Racial isolation (RI)	−0.020 (0.007)	0.005
Mother’s age (mAge)	0.038 (0.007)	<0.001
Birthweight percentile for gestational age (BWTpct)	0.026 (0.006)	<0.001
Mother’s race
NH White	0.173 (0.006)	<0.001
NH Black	−0.320 (0.010)	<0.001
Hispanic	0.258 (0.024)	<0.001
Sex
Male	−0.064 (0.005)	<0.001
Female	0.064 (0.005)	<0.001
Mother’s education level
Did not complete high school	−0.179 (0.011)	<0.001
Completed high school	−0.074 (0.007)	<0.001
At least some postsecondary	0.181 (0.008)	<0.001
Mother’s marital status
Married at time of birth	0.018 (0.006)	0.003
Not married at time of birth	−0.024 (0.008)	0.003
Mother’s smoking status
Smoker	−0.039 (0.013)	0.003
Non-smoker	0.008 (0.003)	0.003
Economically disadvantaged
Yes	−0.109 (0.005)	<0.001
No	0.171 (0.009)	<0.001
BLL × Mother’s race
BLL:NH White	−0.005 (0.005)	0.240
BLL:NH Black	0.002 (0.007)	0.792
BLL:Hispanic	0.042 (0.022)	0.060
PM2.5 × Mother’s race
PM2.5:NH White	−0.011 (0.005)	0.018
PM2.5:NH Black	0.021 (0.007)	0.005
PM2.5:Hispanic	−0.017 (0.023)	0.472
RI × Mother’s race
RI:NH White	0.006 (0.006)	0.250
RI:NH Black	−0.020 (0.008)	0.016
RI:Hispanic	0.058 (0.023)	0.013
mAge × Mother’s race
mAge:NH White	0.028 (0.005)	<0.001
mAge:NH Black	−0.040 (0.007)	<0.001
mAge:Hispanic	−0.032 (0.024)	0.193
BWTpct × Mother’s race
BWTpct:NH White	0.000 (0.005)	0.929
BWTpct:NH Black	0.002 (0.007)	0.766
BWTpct:Hispanic	−0.018 (0.022)	0.427

Data restricted to individuals with 37-42 weeks gestation, mother’s age 15-44 years old at birth, BLL ≤ 80μg/dL (and capped at 10μg/dL), birth order ≤ 4, no current English language learners, and residence in NC at the time of birth and time of 4th end-of-grade test. “Economically disadvantaged” is determined by participation in the National Lunch Program.

ABCs generate output for all main effects, all race-modifier effects, and each group in every categorical variable, which eliminates the presentation bias that would otherwise accompany each categorical variable under RGE. There are highly significant (p < 0.01) negative effects for BLL and RI, where the adverse RI effect doubles for NH Black students $\left({\hat{\mu }}_{\mathrm{R}\mathrm{I}}^{\prime }(\mathrm{N}\mathrm{H}\mathrm{B})={\hat{\alpha }}_{\mathrm{R}\mathrm{I}}+{\hat{\gamma }}_{\mathrm{R}\mathrm{I}:\mathrm{N}\mathrm{H}\mathrm{B}}\right.=-0.020+-0.020=-0.040$ ). This critical result for RI expands upon the previous model fit (Table 1): here, the model adjusts for many additional factors, yet the effect persists. Significantly lower test scores also occur for students who are NH Black, Male, or economically disadvantaged, and whose mothers who are less educated, unmarried, or smokers at time of birth. Significant positive effects are observed for the opposite categories—which is a byproduct of ABCs (e.g., the Male and Female proportions are identical, so the estimated effects must be equal and opposite)—as well as mAge and BWTpct. Finally, PM_2.5 is not identified as a significant main effect (p = 0.403), yet the race-specific effects are significant. Alarmingly, a fitted main-only model (without race-modifiers) conveys an insignificant PM_2.5 effect (Figure 2), which oversimplifies and misleads.

Estimation invariance with ABCs.

Figure 2 presents the estimates and 95% confidence intervals for the main effects that are modified by race. We compare the main-only model (variables only in the left column of Table 2) to the race-modified model (all terms in Table 2), including both ABCs and RGE output for the race-modified models. Remarkably, under ABCs, the estimates and uncertainty quantification for the simpler, main-only model are nearly indistinguishable from those for the expanded, race-modified model. Evidently, ABCs allow estimation and inference for numerous race-specific effects (Table 2, right column) “for free”: the inferential summaries for the main effects are unchanged by the expansion of the model to include race-modifiers. This result empirically confirms the multivariable extension of Eq. (2), despite moderate violations of the equal-variance condition (Table A.2). Unsurprisingly, no such invariance holds for RGE (red): the point and interval estimates are substantially different, with uniformly wider intervals and conflicting conclusions about nonzero coefficients (PM_2.5, RI). These concerning discrepancies occur because the RGE “main effects” are exclusively for NH White students.

Regularized regression with ABCs.

We assess regularized regression and variable selection with ABCs using lasso regression, including all variables from Table 2. We report estimates across tuning parameter values $\mathrm{\lambda }$ for the model coefficients ${\left\{{\hat{\alpha }}_j,{\hat{\gamma }}_{r,j}\right\}}_{r,j}$ and the race-specific slopes $\left\{{\hat{\mu }}_{x_j}^{\prime }(r)=\right.{\left.{\hat{\alpha }}_j+{\hat{\gamma }}_{r,j}\right\}}_{r,j};\mathrm{\lambda }\to 0$ yields OLS estimates, while $\mathrm{\lambda }\to \infty$ yields sparse estimates. Since the penalized estimates depend critically on the parameterization, we compare ABCs and RGE. The estimated $\mathrm{\lambda }$-paths for RI are in Figure 3; results for the remaining race-modified effects (BLL, PM_2.5, mAge, and BWTpct) are in Figures A.2–A.5. RGE fixes ${\hat{\gamma }}_{r,j}=0$ for all $\mathrm{\lambda }$, which results in 1) racially-biased shrinkage of the race-specific effects toward the NH White-specific effect and 2) attenuation of the RI effect ${\hat{\alpha }}_j$ (Figure 3, top right). ABCs resolve these issues. First, the model parameters are separately and equitably pulled toward zero (Figure 3, top left). Second, the RI effect ${\hat{\alpha }}_{\text{RI}}$ is not attenuated, and preserves its magnitude until log $\mathrm{\lambda }\approx 5$ (Figure 3, top left). Finally, the race-specific RI effects merge at a global, and negative, RI effect estimate, and this variable is selected by the one-standard-error rule (36) for choosing $\mathrm{\lambda }$ (Figure 3, bottom left).

Fig. 3.

Estimated lasso paths for RI across varying sparsity levels $(\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{\lambda })$ for the model coefficients ${\hat{\alpha }}_{\mathrm{R}\mathrm{I}}$, ${\hat{\gamma }}_{\mathrm{R}\mathrm{I}:r}$ (top) and the race-specific slopes ${\hat{\mu }}_{\mathrm{R}\mathrm{I}}^{\prime }(r)={\hat{\alpha }}_{\mathrm{R}\mathrm{I}}+{\hat{\gamma }}_{\mathrm{R}\mathrm{I}:r}$ (bottom) under ABCs (left) or RGE (right); vertical lines identify $\mathrm{\lambda }$ for the minimum cross-validated error (solid) and one-standard-error rule (dot-dashed). The outcome is 4th end-of-grade reading score and the covariates include all variables in Table 2. Small $\mathrm{\lambda }$ approximately corresponds to OLS, while increasing $\mathrm{\lambda }$ yields sparsity. Under RGE, the estimates are pulled toward the reference (NH White) estimate—inducing statistical bias by race—and the RI effect is attenuated. By comparison, ABCs offer more equitable shrinkage toward a global RI effect, which is nonzero and detrimental for 4th end-of-grade reading scores.

These themes persist for the remaining race-modified effects (Figures A.2–A.5). We supplement the ABC and RGE lasso paths by including the lasso paths for overparametrized estimation (Over), which does not include any identifiability constraints. The parameters cannot be estimated uniquely by OLS, but can be estimated by lasso regression with $\mathrm{\lambda }>0$. In most cases, Over sets one of the coefficients ${\left\{{\alpha }_j,{\gamma }_{r,j}\right\}}_r$ to zero immediately (small $\mathrm{\lambda }$) for each variable $j$. This effect reproduces RGE and thus Over inherits the same racial biases in estimation and selection. When this implicit selection sets ${\hat{\gamma }}_{j:\mathrm{N}\mathrm{H}\mathrm{W}}=0$, then the Over paths resemble those for RGE (RI, not shown; BWTpct, Figure A.5); when the selection corresponds to the smallest $\left|{\hat{\gamma }}_{r,j}\right|$ among race groups $r$ from ABCs, then the Over and ABC paths are similar (BLL, Figure A.2; BWTpct, Figure A.5). However, when this selection sets the main effect to zero, ${\hat{\alpha }}_j=0$ (PM_2.5, Figure A.3) or overshrinks multiple coefficients toward zero (mAge, Figure A.4), then the Over paths differ substantially from both the RGE and ABC paths and demonstrate erratic behavior (Figure A.4).

Estimation and predictive accuracy for simulated data.

We evaluate estimation and prediction for ABCs, RGE, and Over across several estimation methods: OLS, ridge, and lasso regression (Figure 4). Data are simulated from a Gaussian main-only model with p = 10 covariates and a categorical variable with four levels. For fair comparisons, the data-generating process satisfies both RGE and ABCs. To mimic the challenges of real data analysis, the fitted models are misspecified as Eq. (1), and thus contain extraneous race-modifiers. Root mean squared errors are computed for the regression coefficients ${\left\{{\alpha }_0,{\alpha }_j,{\beta }_r,{\gamma }_{r,j}\right\}}_{r,j}$, the race-specific slopes ${\left\{{\alpha }_j+{\gamma }_{r,j}\right\}}_{r,j}$, and the model expectations $\mu (\mathit{x},r)$ across 500 simulated datasets. In each case, ABCs are substantially more accurate within each estimation method (OLS, ridge, lasso). The estimation invariance of ABCs offers a plausible explanation: whereas each fitted model includes extraneous variables (the race-modifiers), only ABCs reproduce the main effect estimates from the main-only model, which here is the ground truth. This unique statistical property of ABCs is not only convenient for interpreting race-modified models, but also provides more accurate estimates and predictions under both OLS and regularized regression.

Fig. 4.

Estimation and prediction accuracy for the regression coefficients (left), the race-specific slopes (center), and the fitted values (right) for n = 250 (top) and n = 10,000 (bottom) across 500 simulated datasets; nonoverlapping notches indicate significant differences between medians. Data are generated from a Gaussian main-only model with p = 10 covariates and a categorical variable with symmetric proportions $\pi =(0.15,0.35,0.15,0.35{)}^{\mathrm{\top }}$; both RGE and ABCs are satisfied in the true data-generating process. All fitted models use the race-modified model Eq. (1) and thus contain extraneous race-modifiers. ABCs (gold) outperform both RGE (light gray) and Over (dark gray) within each estimation method (ridge, lasso, OLS). By definition, the OLS race-specific slopes and fitted values are invariant to the constraints (ABCs or RGE), and Over cannot be computed for OLS.

Discussion

The path to more equitable decision-making and policy requires a precise and comprehensive understanding of the links between race and health and life outcomes. Alarmingly, the primary statistical tool for this task—regression analysis with race as a covariate and a modifier—in its current form propagates racial bias in both the presentation of results and the estimation of model parameters. We advocated an alternative approach, abundance-based constraints (ABCs), with several unique benefits. First, ABCs eliminate these racial biases in both presentation and statistical estimation of linear regression models. Second, ABCs produce more interpretable parameters for race-modified models. Third, estimation with ABCs features an appealing invariance property: the estimated main effects are approximately unchanged by the inclusion of race-modifiers. Thus, analysts can include and estimate race-specific effects “for free”—without sacrificing parameter interpretability, equitability, or statistical efficiency. Finally, ABCs are especially convenient for regularized regression and variable selection, with meaningful and equitable notions of parameter sparsity and efficient computational algorithms.

Using this new approach, we estimated the effects of multiple environmental, social, and other factors on 4th end-of-grade readings scores for a large cohort of students (n = 27,638) in North Carolina. In aggregate, this analysis 1) identified significant race-specific effects for racial (residential) isolation, PM_2.5 exposure, and mother’s age at birth; 2) showcased the racial biases and potentially misleading results obtained under default approaches; and 3) provided more equitable and interpretable estimates, uncertainty quantification, and selection, both for main effects and race-modified effects. Simulation studies demonstrated substantially more accurate estimates and predictions with OLS, ridge, and lasso regression compared to alternative approaches.

We acknowledge that the interpretation of any “race” effect requires great care (51). Race encompasses a vast array of social and cultural factors and life experiences, with effects that vary across time and geography (27, 52). In some settings, race data are unreliable or partially missing (53, 54). These overarching challenges are not addressed in this paper.

Methods

Abundance-Based Constraints (ABCs) with Multiple Categorical Covariates.

Regression analysis often features multiple continuous covariates $\mathit{X}={\left(X_1,\dots ,X_p\right)}^{\mathrm{\top }}$ and multiple categorical covariates $\mathit{R}={\left(R_1,\dots ,R_L\right)}^{\mathrm{\top }}$ such as race, sex, education level, etc.:

$$\mu (\mathit{x},\mathit{r})={\alpha }_0+{\mathit{x}}^{\top }\mathit{\alpha }+\sum _{\ell =1}^L{\beta }_{\ell ,r_{\ell }}+\sum _{\ell =1}^L{\mathit{x}}^{\top }{\gamma }_{\ell ,r_{\ell }}$$ [3]

where $\mathit{\alpha }={\left({\alpha }_1,\dots ,{\alpha }_p\right)}^{\mathrm{\top }}$ and ${\mathit{\gamma }}_{\ell ,r_{\ell }}={\left({\gamma }_{\ell ,r_{\ell },1},\dots ,{\gamma }_{\ell ,r_{\ell },p}\right)}^{\mathrm{\top }}$ are $p$-dimensional and $r_{\ell }$ denotes the level of the $\ell$ th categorical variable, $\ell =1,...,L$. Eq. (3) includes all continuous-categorical interactions and requires $L(1+p)$ constraints for identification; RGE sets ${\beta }_{\ell ,1}=0$ for all $\ell$ and ${\gamma }_{\ell ,1,j}=0$ for all $\ell$, $j$.

We extend the definition of ABCs based on the joint distribution of the categorical variables $\mathit{R}$. Specifically, let $\stackrel{\mathit{ˆ}}{\mathit{\pi }}={\hat{\pi }}_{r_1,\dots r_L}=\mathbb{P}\left(R_1=r_1,\dots ,R_L=r_L\right)$. If known, the population proportions may be used for $\stackrel{\mathit{ˆ}}{\mathit{\pi }}$; otherwise, we use the sample proportions based on the observed data ${\left\{{\mathit{r}}_i\right\}}_{i=1}^n$, i.e., ${\hat{\pi }}_{r_1,\dots ,r_L}=n^{-1}{\sum }_{i=1}^n\mathbb{I}\left\{r_{i,1}=r_1,\dots ,r_{i,L}=r_L\right\}$. Concisely, the generalized ABCs are

$${\mathbb{E}}_{\hat{\pi }}\left({\mathit{\beta }}_{\mathit{R}}\right)=0_L,{\mathbb{E}}_{\hat{\pi }}\left({\gamma }_{\mathit{R},j}\right)=0_L,j=1,\dots ,p$$ [4]

where ${\mathit{\beta }}_{\mathit{R}}={\left({\beta }_{1,R_1},\dots ,{\beta }_{L,R_L}\right)}^{\mathrm{\top }}$, ${\mathit{\gamma }}_{\mathit{R},j}={\left({\gamma }_{1,R_1,j},\dots ,{\gamma }_{L,R_L,j}\right)}^{\mathrm{\top }}$, and $0_L$ is an $L$-dimensional vector of zeros. Eq. (4) may be equivalently represented via separate marginal expectations for the $L$ sets of categorical covariate parameters: for instance, ${\mathbb{E}}_{\hat{\pi }}\left({\beta }_{\ell ,R_{\ell }}\right)={\mathbb{E}}_{{\hat{\pi }}_{\ell }}\left({\beta }_{\ell ,R_{\ell }}\right)={\sum }_{r_{\ell }}{\hat{\pi }}_{\ell ,r_{\ell }}{\beta }_{\ell ,r_{\ell }}=0$ for each categorical covariate $\ell$.

ABCs in Eq. (4) provide interpretable parameter identifications with equitable presentation and estimation. These interpretations are unchanged if some or all interaction terms are omitted from Eq. (3), which may occur if multiple categorical variables (e.g., sex, education level) are included as covariates, but only race is included as a modifier. ABCs imply that ${\mathbb{E}}_{\stackrel{\mathit{ˆ}}{\mathit{\pi }}}\{\mu (\mathit{x},\mathit{R})\}={\alpha }_0+{\mathit{x}}^{\mathrm{\top }}\mathit{\alpha }$, so that averaging the regression Eq. (3) over all categorical variables (jointly) yields a multivariate regression with only continuous variables. Individually, each $x_j$-effect satisfies

$${\alpha }_j={\mathbb{E}}_{\hat{\pi }}\left\{{\mu }_{x_j}^{\prime }(\mathit{R})\right\}$$ [5]

where ${\mu }_{x_j}^{\prime }(\mathit{r})=\mu \left(x_j+1,{\mathit{x}}_{-j},\mathit{r}\right)-\mu \left(x_j,{\mathit{x}}_{-j},\mathit{r}\right)$ is the slope in the $j$ th direction. To further simplify the interpretation, the expectation under $\stackrel{\mathit{ˆ}}{\mathit{\pi }}$ in Eq. (5) need only be taken with respect to the categorical variables that are interacted with $x_j$ (e.g., race). By comparison, the RGE parametrization yields ${\alpha }_j={\mu }_{x_j}^{\prime }\left(r_1=1,\dots ,r_L=1\right)$, which is the group-specific slope for $x_j$ with each group set to its reference category (e.g., NH White, Male, etc.). Clearly, this representation compounds inequity across each categorical variable and fails to deliver a global interpretation of the $x_j$-effect.

Interpretation of group-specific slopes and the parameters ${\gamma }_{\ell ,r_{\ell },j}$ proceeds by considering partial expectations ${\stackrel{\mathit{ˆ}}{\mathit{\pi }}}_{-\ell }$, which is analogous to the joint distribution $\stackrel{\mathit{ˆ}}{\mathit{\pi }}$ but omits the $\ell$ th categorical variable. Here, as with Eq. (5), this expectation need only consider the categorical variables that are interacted with $x_j$; if the $\ell$ th categorical variable is the only interaction term, then no expectation is needed at all. Then the $x_j$-effect when the $\ell$ th categorical variable has level $r_{\ell }$, averaged over the remaining categorical variables, is

$${\mathbb{E}}_{{\hat{\pi }}_{-\ell }}\left\{{\mu }_{x_j}^{\prime }\left(r_{\ell },{\mathit{R}}_{-\ell }\right)\right\}={\alpha }_j+{\gamma }_{\ell ,r_{\ell },j}$$ [6]

or equivalently, ${\gamma }_{\ell ,r_{\ell },j}={\mathbb{E}}_{{\hat{\pi }}_{-\ell }}\left\{{\mu }_{x_j}^{\prime }\left(r_{\ell },{\mathit{R}}_{-\ell }\right)\right\}-{\mathbb{E}}_{\hat{\pi }}\left\{{\mu }_{x_j}^{\prime }(\mathit{R})\right\}$. The interpretation is simpler than the notation: Eq. (6) directly extends the usual notion of race-specific slopes to average over any other categorical variables that modify $x_j$.

Estimation.

Statistical estimation with ABCs requires solving a linearly-constrained least squares problem given data ${\left\{{\mathit{x}}_i,{\mathit{r}}_i,y_i\right\}}_{i=1}^n$. Define $\mathit{\theta }$ to be the model parameters ${\left\{{\alpha }_0,\mathit{\alpha },{\beta }_{\ell ,r_{\ell }},{\mathit{\gamma }}_{\ell ,r_{\ell }}\right\}}_{r_{\ell },\ell }$ and ${\tilde{\mathit{x}}}_i$ to include the intercept, covariates, race variable indicators (i.e., “dummy variables”), and covariate-race interactions such that Eq. (3) may be written $\mu \left({\mathit{x}}_i,{\mathit{r}}_i\right)={\tilde{\mathit{x}}}_i^{\mathrm{\top }}\mathit{\theta }$. Let $\mathit{C}$ encode ABCs such that $\mathit{C}\mathit{\theta }=0$ enforces Eq. (4), so $\mathit{C}$ has $m=L(1+p)$ rows corresponding to the number of constraints. The OLS estimator under ABCs is

$$\hat{\mathit{\theta }}=\arg \underset{\theta }{\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{i=1}^n{(y_i-{\tilde{\mathit{x}}}_i^{\top }\mathit{\theta })}^2\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}to\mathit{C}\mathit{\theta }=0.$$ [7]

To compute $\stackrel{\mathit{ˆ}}{\mathit{\theta }}$—and subsequently provide inference and penalized estimation—we reparametrize the problem into an unconstrained space with $m$ fewer parameters. Let ${\mathit{C}}^{\mathrm{\top }}=\mathit{Q}\mathit{R}$ be the QR-decomposition of the transposed constraint matrix with columnwise partitioning of the orthogonal matrix $\mathit{Q}=\left({\mathit{Q}}_{1:m}:{\mathit{Q}}_{-(1:m)}\right)$ with ${\mathit{R}}^{\mathrm{\top }}=\left({\mathit{R}}_{1:m,1:m}:0\right)$, since ${\mathit{C}}^{\mathrm{\top }}$ has rank $m$. It is straightforward to verify that $\mathit{\theta }={\mathit{Q}}_{-(1:m)}\mathit{\zeta }$ satisfies $\mathit{C}\mathit{\theta }=0$ for any $\mathit{\zeta }$. Then, using the adjusted covariate matrix $\tilde{\mathit{Z}}=\tilde{\mathit{X}}{\mathit{Q}}_{-(1:m)}$ with $\tilde{\mathit{X}}={\left({\tilde{\mathit{x}}}_1,\dots ,{\tilde{\mathit{x}}}_n\right)}^{\mathrm{\top }}$, the solution to Eq. (7) is equivalently solved using unconstrained OLS:

$$\hat{\mathit{\theta }}={\mathit{Q}}_{-(1:m)}\hat{\mathit{\zeta }},\hat{\zeta }=\arg \underset{\mathit{\zeta }}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{i=1}^n{(y_i-{\tilde{\mathit{z}}}_i^{\top }\mathit{\zeta })}^2.$$ [8]

The QR-decomposition has minimal cost due to the efficiency of Householder rotations and the low dimensionality of $\mathit{C}$ (55).

Although alternative computing strategies are available, the reparametrization in Eq. (8) is especially convenient for generalizations to regularized (lasso, ridge, etc.) estimation. Let $𝒫(\mathit{\theta })$ denote a complexity penalty on the regression coefficients. The penalized least squares estimator under ABCs is

$$\hat{\mathit{\theta }}(\lambda )=\mathrm{a}\mathrm{r}\mathrm{g}\underset{\theta }{\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{i=1}^n{(y_i-{\tilde{\mathit{x}}}_i^{\top }\mathit{\theta })}^2+\lambda 𝒫(\mathit{\theta })\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}\mathit{C}\mathit{\theta }=0$$ [9]

where $\mathrm{\lambda }\ge 0$ controls the tradeoff between goodness-of-fit and complexity (measured via $𝒫$). Following Eq. (8), we instead compute

$$\stackrel{\mathit{ˆ}}{\mathit{\theta }}\left(\lambda \right)={\mathit{Q}}_{-\left(1:m\right)}\stackrel{\mathit{ˆ}}{\mathit{\zeta }}\left(\lambda \right),\stackrel{\mathit{ˆ}}{\mathit{\zeta }}\left(\lambda \right)=\arg \underset{\mathit{\zeta }}{\mathrm{m}\mathrm{i}\mathrm{n}}\sum _{i=1}^n{\left(y_i-{\tilde{\mathit{z}}}_i^{\mathrm{\top }}\mathit{\zeta }\right)}^2+\mathrm{\lambda }𝒫\left({\mathit{Q}}_{-\left(1:m\right)}\mathit{\zeta }\right)$$

which requires the solution to an unconstrained penalized least squares problem.

We focus on complexity penalties of the form

$$𝒫\left(\mathit{\theta }\right)=\sum _j{\omega }_j{\left|{\theta }_j\right|}^{\delta }$$

where ${\omega }_j>0$ are known weights, $\delta =1$ produces sparse coefficients (adaptive lasso regression) and $\delta =2$ guards against collinearity (adaptive ridge regression). Under ridge regression $(\delta =2)$, the solution is

$$\stackrel{\mathit{ˆ}}{\mathit{\theta }}(\lambda )={\mathit{Q}}_{-(1:m)}{\left({\tilde{\mathit{Z}}}^{\mathrm{\top }}\tilde{\mathit{Z}}+\lambda {\mathit{D}}^{\mathrm{\top }}\mathit{D}\right)}^{-1}{\tilde{\mathit{Z}}}^{\mathrm{\top }}\mathit{y}$$

where $\mathit{D}=\mathrm{d}\mathrm{i}\mathrm{a}\mathrm{g}{\{{\omega }_j^{1/2}\}}_{j=1}^p{\mathit{Q}}_{-(1:m)}$. The lasso version $(\delta =1)$ can be solved efficiently using the genlasso package in R (56).

For practical use, we set ${\omega }_j$ to be the sample standard deviation of the $j$ th column of $\tilde{\mathit{X}}$ (with ${\omega }_1=1$ for the intercept). This strategy applies a standardized penalty to each covariate, which is especially important for ABCs. In particular, the magnitudes of the race-specific coefficients vary according to the abundance of the group: by construction, low abundances in group $r$ will correspond to larger group $r$-specific coefficients. The standardized penalty adjusts for this effect to avoid overpenalization of group-specific coefficients for groups with low abundance.

Inference.

The reparametrization strategy in Eq. (8) allows direct application of classical inference theory to the ABC OLS estimator: $\stackrel{\mathit{ˆ}}{\mathit{\theta }}$ is a known, linear function of the (unconstrained) OLS estimator $\stackrel{\mathit{ˆ}}{\mathit{\zeta }}$. Thus, it is straightforward to derive the (Gaussian) sampling distribution of the ABC OLS estimator, which can be used to compute standard errors, hypothesis tests, and confidence intervals, and to establish unbiasedness and efficiency of the estimator. Under minimal assumptions, the (unconstrained) OLS estimator satisfies $\sqrt{n}(\stackrel{\mathit{ˆ}}{\mathit{\zeta }}-\mathit{\zeta })\stackrel{d}{\to }N(0,ℐ(\mathit{\zeta }{)}^{-1})$ and thus the ABC OLS estimator satisfies

$$\sqrt{n}(\stackrel{\mathit{ˆ}}{\mathit{\theta }}-\mathit{\theta })\stackrel{d}{\to }N(0,{\mathit{Q}}_{-\left(1:m\right)}ℐ(\mathit{\zeta }{)}^{-1}{\mathit{Q}}_{-\left(1:m\right)}^{\mathrm{\top }})$$

where $ℐ$ is the Fisher information and $\mathit{\zeta }$, $\mathit{\theta }$ are the true parameter values. When the regression model is paired with independent and identically distributed Gaussian errors ${ϵ}_i:=y_i-\mu \left({\mathit{x}}_i,{\mathit{r}}_i\right)$ with variance ${\sigma }^2$, the unconstrained OLS estimator satisfies $\stackrel{\mathit{ˆ}}{\mathit{\zeta }}~N\{\mathit{\zeta },{\sigma }^2{({\tilde{\mathit{Z}}}^{\mathrm{\top }}\tilde{\mathit{Z}})}^{-1}\}$ and thus

$$\stackrel{\mathit{ˆ}}{\mathit{\theta }}~N\{\mathit{\theta },{\sigma }^2{\mathit{Q}}_{-\left(1:m\right)}{({\tilde{\mathit{Z}}}^{\mathrm{\top }}\tilde{\mathit{Z}})}^{-1}{\mathit{Q}}_{-\left(1:m\right)}^{\mathrm{\top }}\}$$

even in finite samples. This sampling distribution for the OLS estimator under ABCs ensures unbiasedness and efficiency, and provides the means to compute standard errors, hypothesis tests, and confidence intervals.

Simulation design.

Each simulated dataset is constructed from a Gaussian multivariable main-only model

$$\mu \left(\mathit{x},r\right)={\alpha }_0^M+{\mathit{x}}^{\mathrm{\top }}{\mathit{\alpha }}^M+{\beta }_r^M$$ [10]

with $p=10$ continuous covariates and one categorical (race) covariate with four levels. The $p=10$ continuous covariates include six independent covariates, $X_j~N(0,1)$ for $j=\mathrm{1,2},\mathrm{3,6},\mathrm{7,8}$, and four covariates that depend on the categorical variable, $\left[X_j\mid R=r\right]~N(r,1)$, i.e., mean one for group one, mean two for group two, etc., for $j=\mathrm{4,5},\mathrm{9,10}$. The categorical variable is generated based on population proportions $\mathit{\pi }$, which we describe below. In addition to the intercept ${\alpha }_0=1$, the true coefficients for the continuous covariates are ${\alpha }_1=\cdots ={\alpha }_5=1$ (signals) and ${\alpha }_6=\cdots ={\alpha }_{10}=0$ (noise). Only main effects (i.e., no race-modifiers or interactions) are included in the data-generating process. We vary the sample size, $n\in \{250,\mathrm{10,000}\}$, and use a signal-to-noise ratio of one.

The categorical covariate and accompanying coefficients are constructed carefully to ensure fair comparisons. Different identifications correspond to different parameterizations, which may lead to unfair evaluations. Thus, we use a data-generating model that satisfies both RGE and ABCs. We consider two designs for the categorical proportions: symmetric weights $\mathit{\pi }=(0.15,0.35,0.15,0.35{)}^{\mathrm{\top }}$ (Figure 4) and uniform weights $\mathit{\pi }=(0.25,0.25,0.25,0.25{)}^{\mathrm{\top }}$ (Figure A.6). The true categorical variable coefficients are $\mathit{\beta }=(0,1,0,-1{)}^{\mathrm{\top }}$, so RGE is satisfied, ${\beta }_1=0$. ABCs are satisfied for the true population proportions, ${\sum }_r{\pi }_r{\beta }_r=0$, but we use the sample proportions $\stackrel{\mathit{ˆ}}{\mathit{\pi }}$ for estimation. The omission of race-modifiers from the data-generating process satisfies both RGE and ABCs. Finally, we require at least $p+1$ observations for each categorical level, which is necessary for OLS estimation of the interaction effects. We simulate 500 such datasets for each $(n,\mathit{\pi })$ design.

All competing methods follow Eq. (1), which includes all continuous covariates, race, and all race-modifiers. Thus, all competing models are overparametrized relative to the ground truth, with 55 columns of the unconstrained designed matrix $\tilde{\mathit{X}}$ and 44 identifiable model parameters to estimate. The estimation approaches are OLS, ridge regression, and lasso regression. The tuning parameter for ridge and lasso regression is selected using the one-standard-error rule (36). The parametrizations determine the identification constraints on ${\beta }_r$ and ${\gamma }_{r,j}$: we consider ABCs, RGE, and Over. Over is not identified for OLS, and thus is presented only for ridge and lasso regression.

Empirical verification of estimation invariance with ABCs.

To empirically verify estimation invariance with ABCs, we generate 500 synthetic datasets that mildly violate the equal-variance condition. Iteratively, we sample a categorical variable $R$ with groups {A,B,C,D} and respective probabilities $\mathit{\pi }=(0.55,0.20,0.10,0.15{)}^{\mathrm{\top }}$ and then sample a continuous variable with the distribution determined by the group:

$$\left[X∣R=r\right]~\left\{\begin{array}{ll}5+N\left(0,1\right)& r=\mathrm{A}\\ \sqrt{12}\mathrm{U}\mathrm{n}\mathrm{i}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}\left(0,1\right)& r=\mathrm{B}\\ 5+t_4\left(0,1\right)& r=\mathrm{C}\\ \mathrm{G}\mathrm{a}\mathrm{m}\mathrm{m}\mathrm{a}\left(1,1\right)& r=\mathrm{D}\end{array}\right.$$ [11]

By design, $X$ depends on R in both mean and distribution. The $R$-specific population variances of $X$ are each one, but Eq. (2) requires that the $R$-specific sample variances are identical, which will not be satisfied for any simulated dataset. Thus, Eq. (11) includes a mild deviation from the equal-variance condition Eq. (2).

The response variable $Y$ is simulated with expectation

$$\mu \left(x,r\right)=1+x+\gamma x\mathbb{I}\left\{r=\mathrm{A}\right\}-\gamma x\mathbb{I}\left\{r=\mathrm{B}\right\}$$

plus $t_4(0,1)$-distributed errors, i.e., standard $t$-distributions with 4 degrees of freedom. The coefficient $\gamma$ determines the strength of the race-modifier effect: we consider $\gamma =0$ (no race-modifier effect), $\gamma =0.5$ (50% of the main $x$-effect), and $\gamma =1.5$ (150% of the main $x$-effect). This data-generating process includes a $R$-modifier (when $\gamma \ne 0$) but does not satisfy ABCs or traditional Gaussian error assumptions. Repeating this process 500 times, each simulated dataset contains ${\left\{x_i,r_i,y_i\right\}}_{i=1}^n$. We consider $n\in \{100,500\}$.

We fit the main-only and race-modified models and record the estimated $x$-effects ${\hat{\alpha }}_1^M$ and ${\hat{\alpha }}_1$, respectively, for each simulated dataset. These estimated coefficients depend on the constraints: we compare ABCs, RGE (with reference group A), and STZ constraints for $\left\{{\beta }_r,{\gamma }_r\right\}$ (Figure A.1). Although the conditions in Eq. (2) are not satisfied, the ABC estimates lie along the 45 degree line with ${\hat{\alpha }}_1={\hat{\alpha }}^M$; the estimated x-effect is nearly unchanged by the addition of the race-modifier. This invariance is not satisfied for RGE or STZ. The estimated $x$-effects under RGE or STZ vary considerably between the main-only and race-modified models, with greater discrepancies as the magnitude of the race-modifier effect increases. By comparison, the estimation invariance of ABCs is robust to the magnitude of the race-modifier effect. The mild deviations from the equal-variance condition Eq. (2) are most impactful when $\gamma =1.5$, which represents the unusual setting in which the interaction effect is much larger than the main effect. Even in this challenging case, the ABC estimates remain nearly invariant between the main-only and race-modified models, especially when compared to the RGE and STZ counterparts.

Contrast coding.

For OLS estimation, identifiability constraints may be imposed using contrasts. In this approach, the linear model is fit under any minimally sufficient identification (RGE, STZ, ABCs, etc.) and the categorical variable coefficients are post-processed using linear contrast matrices. Examples include dummy coding (akin to RGE), effects coding (akin to STZ), weighted effects coding (WEC; akin to ABCs), and Helmert coding (for ordered categories). However, contrasts are typically reserved for main-only models and are difficult to combine with regularized regression and variable selection. Further, these previous approaches do not consider or resolve the inequities of reporting or estimating race-specific effects. In particular, WEC has been advocated only in cases when “a categorical variable has categories of different sizes, and if these differences are considered relevant” (57) or “certain types of unbalanced data that are missing not at random” (58), with regression output that suffers from the same presentation bias that afflicts RGE (59). We do not agree with such restrictions for ABCs, and instead argue that this approach offers an equitable and interpretable parametrization with unique and appealing statistical properties, including both estimation invariance and regularized regression. These estimation invariance results and regularized regression analyses are notably absent from previous contrast coding approaches.

Data Availability

The North Carolina dataset cannot be released due to privacy protections. However, access to the data can occur through establishing affiliation with the Children’s Environmental Health Initiative (contact cehi@nd.edu). Additional data documentation is available at https://www.cehidatahub.org.