Towards a Better Understanding of When to Apply Propensity Scoring: A Comparison with Conventional Regression in Ethnic Disparities Research

Abstract

Purpose

Despite growing popularity of Propensity Score (PS) methods used in ethnic disparities studies, many researchers lack clear understanding of when to use PS in place of conventional regression models. One such scenario is presented here: when the relationship between ethnicity and primary care utilization is confounded with and modified by socioeconomic status. Here, standard regression fails to produce an overall disparity estimate, whereas PS methods can through the choice of a reference sample (RS) to which the effect estimate is generalized.

Methods

Using data from the National Alcohol Surveys, ethnic disparities between white and Hispanics in access to primary care were estimated using PS methods (PS stratification and weighting), standard logistic regression, and the marginal effects from logistic regression models incorporating effect modification.

Results

Whites, Hispanics and combined white/Hispanic samples were used separately as the RS. Two strategies utilizing PS generated disparities estimates different from those from standard logistic regression, but similar to marginal ORs from logistic regression with ethnicity by covariate interactions included in the model.

Conclusions

When effect modification is present, PS estimates are comparable to marginal estimates from regression models incorporating effect modification. The estimation process requires a-priori hypotheses to guide selection of the RS.

INTRODUCTION

Epidemiological studies investigating racial/ethnic disparities in health have grown exponentially over the past few decades (1), in conjunction with the Institute of Medicine’s groundbreaking report on disparities in health care (2) and U.S. national objectives put forth in Healthy People 2000–2020 (3–5). Until recently, disparities research relied heavily upon conventional regression modeling to document inequalities in health and health care across racial/ethnic groups (1). One common problem with regression, however, is that the relationship between ethnicity and the health outcomes of interest are often confounded with, and modified by, socioeconomic status (6). This study addresses why and how newer methods based on Propensity Scores (PS) are particularly well suited for disparities research, although they are relevant to any area of epidemiologic research where effect modification is similarly of concern.

There is lively debate over when PS methods offer benefits over standard multivariable linear or logistic regression typically used in disparities research (7–10). Although there is empirical support for PS’s advantages (11, 12), in practice, PS methods often seem to generate results quite similar to those from multivariable regression (13, 14). This paper is preoccupied with one common scenario in which PS methods should theoretically produce more interpretable effect estimates than those from multivariable regression. This occurs when the distribution of one or more confounding covariates (e.g., SES), as well as the relationship between these confounding covariates and the health outcome, varies across the ethnic groups being compared. Entering relevant interaction terms addresses the effect modification, but the regression model can only produce disparity estimates at specific SES levels and fails to generate a single overall disparity estimate which, in most practical cases, is desired. Without including interactions, disparity estimates will, in general, be biased.

With control variables in the model, standard regression generates conditional effect estimates. In contrast, PS methods produce marginal effect estimates that can be interpreted counterfactually. In the language of causal inference, which often focuses on the effect of a treatment or of exposure to a risk factor, PS methods estimate the difference in an outcome for a given population when it is treated/exposed, and the outcome for the same population when it is not treated/exposed. PS methods thus estimate the marginal effect by design, largely through specifying a Reference Sample (RS) to which the effect estimate is generalized. The RS often used is either the “treated” (“risk-exposed”) sample or the sample combining those who were and were not treated/risk-exposed (15–17). With regression generating conditional effect estimates and PS methods producing marginal estimates, questions have been raised regarding the comparability of the two approaches in non-linear models (10, 18). Specifically, for a dichotomous outcome (as in the present study), it was suggested that marginal Odds Ratios (ORs) should be generated from both PS and logistic regression for the two methods to be comparable (19, 20). Little is known about the comparability of the two approaches, however, if effect modification is present.

In the current study of ethnic disparities, we compare estimates from PS approaches generated using various RSs to both conditional and marginal ORs produced from ordinary logistic regression. We show how marginal ORs from logistic regression incorporating effect modification can be produced and comparable to PS estimates. While some recent work has examined the performance of PS methods under effect modification (21–23), this study compares substantive results obtained from PS and logistic regression models; and also shows how varying the specification of the RS can influence results.

MATERIALS AND METHODS

Dataset and measures

Our empirical analysis focuses on the timely issue of racial/ethnic disparities between whites and Hispanic Americans in access to primary care interventions for alcohol problems (24). Data come from the combined 2000 and 2005 U.S. National Alcohol Surveys (NAS), two comparable probability samples of U.S. adults collected using computer-assisted telephone interviews via random digit dialing (25). Included in this analysis are “at risk” drinkers who meet the NIAAA drinking guidelines defined as: men/women drinking more than 4/3 drinks on any day or more than 14/7 drinks per week (26). Self-identified ethnicity, based on the current U.S. Census definition, is the key risk-exposure variable in this analysis. Only those self-identifying as white (N=2798) or Hispanic (N=684) were retained for analysis as the greatest disparities have been found between these two groups (24, 27, 28). The outcome of interest is the subject’s report of whether she or he had one or more primary care visits in the prior year. Any visit with a private doctor, clinic or non-emergency medical setting during the prior year counted as a primary care visit. Demographic and SES covariates were used as potential confounders and effect modifiers of interest, including gender, age, education, annual household income and health insurance coverage.

Statistical Analysis

PS Stratification and Weighting

A PS is defined as the estimated probability of being Hispanic versus white and is modeled as a function of gender, age, education, annual household income and health insurance coverage using logistic regression. Two approaches utilizing the estimated PS to estimate ethnic disparities are considered in the present study, PS stratification and PS weighting.

PS-Stratification (15, 16) classifies subjects into five strata (quintiles) using their estimated PS. This process is repeated several times, using “Hispanics”, “whites”, and the “combined white and Hispanic sample” separately as the RS. When Hispanics are treated as the RS, the Hispanic sample is divided into 5 equal size strata based on the sorted PS distribution within the Hispanic sample. Using the PS thresholds for quintile generation, the comparison white sample is then divided into five groups, presumably of unequal sizes given the distribution of PS is different between whites and Hispanics. This is done analogously for whites as the RS. When the combined whites/Hispanic sample is treated as the RS, the pooled sample is divided into 5 equal size strata based on the sorted PS distribution for the total sample. The overall response probability, for each choice of RS, is simply a weighted average of stratum-specific response probabilities. Since the RS is divided into five equal size strata in the current design, equal weights are assigned across the five strata and the overall response probability is the simple average of the stratum-specific probabilities. The marginal OR estimate is then derived using the overall response probabilities (see the Appendix for details as well as (19)).

For PS weighting (29), subjects in the two ethnic comparison groups are weighted based on their estimated PS to construct a ‘pseudo-population’ in which confounders are no longer associated with ethnicity. When using either the white or the Hispanic group as the RS, “Standardized Mortality Ratio” (SMR) weights are used that assign a weight of 1 to the group chosen as the RS and the propensity odds ê(X)/(1 − ê(X) (where ê(X) is the estimated PS) to the non-RS. When using the combined sample as the RS, the Inverse-Probability-of-Treatment-Weighted (IPTW) estimator must be used and derives the effect estimate by using the inverse of the PS (1/ê(X)) as weights for one group and the inverse of 1 minus the PS (1/(1 − ê(X)) for the other. The marginal ORs are estimated by fitting logistic regression using the ethnic indicator as predictor, with the PS weights used as sampling weights in the model estimation (29).

Standardization and Reference Sample

Standardization is the traditional approach in epidemiology to obtain an overall effect estimate when a confounder modifies the relationship between the outcome and a risk factor. Standardization takes a weighted average of the stratum-specific rates or risks, with the strata defined by the effect modifier and with weights corresponding to the number of persons in the RS falling into each category of the effect modifier (30). As noted above, disparity estimates from the PS stratification method are created from a weighted average of stratum-specific estimates which is essentially a standardization process where the PS serves as the effect modifier (see an illustration in (29) Appendix 1). Sato and Matsuyama (31) also show that PS weighting allows for nonparametric standardization using either the exposure group or the total combined groups as the RS.

Conditional and Marginal ORs from Logistic Regression

We first fit a standard logistic regression model which generates the conditional OR for the ethnic disparity estimate. Using the fitted model, two predicted response probabilities are generated for each individual by plugging in the actual values of their observed covariates (other than ethnicity) into the regression model. Regardless of an individual’s actual ethnicity, the first predicted response probability is generated by assuming the individual is white; the second by assuming he/she is Hispanic. Using these two sets of estimated response probabilities for each individual, the separate “white”/“Hispanic” marginal response probabilities are estimated as the simple average across all respondents (thus using the total combined sample as the RS) where everyone was assumed to be white, and (separately) Hispanic. Finally, the marginal OR is then derived using these two marginal response probabilities. Alternatively, the marginal response probabilities, as well as the corresponding marginal ORs, can also be generated by averaging the individual predicted probabilities across the white or Hispanic sample only, thus using the white or Hispanic sample as the RS. See the Appendix for details and also see (19).

Marginal ORs from logistic regression incorporating effect modification

Here, interaction terms are explicitly built into the regression model to generate ethnic-specific coefficients. Individual predicted response probabilities for the white and Hispanic effects, marginal response probabilities and the corresponding marginal ORs are all derived in a similar manner to that described above using standard logistic regression models. More details for this procedure can be found in the Appendix. Unlike the above procedure, the presence of effect modification (e.g., SES with ethnicity) requires the appropriate covariate and coefficient values to be used in the construction of marginal probabilities. Considering a white female case, her estimated counterfactual Hispanic outcome would be obtained by plugging in her actual SES into the regression but using the estimated Hispanic SES coefficient (rather than that estimated for whites). The marginal response probability thus derived can then be interpreted as the predicted outcome for Hispanics had they the same covariate distribution as whites.

RESULTS

We first examined differences in the distributions of demographic and SES covariates as well as effect modification between white and Hispanic at-risk drinkers. Table 1 shows pronounced differences in socio-demographic estimates between the two ethnic groups, with Hispanics more likely to be male, younger, have lower education, income and no health insurance compared to whites. The right-hand panel of Table 1 also shows the percentages of white and Hispanic at-risk drinkers receiving any primary care across the levels of each covariate. The magnitude of the ethnic difference in outcome rates is smaller among those with higher income and education vs. those with lower levels. The presence of effect modification was also indicated by the large differences in the magnitude of the χ² statistics across the two ethnic groups.

Table 1.

Distribution of Covariates across White and Hispanic at Risk Drinkers and Their Association with Primary Care Use within Ethnic Groups

	Distribution of covariates (%)		% Receiving any primary care
	White (N=2798)	Hispanics (N=684)	White (N=2798)	Hispanics (N=684)
Sex	χ2(1)=26.9***		χ2(1)=1.21	χ2(1)=7.3**
Male	52.4	63.5	33.1	15.0
Female	47.6	36.5	37.3	23.2
Age	χ2(3)=108.1***		χ2(3)=0.71	χ2(3)=2.3
18–29	26.7	45.9	32.7	15.9
30–49	51.4	42.8	36.4	20.1
50–64	16.5	9.5	35.5	16.9
65+	5.4	1.8	34.4	25.0
Education	χ2(3)=402.8***		χ2(3)=3.91	χ2(3)=30.6***
<HS grad	5.3	28.9	33.3	8.1
HS grad	25.4	30.7	29.6	15.7
Some college	30.4	23.2	35.6	23.9
College grad	39.0	17.1	38.6	30.8
Income	χ2(6)=233.7***		χ2(6)=4.21	χ2(6)=27.3***
≤ 10k	6.8	17.0	30.9	10.3
10,001–20k	8.5	21.2	27.8	13.8
20,001–30k	11.1	14.5	34.4	12.1
30,001–40k	12.1	11.3	31.1	22.5
40,001–60k	19.1	11.3	39.1	32.5
>60k	34.6	15.8	37.7	26.9
Missing	7.9	9.1	32.7	12.9
Insurance	χ2(4)=304.6***		χ2(4)=5.31	χ2(4)=33.3***
None	10.0	35.7	25.7	8.2
Private	74.5	48.7	36.5	26.4
Medicaid	1.8	2.8	41.2	10.5
Medicare/Fed	9.7	7.0	38.2	14.6
Other	3.4	5.8	22.5	15.0

^*P<0.05,

^**P<0.01,

^***P<0.001

¹χ² test statistics for whites are derived by normalizing sample size of whites to that of Hispanics (N=684)

We examined the common support region and evaluated the balancing quality of the two PS approaches. The estimated PS, the probability of being Hispanics, ranged from 0.039 to 0.823 for the Hispanic sample, and from 0.024 to 0.823 for whites. The non-overlapping region was very small and sensitivity analysis restricting to the common support generated similar results. Overall, PS weighting performed better than PS stratification in balancing the covariates. Of the 21 dummy variables from the demographic and SES measures, and assessed separately for Hispanics, whites, and the combined sample as the RS (63 tests altogether), no significant differences were found between whites and Hispanics using PS weighting for any of the covariates. All comparisons had p>0.10 and only 2 out of 63 had p<0.20. In contrast, using PS stratification assessing balancing for all covariates separately across the 5 strata, about 9–13% of comparisons showed significant differences at p<0.05 for the three RSs, with the 5^th stratum (i.e., those with the largest PS) producing the majority of the unbalanced covariates.

Table 2 shows the effect estimates, both overall and within each stratum, using the PS stratification method. With Hispanics used as the RS, shown in the top third of Table 2, as expected, the five strata each contained a similar number of Hispanics. The middle of Table 2 shows results when whites are used as the RS, for which the strata contain a similar proportion of whites and analogously for the combined white/Hispanic groups used as the RS as shown in the bottom third of Table 2. In general, larger ORs, and hence larger disparity estimates, were observed for the strata with higher PS estimates for all three choices of the RS. However, with Hispanics as the RS, four strata had ORs significantly different from one. By contrast, with whites as the RS, only two strata had ORs significantly different from 1.00. This suggests that the overall ethnic effect would be larger with Hispanics used as the RS compared to whites as the RS. As shown in the last rows of the three sections in Table 2, the overall marginal ORs were 1.99, 1.44, and 1.57 using Hispanics, whites, and the combined white/Hispanic sample as the RS, respectively in separate procedures.

Table 2.

Estimated Proportions and Odds Ratios (ORs) of any Primary Care Use for whites and Hispanics, for Strata and Combined Overall across Strata for the PS Stratification Method, using Hispanics, Whites and the Combined Sample as Reference Samples

	Ethnicity	N	Mean (se)	Difference (se)	Odds ratios (whites vs Hispanics)
PS Stratification, Hispanics as reference sample
Stratum (from lowest to highest propensity for being Hispanic)
1	White	1470	0.381 (0.013)	0.022 (0.042)	1.10
	Hispanics	142	0.359 (0.040)
2	White	789	0.346 (0.017)	0.128 (0.040)**	1.90**
	Hispanics	133	0.218 (0.036)
3	White	321	0.308 (0.026)	0.157 (0.040)***	2.51**
	Hispanics	139	0.151 (0.030)
4	White	160	0.231 (0.033)	0.134 (0.042)**	2.80***
	Hispanics	134	0.097 (0.026)
5	White	58	0.241 (0.056)	0.175 (0.060)**	4.49**
	Hispanics	136	0.066 (0.021)
Overall	White	2798	0.302 (0.033) 1
	Hisp	684	0.178 (0.031) 1	0.123 (0.045)**	1.99 (1.58, 2.51) 2
PS Stratification, whites as reference sample
Stratum (from lowest to highest propensity for being Hispanic)
1	White	572	0.406 (0.021)	0.011 (0.082)	1.05
	Hispanics	38	0.395 (0.079)
2	White	553	0.354 (0.020)	0.054 (0.063)	1.28
	Hispanics	60	0.300 (0.059)
3	White	562	0.386 (0.021)	0.035 (0.059)	1.16
	Hispanics	74	0.351 (0.055)
4	White	552	0.330 (0.020)	0.118 (0.046)*	1.83*
	Hispanics	99	0.212 (0.041)
5	White	559	0.279 (0.019)	0.175 (0.024)***	3.33***
	Hispanics	413	0.104 (0.015)
Overall	White	2798	0.351 (0.020) 1
	Hisp	684	0.272 (0.054) 1	0.079 (0.058)	1.44 (1.12, 1.87) 2
PS Stratification, combined whites/Hispanics as reference sample
Stratum (from lowest to highest propensity for being Hispanic)
1	White	751	0.403 (0.018)	0.101 (0.072)	1.56
	Hispanics	43	0.302 (0.070)
2	White	571	0.354 (0.020)	0.009 (0.066)	1.04
	Hispanics	58	0.345 (0.062)
3	White	581	0.348 (0.020)	0.069 (0.050)	1.38
	Hispanics	97	0.278 (0.043)
4	White	553	0.340 (0.020)	0.120 (0.040)**	1.83**
	Hispanics	141	0.220 (0.035)
5	White	342	0.257 (0.024)	0.165 (0.028)***	3.39***
	Hispanics	345	0.093 (0.015)
Overall	White	2798	0.340 (0.020) 1
	Hisp	684	0.248 (0.050) 1	0.093 (0.054)†	1.57 (1.23, 2.00) 2

¹The overall mean is simple average of the means from the five strata and pooled SE is calculated based on (16).

²Pooled OR is calculated from the pooled proportion from white/Hispanic groups; 95% CIs were generated using bootstrap method.

^†p<0.10,

^*p<0.05,

^**p<0.01,

^***p<0.001

Table 3 shows that, before adjustment, whites had 2.47 times the odds compared to Hispanics to report a primary care visit. Next, adjusting for demographics and SES using logistic regression, the adjusted conditional OR dropped to 1.84. The marginal OR from the logistic regression adjusting for covariates was 1.81. This marginal OR was derived by averaging across the total combined sample. For illustrative purpose, we estimated the marginal OR for whites and, separately, Hispanics as the RS as 1.82 and 1.81, respectively. Note that similar results are expected across different RSs here, as effect modification was not incorporated in the model.

Table 3.

Estimated Disparities in Primary Care Use for the PS Weighting Method Standardized to Different Reference Samples, Compared to Raw and Multivariable Logistic Regression, Whites versus Hispanics

Raw OR (95% CI)	2.47 (2.00, 3.05)
Multivariable logistic regression, conditional OR (95% CI)	1.84 (1.47,2.31)
Multivariable logistic regression, marginal OR (95% CI)	1.81 (1.45, 2.27)1
PS weighting, marginal ORs (95% CI)
- Hispanics as reference sample	2.01 (1.58, 2.57)
- Whites as reference sample	1.41 (1.09, 1.83)
- Combined sample as reference sample	1.54 (1.19, 1.99)

¹Marginal OR was derived for the total combined sample; 95% CI was generated using bootstrap method

For a specific choice of RS, PS weighting methods produced results quite similar to those from PS stratification. Using PS weighting, the marginal ORs were 2.01, 1.41, and 1.54 using Hispanics, whites, and the combined white/Hispanic sample as the RS, separately.

Table 4 shows the marginal effects from the logistic regressions incorporating effect modification by SES, including the marginal response probabilities and marginal ORs for the three RS choices. Using Hispanics, whites, and the combined white/Hispanic sample as the RS, the marginal ORs were 2.01, 1.48, and 1.56, respectively, each of which are similar to those produced from PS methods for the corresponding RS.

Table 4.

Estimated Disparities in Primary Care Using Logistic Regression Incorporating Effect Modification

Hispanics as reference sample
Hispanics proportion	0.180
Counterfactual whites proportion	0.305
Difference in proportion (whites vs Hispanics)	0.126
Marginal OR (whites vs Hispanics)	2.01 (1.54, 2.61) 1
Whites as reference sample
Counterfactual Hispanics proportion	0.268
Whites proportion	0.351
Difference in proportion (whites vs Hispanics)	0.083
Marginal OR (whites vs Hispanics)	1.48 (1.18, 1.86) 1
Combined whites/Hispanics as reference sample
Counterfactual Hispanics proportion	0.251
Counterfactual whites proportion	0.342
Difference in proportion (whites vs Hispanics)	0.092
Marginal OR (whites vs Hispanics)	1.56 (1.25, 1.93) 1

¹95% CIs were generated using bootstrap method

DISCUSSION

Our goal was to compare standard logistic regression and Propensity Score (PS) methods to estimate the main effect of ethnicity on a health-related outcome in the presence of effect modification. This is a common problem in epidemiological research, particularly in disparities research where ethnicity and SES are typically confounded. Standard regression analyses, controlling for SES and other demographics, estimated that whites had 1.84 times the odds of receiving primary care compared to Hispanics. However, when effect modifiers are present, this conditional estimate assumes, incorrectly, the ethnicity effects are constant across SES levels. In contrast, PS methods produced marginal estimates for a given reference sample (RS), using a standardization process that used weighting to reflect the covariate distribution in the RS. In this study, if our substantive research aim is the ethnic difference between whites and Hispanics if Hispanics were like whites in terms of measured SES characteristics (i.e., using whites as the RS), the results from PS methods showed whites had 1.41 or 1.44 times the odds (using PS weighting or stratification) of visiting primary care than Hispanics, markedly different from the conditional OR of 1.84 using standard logistic regression.

Our findings highlight the importance of careful selection of the RS to generate appropriate marginal effects when potential confounders are also effect modifiers. In the disparities research, the decisions about the RS should be informed by the substantive research question at hand and should also consider policy implications of such a choice. Findings from disparities research are often politically charged and rather technical choices concerning the selection of RS can substantially impact the “take home” message of one’s research. As illustrated above, ethnic disparity estimate using whites as the RS (OR=1.41 or 1.44 using two PS approaches) was substantially different from those using Hispanics as the RS (OR=1.99 or 2.01). In this study, as in many other disparities investigations, a comparison of greater interest is the Hispanics-white disparity where the minority population (Hispanics) had the same demographic and SES characteristics as whites, when whites, an advantaged group, serve as the RS.

Given the importance of the selection of RS, it should be carefully thought through by researchers designing PS analyses. One relevant issue is incomplete matching (32), when it is not possible to find matches for each case in the RS. PS methods, particularly the PS matching approach, often produce a matched subsample in order to achieve well balanced groups. When the number of unmatched individuals is large, there is concern whether the matched individuals are representative of the population being studied (33). This is particularly problematic when effect modification is present and the covariate distribution in the matched subsample is different from that in the total RS. Recent work has shown inconsistent estimates when different matched sub-samples were chosen (21, 23). In this study, to use PS matching with whites as the RS, one would need to identify enough individuals from a much smaller sample (Hispanics) to match with all subjects from the much larger sample (whites). Due to this potential problem, the PS matching method was not implemented here and left for future research.

PS is a complex approach and researchers implementing it are confronted with technical choices that can potentially impact the results. Other than the choice of RS, there are questions regarding choice of specific PS approach to apply (matching, stratification or weighting), evaluation of common support, and assessment of balancing quality, etc. (33) In this analysis, we found that the PS stratification approach might not fully balance the covariates due to uncontrolled intra-stratum confounding. One remedy is to further adjust for the SES covariates within each PS stratum. For example, using the combined sample as the RS, the ORs adjusting for covariates are 1.62, 0.96, 1.42, 1.81 and 3.01 for the five strata 1–5, in comparison to 1.56, 1.04, 1.38, 1.83 and 3.39 without covariate adjustment as in the bottom section of Table 2. Comparing the two sets of results suggests the overall OR using the new approach might be slightly lower than we previously found (1.57). A problem of this adjusting approach is that, given the non-collapsibility property of ORs, the overall OR cannot be derived simply by averaging the stratum-specific ORs (=1.76) as was done for stratum-specific response probabilities which are collapsible.

Finally, we found that the marginal effects derived from logistic regression that included interaction terms appeared quite similar to those produced from the PS methods. We stress the comparability between the PS estimates and regression estimates for marginal effects only. Conceptually, PS methods produce counterfactual outcome estimates for a specific RS and generate a marginal effect. Quantitatively, the marginal and conditional ORs from logistic regressions do not always converge due to the non-collapsibility property of ORs (34, 35). Furthermore, when interaction terms are entered into the regression model, marginal ORs for a specific RS can be produced in a process involving the averaging across a given population. This generalized regression approach incorporating effect modification has its roots both in the Rubin casual model as well as the economics literature. Rubin (36) developed a formula estimating the treatment effect using linear regression for the combined treated and non-treated samples, basically equivalent to the marginal effects we use here for the combined sample as the RS. Separately, labor economists devised the so-called Oaxaca-Blinder (OB) decomposition (37, 38) to estimate gender and racial disparities due to discrimination. Its generalized form for dichotomous outcomes (39) is equivalent to what we present here using whites or Hispanics as the RS. Last, while a model specification including interaction terms is more flexible than one only having main effects, all regressions are parametric models which assume some underlying data generation processes which might not be correct and may have potential problems stemming from untrustworthy extrapolation (7). In this regard, PS methods as a quasi-parametric approach may be more robust.

List of Abbreviations

PS

Propensity score

RS

Reference sample

SES

Socioeconomic status

OR

Odds ratio

NAS

National Alcohol Survey

SMR

Standardized Mortality Ratio

IPTW

Inverse-Probability-of-Treatment-Weighted

Appendix

1. The calculation of PS stratification marginal response probability and marginal ORs

For each of the three RS selections (Hispanics, whites and the combined sample), the overall (marginal) response probability for the white and Hispanic effect is estimated by weighting the stratum-specific response probability

$${\widehat{p}}_{1,\mathit{Resp}}={\sum }_{s=1}^5{w}_s{\widehat{p}}_{1s}\&{\widehat{p}}_{0,\mathit{Resp}}={\sum }_{s=1}^5{w}_s{\widehat{p}}_{0s}$$ (1)

Where p̂_1,Resp and p̂_0,Resp are the overall response probabilities for white effect and Hispanic effect separately, and p̂_1s and p̂_0s the stratum-specific response probabilities for whites and Hispanics separately, where s=1 to 5. Since the reference sample (RS) is divided into five equal size strata in the current design, equal weights are assigned across the five strata and the overall response probability is just the simple average of the stratum-specific probability. The overall (marginal) OR is defined for whites versus Hispanics as:

$$OR=\frac{{\widehat{p}}_{1,\mathit{Resp}}/(1-{\widehat{p}}_{1,\mathit{Resp}})}{{\widehat{p}}_{0,\mathit{Resp}}/(1-{\widehat{p}}_{0,\mathit{Resp}})}$$ (2)

2. The calculation of marginal ORs from standard logistic regression

Consider a standard logistic regression model

$$\mathit{logit}\{P(Y=1\mid Z,X)\}={\beta }_0+Z{\beta }_Z+X{\beta }_x$$ (3)

where Y is the dichotomous observed response, Z is indicator for ethnicity (coded as 1 if white and 0 if Hispanic) and X are covariates. For this model, the conditional OR is estimated as exp (β̂_Z). Using the fitted model (3) above, for each subject i, two predicted response probabilities are generated given individual observed covariates x_i. The first is generated by assuming the respondent is white (i.e., using Z=1); the second by assuming the respondent is Hispanic (i.e., Z=0), regardless of the respondent’s actual ethnicity. The response probability can be represented as

$${\widehat{p}}_l^i=\frac{1}{(1+\mathit{exp}\{-({\widehat{\beta }}_0+{\widehat{\beta }}_{Z}l+x_i{\widehat{\beta }}_x)\})}$$ (4)

where l=1, then =0 for each respondent for the white and Hispanic effects.

The marginal response probability for the total combined sample is just the average across the whole sample of individual predicted probability for both white and Hispanic effects (l=1 and 0 respectively):

$${\widehat{p}}_l=\frac{1}{n}{\sum }_{i=1}^n{\widehat{p}}_l^i$$ (5)

Finally the marginal OR for the total combined sample is as below.

$$OR=\frac{{\widehat{p}}_1/(1-{\widehat{p}}_1)}{{\widehat{p}}_0/(1-{\widehat{p}}_0)}$$ (6)

Alternatively, the marginal response probabilities, as well as the corresponding marginal ORs, can also be estimated for the white or Hispanic sample only as the RS (Z=1 or 0, respectively), for both the white and Hispanic effects (l=1 and 0 respectively):

$${\widehat{p}}_l^Z=\frac{1}{n_z}{\sum }_{i=1}^{n_z}{\widehat{p}}_l^{zi}$$ (7)

3. The calculation of marginal ORs from standard regression incorporating effect modification

The logistic regression (3) above is generalized to include ethnic-specific coefficients in the model

$$\mathit{logit}\{P(Y=1\mid Z_1,Z_0,X)\}={\beta }_{01}{Z}_1+{\beta }_{00}{Z}_0+{XZ}_1{\beta }_{x1}+{XZ}_0{\beta }_{x0}$$ (8)

where Z₁ is the indicator for whites and Z₀ is the indicator for Hispanics, and the intercept is removed from the model. Similar to (4) above, for each subject i, two predicted response probabilities can be generated that, respectively, assuming the respondent is white or Hispanic (regardless of their true ethnicity) to generate the white and Hispanic effects (l=1 and 0 respectively). For example, for a white individual, the predicted response probability for the white effect (l=1) is

$${\widehat{p}}_{l=1}^{wi}=\frac{1}{(1+\mathit{exp}\{-({\widehat{\beta }}_{01}+x_{wi}{\widehat{\beta }}_{x1})\})}$$ (9)

For this same white individual, the predicted response probability for the Hispanic effect (l=0) is

$${\widehat{p}}_{l=0}^{wi}=\frac{1}{(1+\mathit{exp}\{-({\widehat{\beta }}_{00}+x_{wi}{\widehat{\beta }}_{x0})\})}$$ (10)

Note in (10) while this white individual’s own covariates x_wi are used, the coefficients for Hispanics (β̂₀₀ and β̂_x0) are plugged into the model to generate the counterfactual predicted outcome if the respondent were Hispanic. The predicted response probability in (10) can also be interpreted as the predicted outcome for Hispanics should they have the same covariate distribution as whites.

Similarly, the predicted response outcome for each Hispanic individual can be estimated to generate the white and Hispanic effects (l=1 and 0, respectively). The marginal response probability can then be calculated by taking the average of the individual predicted probabilities as shown in (5) for the combined overall sample as the RS or (7) for whites or Hispanics as the RS. Finally, the marginal OR for each RS is derived as shown in (6) using the corresponding marginal response probabilities.