Sources of Biased Inference in Alcohol and Drug Services Research: An Instrumental Variable Approach

Abstract

Objective:

This study examined the potential for biased inference due to endogeneity when using standard approaches for modeling the utilization of alcohol and drug treatment.

Method:

Results from standard regression analysis were compared with those that controlled for endogeneity using instrumental variables estimation. Comparable models predicted the likelihood of receiving alcohol treatment based on the widely used Aday and Andersen medical care–seeking model. Data were from the National Epidemiologic Survey on Alcohol and Related Conditions and included a representative sample of adults in households and group quarters throughout the contiguous United States.

Results:

Findings suggested that standard approaches for modeling treatment utilization are prone to bias because of uncontrolled reverse causation and omitted variables. Compared with instrumental variables estimation, standard regression analyses produced downwardly biased estimates of the impact of alcohol problem severity on the likelihood of receiving care.

Conclusions:

Standard approaches for modeling service utilization are prone to underestimating the true effects of problem severity on service use. Biased inference could lead to inaccurate policy recommendations, for example, by suggesting that people with milder forms of substance use disorder are more likely to receive care than is actually the case.

Since the early 1980s, alcohol and drug services researchers, guided by Aday and Andersen's model of medical care seeking, have pursued questions about the determinants of receiving care for alcohol and other drug problems (for reviews, see Andersen, 1995; Booth et al., 2001; Schmidt and Weisner, 1999; Weisner and Schmidt, 2001). A large body of empirical research has identified what appear to be robust predictors of entry into treatment. “Predisposing characteristics,” such as being middle aged, being unmarried, and having a treatment history, increase the propensity to enter care in studies spanning many countries and service venues (Booth et al., 1991; Cavior et al., 1967; Kaskutas et al., 1997; Klingemann et al., 1992; McKirnan, 1977; Ritson and Roumanie, 1984; Room et al., 1996; Weisner et al., 2001). Meanwhile, “enabling factors,” such as income and insurance coverage, appear to have modest or mixed effects on treatment use despite the fact that they are important predictors of seeking other types of medical services (Harwood et al., 2001; Schmidt and Weisner, 2005; Weisner, et al, 2002; Wu et al., 2003). Finally, factors representing the “severity of need,” including the severity of dependence, social consequences, and psychiatric comorbidities, are among the strongest, most robust determinants of treatment entry. The presence of a higher-severity disorder can increase the likelihood of entering services by three- to fivefold after controlling for factors that predispose and enable treatment. Substance-related legal, family, and work problems usually precipitate treatment (Weisner and Matzger, 2003), and a psychiatric comorbidity appears to increase treatment use as well (Booth et al., 1991; Orwin et al., 1999; Weisner et al., 2001).

The present study addresses methodological concerns in this body of research and, particularly, how bias in the results of this literature could affect the conclusions we draw about access to care. Services researchers typically use cross-sectional multivariate regression models to study the impact of predisposing, enabling, and need factors on the likelihood of individual treatment entry. By unpacking the factors that determine if and how people come into contact with service providers, these studies have provided an important evidence base for policymakers concerned with high rates of unmet treatment need. They show that only a fraction of people in need of alcohol and drug services—between 9% and 15%—actually obtain care for the problem (Green-Hennessy, 2002; Office of Applied Studies, 2002; Woodward et al., 1997). Studies also point to specific population groups with the greatest unmet need, such as young adults and drinkers with milder alcohol use disorders (Dawson et al., 2006; Schmidt et al., 2007). Such findings have been used to support the current thrust of national and international policy toward targeting these underserved groups through promotion of screening and brief intervention programs (Babor and Higgins-Biddle, 2000; Fiellin et al., 2000; National Institute on Alcohol Abuse and Alcoholism, 2006; Parish, 1997; WHO Brief Intervention Study Group, 1996).

Policies to promote access to alcohol and drug services, however, can only prove as good as the evidence base that guides them. Alongside the growing body of research on treatment utilization is a growing methodological critique of the literature (Andersen, 1995; Mechanic et al., 1995; Weisner and Schmidt, 2001). Some critics point to overreliance on cross-sectional designs as a fundamental flaw, spurring a new emphasis on prospective designs (Booth et al., 2000; Hser et al., 2006; Weisner and Matzger, 2002; Weisner et al., 2003). Others raise concerns about researchers’ emphasis on individual-level factors and concomitant neglect of the larger social contexts that shape help-seeking behaviors (Weisner and Schmidt, 2001). Still others raise questions about the growing reliance on general population samples that often fail to capture nonhousehold populations, where rates of substance use disorders and treatment utilization are disproportionately high (United States General Accounting Office, 1993; Weisner et al., 1995).

Another concern—one that motivates this study—involves the potential for biased inference because of unob served systematic variation in models of treatment entry, or the problem of endogeneity. Endogeneity can arise when one or more predictors in the treatment utilization model (e.g., severity of need) are correlated with important unobserved variables omitted from the model, called omitted variable bias. It can also occur when the outcome variable (e.g., treatment entry) has causal impact on one or more predictors (e.g., severity of need), called the problem of reverse causality (Blalock, 1985; French and Popovici, 2011; Hu, 1982).

Omitted variable bias could be a problem, for example, if there were an unobserved generalized vulnerability for “getting into trouble” that confounded the effects of alcohol problem severity on service use. This idea crops up in studies of social marginality, for example, in the observation that socially disenfranchised “spare people” predominate in alcohol treatment programs (Room, 1980). It is also suggested by psychological concepts related to the risk personality, which propose that some people have a general propensity for risk taking (Gruenewald et al., 1996; Kandel, 1980). If an individual has a general propensity to get into trouble—whether because of his or her social status or personality factors—that propensity could contribute to a disproportionately high risk of experiencing alcohol and drug problems and, at the same, could independently contribute to a higher likelihood of treatment entry. Because direct measures of risk personality and marginalization are not usually represented in studies of treatment utilization, they are likely to go unobserved, and their omission could bias results.

Endogeneity can also result from unobserved reverse causation. Services researchers usually rely on self-report survey data, in which errors in the time ordering of severity of need and treatment entry are seldom obvious or controlled. Researchers typically assume that measurements of treatment need precede entry into treatment in chronological time. However, the opposite could be true if careful data collection procedures are not in place to ensure the accuracy of time ordering. Even if given an explicit 12-month time frame, respondents in a survey are likely to report symptoms of dependence and any episodes of treatment that occurred over the past 12 months without differentiation of the time ordering of events. Because treatment can (and, under the best circumstances, does) have an impact on problem severity, the actual time ordering of “need” and “treatment use” in this scenario could be reversed. The potential for reverse causation bias is likely to be greatest in cross-sectional studies that capture symptoms and treatment episodes over longer time frames, such as the lifetime or past 12 months. Studies of this kind leave ample room for reporting error yet predominate in the literature on service utilization. Still, prospective designs are not immune from the problem unless appropriate care is taken to ensure that event sequencing is accurate and time frames are narrowly defined.

The potential impact of these two sources of endogeneity on utilization research has not, to our knowledge, been investigated in much depth. In this study, we used instrumental variables estimation, a technique commonly used to control for endogeneity in econometric research, to explore the potential for such bias and its ramifications. Our objectives were both to apply the instrumental variables approach to explore the extent of bias due to endogeneity and to interpret what changes occur in substantive results when one controls for endogeneity. The analysis compared results from standard regression analyses and the instrumental variables approach using the same data set and measures. This allowed us to explore the potential for endogeneity and to consider how this methodological problem could be biasing the current picture drawn from utilization research.

Method

Sample

Cross-sectional analyses were conducted on the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC) sponsored by the National Institute on Alcohol Abuse and Alcoholism. The NESARC includes a representative sample of the civilian, noninstitutionalized adult population in the United States, including all 50 states and the District of Columbia. The target population included adults living in households; military personnel living off base; and residents of group quarters such as boarding houses, rooming houses, non-transient hotels and motels, shelters, facilities for housing workers, college quarters, and group homes. Sampling frames included the U.S. Census 2000/2001 Supplementary Survey and Group Quarters Inventory. The NESARC used a multistage stratified design and contains intentional oversamples of African Americans, Hispanics, and young adults 18–24 years old. The 2001–2002 Wave 1 survey resulted in 43,093 face-to-face interviews, achieving a response rate of 81%. The Wave 2 survey re-interviewed 34,653 adults, yielding a cumulative response rate of 70%.

We focused on two relevant subsamples of participants in the NESARC who met different definitions of having a current need for alcohol and drug treatment. “Subsample 1” included all respondents who met the American Psychiatric Association's (1994) Diagnostic and Statistical Manual, Fourth Edition (DSM-IV), criteria for an alcohol use disorder or dependence in the past year, with or without a co-occurring illicit drug use disorder. “Subsample 2” included study participants with an alcohol use disorder but no current drug use disorder. To qualify as alcohol dependent, a respondent must have fulfilled at least three of seven DSM-IV criteria in the past year: tolerance, withdrawal, drinking larger amounts over a longer time, persistent efforts to cut down, time spent in activities to obtain alcohol or recover from drinking, social/occupational/recreational activities given up because of drinking, and drinking despite physical or psychosocial problems. An alcohol use disorder was defined as meeting one or more of the four DSM-IV disorder criteria, including hazardous use, failure to fulfill roles, social and legal problems, and not dependent on the same substance.

Measures

All variables included in the analysis were measured within the commonly used time frames of the lifetime and 12 months before interview. The outcome of interest was alcohol treatment utilization during the past 12 months. Alcohol services could have been provided in specialty settings, such as alcohol detoxification, inpatient programs, outpatient clinics, or rehabilitation programs. They could also have been provided in nonspecialty settings, including in the offices of private physicians or other professionals, Alcoholics Anonymous or other 12-step programs, family or social service agencies, emergency rooms, halfway house or therapeutic community, crisis centers, employee assistance programs, or by clergy.

The explanatory variable of primary interest indicated the severity of need for treatment. “Alcohol problem severity” was measured as the total number of alcohol dependence criteria reported in the past 12 months. Although this variable had a discrete range of 0–7, we chose to model it as a continuous variable after observing that the estimated residuals did not indicate large deviations from normality (skewness = 0.3, kurtosis = 2.6). Other factors related to severity of need included the presence of co-occurring drug use disorder or a mood disorder, based on DSM-IV criteria, in the past year.

Alcohol-related social consequences were specified as a dichotomous measure based on reporting one or more of the following in the past 12 months: drinking interfered with taking care of home or family, school or job troubles because of drinking, family or friendship troubles, physical fights, or arrests and other legal problems because of drinking. Other covariates reflected standard predisposing factors used in the care-seeking model: gender, age, ethnicity, education, and marital status (Table 1). Enabling factors were represented by annual family income (range from 1 to 9 with indicators of 1 ≤ $10,000, 2 = $10,000–$19,000,… 8 = $70,000–$79,000, 9 ≥ $80,000), type of health insurance coverage, and prior use of alcohol treatment.

Table 1.

Characteristics of two NESARC subsamples by alcohol treatment utilization

	Subsample 1 Alcohol use disorder (n = 2,648)		Subsample 2 Alcohol use disorder without co-occurring drug use disorder (n = 2,324)
Variable	No treatment (93.2%)	Alcohol treatment past year (6.8%)	No treatment (94.5%)	Alcohol treatment past year (5.5%)
Male, %	68.8	72.7	68.2	74.0
Age, in years, M (SE)	35.1 (0.3)	35.1 (0.9)	36.1 (0.3)	36.8 (1.2)
Ethnicity, %
White	76.7	72.7	77.3	74.1
Black	8.7	8.7	8.5	10.2
Hispanic	9.6	12.9	9.9	10.5
Other	5.0	5.8	4.4	5.2
Married/living together, %	47.3	34.6**	50.6	34.1**
Education, %
No high school graduation	11.9	19.0*	11.1	18.1
High school graduate	28.4	29.4	28.1	26.8
College or more	59.7	51.6	60.8	55.0
Family income,aM (SE)	5.0 (0.1)	4.1 (0.2)***	5.2 (0.1)	4.2 (0.3)***
Insurance, %
No health insurance	23.9	24.5***	21.6	23.2*
Public health insurance	9.9	23.1	10.1	18.7
Private health insurance	66.2	52.4	68.3	58.1
Any prior alcohol treatment, %	9.3	45.6***	8.7	43.0***
Any alcohol-related social consequences, %	21.7	81.7***	19.0	77.1***
Drug use disorder in past year, %	11.4	29.5***	–	–
Mood disorder in past year, %	18.3	42.5***	16.1	38.1***
Alcohol problem severity,bM (SE)	2.1 (0.1)	4.1 (0.2)***	1.9 (0.1)	3.7 (0.2)***

Notes: Statistical significance was determined using chi-square and t tests by treatment status within each subsample. Dashes (–) indicate that the variable was not included in model. NESARC = National Epidemiologic Survey on Alcohol and Related Conditions.

^aFamily income was scored on a range from 1 to 9 (1 = less than $10,000, 2 = $10,000–$ 19,000,… 8 = $70,000–$79,000, 9 = $80,000 or more);

^balcohol problem severity was scored as the total number of alcohol dependence criteria reported in the past 12 months (range: 0–7).

^*p <.05;

^**p <.01;

^***p < .001.

Two variables—family alcoholism history and having resided with an alcoholic adult before age 18—were used as the instruments in our instrumental variables analyses. The first was based on asking respondents to report whether there were any alcoholics or problem drinkers among first-degree relatives, including biological parents and full siblings. The second asked them to respond affirmatively or negatively to the following question: “Before you were 18 years old, was a parent or other adult living in your home a problem drinker or alcoholic?”

Analysis

We used cross-sectional data from the NESARC to compare results obtained by using instrumental variables estimation with those derived using the standard regression approach predominant in the service utilization literature. Although instrumental variables estimation is commonly used to address bias due to endogeneity in econometric research (Baltagi and Griffin, 2001; French, 2000), to our knowledge, treatment entry researchers have not yet used this approach to assess the potential for similar forms of bias in help-seeking research.

Instrumental variables estimation is a creative, albeit somewhat controversial, approach to addressing endogeneity in econometric research (French and Popovici, 2011). An instrumental variables analysis begins with a first-stage model that predicts the endogenous explanatory variable (in our case, alcohol problem severity) using “instrumental variables” or “instruments.” Instruments are variables that are directly associated with problem severity but are not associated with the ultimate outcome of interest, treatment utilization. The predicted values resulting from the first-stage model are presumed to be an unbiased measure of problem severity. In a second-stage equation, the researcher then substitutes the predicted values of problem severity (generated from the first-stage regression) for the original observed variable as a way to control for the endogeneity (Wooldridge, 2000).

As noted, instrumental variables estimation is designed to reduce bias in regression coefficient estimates caused by reverse causation and omitted variables. From a more technical standpoint, the concern it addresses is that standard regression could produce a biased estimate of the effect of alcohol problem severity on treatment utilization because it is correlated with the model's error term. By using the first-stage model's predicted values for alcohol problem severity, that is, values that are uncorrelated with treatment utilization, this source of bias is presumably eliminated. Our instrumental variables estimation was thus specified in terms of the following:

where Z_i is a vector of instrumental variables that identifies alcohol problem severity in the first-stage regression, and X_i is a vector of exogenous explanatory variables. A_i represents the predicted values of alcohol problem severity from the first-stage regression, and Y_i is the likelihood of using alcohol treatment services in the second-stage regression. In the first-stage model (1), alcohol problem severity was predicted as a function of the two instruments (Z_i) and other exogenous explanatory variables (X_i) using ordinary least squares regression. In the second-stage regression (2), the dichotomous measure of alcohol treatment use in the past year was estimated as a function of the predicted value of alcohol problem severity obtained from the first-stage regression. To obtain unbiased estimates for all coefficients in the second-stage model, all other predictors were included in the first-stage regression as well (Baltagi, 2002).

Accurate instrumental variables estimation relies on the careful identification of instruments that meet two conditions: (a) the instruments have a statistically significant correlation with the endogenous variable, alcohol severity (called the instrument's “strength”), and (b) the instruments are not directly correlated with the outcome measure, alcohol treatment entry (called the instrument's “validity”). A growing body of econometric research underscores the importance of assessing the suitability of instruments based on these two criteria (Murray, 2006; Newhouse and McClellan, 1998).

To do so, we first confirmed the instruments’ strength using the Wald test of exogeneity, which tests the null hypothesis that alcohol problem severity is exogenous (see Wooldridge 2000). Another test involves regressing the endogenous explanatory variable, alcohol problem severity, on the instrumental variables and all other exogenous predictors. Statistically significant estimated coefficients for the instruments in this regression model indicate the explanatory power, or strength, of the instruments.

Finally, to evaluate the “validity” of our two instruments, we followed the method proposed by Bollen et al. (1995) that applies over-identification restrictions on instruments for models using dichotomous outcomes. This test involved specifying a standard probit regression on alcohol treatment utilization that included the actual observed alcohol-problem severity measure, along with the instruments and other exogenous explanatory variables. If valid, the instruments should not be significant predictors of alcohol treatment utilization after controlling for problem severity. To further assess the validity of our instruments, we conducted the instrumental variables estimation using the two-stage least square (2SLS) estimator to predict alcohol treatment utilization in its continuous form. Following this, we conducted Sargan's test (Sargan, 1958; StataCorp LP, College Station, TX), which tests the null hypothesis that all instruments are uncorrelated with the error term in an equation predicting the outcome variable, treatment utilization (Wooldridge, 2000).

After having established the suitability of the instruments, we were able to proceed with a comparison of results from the standard and instrumental variables approaches for modeling treatment entry. Comparable cross-sectional probit regression and instrumental variables models on service utilization were compared for all respondents with an alcohol use disorder (Subsample 1) and separately for the subset of alcohol-use-disorder–only respondents, who did not meet criteria for a co-occurring drug use disorder (Subsample 2). Because of small sample sizes and the inability to identify suitable instruments, we were unable to generate stable estimates for respondents with illicit drug disorders alone.

All analyses were performed using survey data commands in Stata Version 10, which incorporate sampling weights and survey design variables to obtain correct standard errors for estimates. We used the probit procedure for standard regression models and the ivprobit procedure for the instrumental variables estimation. All analyses incorporate sampling weights that adjust for differential probabilities of selection across primary sampling units within strata and among housing units within the selected primary sampling units. Weights also adjust for the within-household probability of selection, nonresponse, and selection of two primary sampling units to represent an entire stratum. A raking process was used to develop weights that further enhanced representativeness of the sample's socioeconomic distribution (Grant et al., 2003).

Results

Sample characteristics

Table 1 shows the characteristics of the two subsamples with alcohol use disorders by alcohol treatment utilization. In both subsamples, alcohol problem severity was significantly associated with the likelihood of receiving treatment during the past year. Statistically significant associations were also observed for prior treatment experiences and markers for severity of need, including alcohol-related social consequences, co-occurring drug use disorders, and mood disorders. In addition, those with an episode of alcohol treatment in the past year were more likely to be unmarried/not cohabitating, to have lower incomes, and to have public-sector health insurance (e.g., Medicaid).

Assessing the suitability of instruments

Probit regression and instrumental variables models predicting alcohol treatment utilization were conducted for the two subsamples separately. The Wald tests of exogeneity for the instrumental variables models were statistically significant in both samples, suggesting that instrumental variables estimation was both warranted and appropriate, Subsample 1: χ²(1) = 4.56, p < .001; Subsample 2: χ²(1) = 6.77, p < .001. Rejection of the null hypothesis indicated that alcohol problem severity is indeed endogenous or, otherwise put, that there are correlations between problem severity and the error terms in both regressions that need to be controlled.

Two variables—family alcoholism history and having resided with an alcoholic adult before age 18—met both the “strength” and “validity” criteria as instruments for this analysis. According to French and Popovici (2011), these measures are two of the most commonly used instruments in econometric research on substance use disorders. Family alcoholism history in part reflects a biological predisposition to alcoholism and is therefore a reasonable predictor of adult problem severity. Living with a problem-drinking adult during childhood could influence attitudes and choices about alcohol use later in adult life (Dawson et al., 2005; Grucza et al., 2008; Hingson et al., 2006; Linsky et al., 1985; Moss et al., 2008; Wilsnack et al., 1991).

These arguments provided theoretical support for the strength of the instruments, which was further supported by statistically significant regression coefficients for the instruments in models predicting alcohol problem severity. Table 2 shows results of the ordinary least square regressions on alcohol problem severity using the instruments and other exogenous predictors. Both family alcoholism history and residing with alcoholic adults before age 18 were predictive of alcohol problem severity at the α = .05 level, suggesting that they are sufficiently strong instruments.

Table 2.

Regressions on alcohol problem severity in two NESARC subsamples

Variable	Subsample 1 Alcohol use disorder (n = 2,648) Coefficient b(SE)	Subsample 2 Alcohol use disorder without co-occurring drug use disorder (n = 2,324) Coefficient b (SE)
Male	0.016 (0.070)	0.014 (0.075)
Age	−0.023*** (0.003)	−0.023*** (0.003)
Ethnicity, ref.: White
Black	0.251 (0.095)	0.322** (0.101)
Hispanic	0.144 (0.089)	0.230* (0.095)
Other	−0.126 (0.170)	−0.132 (0.191)
Married/living together	−0.073 (0.074)	−0.083 (0.078)
Education, ref.: no high school graduation
High school graduate	−0.188 (0.110)	−0.236 (0.121)
College or more	−0.165 (0.107)	−0.209 (0.118)
Family income	−0.039 (0.014)	−0.043** (0.016)
Insurance, ref.: uninsured
Public	0.057 (0.119)	−0.006 (0.128)
Private	−0.145 (0.083)	−0.154 (0.090)
Any prior alcohol treatment	0.829*** (0.103)	0.817*** (0.114)
Any alcohol−related social consequences	0.892*** (0.081)	0.750*** (0.09)
Drug use disorder in past year	0.674*** (0.104)	–
Mood disorder in past year	0.724*** (0.083)	0.748*** (0.091)
Family alcoholism history	0.663*** (0.147)	0.633*** (0.161)
Resided with alcoholic adult during childhood	−0.178* (0.079)	−0.176* (0.085)
R2	.248	.184
F(df)	51.02 (17,2630)	32.50 (16,2307)

Notes: A dash (–) indicates that the variable was not included in model. NESARC = National Epidemiologic Survey on Alcohol and Related Conditions; ref. = reference.

^*p < .05;

^**p < .01;

^***p < .001.

To test the validity of the two instruments, we used a standard probit regression on alcohol treatment utilization that included the observed problem severity measure as well as the instruments and other explanatory variables. Results indicated that both family alcoholism history (β = .347, SE = .177, p > .055) and childhood residence with a problem-drinking adult (β = − 025, SE = .109, p > .815) were not significant predictors of alcohol treatment at the α = .05 level of significance. As a final check on instrument validity, Sargan's test statistics were not statistically significant for the instrumental variables analyses performed on both subsamples, suggesting that the instruments were uncorrelated with the error term in these models, Subsample 1: χ²(1) = 0.441, p < .507; Subsample 2: χ²(1) = 0.432, p < .511.

Comparison of results from standard and instrumental variable estimation

Table 3 compares results obtained using standard cross-sectional regression and instrumental variables models based on the Aday and Andersen help-seeking model. Results for the standard approach using probit regression indicated that, in both subsamples, alcohol problem severity as measured by the number of alcohol-dependence criteria had a statistically significant, positive association with treatment utilization (see the “Probit estimate” columns in Table 3). Other results were consistent with prior research, suggesting that this analysis replicates those found in the wider literature of treatment utilization using standard approaches. Other indicators of severity of need, including alcohol-related social consequences and a co-occurring drug use disorder, were positively correlated with treatment utilization in the probit model. Having an alcohol treatment history elevated the likelihood of obtaining treatment, in keeping with prior studies of this kind. Finally, for Subsample 1, having public insurance was positively associated with alcohol treatment use, again consistent with findings from other utilization studies (Harwood et al., 2001; Schmidt and Weisner, 2005; Weisner, 2002; Wu et al., 2003).

Table 3.

Regressions on alcohol service utilization: Standard probit and instrumental variable estimates in two NESARC subsamples

	Subsample 1 Alcohol use disorder (n = 2,648)		Subsample 2 Alcohol use disorder without co-occurring drug use disorder (n = 2,324)
Variable	Probit estimates Coefficient b (SE)	IV estimates Coefficient b (SE)	Probit estimates Coefficient b (SE)	IV estimates Coefficient b (SE)
Male	0.001 (0.101)	0.012 (0.082)	0.061 (0.115)	0.054 (0.082)
Age	0.008* (0.004)	0.015*** (0.004)	0.007 (0.004)	0.016*** (0.003)
Ethnicity, ref. White
Black	−0.127 (0.139)	−0.202 (0.113)	−0.149 (0.156)	−0.250* (0.111)
Hispanic	0.039 (0.125)	−0.024 (0.104)	−0.064 (0.145)	−0.142 (0.103)
Other	−0.237 (0.231)	−0.142 (0.196)	−0.222 (0.274)	−0.093 (0.205)
Married/living together	−0.162 (0.110)	−0.097 (0.095)	−0.172 (0.122)	−0.075 (0.095)
Education, ref.: no high school graduation
High school graduate	−0.038 (0.143)	0.036 (0.120)	−0.153 (0.163)	0.012 (0.130)
College or more	0.117 (0.137)	0.148 (0.112)	0.001 (0.156)	0.099 (0.116)
Family income	0.003 (0.021)	0.017 (0.017)	−0.010 (0.024)	0.012 (0.018)
Insurance, ref.: uninsured
Public	0.366* (0.149)	0.255 (0.142)	0.152 (0.175)	0.108 (0.130)
Private	0.180 (0.115)	0.196* (0.094)	0.143 (0.132)	0.171 (0.095)
Prior alcohol treatment	0.885*** (0.103)	0.332 (0.283)	0.835*** (0.118)	0.137 (0.260)
Any alcohol−related social consequences	1.008*** (0.103)	0.400 (0.310)	0.998*** (0.112)	0.276 (0.279)
Drug use disorder in past year	0.273* (0.118)	−0.069 (0.169)	–	–
Mood disorder in past year	0.186 (0.104)	−0.154 (0.154)	0.199 (0.121)	−0.230 (0.143)
Alcohol problem severity	0.131*** (0.025)	–	0.115*** (0.027)	–
Predicted values of alcohol problem severity		0.493*** (0.105)	–	0.539*** (0.078)
Wald statistics of model fit, χ2 (df)	429.87 (16)***	742.72 (16)***	274.90 (15)***	808.72 (15)***
Wald test of exogeneity, χ2 (df)		4.56 (1)*		6.77 (1)**

Notes: A dash (–) indicates that the variable was not included in model. NESARC = National Epidemiologic Survey on Alcohol and Related Conditions; IV = instrumental variables; ref. = reference.

^*p < .05;

^**p < .01;

^***p < .001.

The instrumental variables models (see the “IV estimates” columns in Table 3) were identical to the standard probit models except for inclusion of the predicted values of alcohol problem severity from the first-stage regression rather than the observed values. By comparing the estimates from the standard and instrumental variables models, it becomes possible to examine the degree to which results change when endogeneity is controlled. Although measures of problem severity in both the standard and instrumental variables models were statistically significant, the effect sizes differed to a substantial degree. The regression coefficients for problem severity in the instrumental variables models were about four times larger than in the standard probit regression models (.493 vs. .131 for Subsample 1 and .539 vs. .115 for Subsample 2). This suggests that the standard estimation approach underestimates the true effects of problem severity on service utilization. Note that the alcohol-problem-severity coefficients in the instrumental variables models also have larger standard errors than those in the standard probit models. This indicates a loss of efficiency in fitting these more complicated models.

Results from Table 3 further indicate that the effects of other variables in the utilization model, besides alcohol problem severity, change upon controlling for endogeneity. The effects of insurance on treatment entry differ between standard and instrumental variables models, thereby leading one to draw rather different conclusions about how insurance affects access to alcohol treatment. For Subsample 1, the instrumental variables model now leads us to conclude that having private insurance—not public insurance as before—significantly increases the likelihood of treatment entry. Similarly, we find changes in the apparent role played by alcohol treatment history: In the standard regression models, prior use of alcohol treatment appeared to have large, significant effects on the likelihood of treatment entry. However, in the instrumental variables models, this effect was rendered not statistically significant. Finally, following the instrumental variables controls on endogeneity, we no longer saw statistically significant effects for alcohol-related social consequences or for a co-occurring drug use disorder.

Discussion

The results of this study suggest that standard approaches to the cross-sectional modeling of service utilization may be prone to bias due to the problem of endogeneity. We compared results from analyses using a standard regression approach for modeling alcohol treatment utilization with analyses designed to control for endogeneity using instrumental variables estimation. Correcting for endogeneity had a meaningful impact on results—one that could significantly alter the picture of how people with substance use disorders come into contact with treatment services.

In particular, our analyses underscore the potential for standard approaches to underestimate the true effects of alcohol problem severity on treatment entry. In the services research literature, alcohol problem severity is considered one of the most robust predictors for obtaining treatment: The more severe one's alcohol problem, presumably the better one's chances are for entering treatment. This finding helps justify concerns that drinkers with less severe alcohol problems may have difficulty obtaining needed care, thereby supporting the current thrust of treatment policy toward more aggressive early intervention, screening, and brief intervention. Our results suggest, however, that standard analytic approaches, if they fail to address endogeneity, could significantly underestimate the true extent of this problem. After making adjustments for endogeneity, the effects of problem severity on treatment utilization increased by a factor of four. Controlling for endogeneity meant that, for each added report of an alcohol dependence symptom in the past year, the likelihood of entering treatment grew from .131 to .493 among those with an alcohol use disorder in Subsample 1.

We also found evidence that addressing bias in estimates of problem severity could affect other relationships in the treatment-seeking model. Introducing a presumably unbiased estimate of problem severity produced significant changes in the effects of prior treatment use, alcohol-related social consequences, and co-occurring drug and psychiatric disorders. In prior studies, these variables have appeared to be important predictors increasing the likelihood of treatment entry. Yet in all cases we considered, their effects were either rendered not statistically significant or actually changed direction after controlling for endogeneity.

Notably, our comparisons of standard and instrumental variables estimation also produced substantive changes in the effects of insurance coverage on service utilization. Prior studies have reported modest or null findings with respect to the impact of private insurance coverage on entry into alcohol and other drug services (Schmidt and Weisner, 2005; Weisner et al., 2002; Wu et al., 2003). However, our results suggest that uncontrolled endogeneity could be one explanation for why the effects of insurance coverage on treatment seem rather modest, particularly when compared with the rather robust effects of insurance on access to other kinds of health care services (Aday, 2001). Once we adjusted for endogeneity using instrumental variables estimation, having private insurance became a positive, statistically significant predictor of entry into alcohol treatment, much as it is for most other forms of health care.

Implications for research

Our results underscore the need for greater precision and methodological sophistication in the coming generation of studies on utilization of care. They suggest that endogeneity is an important concern because it could lead to systematic bias in the presumed effects of key determinants of alcohol treatment entry, including problem severity, past treatment history, social consequences, co-occurring disorders, and insurance coverage. Services researchers need to rethink research designs, data collection procedures, and statistical approaches as they look for ways to mitigate this problem in future research. Instrumental variables estimation is just one member of a family of methodological approaches that can improve accuracy in estimation. The next generation of research should push the standard toward longitudinal designs using Timeline Followback techniques that disentangle the time ordering of events, and toward research designs that can ultimately unpack the roles of causation and selection in service utilization.

Implications for services

Disentangling the relationship between severity of need and service utilization is not simply an academic exercise but has practical value as well. Biased inference is among the most crucial methodological concerns for applied research (Babbie, 1983), particularly if flawed approaches are propagated across numerous studies, thereby leading to the erroneous conclusion that findings are robust. Services research is the policymaker's window into the underlying sources of unmet need and helps to guide real-world efforts to close the alcohol and drug “treatment gap” (McAuliffe et al., 2003; Office of Applied Studies, 2002; Woodward et al., 1997).

Our findings have substantive importance because they suggest that utilization research may be underestimating the effects of problem severity on treatment entry. By ignoring the potential for endogeneity, we run the gamble of underestimating the true size of the treatment gap for drinkers with lower-severity alcohol problems—as suggested by this study, to a fourfold degree. Biased research findings could lead policymakers to erroneously conclude that the current treatment system is better able to serve people with earlier and milder forms of substance use disorders than is actually the case. As noted at the outset, policymakers at the national and international levels are making concerted efforts to increase access to early and brief intervention services that target these groups. The results of this study suggest that, if provided with more accurate research findings, they might in fact be encouraged to redouble their efforts.