Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2011 Sep 1.
Published in final edited form as: J Psychoactive Drugs. 2010 Sep;42(3):315–325. doi: 10.1080/02791072.2010.10400694

Inpatient Hospitalization in Addiction Treatment for Patients with a History of Suicide Attempt: A Case of Support for Treatment Performance Measures†

Joseph E Glass 1, Mark A Ilgen 2, Jamie J Winters 3, Regan L Murray 4, Brian E Perron 5, Stephen T Chermack 6
PMCID: PMC3071023  NIHMSID: NIHMS253923  PMID: 21053754

Abstract

This study attempts to validate substance use disorder (SUD) treatment performance measures (PM) in a naturalistic treatment setting. Despite its significance in healthcare systems and in SUD populations, suicidality is one patient characteristic that remains unexplored in the context of SUD PMs. The current study focused on the extent to which the care processes encouraged by SUD PMs were associated with improved outcomes in patients with a prior suicide attempt as compared to those without. We abstracted Addiction Severity Index and health services data from the VA medical record for 381 veterans who initiated outpatient SUD treatment and completed baseline intake measures at a Midwestern VA hospital. Cox proportional hazard regressions examined how baseline characteristics, prior suicide attempts, and PM status predicted the time until hospitalization for psychiatric or substance use problems. Prior suicide attempts significantly interacted with treatment engagement, and hospitalization risk was significantly higher among individuals with a prior suicide attempt who did not meet PMs. This study provides initial observational evidence that past suicide attempts may be a factor that should be considered when defining performance standards that influence the processes of SUD treatment. Future research on PMs should take into account the differences on indicators of high risk and poor treatment outcomes.

Keywords: continuity of care, substance use disorders, suicide, treatment engagement, treatment performance measures

In the field of substance use disorder (SUD) treatment, policy makers and researchers have emphasized the importance of increased involvement in early treatment (Garnick et al., 2006), as well as continued treatment engagement throughout longer periods of time (McKay, 2005). These recommendations arise from evidence suggesting that increased treatment contact throughout the continuum of care produces better long-term treatment outcomes for patients in addiction treatment. Such beneficial outcomes have included reductions in alcohol/drug use (Finney and Moos, 2002; McKay et al., 2002; Moos and Moos, 2003), decreased criminal involvement (Garnick et al., 2007; Simpson et al., 1997), fewer treatment readmissions (Luchansky et al., 2000), and fewer inpatient hospitalizations (Foote and Erfurt, 1991; Kristenson et al., 1983).

CURRENT USE OF SUD PERFORMANCE MEASURES

Building on these findings, large health care systems often employ treatment performance measures that promote an increased number of SUD treatment contacts during the early stages of care (Garnick et al., 2006). Two of the most widely used SUD treatment performance measures are Washington Circle's treatment engagement (Garnick et al., 2002) and Department of Veterans Affairs’ (VA) continuity of care monitor (VA Office of Quality and Performance, 2005). Washington Circle, a group established by the Substance Abuse and Mental Health Services Administration (SAMHSA) to develop and disseminate SUD treatment performance measures, defines treatment engagement as attending a minimum of three SUD sessions in the first month of care (Garnick et al., 2002). Continuity of care is defined by the VA as engaging in the first month of care by attending at least three sessions, then attending at least two visits per month in the following two months of care (VA Office of Quality and Performance, 2005). Health care systems have incentive to perform well on these measures. For example, the Health Plan Employer Data and Information Set (HEDIS) includes treatment engagement as a measure to compare the performance of third-party payers (National Committee for Quality Assurance, 2006), allowing large companies to use HEDIS data to evaluate SUD treatment providers. Similarly, the VA incentivizes its performance measures by providing monetary rewards to front-line physicians and regional VA leadership for meeting or exceeding these predetermined standards of care (Kerr and Fleming, 2007).

CLINICAL OUTCOMES ASSOCIATED WITH SUD PERFORMANCE MEASURES: MIXED FINDINGS

Despite the widespread use of SUD performance measures, there has been conflicting evidence on their association with post-treatment outcomes (McCarty, 2007). Two recent validation studies have provided support for the treatment engagement measure, finding an association with lower rates of post-treatment arrests and incarcerations (Garnick et al. 2007) and clinically modest improvements in addiction-related problems (Harris et al., 2010). However, other studies attempting to validate the treatment engagement and continuity of care measures using aggregate treatment facility scores on continuity and clinical outcomes have failed to provide support for their use (Harris et al., 2009; Harris et al., 2007). These mixed findings suggest the need for further work to evaluate how the features that are encouraged by SUD performance measures (i.e. the number of treatment contacts in the early phases of care) relate to post-treatment outcomes.

SUBGROUP DIFFERENCES MAY EXPLAIN MIXED FINDINGS

One possible reason for the mixed findings regarding the association between SUD performance measures and post-treatment outcomes is that treatment samples are extremely heterogeneous. Patients tend to vary greatly in their level of substance use severity and psychopathology (Tiet et al. 2006a; Harrison & Asche 2001). There is also significant between-patient variability in the amount of benefit that is received from interventions provided in the early stages of SUD care (Morgenstern and McKay, 2007). For example, a review of controlled studies of psychosocial interventions for SUD patients with co-occurring psychiatric illness found that early treatment engagement was particularly beneficial for this population, and was associated with improved outcomes such as decreased substance use and hospitalizations (Drake et al., 2004). Thus, increasing the number of sessions in the early phase of treatment may provide more benefit to some individuals than for others. With respect to SUD performance measures, recent studies that have used interaction and subgroup analyses seem to support this conceptual argument, where stronger associations between performance measures and post-treatment outcomes were found for particular groups of patients such as those of older age (Garnick et al., 2007) and those who were not abstinent at intake (Harris et al., 2009). As has been noted more generally about research on SUD treatment, it is essential to identify the aspects of care that are especially important to subgroups of patients (Finney and Moos, 1986; Finney and Moos, 2002). With the exception of the findings in two studies discussed above, limited data exist about subgroups of SUD patients for whom early treatment engagement and continuous care might be particularly important. Further research is needed to identify patient characteristics that are associated with differing levels of benefit with regard to SUD performance measures (Harris et al., 2009).

SUD PATIENTS WITH SUICIDALITY AS A DISTINCT SUBGROUP

Despite its significance in healthcare systems and in SUD populations, suicidality is one patient characteristic that remains unexplored in research on SUD performance measures. Individuals with substance use disorders are clearly at increased risk for suicide (Wilcox et al., 2004). In SUD treatment settings, rates of prior suicidality are high with over 40% of patients reporting either suicidal ideation or a suicide attempt at some point prior to seeking SUD treatment (Ilgen et al., 2010; Johnsson and Fridell, 1997). Prior research has identified that individuals who present to SUD treatment with a prior suicide attempt are a unique subgroup of patients who might require more intensive services (Tiet et al., 2006b). At treatment entry, those with a recent suicide attempt report substantially more psychopathology and more problematic use of substances than those not reporting a recent attempt (Ilgen et al., 2004). Additionally, observational evidence suggests that individuals with a recent suicide attempt respond better to inpatient treatment than outpatient SUD treatment (Ilgen et al., 2005). This likely reflects the overall higher severity of SUD patients with a recent suicide attempt and their need for more intensive monitoring. Similarly, the VA healthcare system has taken action by commissioning suicide prevention coordinators to promote increased treatment contact for all veterans at risk for suicide (Department of Veterans Affairs, 2008), and recently instituted a performance measure promoting increased mental health treatment contact for patients after discharge from SUD or psychiatric hospitalization (VA Office of Quality and Performance, 2009). However, with respect to SUD treatment, the extent to which patients with a prior suicide attempt might be uniquely responsive to treatment engagement and continuity of care remains unknown.

In this context, the present study examines whether SUD treatment performance measures are associated with an improved treatment response, particularly for patients with a previous suicide attempt. In a time-to-event analysis, inpatient hospitalizations related to substance use and/or psychiatric problems represent the primary outcome variable. Understanding if SUD performance measures are uniquely associated with hospitalization hazard for this subgroup will help inform whether or not increased treatment contact in early treatment phases may help keep these patients in routine (i.e. rather than acute) care for longer periods. In addition to informing the literature on SUD performance measures, understanding this care process is related conceptually to the nature of adaptive treatment algorithms, which individually adjust (i.e. increase or decrease) the intensity of SUD treatment for patients in a given treatment phase, based on individual characteristics and prior treatment response (McKay 2009). Thus, this research could help inform the development of these algorithms.

HYPOTHESES

Prior suicide attempts are common in adults seeking treatment for substance use disorders (Anderson et al., 1995; Johnsson and Fridell, 1997) and those with prior suicide attempts have poorer post-treatment mental health functioning than those without (Ilgen, Tiet, & Moos 2004). Importantly, preliminary evidence indicates that those with recent suicide attempts report better treatment outcomes following the receipt of residential SUD treatment (Ilgen et al. 2005). Given that SUD patients with suicidality may have better treatment outcomes when subjected to an increased intensity of care, we suspected that the care processes encouraged by SUD performance measures would be particularly beneficial for patients with past suicide attempts. Meeting the treatment engagement measure was operationalized as attending a minimum of three sessions in the first month of care (versus attending fewer than three), and meeting a measure of continuous care was operationalized as attending a minimum of three sessions in the first month of care and additionally two sessions in the subsequent two months of care (versus attending a fewer number of sessions in any of the first three months of care). We specifically expected that meeting either measure would be associated with a decreased risk of hospitalization, and the decrease in risk would be even greater for those past suicide attempts.

METHODS

Data Source

This study involves the use of routinely collected patient care data from a Midwestern VA Medical Center. Researchers identified and prepared data from the electronic patient record system using methodology that is equivalent to that published by the VA (VA Information Resource Center, 2007).

Study Population

Participants were 369 men and 12 women initiating SUD treatment in an outpatient specialty SUD clinic at a Midwestern VA hospital in fiscal years 2003 to 2006 who completed baseline intake measures within two weeks of initiating treatment. The average age of participants was 49.49 (SD = 10.88), 298 (78.2%) were Caucasian, 79 (20.7%) were African American, and four (1.0%) were of other racial/ethnic minority groups (e.g. Asian or Pacific Islanders, Alaskan Natives). Additionally, 73 (19.2%) participants reported that they were currently married or remarried. The average completed years of education were 13.06 (SD = 2.12).

Measures

Baseline characteristics

The Addiction Severity Index (ASI; McLellan et al., 1992) was used in the baseline interview to assess patient characteristics upon treatment entry. Clinicians administering the ASI received group and individual training to ensure proper use of the instrument. The ASI is a semistructured interview that assesses alcohol and drug use, psychiatric symptoms, medical problems, legal problems, employment problems, and interpersonal conflict over an individual's lifetime and within the 30 days prior to assessment. The ASI is commonly used in VA and non-VA facilities and has been validated as an assessment and research tool for substance use disorders (McLellan et al., 2006). Although the ASI can be used to generate composite scores related to domains of interest (e.g., alcohol-related problems), the primary aims of the present study (i.e., to examine the impact of prior suicide attempts on hospitalization hazard while adjusting for a variety of specific psychiatric symptoms, substance use variables and psychosocial characteristics) necessitated the use of individual items instead of these composite measures.

Dependent variable

The primary outcome in this study was days until hospitalization for psychiatric and/or substance use related inpatient admissions that occurred within one year of initiating SUD treatment. Hospital admissions were obtained from the medical record. Survival analyses (see section titled “Survival Analysis” below) modeled days until inpatient admission with a time-to-event analysis.

SUD performance measures

Treatment records were examined to determine performance measure status. Washington Circle defines treatment engagement as participating in three SUD treatment visits during the first month of care (Garnick et al., 2002). For our measure of treatment engagement, patients who received three or more visits met the measure, and patients who received fewer than three visits failed the measure. Our definition of meeting continuity of care, hereafter referred to as “continuous care,” was based on the VA definition (VA Office of Quality and Performance, 2005). The VA monitors SUD outpatient treatment contacts over the first three months of care. To be monitored, patients must engage in the first month of care by attending three visits. Patients pass the measure if they attend at least two visits in months two and three. It should be noted that because this study was interested in outcomes for all SUD patients, our continuous care measure was calculated for all patients, irrespective of their engagement in the first month.

Substance use, psychiatric symptoms, and suicide attempt

The ASI assessed days of use in the past month of alcohol (to intoxication), cocaine, and cannabis. We dichotomized these measures into a variable indicating past-month use, or no past-month use (i.e. abstinence). Other substances measured by the ASI (heroin, barbiturates, inhalants, hallucinogens, etc.) had low rates of past-month use (0% to 4.5%), and thus were excluded from analyses. Recent psychiatric symptoms, also identified by the ASI, included the presence of past-month depression, anxiety, and suicidal ideation. The ASI also assessed the presence of lifetime suicide attempts (any versus none).

Demographics

The ASI assessed demographics and included age, homelessness, race, marital status, and court prompted treatment. Marital status categories were collapsed into a dichotomous variable representing “single” versus “presently married.” Given the low rate of non African-American racial/ethnic minorities in this sample, race/ethnicity was dichotomized into two groups indicating whether the participant was Caucasian or of a racial/ethnic minority group.

Analysis Strategy

Association between predictors and performance measure status

Analyses began by examining the bivariate relationships between demographic variables, recent substance use patterns, recent psychiatric symptoms, suicide attempt history, and the two SUD performance measures using chi-square (χ2) tests for categorical variables and t-tests for continuous variables.

Survival analysis

Using survival analysis, we were able to determine characteristics that were associated with the length of time until hospitalization for psychiatric or substance use problems. This length of time is considered a survival period, and is measured by counting the number of days, starting from an index date, for which the event (hospitalization) did not occur. In survival analysis, censoring allows for information from all individuals to be included in the analysis, regardless of whether or not the event of interest occurs. In the current study, censoring occurred at one year (360 days) for individuals who were not hospitalized within the one-year study period. Censoring also occurred for those who died, at their date of death.

Specific details regarding the use of survival analysis to accomplish the aims of the current study can be described as follows. Cox proportional hazard regressions modeled the relative impact of performance measure status, baseline characteristics, and prior suicide attempts on the risk of hospitalizations for psychiatric or substance abuse problems that occurred within one year after initiating treatment. Since two performance measures were of interest in this study, we ran separate regressions for each performance measure. An interaction term (Baron and Kenny, 1986) examined whether suicide attempt history moderated the association between performance measure status and risk of hospitalization. Models controlled for basic demographics (age, race, gender, marital status, homelessness, court prompted treatment), past-month psychiatric symptoms (suicidal ideation, depression, and anxiety), and past-month substance use patterns (alcohol use to intoxication, cocaine, and cannabis). The outcome variable for both models was days until hospitalization for substance use or psychiatric problems. For analyses of treatment engagement, the index time started one month (30 days) after treatment initiation. For continuous care, the index time started at three months (90 days). If participants were hospitalized for any reason before the index time started (before full observation of the performance measure period), censoring occurred upon the date of hospital admission. Assumptions of proportional hazards were checked for these models. Finally, plots were generated from the hazard ratios of the interaction terms in order to observe the nature of the moderation.

In addition to the above models testing for interaction effects, further analyses examined the differences between four (2×2) risk groups. The combination of suicide attempt history status (yes vs. no) and performance measure status (met vs. failed) determined group membership. Separate models tested each performance measure. Other than replacing the primary variables of interest (suicide attempt history, performance measure status, and the interaction term) with the risk group variable, the analytic procedure remained identical. Additionally, graphs utilized unadjusted Kaplan-Meier estimates to plot survival probability curves of risk groups.

The above modeling approaches were complementary. Under optimal conditions, tests of moderation use interaction terms to examine the conditional probability of an outcome based on differing levels of two predictors (Finney, 1995; Kraemer et al., 2001). However, in instances where the predictors are dichotomous and the power to detect potential effects is low, a risk group analysis can provide information to complement statistical findings when relationships are conditional on a third variable (e.g. suicide attempt history) (Aneshensel, 2002; Lagakos, 2006).

RESULTS

Prevalence of Primary Predictors, Hospitalization, and Censored Cases

Prior suicide attempts were present for 71 (18.6%) participants. The treatment engagement measure was met by 284 (74.5%) participants, and 72 (18.9%) met the continuous care measure. Forty-one (10.8%) participants were hospitalized for psychiatric or substance use problems within one year of treatment initiation. Censoring occurred for a total of four cases (1.0%) of death and 16 cases (4.2%) for which hospitalization occurred during the performance measure observation periods.

Characteristics of Participants Who Meet Performance Measures

For each performance measure, odds ratios are reported in Table 1 to show the bivariate associations between demographics, recent substance use patterns, recent psychiatric symptoms, suicide attempt history, and performance measure status. Significant associations included the following. Being prompted by court systems to receive treatment was positively associated with treatment engagement status, χ2 (1, N=381) = 6.65, p < .05 and continuous care status, χ2 (1, N=381) = 8.00, p < .01. Recent alcohol use to intoxication was associated with a lower likelihood of meeting the continuous care performance measure, χ2 (1, N = 381) = 5.41, p < .05, and recent cannabis use was associated with lower likelihood of meeting the engagement monitor, χ2 (1, N = 381) = 8.07, p < .01. Age was not significantly associated with either performance measure.

TABLE 1.

Bivariate Associations between Baseline Presenting Characteristics and SUD Performance Measures

Independent Variable Treatment Engagement Odds Ratio (95% CI) Continuous Care Odds Ratio (95% CI)
Racial/Ethnic Minority 0.91 (0.52-1.62) 0.88 (0.50-1.55)
Female 0.58 (0.17-1.98) 0.25 (0.32-1.97)
Unmarried 0.69 (0.36-1.32) 0.77 (0.44-1.36)
Homeless 0.84 (0.39-1.81) 1.49 (0.73-3.03)
Court Prompted Treatment 2.09 (1.19-3.72)* 1.97 (1.23-3.17)*
Recent Alcohol Use 0.69 (0.43-1.11) 0.54 (0.35-0.92)*
Recent Cannabis Use 0.45 (0.26-0.79)* 0.53 (0.27-1.04)
Recent Cocaine Use 1.57 (0.65-3.84) 0.72 (0.35-1.51)
Recent Suicidal Ideation 0.87 (0.44-1.72) 0.56 (0.26-1.19)
Recent Anxiety 0.92 (0.57-1.48) 0.73 (0.46-1.61)
Recent Depression 0.61 (0.37-1.00) 0.64 (0.40-1.01)
Prior Suicide Attempt 0.97 (0.53-1.78) 1.00 (0.56-1.80)
Open in a new tab

For each performance measure (PM), unadjusted odds ratios (with 95% confidence intervals, CI) are reported. When odds ratios are significant and above 1.0, they indicate the independent variable is associated with an increased likelihood of meeting the performance measure. Independent variables were dummy coded.

*

p < .05 for 2×2 chi-square tests (test statistics are reported in the text.)

Models Examining the Interaction Between Suicide Attempt History and Performance Measure Status

Treatment engagement

Adjusted estimates from the Cox proportional hazards regression interaction model for treatment engagement are presented in Table 2. The interaction between suicide attempt history and treatment engagement was significant. In addition, suicide attempt history was significantly associated with greater hazard of hospitalization. Figure 1a shows a plot of hazard ratios for this interaction (Allison, 1995). This plot reveals that failing the treatment engagement measure was associated with a higher hospitalization hazard, but only for those with a lifetime history of suicide attempt. In contrast, for those without a suicide attempt, meeting the treatment engagement measure did not appear to relate to a change in hospitalization hazard.

TABLE 2.

Interaction Models of Prior Suicide Attempt X Performance Measure: Time-to-Event Analysis of Inpatient Hospitalization

Independent variable Treatment Engagement Hazard Ratio (95% CI) Continuous Care Hazard Ratio (95% CI)
Age 0.99 (0.96-1.02) 0.99 (0.95-1.03)
Racial/Ethnic Minority 0.67 (0.27-1.63) 0.37 (0.11-1.26)
Female 1.92 (0.48-7.70) 1.70 (0.35-8.30)
Unmarried 1.01 (0.43-2.39) 1.24 (0.48-3.20)
Homeless 1.72 (0.72-4.11) 1.43 (0.49-4.16)
Court Prompted Treatment 1.53 (0.64-3.61) 1.89 (0.67-5.31)
Recent Alcohol Use 1.20 (0.63-2.31) 1.33 (0.64-5.31)
Recent Cannabis Use 0.47 (0.17-1.32) 0.60 (0.21-1.74)
Recent Cocaine Use 1.57 (0.65-3.84) 1.67 (0.61-4.60)
Recent Suicidal Ideation 0.87 (0.34-2.21) 0.52 (0.18-1.51)
Recent Anxiety 1.47 (0.44-4.90) 1.17 (0.34-4.14)
Recent Depression 0.63 (0.18-2.20) 1.05 (0.28-3.92)
Prior Suicide Attempt 2.56 (1.08-6.08)* 2.74 (0.43-17.40)
Met SUD Performance Measure 0.54 (0.16-1.87) 1.41 (0.40-5.02)
Prior Suicide Attempt X SUD 5.61 (1.19-26.47)* 1.60 (0.22-11.51)
Performance Measure (Interaction)
Open in a new tab

Hazard ratios (with 95% confidence intervals, CI) are reported for each performance measure in separate columns to show the adjusted associations between predictors and hospitalization hazard. When hazard ratios are significant and above 1.0, they indicate the predictor is associated with an increased likelihood of hospitalization. Categorical variables were dummy coded.

*

p < .05.

FIGURE 1.

FIGURE 1

Open in a new tab

Interaction graph: Hazard of inpatient hospitalization, by performance measure status (met versus failed) and presence of prior suicide attempts (y/n)

Continuous care

Adjusted estimates from the continuous care interaction model are presented in the third column of Table 2. These results reveal that for continuous care, there were no significant associations between predictor variables and hospitalization hazard. The interaction between suicide attempt history and continuity of care was not significant. For consistency in presentation of the results, a plot of interaction variable is shown in Figure 1b. This plot shows a similar (but not a significant) trend to what was found for treatment engagement (Figure 1a).

Two-By-Two Risk Group Models for the Combined Effect of Suicide Attempt History and Performance Measure Status

Treatment engagement

The results of the Cox proportional hazard regression 2x2 risk group model for treatment engagement are described as follows.

In regards to risk groups, the conceptually highest-risk group (individuals with a prior suicide attempt who failed to engage) had a significantly higher risk of hospitalization than lowest-risk group (individuals without a suicide attempt history, who did engage) (HR = 8.59, CI = 3.56-20.73, p < .01). None of the other variables in the model were significantly associated with hospitalization. Figure 2a provides a graphical representation of the unadjusted Kaplan-Meier survival probabilities for treatment engagement, stratified by risk-groups. The graph shows, for each risk group, the estimated probability of survival (i.e. not being hospitalized) throughout each day over the remainder of the 12-month period. A likelihood ratio test found significant differences in the survival curves, suggesting that risk-group membership was associated with survival time (p < .001).

FIGURE 2.

FIGURE 2

Open in a new tab

Unadjusted Kaplan-Meier survival probabilities (probability of not being hospitalized over time), by performance measure status (met versus failed) and presence of prior suicide attempts (y/n)

Continuous care

The results of the continuous care risk group model provided similar results to those found for treatment engagement. The conceptually highest-risk group (individuals with a prior suicide attempt who did not receive continuous care) had a significantly higher risk of hospitalization than conceptually lowest-risk group (individuals without a suicide attempt history, who did receive continuous care; HR = 6.18, CI = 1.61-23.65, p < .05). Other variables in the model were not significant. The unadjusted Kaplan-Meier survival probability curves (described in “Survival analysis” above) for continuous care, stratified by risk groups, are reported in Figure 2b.

Supplemental Analyses

We confirmed the results of time-to-event analyses (for all models described) using logistic regression. The outcome variable (time until hospitalization) was changed to a dichotomous variable that indicated the presence of hospitalization (hospitalized or not) within one year of initiating treatment. Rather than censoring cases (see “Survival analysis” above), these cases were excluded. Logistic regression analyses yielded significant findings that were consistent with our primary analyses.

DISCUSSION

Overall, the association between meeting performance measures and risk of hospitalization varied depending on whether or not individuals had a history of prior suicide attempts. For patients with prior suicide attempts, failing to meet either the treatment engagement or continuous care performance measure was associated with an elevated risk of hospitalization. For those without suicide attempts, risk of hospitalization did not differ with respect to performance measure status. To the best of our knowledge, this is the first study to illustrate that SUD patients with a history of suicide attempt may have an increased likelihood of favorable treatment response (i.e. a reduced hazard of inpatient hospitalization) when they meet performance indicators targeting increased engagement in early and continuous care. Additionally, this study supports the argument that SUD performance measures may offer greater benefit to patients with higher problem severity at baseline, consistent with previous research (Harris et al., 2009) that found positive associations between treatment engagement and post-treatment outcome for patients who were using substances at intake (but for not those who were abstinent at intake).

Past research suggests that those with prior suicide attempts are a unique subgroup of patients who might require more intensive services (Tiet et al., 2006b). The current findings further suggest that increasing treatment contact in early phases of care may help address these needs. Given the high prevalence of past suicide attempts in SUD treatment samples (Anderson et al., 1995; Johnsson and Fridell, 1997), and the knowledge that patients with a past suicide attempt present to treatment with higher problem severity (Harrison and Asche, 2001; Tiet et al., 2006b), it is important to understand the treatment needs of this subgroup. To the best of our knowledge, only one prior study has examined the extent to which more intensive treatment is particularly important in those with elevated suicidality (Ilgen et al., 2005). This prior study found that those with a recent suicide attempt reported better outcomes if they were treated in residential, as compared to outpatient, settings even after accounting for other co-occurring substance use and psychiatric symptoms. The current study focused on the extent to which increased contact with outpatient care could be associated with improved outcomes in those with a prior suicide attempt. These analyses accounted for recent psychiatric and substance use problems, and indicate that over and above baseline psychiatric symptoms, suicide attempt history may be a unique characteristic that represents treatment need. For example, previous research identifies impulsivity and aggression as factors that distinguish persons with past suicide attempts from those without attempts (Mann et al., 1999); these factors also may be particularly important to address in early SUD treatment. Although our measures may not have fully captured the extent of recent psychopathology, these findings suggest that those with a prior suicide attempt represent a unique subgroup of individuals for whom the benefits of treatment engagement and continuous care (i.e. increased contact in early treatment) may be particularly indicated.

In addition to indicators of problem severity at baseline (suicide attempt history in the current study, and substance use in a previous study; Harris et al. 2009), the impact of meeting performance measures also likely depends on the specific outcome of interest. Previous studies have used abstinence (e.g. Schaefer et al., 2008), criminal involvement (Garnick et al., 2006), and ASI composite scores (Harris et al., 2009; Harris et al., 2008) as indicators of treatment outcome, whereas we focused on psychiatric and substance use related inpatient hospitalizations. It is important to consider that the treatment outcome of interest may vary depending on stakeholder audiences. Since third-party payers and administrators are likely to be the audience most intimately familiar with performance measures and healthcare costs, it can be assumed that hospitalizations are highly salient outcomes for healthcare systems.

To summarize, the empirical support for the impact of SUD performance measures on subsequent treatment outcomes has been mixed (McCarty 2007). As described previously, the heterogeneity of SUD patients may explain these differences. Characteristics such as older age (Garnick et al., 2007), baseline substance use (Harris et al., 2009), and in the current study, suicide attempt history have been associated with a better treatment response with regard to meeting performance measures. If certain patient subgroups are consistently found to have better outcomes associated with meeting certain performance measures, treatment systems may be able to refine or adapt their performance goals and/or clinical programming based on the characteristics of their patient population. As well, such findings indicating differential treatment response for patient subgroups could help inform the design of adaptive treatment algorithms (McKay, 2009) to be used in the early phases of SUD care.

Implications for Clinical Practice

The present findings suggest that SUD treatment providers should take additional steps to provide frequent treatment contact in the early phases of care for patients who present to treatment with prior suicide attempts. Previous research has provided evidence that certain interventions enhance engagement and retention/continuity in SUD treatment, such as motivational interviewing (Dunn et al., 2001; Miller and Rollnick, 2002), contingency management (Prendergast et al., 2006), and continuing care/monitoring approaches (McKay, 2009). Additionally, approaches that address psychosocial needs have been found to increase treatment engagement, including problem-service matching (McLellan et al., 1997) and individualized case management (McLellan et al., 1999). Although the authors are unaware of studies that test these interventions in subgroups of patients with prior suicide attempts, cross-sectional research suggests that SUD patients with suicidality are more likely to present to treatment with a variety of interrelated psychosocial needs (Perron et al., 2008).

Limitations

Due to the observational nature of the study design, causal relationships between patient/psychosocial factors, treatment, and subsequent hospitalizations remain unknown. However, the present analysis strategy has a number of strengths, including a naturalistic design to study the relationships among performance measures, suicide attempt history, and inpatient hospitalization, which are all of significant importance to patients and healthcare systems in their natural clinical environment. More specifically, we collected data from a treatment setting rather than part of a clinical trial or an outcomes monitoring project, and we had complete data on inpatient hospitalizations. Nevertheless, the use of psychiatric and SUD related hospitalizations as an outcome variable, as opposed to using a psychometric measure, should be interpreted with caution. For mental health treatment populations, SAMHSA defines inpatient psychiatric hospitalization as a valid indicator of poor mental health treatment outcome (SAMHSA 2006). In SUD treatment, previous literature has discussed the use of inpatient hospitalizations as an indicator of treatment outcome (McLellan et al., 2005; O'Brien and McLellan, 1996), partially because inpatient hospitalizations result in high costs to the individual and the treatment system (Ettner et al. 2006; Nordt, Muller, Rossler & Lauber 2007). However, others have argued that hospitalization could also be an indicator of better patient self awareness and may reflect a treatment system that is more responsive to the patients’ needs (Humphreys and Weingardt, 2000).

It is also important to note that this sample was comprised mostly of male veterans treated in a single VA SUD treatment facility. Clearly, more work is needed to determine the extent to which these findings apply to other patient populations or clinical settings. We hope that the present study illustrates the feasibility of examining such issues under naturalistic clinical conditions that could be applied on a greater scale or in other healthcare settings/organizations. In addition, the assessment of prior suicide attempt was based on a single item of the ASI. Although the composite measures in the ASI have sound psychometric validity (McLellan et al., 2006), the reliability and validity of the suicide attempt item is not known. However, many other studies have used the ASI to assess past suicide attempts (Ilgen et al., 2007; Ilgen et al., 2005; Saffier et al., 2007; Tiet et al., 2006a; Tiet et al., 2006b). Still, a more comprehensive measure of suicide attempt history may have yielded different results.

The present findings illustrate some of the challenges associated with studying statistical moderation effects when using relatively low base rate patient factors (suicide attempt history) and outcomes (inpatient hospitalization). Full tests of interaction effects require a considerable amount of power to detect a significant effect (Jaccard and Dodge, 2004). Thus, the present study relied on risk group models to complement interaction analyses. A side benefit of using a risk group approach is that clinicians may conceptualizes risk factors in a similar way when formulating clinical interventions, such as in problem-service matching.

Finally, since collection of data for this study, the VA has instituted new treatment initiatives such as suicide prevention interventions (e.g. increased identification, tracking, and follow-up) for high-risk veterans (Department of Veterans Affairs 2008), and performance measures that aim to increase mental health treatment contact after inpatient hospitalizations (VA Office of Quality and Performance 2009). Future research should determine whether these new VA treatment initiatives have had a positive impact on increasing SUD continuity of care and reducing hospitalizations for high-risk SUD patients.

CONCLUSIONS

The findings of this study provide a clinical example to support the use of SUD performance measures that encourage regular session attendance during the early periods of treatment. In order to inform the development and refinement of performance measures and adaptive treatment algorithms, future research is needed to delineate the patient characteristics that are associated with improved treatment response. This study provides initial observational evidence that past suicide attempts may be among the factors that should be considered when defining performance standards that influence processes of care.

In order to improve treatment services for SUD patients with a history of suicide attempt, future research is needed to delineate the characteristics of this subgroup that may respond well to treatment, particularly in the early stages of care. Furthermore, until engagement-enhancing interventions are validated for this subgroup, current interventions such as motivational interviewing, case management, contingency management, and problem-service matching, all of which have been shown to increase treatment engagement for SUD patients, should be considered as a way to potentially improve early treatment engagement and post-treatment outcomes.

Footnotes

†

This investigation was supported by the National Institutes of Health under Ruth L. Kirschtein National Research Service Award (T32 MH019960) (JG). Parts of this study were presented as an abstract at the 2008 Research Society on Alcoholism Conference and as a paper presentation at the 2008 VA Mental Health Conference. The opinions expressed are those of the authors and do not necessarily represent the views of the VA, Washington University in St Louis, or the University of Michigan. The authors would like to thank Doug Carson and the staff of the VA Ann Arbor SUD clinic for their efforts that led to the availability of data for this study.

REFERENCES

  1. Allison PD. Survival Analysis Using the SAS System: A Practical Guide. SAS Institute; Cary, NC: 1995. [Google Scholar]
  2. Anderson BA, Howard MO, Walker RD, Suchinsky RT. Characteristics of substance-abusing veterans attempting suicide: A national study. Psychological Reports. 1995;77(3 Pt 2):1231–42. doi: 10.2466/pr0.1995.77.3f.1231. [DOI] [PubMed] [Google Scholar]
  3. Aneshensel CS. Theory-Based Data Analysis for the Social Sciences. Pine Forge Press; Thousand Oaks, CA: 2002. [Google Scholar]
  4. Baron RM, Kenny DA. The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology. 1986;51(6):1173–82. doi: 10.1037//0022-3514.51.6.1173. [DOI] [PubMed] [Google Scholar]
  5. Department of Veterans Affairs . Uniform Mental Health Services in VA Medical Centers and Clinics. Veterans Health Administration; Washington, DC: 2008. [Google Scholar]
  6. Drake RE, Mueser KT, Brunette MF, Mchugo GJ. A review of treatments for people with severe mental illnesses and co-occurring substance use disorders. Psychiatric Rehabilitation Journal. 2004;27(4):360–74. doi: 10.2975/27.2004.360.374. [DOI] [PubMed] [Google Scholar]
  7. Dunn C, Deroo L, Rivara FP. The use of brief interventions adapted from motivational interviewing across behavioral domains: A systematic review. Addiction. 2001;96(12):1725–42. doi: 10.1046/j.1360-0443.2001.961217253.x. [DOI] [PubMed] [Google Scholar]
  8. Ettner SL, Huang D, Evans E, Ash DR, Hardy M, Jourabchi M, Hser YI. Benefit-cost in the California treatment outcome project: does substance abuse treatment “pay for itself”? Health Serv Res. 2006;41(1):192–213. doi: 10.1111/j.1475-6773.2005.00466.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Finney JW. Enhancing substance abuse treatment evaluations: Examining mediators and moderators of treatment effects. Journal of Substance Abuse. 1995;7(1):135–50. doi: 10.1016/0899-3289(95)90310-0. [DOI] [PubMed] [Google Scholar]
  10. Finney JW, Moos RH. Psychosocial treatment for alcohol use disorders. In: Nathan PE, Gorman JM, editors. A Guide to Treatments That Work. Oxford; New York: 2002. [Google Scholar]
  11. Finney JW, Moos RH. Matching patients with treatments: Conceptual and methodological issues. Journal of Studies on Alcohol. 1986;47(2):122–34. doi: 10.15288/jsa.1986.47.122. [DOI] [PubMed] [Google Scholar]
  12. Foote A, Erfurt JC. Effects of EAP follow-up on prevention of relapse among substance abuse clients. Journal of Studies on Alcohol. 1991;52(3):241–48. doi: 10.15288/jsa.1991.52.241. [DOI] [PubMed] [Google Scholar]
  13. Garnick DW, Horgan CM, Chalk M. Performance measures for alcohol and other drug services. Alcohol Research & Health. 2006;29(1):19–26. [PMC free article] [PubMed] [Google Scholar]
  14. Garnick DW, Horgan CM, Lee MT, Panas L, Ritter GA, Davis S, Leeper T, Moore R, Reynolds M. Are Washington Circle Performance Measures associated with decreased criminal activity following treatment? Journal of Substance Abuse Treatment. 2007;33(4):341–52. doi: 10.1016/j.jsat.2007.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Garnick DW, Lee MT, Chalk M, Gastfriend D, Horgan CM, Mccorry F, McLellan AT, Merrick EL. Establishing the feasibility of performance measures for alcohol and other drugs. Journal of Substance Abuse Treatment. 2002;23(4):375–85. doi: 10.1016/s0740-5472(02)00303-3. [DOI] [PubMed] [Google Scholar]
  16. Harris AHS, Humphreys K, Bowe T, Kivlahan DR, Finney JW. Measuring the quality of substance use disorder treatment: Evaluating the validity of the Department of Veterans Affairs Continuity of Care Performance Measure. Journal of Substance Abuse Treatment. 2009;36(3):294–305. doi: 10.1016/j.jsat.2008.05.011. [DOI] [PubMed] [Google Scholar]
  17. Harris AHS, Humphreys K, Finney JW. Veterans Affairs facility performance on Washington Circle indicators and casemix-adjusted effectiveness. Journal of Substance Abuse Treatment. 2007;33(4):333–39. doi: 10.1016/j.jsat.2006.12.015. [DOI] [PubMed] [Google Scholar]
  18. Harris AHS, Humphreys K, Bowe T, Tiet Q, Finney JW. Does Meeting the HEDIS Substance Abuse Treatment Engagement Criterion Predict Patient Outcomes? Journal of Behavioral Health Services & Research. 2010;37(1):25–39. doi: 10.1007/s11414-008-9142-2. [DOI] [PubMed] [Google Scholar]
  19. Harrison PA, Asche SE. Outcomes monitoring in Minnesota: Treatment implications, practical limitations. Journal of Substance Abuse Treatment. 2001;21(4):173–83. doi: 10.1016/s0740-5472(01)00199-4. [DOI] [PubMed] [Google Scholar]
  20. Humphreys K, Weingardt KR. Assessing readmission to substance abuse treatment as an indicator of outcome and program performance. Psychiatric Services. 2000;51(12):1568–69. doi: 10.1176/appi.ps.51.12.1568. [DOI] [PubMed] [Google Scholar]
  21. Ilgen MA, Tiet Q, Moos R. Outcomes of substance use disorder treatment in suicidal and nonsuicidal male patients. Journal of Studies on Alcohol. 2004;65(5):643–50. doi: 10.15288/jsa.2004.65.643. [DOI] [PubMed] [Google Scholar]
  22. Ilgen MA, Burnette ML, Conner KR, Czyz E, Murray R, Chermack S. The association between violence and lifetime suicidal thoughts and behaviors in individuals treated for substance use disorders. Addictive Behavior. 2010;35(2):111–15. doi: 10.1016/j.addbeh.2009.09.010. [DOI] [PubMed] [Google Scholar]
  23. Ilgen MA, Harris AH, Moos RH, Tiet QQ. Predictors of a suicide attempt one year after entry into substance use disorder treatment. Alcoholism: Clinical and Experimental Research. 2007;31(4):635–42. doi: 10.1111/j.1530-0277.2007.00348.x. [DOI] [PubMed] [Google Scholar]
  24. Ilgen MA, Tiet Q, Finney JW, Harris AHS. Recent suicide attempt and the effectiveness of inpatient and outpatient substance use disorder treatment. Alcoholism: Clinical and Experimental Research. 2005;29(9):1664–71. doi: 10.1097/01.alc.0000179411.88057.3a. [DOI] [PubMed] [Google Scholar]
  25. Jaccard J, Dodge T. Analyzing contingent effects in regression models. In: Hardy M, Bryman A, editors. Handbook of Data Analysis. Sage Publications Ltd.; London: 2004. [Google Scholar]
  26. Johnsson E, Fridell M. Suicide attempts in a cohort of drug abusers: A 5-year follow-up study. Acta Psychiatrica Scandinavica. 1997;96(5):362–66. doi: 10.1111/j.1600-0447.1997.tb09930.x. [DOI] [PubMed] [Google Scholar]
  27. Kerr EA, Fleming B. Making performance indicators work: Experiences of US Veterans Health Administration. British Medical Journal. 2007;335(7627):971–73. doi: 10.1136/bmj.39358.498889.94. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Kraemer HC, Stice E, Kazdin A, Offord D, Kupfer D. How do risk factors work together? Mediators, moderators, and independent, overlapping, and proxy risk factors. American Journal of Psychiatry. 2001;158(6):848–56. doi: 10.1176/appi.ajp.158.6.848. [DOI] [PubMed] [Google Scholar]
  29. Kristenson H, Ohlin H, Hulten-Nosslin MB, Trell E, Hood B. Identification and intervention of heavy drinking in middle-aged men: Results and follow-up of 24-60 months of long-term study with randomized controls. Alcoholism: Clinical and Experimental Research. 1983;7(2):203–9. doi: 10.1111/j.1530-0277.1983.tb05441.x. [DOI] [PubMed] [Google Scholar]
  30. Lagakos SW. The challenge of subgroup analyses—Reporting without distorting. New England Journal of Medicine. 2006;354(16):1667–69. doi: 10.1056/NEJMp068070. [DOI] [PubMed] [Google Scholar]
  31. Luchansky B, He L, Krupski A, Stark KD. Predicting readmission to substance abuse treatment using state information systems. The impact of client and treatment characteristics. Journal of Substance Abuse. 2000;12(3):255–70. doi: 10.1016/s0899-3289(00)00055-9. [DOI] [PubMed] [Google Scholar]
  32. Mann JJ, Waternaux C, Haas GL, Malone KM. Toward a clinical model of suicidal behavior in psychiatric patients. American Journal of Psychiatry. 1999;156(2):181–89. doi: 10.1176/ajp.156.2.181. [DOI] [PubMed] [Google Scholar]
  33. McCarty D. Performance measurement for systems treating alcohol and drug use disorders. Journal of Substance Abuse Treatment. 2007;33(4):353–54. doi: 10.1016/j.jsat.2007.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. McKay JR. Continuing care research: What we have learned and where we are going. Journal of Substance Abuse Treatment. 2009;36(2):131–45. doi: 10.1016/j.jsat.2008.10.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. McKay JR. Is there a case for extended interventions for alcohol and drug use disorders? Addiction. 2005;100(11):1594–610. doi: 10.1111/j.1360-0443.2005.01208.x. [DOI] [PubMed] [Google Scholar]
  36. McKay JR, Pettinati HM, Morrison R, Feeley M, Mulvaney FD, Gallop R. Relation of depression diagnoses to 2-year outcomes in cocaine-dependent patients in a randomized continuing care study. Psychology of Addictive Behaviors. 2002;16(3):225–35. [PubMed] [Google Scholar]
  37. McLellan AT, Cacciola JC, Alterman AI, Rikoon SH, Carise D. The Addiction Severity Index at 25: Origins, contributions and transitions. American Journal on Addictions. 2006;15(2):113–24. doi: 10.1080/10550490500528316. [DOI] [PubMed] [Google Scholar]
  38. McLellan AT, Mckay JR, Forman R, Cacciola J, Kemp J. Reconsidering the evaluation of addiction treatment: From retrospective follow-up to concurrent recovery monitoring. Addiction. 2005;100(4):447–58. doi: 10.1111/j.1360-0443.2005.01012.x. [DOI] [PubMed] [Google Scholar]
  39. McLellan AT, Hagan TA, Levine M, Meyers K, Gould F, Bencivengo M, Durell J, Jaffe J. Does clinical case management improve outpatient addiction treatment. Drug and Alcohol Dependence. 1999;55(1-2):91–103. doi: 10.1016/s0376-8716(98)00183-5. [DOI] [PubMed] [Google Scholar]
  40. McLellan AT, Grissom GR, Zanis D, Randall M, Brill P, O'Brien CP. Problem-service “matching” in addiction treatment. A prospective study in 4 programs. Archives of General Psychiatry. 1997;54(8):730–35. doi: 10.1001/archpsyc.1997.01830200062008. [DOI] [PubMed] [Google Scholar]
  41. McLellan AT, Kushner H, Metzger D, Peters R, Smith I, Grissom G, Pettinati H, Argeriou M. The fifth edition of the Addiction Severity Index. Journal of Substance Abuse Treatment. 1992;9(3):199–213. doi: 10.1016/0740-5472(92)90062-s. [DOI] [PubMed] [Google Scholar]
  42. Miller WR, Rollnick S. Motivational Interviewing: Preparing People For Change. Guilford Press; New York: 2002. [Google Scholar]
  43. Moos RH, Moos BS. Long-term influence of duration and intensity of treatment on previously untreated individuals with alcohol use disorders. Addiction. 2003;98(3):325–37. doi: 10.1046/j.1360-0443.2003.00327.x. [DOI] [PubMed] [Google Scholar]
  44. Morgenstern J, McKay JR. Rethinking the paradigms that inform behavioral treatment research for substance use disorders. Addiction. 2007;102(9):1377–89. doi: 10.1111/j.1360-0443.2007.01882.x. [DOI] [PubMed] [Google Scholar]
  45. National Committee for Quality Assurance . HEDIS 2006 Volume 2: Technical Specifications. Author; Washington, DC: 2006. [Google Scholar]
  46. Nordt C, Muller B, Rossler W, Lauber C. Predictors and course of vocational status, income, and quality of life in people with severe mental illness: a naturalistic study. Social Science & Medicine. 2007;65(7):1420–9. doi: 10.1016/j.socscimed.2007.05.024. [DOI] [PubMed] [Google Scholar]
  47. O'Brien CP, McLellan AT. Myths about the treatment of addiction. Lancet. 1996;347(8996):237–40. doi: 10.1016/s0140-6736(96)90409-2. [DOI] [PubMed] [Google Scholar]
  48. Perron BE, Ilgen MA, Hasche L, Howard MO. Service needs of clients in outpatient substance-use disorder treatment: A latent class analysis. Journal of Studies on Alcohol and Drugs. 2008;69(3):449–53. doi: 10.15288/jsad.2008.69.449. [DOI] [PubMed] [Google Scholar]
  49. Prendergast M, Podus D, Finney J, Greenwell L, Roll J. Contingency management for treatment of substance use disorders: A meta-analysis. Addiction. 2006;101(11):1546–60. doi: 10.1111/j.1360-0443.2006.01581.x. [DOI] [PubMed] [Google Scholar]
  50. Saffier K, Colombo C, Brown D, Mundt MP, Fleming MF. Addiction Severity Index in a chronic pain sample receiving opioid therapy. Journal of Substance Abuse Treatment. 2007;33(3):303–11. doi: 10.1016/j.jsat.2006.12.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Schaefer JA, Harris AH, Cronkite RC, Turrubiartes P. Treatment staff's continuity of care practices, patients’ engagement in continuing care, and abstinence following outpatient substance-use disorder treatment. Journal of Studies on Alcohol and Drugs. 2008;69(5):747–56. doi: 10.15288/jsad.2008.69.747. [DOI] [PubMed] [Google Scholar]
  52. Simpson DD, Joe GW, Rowan-Szal GA. Drug abuse treatment retention and process effects on follow-up outcomes. Drug and Alcohol Dependence. 1997;47(3):227–35. doi: 10.1016/s0376-8716(97)00099-9. [DOI] [PubMed] [Google Scholar]
  53. Substance Abuse and Mental Health Services Administration . National Outcome Measures (NOMs): NOMs Grid. Center for Mental Health Services, Substance Abuse and Mental Health Services Administration, U.S. Department of Health and Human Services; Rockville, MD: 2006. [Google Scholar]
  54. Tiet QQ, Finney JW, Moos RH. Recent sexual abuse, physical abuse, and suicide attempts among male veterans seeking psychiatric treatment. Psychiatric Services. 2006;57(1):107–13. doi: 10.1176/appi.ps.57.1.107. [DOI] [PubMed] [Google Scholar]
  55. Tiet QQ, Byrnes HF, Barnett P, Finney JW. A practical system for monitoring the outcomes of substance use disorder patients. Journal of Substance Abuse Treatment. 2006a;30(4):337–47. doi: 10.1016/j.jsat.2006.03.002. [DOI] [PubMed] [Google Scholar]
  56. Tiet QQ, Ilgen MA, Byrnes HF, Moos RH. Suicide attempts among substance use disorder patients: An initial step toward a decision tree for suicide management. Alcoholism: Clinical and Experimental Research. 2006b;30(6):998–1005. doi: 10.1111/j.1530-0277.2006.00114.x. [DOI] [PubMed] [Google Scholar]
  57. VA Information Resource Center . Virec Research User Guide: VHA Medical SAS Outpatient Datasets FY2006. VA Information Resource Center; Hines, IL: 2007. [Google Scholar]
  58. VA Office of Quality and Performance . Mental Health Inpatient 7 Day Follow-up Monitor. Author; Washington, DC: 2009. [Google Scholar]
  59. VA Office of Quality and Performance . Continuity of Care VHA Performance Measure. Author; Washington, DC: 2005. [Google Scholar]
  60. Wilcox HC, Conner KR, Caine ED. Association of alcohol and drug use disorders and completed suicide: An empirical review of cohort studies. Drug and Alcohol Dependence. 2004;76(Suppl):S11–19. doi: 10.1016/j.drugalcdep.2004.08.003. [DOI] [PubMed] [Google Scholar]

RESOURCES