Abstract
Objective
To evaluate community-based outpatient mental health services for young adults.
Methods
Participants were interviewed at ages 21, 24, 27, and 30. Outcomes included: (1) symptoms of depression, generalized anxiety, social phobia, dysthymia and post traumatic stress individually and as a global scale; and (2) a dichotomous diagnosis variable inclusive of all above disorders. Treatment was indicated by an outpatient visit to a psychiatrist or other professional.
Results
Treatment did not reduce mental disorder or symptoms. Substance use, violence, poverty, community disorganization, and family history of antisocial behavior increased risks for negative outcomes, while social support was protective.
Conclusions
The absence of positive findings associated with outpatient treatment is troubling given the empirically supported interventions for the conditions examined. Practitioners, agencies, and managed care organizations share a responsibility to implement effective and comprehensive interventions.
Keywords: Outpatient treatment, mental health services, young adults
The call for objective evidence regarding community-based mental health treatment outcomes has increased over the years (Garland, Kruse, & Aarons, 2003); however, the realization of this goal, which should be viewed as an ethical imperative for service providers (Madden, 1998), has been slow to develop. Too often treatment as usual (TAU), which may have institutional history but has not likely been evaluated, is delivered instead of services with an established evidence base (Aarons, 2004). A reliance on non-evidence based treatments is troubling on two levels. First, this is likely to be an inferior form of treatment compared to interventions with an evidence base. Second, clinicians who do not use evidence-based treatments are less likely to evaluate and make needed changes to their practice (Bickman, 2008a). These factors recently led one researcher to conclude that the best thing that can be said about mental health services is that they are not “visibly harmful” (Bickman, 2008b, p. 437). Not only are evidence-based treatments not being widely used, or implemented with fidelity (Bickman, 2008a), but information on treatment outcomes for young adults – a time when mental health risk may be particularly elevated (Tubman, Gil, Wagner, & Artigues, 2003) – lags behind information about other age groups, which is a consistent trend in the social and behavioral sciences (Van Dorn, Williams, Del-Colle, & Hawkins, 2009).
Mental Health Problems, Substance Use and Outpatient Treatment Effectiveness among Young Adults
Traditional “outpatient disorders,” including mood, anxiety and substance use disorders are prevalent among young adults. A recent study found that approximately one-sixth of college students and recent graduates met criteria for an anxiety or mood disorder (Eisenberg, Gollust, Golberstein, & Hefner, 2007). Other estimates place the prevalence of mood and anxiety disorders for young adults in the range of one-fifth of the population; rates for sub-threshold indications of these disorders are substantially higher (Wittchen, Nelson, & Lachner, 1998). Substance use disorders are the most common disorders for young adults. Over one-quarter of young adult men and almost 10% of young adult women met criteria for a lifetime alcohol or illicit substance use disorder (Wittchen, et al., 1998). The prevalence of substance use, short of disorder, in young adults, including binge drinking, marijuana use and other drug use within the past 30 days is particularly high (Andrews, Tildesley, Hops, & Fuzhong, 2002).
It is also often the case that mood, anxiety and substance use problems co-exist (Brown & Barlow, 2009; Van Dorn, et al., 2009), which can hinder successful treatment outcomes. The nexus of these co-morbid conditions, in both magnitude and effect, presents multiple difficulties for both clinicians and clients. Further, the social and treatment costs associated with comorbid conditions remain high (Dickey & Azeni, 1996; Jerrell, Wilson, & Hiller, 2000; Rice, Kelman, & Miller, 1991). For example, the combination of mental illness and substance abuse has been correlated with other negative outcomes, including violence, criminal justice involvement, and inpatient hospitalization (Steadman, et al., 1998; Swanson, et al., 1997; Swartz & Lurigio, 2007). In order to forestall these negative outcomes, it is necessary to gain a better understanding of outcomes associated with the outpatient services being delivered.
Behavioral health practitioners are varied in their beliefs about what constitutes a “best practice,” yet general agreement indicates that care should improve quality of life, which includes a reduction of “symptoms” (Daleiden, Chorpita, Donkervoet, Arensdorf, & Brogan, 2006). Over the past decade, systems of care have been urged to implement and then evaluate practices with an evidence base (Chambless & Ollendick, 2001). While this evidence-based effort has yet to fully take hold in clinical systems of care, recent evidence suggests that both cognitive-behavioral therapy and interpersonal therapy are effective in treating both depressive disorders and dysthymia (David-Ferdon & Kaslov, 2008). Empirically supported treatments based on cognitive and behavioral principles have also been demonstrated effective in improving outcomes associated with anxiety disorders, including generalized anxiety disorder, social phobia and posttraumatic stress disorder (Antony & Rowa, 2005; Lawyer & Smitherman, 2004). Although behavioral health practitioners are in general agreement that such empirically supported treatments provide the best opportunity for clients to achieve optimal treatment outcomes, there has been little progress toward treatment standardization, which hinders the routine uptake of empirically supported treatments (Antony & Rowa, 2005; Chambless & Ollendick, 2001).
Self-Selection into Treatment in Observational Studies and the Use of Propensity Scores
While the importance of evaluating treatment outcomes from both clinical and ethical perspectives is well established, there are difficulties in doing so. The primary difficulty is that in many studies, including this one, participants self-select into to treatment condition, which can result in bias (Lee, 2000). However, in the absence of random assignment to treatment, statistical methods can be used to control for self-selection bias, including propensity scoring methods (Rosenbaum & Rubin, 1983), which we have done, and describe in more detail below. Our goal in creating a propensity score is to factor out variance in the outcome associated with the decision to engage in treatment. Propensity scoring also allows for the examination of how well the covariates are “balanced” after inclusion of the propensity weight (Rosenbaum & Rubin, 1984). The use of propensity scoring approaches in studies of adult mental health outcomes is becoming more common (Swanson, et al., 2008; Swanson, Van Dorn, & Swartz, 2007; Swartz, et al., Under Review). In sum, our propensity scoring methodology was intended to create a situation wherein self-selection into treatment did not unduly influence the outcomes.
The present study attempts to address many questions that remain unanswered regarding outpatient services and their relationship to subsequent mental disorder and mental health symptoms in young adults. For example, does typical community-based outpatient treatment reduce the likelihood of disorders and symptoms? Are more treatment encounters associated with positive outcomes compared to fewer encounters? How does concurrent substance use affect risks for mental disorder and mental health symptoms in this population?
Methods
Sample, Retention, and Exclusion Criteria
This study utilized data from the Seattle Social Development Project (SSDP), an ongoing longitudinal study. Details about the study design and methodology have been presented elsewhere (Hawkins, Catalano, & Miller, 1992). Briefly, the sampling frame included all fifth graders in eighteen Seattle elementary schools in 1985 (N=1,053). A total of 808 participants (77%) consented to the longitudinal study. The SSDP panel has been interviewed in 12 waves from 1985 through 2005, when most subjects were 30 years old. Since 1986, panel members have been interviewed individually and in person. Respondents who moved out of state were tracked and interviewed. A substantial proportion of participants are from low-income families. Forty-six percent of the participants’ parents reported a maximum family income of less than $20,000 per year in 1986. About 52% of the panel members participated in the National School Lunch/School Breakfast Program between the ages of 10 and 12, indicating low-income status. Forty-two percent reported only one parent present in the home when the student was in fifth grade (Oesterle, Hill, Hawkins, & Abbott, 2008).
Data for the current study were collected between 1996 and 2005, when participants were 21, 24, 27, and 30 years old. Retention rates have remained above 90% since 1989, when participants were 14 years old. Nonparticipation by wave was not related to sex, ethnicity, or poverty. Participants with a mental disorder were no more likely than those without a mental disorder to drop out of the study.
Two analytic samples from the original longitudinal sample of N=808 were utilized in this study. First, all participants who met diagnostic criteria for a mental disorder or who attended any outpatient treatment across the four waves of data were included in the mental disorder analytic sample (N=493 participants; N=1,474 person-period observations). Second, the mental health symptom-based sample (all those reporting at least one symptom) included 774 participants and 2,322 person-periods. Because the present study examined the relationship between outpatient treatment and subsequent mental disorder and symptoms, all participants with a psychiatric or drug-related hospitalization were excluded (n=34) from both samples. There were no additional cases excluded from the mental health symptom sample as all subjects were positive for at least one symptom over the four waves of data. The University of Washington Institutional Review Board approved the protocol.
Measures
Dependent Variables
Mental disorder and symptom counts were assessed using the Diagnostic Interview Schedule (DIS) (Robins, Helzer, Croughan, Williams, & Spitzer, 1981) to measure Diagnostic and Statistical Manual of Mental Disorders (4th ed.) (DSM-IV) (American Psychiatric Association, 1994) criteria at ages 21, 24, 27, and 30. A dichotomous variable indicated fulfillment of past year DSM-IV criteria for: (a) depression, (b) generalized anxiety, (c) social phobia, (d) dysthymia, or (e) post traumatic stress. Symptom counts corresponded to DSM-IV criteria for each of the five disorders. For example, for depression, counts of symptom criteria were based on individual items assessing (a) depressed mood, (b) anhedonia, (c) weight changes, (d) sleep problems, (e) psychomotor agitation or retardation, (f) fatigue, (g) feelings of worthlessness, (h) concentration difficulties, and (i) suicidal ideation.
Independent Variables
Outpatient treatment was defined as an outpatient visit, in the prior data collection wave, to a psychiatrist or other professional for some emotional or psychological problem or symptom and was coded two ways: (a) a dummy code comparing those with any treatment to those with no treatment, and (b) dummy codes assessing the frequency of outpatient visits (i.e., between one and seven visits and eight or more visits, compared to those with no visits). Receipt of a psychiatric prescription for medication was indicated by a dichotomous variable.
The presence of a mental disorder in the prior interview wave, as assessed by the DIS, was operationalized as: (a) no mental disorder, (b) a unitary mental disorder, and (c) comorbid (i.e., two or more) mental disorders.
Severity of mental disorder was assessed as a follow-up to the DIS where participants were asked the following question: How much have these problems interfered with your life or everyday activities? In 1996 (at age 21) and 2002 (at age 27) responses ranged from 1 (very little) to 5 (very much). However, the 1999 (age 24) data presented a binary response option, which required that responses from 1996 and 2002 also be dichotomized. Anyone indicating severity of 4 or 5 in 1996 or 2002 was coded as 1 and compared to those not meeting DIS criteria or with very few symptoms.
Variables assessing outpatient treatment, psychiatric prescription, prior mental disorder, and severity of mental disorder (all of which referred to the past year) were temporally lagged from the prior data collection wave to ensure directionality with the outcome. Other independent variables (described next) were assessed concurrently with the outcome.
Concurrent Behavior Problems
Substance use was evidenced by any binge drinking (i.e., five or more drinks in one sitting) or marijuana use in the previous month or other drug use in the previous year. Violence during the previous year was measured via six items (e.g., beat up someone so badly they probably needed a doctor). Both of these variables were dichotomized with any indication of the behavior coded as 1.
Sociodemographic Characteristics
All models controlled for sex, race, education, and poverty. Poverty was measured via a single indicator that assessed receipt of public welfare benefits in the previous 12 months.
Family and Community Characteristics
Recent household antisocial behavior assessed whether any housemate was convicted of a crime, was an alcoholic or problem drinker, or was addicted to drugs or had a drug abuse problem during the previous three years.
Three types of social support were assessed. Referring to the previous year, respondents were asked how much support and encouragement they received, if they were helped when help was needed, and how much warmth and affection they received. The sources of support were the participants’ parents and partner. Respondents indicating that they pretty much received support or that they received a lot of support were coded as 1 while those indicating that they received very little or only some support were coded a 0. This coding scheme was repeated for each type and source of support. These indicators were summed into a continuous scale (mean alpha across waves=0.87).
Community disorganization was measured via six items (e.g., neighborhood crime or drug selling, shootings, or knifings). Response options for these items were not at all, not much, pretty much, and a lot. Responses were summed to create a continuous measure (mean alpha across waves=0.86).
Childhood Abuse
A variable was created to assess physical or sexual abuse prior to the age of 18. Three retrospective questions assessed physical abuse (e.g., punished with a belt, a board, a cord, or some other hard object) and four questions assessed sexual abuse (e.g., tried to make me do sexual things or watch sexual things). Respondents reporting any abuse were coded as 1 and compared to those reporting no abuse.
Propensity Scores
In addition to the above “categories” of measures (i.e., mental health, concurrent behavior problems, demographic and socioeconomic, and household and community characteristics, and childhood abuse), the following variables (i.e., rebelliousness, sensation seeking, antisocial rewards, opportunities for antisocial involvement) were used to calculate propensity scores based on the likelihood of outpatient service use at age 21. Separate propensity scores were created for the mental disorder and mental health symptom samples. The propensity scores from each model were output and used to weight the longitudinal data to account for baseline differences in the likelihood of receiving outpatient treatment in each sample. This approach, inverse probability of treatment weighting, is used to “predict” the propensity of an individual receiving the treatment they actually received (Robins, Hernan, & Brumback, 2000). Creation of the weight was based on numerator and denominator calculations predicated on exposure to initial treatment.
Rebelliousness was measured using the mean score of three items indicating respondents’ noncompliant attitudes toward societal norms (α = 0.77) (e.g., “I ignore rules that get in my way”). Response categories for these items were almost always, fairly often, sometimes, seldom, and almost never.
Sensation seeking was assessed using the mean score of two items (e.g., “How many times have you done something dangerous because someone dared you to do it?”). Response categories for these items were never; I’ve done it, but not in the past year; less than once a month; two or three times a month; and once a week or more.
Antisocial rewards were assessed by a five-item scale (α = 0.76) (e.g., “What are the chances you would be seen as cool if you carried a handgun?”). Response categories were no chance, little chance, pretty good chance, and very good chance.
Opportunities for antisocial involvement were assessed by summing two dichotomous items: During the past year, have you been asked to (a) sell drugs or (b) buy or sell stolen goods?
Analysis
First, results from a regression model predicting outpatient mental health treatment at age 21 were output to create propensity scores that were then used to weight the longitudinal data. (Two propensity models were run, one that was included in the multivariable regression models assessing diagnostic outcomes and another that was included in the models assessing symptom counts. The difference between the two propensity weights, in addition to the analytic samples, was that the former included dummy codes for diagnosis as an independent variable while the latter included symptoms as an independent variable.) Second, bivariate relationships were examined between independent variables and the two outcomes. Third, the effect of prior outpatient treatment on subsequent mental disorder and symptoms was examined two ways, using multivariable repeated-measures logistic regression (Fleiss, Williams, & Dubro, 1986; Hosmer & Lemeshow, 2000): (a) prior treatment was included as a dummy code representing any treatment compared to no treatment; and (b) dummy codes assessing treatment intensity (i.e., one to seven visits and eight or more visits compared to no treatment) were included. Both models controlled for mental health diagnosis (or symptoms for models that assessed the impact of treatment on symptoms counts), severity of mental disorder, and receipt of a psychiatric prescription concurrent to treatment and time.
For the binary outcomes (diagnoses), logistic models were estimated and for the count outcomes (symptoms), a negative binomial model was estimated due to overdispersion of the data. The negative binomial model is similar to the Poisson model, but is used in cases of overdispersed count data (Agresti, 2002). Also of note, when analyzing symptom counts, both individually and as a global scale, all regression models included an offset statement with the log of the number of possible symptoms. The inclusion of this statement was necessary as disorders had different “numbers” of symptoms, both across disorders and within disorder across waves. The latter was due to minor changes in the number of DIS symptoms assessed between administrations.
Estimates from all regression models represent an average treatment effect (ATE), which include both treated and untreated members from each of the samples. The ATE, as opposed, for example, to an average treatment on the treated effect (ATT), was deemed the most appropriate estimate for this study for two reasons. First, throughout the four waves, participants often varied in their use of treatment. For example, it was not uncommon for someone to report being in treatment at age 21, then not in treatment at age 24, but back in treatment at age 27, or any other number of treatment combinations. This natural fluctuation in the receipt of treatment makes ATT models difficult to conceptualize and interpret. Second, ATT models, or even average effect of treatment on the untreated (ATU) models, are often used to evaluate the effectiveness, or in the case of an ATU model, the potential effectiveness, of a specific “program”. However, as described above, participants did not participate in a specific treatment program; therefore, the ATE was the most appropriate treatment effect to examine for this study.
Because some cases were missing data at some waves, PROC MI was used to impute missing data with imputations set to the default of 5; PROC MIANALYZE accounted for the imputed data and output a summary of the models (SAS, 2000). This method provides less biased parameter estimates than other missing data strategies such as listwise or pairwise deletion (Schafer & Olsen, 1999). All analyses were conducted with SAS 9.1.
Results
Sample Characteristics
Baseline characteristics of both samples are shown in Table 1. The mental disorder sample had a slightly higher proportion of non-Whites, those with less than a high school education and those reporting receipt of public welfare benefits than did the larger, mental health symptoms sample. The mental disorder sample also had a slightly higher proportion of subjects who reported substance use, violence, household history of antisocial behavior, community disorganization and childhood abuse than did the mental health symptoms sample. Finally, subjects in the mental disorder sample reported having, on average, slightly fewer social supports than did those in the mental health symptoms sample.
Table 1.
Baseline demographic characterisitcs in the mental disorder and mental health symptom samples
| Mental Disorder Sample (N=493) |
Mental Health Symptoms Sample (N=774) |
|||
|---|---|---|---|---|
| N | % | N | % | |
| Demographic and socioeconomic characteristics | ||||
| Male | 246 | 49.90 | 392 | 50.65 |
| Nonwhite | 288 | 58.42 | 438 | 56.59 |
| High school and beyond | 423 | 85.80 | 683 | 88.24 |
| Poverty | 83 | 16.84 | 110 | 14.21 |
| Social problems | ||||
| Substance use | 240 | 48.68 | 339 | 43.80 |
| Violence | 95 | 19.35 | 118 | 15.25 |
| Family and community characteristics | ||||
| Recent household antisocial behavior | 216 | 43.81 | 305 | 39.41 |
| Social support (Mean and SD) | 4.92 | 2.18 | 5.01 | 2.12 |
| Community disorganization (Mean and SD) | 1.77 | 0.73 | 1.73 | 0.70 |
| Childhood Abuse | ||||
| Physical or sexual abuse | 291 | 59.03 | 448 | 57.96 |
One-hundred and fifty eight participants (32% of the mental disorder sample and 20% of the mental health symptom sample) met DIS criteria for a mental disorder at baseline (126 and 32 met criteria for one and two or more disorders, respectively); 108 of these (68%) classified their mental health problems as “severe”; and 66 of these participants (42%) reported at least one outpatient visit (47 and 19 reported between one and seven visits and eight or more visits, respectively). (Results not shown.)
Propensity Model
Table 2 displays the results of the propensity regression models that included dummy codes for unitary or comorbid diagnoses and a separate model that included a covariate for symptoms. For the former model, those with a mental disorder, those reporting a household history of antisocial behavior and those reporting recent violence were more likely to report receiving outpatient treatment. Non-Whites, males, those reporting substance use, opportunities for antisocial involvement, social support and community disorganization were less likely to use outpatient treatment. Somewhat similar findings were present for the model that included an assessment of symptoms; notably, however, substance use, social support and community disorganization were not related to outpatient treatment use in the symptom-count sample (the latter two variables approached statistical significance). Finally, the propensity scores were balanced. That is, individual two-way ANOVA models for each covariate, with a stratified propensity quintile variable added as a control factor, showed no significant differences between the two treatment conditions. The most unbalanced factor was education; however, the associated p value from the F-statistic was still greater than 0.10.
Table 2.
Factors associated with baseline (21 year old) outpatient treatment use: Propensity score creation1
| Outpatient Service Use |
||||
|---|---|---|---|---|
| OR | 95% CI | OR | 95% CI | |
| Mental health characteristics | ||||
| Mental disorder | ||||
| No mental disorder [Reference] | - | - - | ||
| Unitary mental disorder | 2.54 | (1.89– 3.43) *** | ||
| Co-morbid mental disorder | 3.49 | (2.17– 5.61) *** | ||
| Symptom count | 1.12 | (1.06– 1.18) *** | ||
| Severity of mental disorder | 1.25 | (0.90– 1.72) | 1.90 | (0.98– 3.68) † |
| Concurrent social problems | ||||
| Substance use | 0.71 | (0.55– 0.92) ** | 0.79 | (0.44– 1.43) |
| Violence | 1.77 | (1.29– 2.42) ** | 1.99 | (0.98– 4.02) † |
| Demographic and socioeconomic characteristics | ||||
| Male | 0.59 | (0.43– 0.80) *** | 0.50 | (0.25– 1.01) † |
| Nonwhite | 0.42 | (0.32– 0.55) *** | 0.35 | (0.19– 0.67) ** |
| Education | ||||
| Less than high school [Reference] | - | - - | - | - - |
| High school or Beyond | 1.28 | (0.88– 1.86) | 1.29 | (0.55– 3.02) |
| Poverty | 1.46 | (1.06– 2.01) * | 1.56 | (0.77– 3.18) |
| Family and community characteristics | ||||
| Recent household antisocial behavior | 1.79 | (1.38– 2.34) *** | 1.74 | (0.97– 3.10) * |
| Social support | 0.91 | (0.86– 0.96) *** | 0.89 | (0.78– 1.01) † |
| Community disorganization | 0.69 | (0.58– 0.83) *** | 0.67 | (0.45– 1.02) † |
| Antisocial attitudes and opportunities | ||||
| Rebelliousness | 0.96 | (0.80– 1.14) | 1.01 | (0.69– 1.47) |
| Sensation seeking | 0.92 | (0.81– 1.04) | 0.91 | (0.68– 1.21) |
| Antisocial rewards | 0.96 | (0.83– 1.12) | 0.49 | (0.32– 0.76) *** |
| Opportunities for antisocial involvement | 0.51 | (0.42– 0.62) *** | 0.46 | (0.30– 0.72) *** |
| Childhood Abuse | ||||
| Physical or sexual abuse | 0.97 | (0.74– 1.26) | 0.96 | (0.53– 1.74) |
| N individual persons | N=493 | N=774 | ||
| Pseudo R2= | 17% | 23% | ||
Statistical Significance:
p<0.10 (trend);
p<.05;
p<.01;
p<.001
Results represent summary of five imputations using PROC MI and PROC MIanalyze
Bivariate Associations
Table 3 displays bivariate associations between participants’ characteristics and both mental disorder and global symptom count over time. Those with treatment in a prior wave were more likely to subsequently report a mental disorder and symptoms of a mental disorder. There was a significant and positive effect for reporting between one and seven outpatient visits in the mental disorder model; however, there was not a significant effect for those reporting eight or more visits in this model. In the mental health symptom model both indicators of treatment were significant predictors of more symptoms; however, the effect was stronger for those with less than eight visits. Those with a prior mental disorder were more likely to report the same three years later; the association was stronger for those with two or more comorbid mental disorders. Prior mental disorder symptoms were also predictive of subsequent symptoms. Finally, those classifying their mental disorder(s) and symptoms as severe were more likely to have a mental disorder or increased symptoms three years later.
Table 3.
Factors associated with mental disorder and symptoms over time: Unadjusted odds ratios 1
| Mental Disorder | Count of Mental Health Symptoms |
|||
|---|---|---|---|---|
| OR | 95% CI | OR | 95% CI | |
| Mental health treatment and condition in prior wave | ||||
| Outpatient treatment | ||||
| None [Reference] | - | - - | - | - - |
| Any outpatient treatment | 1.78 | (1.19– 2.67) ** | 1.50 | (1.23– 1.82) ** |
| None [Reference] | - | - - | - | - - |
| Between 1 and 7 visits | 1.89 | (1.15– 3.11) * | 1.51 | (1.30– 1.76) ** |
| Eight or more visits | 1.49 | (0.69– 3.22) | 1.46 | (1.18– 1.80) ** |
| Mental health prescription | 2.00 | (0.96– 4.18) † | 1.74 | (1.41– 2.14) *** |
| Mental Disorder | ||||
| No mental Disorder [Reference] | - | - - | ||
| Unitary mental disorder | 1.75 | (1.12– 2.72) * | ||
| Co-morbid mental disorder | 3.91 | (2.18– 7.01) *** | ||
| Symptom count | 1.06 | (1.05– 1.07) *** | ||
| Severity of mental disorder | 1.98 | (1.34– 2.93) *** | 1.72 | (1.52– 1.94) *** |
| Concurrent social problems | ||||
| Substance use | 1.94 | (1.40– 2.68) *** | 1.52 | (1.37– 1.68) *** |
| Violence | 1.66 | (1.03– 2.66) * | 1.31 | (1.13– 1.53) *** |
| Demographic and socioeconomic characteristics | ||||
| Male | 0.60 | (0.40– 0.91) * | 0.70 | (0.63– 0.77) *** |
| Nonwhite | 1.46 | (0.92– 2.33) | 1.42 | (1.29– 1.58) *** |
| Education | ||||
| Less than high school [Reference] | - | - - | - | - - |
| High school and beyond | 1.25 | (0.76– 2.08) | 0.85 | (0.74– 0.99) * |
| Poverty (concurrent) | 2.26 | (1.50– 3.40) *** | 2.03 | (1.77– 2.32) *** |
| Concurrent family and community characteristics | ||||
| Recent household antisocial behavior | 1.61 | (1.12– 2.31) ** | 1.47 | (1.33– 1.63) *** |
| Social support | 0.87 | (0.80– 0.93) *** | 0.88 | (0.86– 0.90) *** |
| Community disorganization | 1.35 | (1.04– 1.76) * | 1.37 | (1.28– 1.47) *** |
| Childhood Abuse | ||||
| Physical or sexual abuse | 1.51 | (0.98– 2.34) † | 1.42 | (1.28– 1.57) *** |
| N=493 | N=774 | |||
Statistical Significance:
p<0.10 (trend);
p<.05;
p<.01;
p<.001
Results represent summary of five imputations using PROC MI and PROC MIanalyze
Both substance use and violence were also positively related to mental disorder and symptoms. Males were less likely to be classified with a mental disorder whereas those receiving public welfare benefits had an increased risk of mental disorder; these same covariates, in addition to non-White race, were also positively associated with increased symptoms. Participants reporting social support had lower odds of mental disorder and symptoms. Reporting household antisocial behavior or living in a disorganized community increased the odds of mental disorder and mental disorder symptoms. Reporting childhood abuse was associated with increased odds of mental disorder symptoms.
Treatment Outcomes
Table 4 and Table 5 display the results of the multivariable models for both mental disorder and symptom count. Table 4 includes a dummy code for any mental health treatment, whereas Table 5 includes dummy codes for treatment frequency; all models control for time. Prior outpatient treatment was positively and significantly related to subsequent mental health symptoms; however, the same effect only approached statistical significance in the mental disorder model (p=0.054) (Table 4). Those with between one and seven outpatient visits were more likely to report subsequent mental disorder. However, the effect was not significant for those with eight or more visits. These latter two findings regarding treatment intensity hold for the global symptom-based outcome as well (Table 5).
Table 4.
Outpatient treatment effects on mental disorder: Weighted multivariable analysis1
| Mental Disorder | Count of Mental Health Symptoms |
|||
|---|---|---|---|---|
| OR | 95% CI | OR | 95% CI | |
| Mental health treatment and condition in prior wave | ||||
| Outpatient treatment | 1.54 | (0.99– 2.39) † | 1.22 | (1.08– 1.38) ** |
| Mental disorder prescription | 1.66 | (0.97– 2.84) † | 0.94 | (0.76– 1.18) |
| Mental disorder | ||||
| No mental disorder [Reference] | - | - - | ||
| Unitary mental disorder | 1.42 | (0.99– 2.04) † | ||
| Co-morbid mental disorder | 1.98 | (1.20– 3.29) ** | ||
| Symptom count | 1.04 | (1.03– 1.05) *** | ||
| Severity of mental disorder | 1.31 | (0.80– 2.15) | 1.12 | (0.98– 1.28) † |
| Concurrent social problems | ||||
| Substance use | 2.03 | (1.43– 2.88) *** | 1.37 | (1.23– 1.52) *** |
| Violence | 1.23 | (0.69– 2.21) | 1.24 | (1.08– 1.44) ** |
| Demographic and socioeconomic characteristics | ||||
| Male | 0.55 | (0.37– 0.82) ** | 0.70 | (0.62– 0.78) *** |
| Nonwhite | 1.33 | (0.90– 1.97) | 1.11 | (1.00– 1.22) † |
| Education | ||||
| Less than high school [Reference] | - | - - | - | - - |
| High school or beyond | 1.33 | (0.80– 2.22) | 0.85 | (0.72– 1.00) † |
| Poverty (concurrent) | 1.87 | (1.17– 2.99) ** | 1.29 | (1.13– 1.48) *** |
| Concurrent family and community characteristics | ||||
| Recent household antisocial behavior | 1.35 | (0.91– 1.98) | 1.13 | (1.02– 1.25) * |
| Social support | 0.87 | (0.80– 0.95) ** | 0.94 | (0.92– 0.96) *** |
| Community disorganization | 1.26 | (0.99– 1.59) † | 1.14 | (1.07– 1.22) *** |
| Childhood Abuse | ||||
| Physical or sexual abuse | 1.14 | (0.87– 1.49) | 1.26 | (1.14– 1.39) *** |
| N individual persons | N=493 | N=774 | ||
| N person/period observations | N=1,474 | N=2,322 | ||
Statistical Significance:
p<0.10 (trend);
p<.05;
p<.01;
p<.001
Results represent summary of five imputations using PROC MI and PROC MIanalyze
Table 5.
Frequency of outpatient treatment effects on mental disorder: Weighted multivariable analysis1
| Mental Disorder | Count of Mental Health Symptoms |
|||
|---|---|---|---|---|
| OR | 95% CI | OR | 95% CI | |
| Mental health treatment and condition in prior wave | ||||
| Outpatient treatment | ||||
| None [Reference] | - | - - | - | - - |
| Between 1 and 7 visits | 1.47 | (1.01– 2.15) * | 1.30 | (1.06– 1.59) * |
| Eight or more visits | 1.13 | (0.52– 2.47) | 1.02 | (0.77– 1.35) |
| Mental disorder prescription | 1.73 | (0.81– 3.69) | 0.93 | (0.73– 1.19) |
| Mental disorder | ||||
| No mental disorder [Reference] | - | - - | ||
| Unitary mental disorder | 1.46 | (0.90– 2.36) | ||
| Co-morbid mental disorder | 1.90 | (1.01– 3.57) * | ||
| Symptom count | 1.04 | (1.03– 1.05) *** | ||
| Severity of mental disorder | 1.47 | (0.89– 2.42) | 1.11 | (0.97– 1.27) † |
| Concurrent social problems | ||||
| Substance use | 1.97 | (1.40– 2.78) *** | 1.35 | (1.23– 1.49) *** |
| Violence | 1.22 | (0.68– 2.17) | 1.24 | (1.07– 1.43) ** |
| Demographic and socioeconomic characteristics | ||||
| Male | 0.58 | (0.39– 0.85) ** | 0.69 | (0.62– 0.77) *** |
| Nonwhite | 1.33 | (0.89– 1.98) | 1.11 | (1.00– 1.23) † |
| Education | ||||
| Less than high school [Reference] | - | - - | - | - - |
| High school and beyond | 1.25 | (0.76– 2.07) | 0.85 | (0.74– 0.97) * |
| Poverty (concurrent) | 1.80 | (1.14– 2.84) ** | 1.27 | (1.10– 1.45) ** |
| Concurrent family and community characteristics | ||||
| Recent household antisocial behavior | 1.36 | (0.92– 1.99) | 1.14 | (1.04– 1.26) ** |
| Social support | 0.87 | (0.80– 0.95) ** | 0.94 | (0.92– 0.96) *** |
| Community disorganization | 1.24 | (0.98– 1.58) † | 1.16 | (1.08– 1.24) *** |
| Childhood Abuse | ||||
| Physical or sexual abuse | 1.15 | (0.78– 1.71) | 1.24 | (1.12– 1.37) *** |
| N individual persons | N=493 | N=774 | ||
| N person/period observations | N=1,474 | N=2,322 | ||
Statistical Significance:
p<0.10 (trend);
p<.05;
p<.01;
p<.001
Results represent summary of five imputations using PROC MI and PROC MIanalyze
Other factors were similar between the models and were consistent with the bivariate results. Reporting two or more prior mental disorders and prior mental health symptoms were predictive of those subsequent and respective outcomes. Those indicating any binge drinking, marijuana use or other drug use were more likely to also report mental disorder and symptoms; violence was also significantly and positively associated with symptoms, but not mental disorder. Males and those with social support were less likely to report mental disorder and symptoms. Poverty and community disorganization were positively associated with both mental disorder and symptoms; although the latter finding only approached significance in the models assessing mental disorder. Household antisocial behavior was significantly and positively related to mental health symptoms. Finally, illness severity and non-White racial or ethnic group were associated with a trend toward increased risk of mental health symptoms but only at a level that approached statistical significance.
Treatment Outcomes for Individual Symptom Counts
In addition to the results shown in Table 4 and Table 5, we also ran models assessing the impact of outpatient treatment on individual symptom counts for each disorder (i.e., depression, generalized anxiety, social phobia, dysthymia, and post traumatic stress) (results not shown). The overall findings from above remain consistent. That is, for each individual symptom count, outpatient treatment did not significantly reduce disorder-specific symptoms in the next wave.
Analysis of Treatment Initiation, Continuation, and Discontinuation
Given the consistent findings that outpatient treatment, including more intense treatment, conferred no significant advantage in the reduction of mental disorder or mental health symptoms in the two analytic samples, we also wanted to test whether or not there might have been differential effects for those who initiated treatment compared to those who remained untreated, and for those who continued treatment compared to those who discontinued treatment. Specifically, for those with a mental disorder, and by wave, we coded various cohorts based on whether or not they (1) initiated treatment in that wave; and (2) continued treatment in the subsequent wave; or (3) remained untreated; or (4) discontinued treatment in the subsequent wave. These chi-square analyses were conducted with the mental disorder analytic sample and confirm our prior multivariable analyses. Those with a mental disorder who initiated treatment were more likely to report a mental disorder in a subsequent wave than those with a mental disorder who remained untreated (χ2=15.38, p<0.001). The cohort of subjects with a mental disorder who continued treatment were also more likely to report a subsequent mental disorder than the cohort of subjects with a mental disorder who discontinued treatment (χ2=15.98, p<0.001). Finally, there were no notable differences between the diagnoses that comprised the untreated and discontinued cohorts compared to the diagnoses that comprised the initiated and continued cohorts, which indicate that these significant differences were not due to specific diagnoses being clustered in the untreated or discontinued cohorts.
Discussion
The key finding of the present study was that outpatient treatment—in the context of these data, perhaps best characterized as treatment as usual, or “typical” treatment for an urban-derived sample that has spread to diverse communities—did not significantly reduce mental disorder or mental disorder symptoms and in fact was related to worse mental health outcomes in young adults. This was particularly true for short-term (i.e., between 1 and 7 visits) treatment. However, we hesitate to conclude that the effects are truly iatrogenic given limitations of these models. This point is discussed further below.
This lack of positive findings for outpatient treatment is troubling, given the existence of empirically supported therapy-based treatments for the primarily outpatient-based mental disorders examined in this research (Clark, et al., 2003; Ehlers, et al., 2003; Jacobson, Martell, & Dimidjian, 2001). The results associated with short-term treatment should crystallize clinicians’ attention to the importance of developing therapeutic rapport early in the treatment process (Barrett, Chua, Crits-Christoph, Gibbons, & Thompson, 2008). The negative outcomes associated with short treatment duration is consistent with prior research positing that fewer than eight sessions does not equate to effective treatment exposure (Howard, Kopta, Krause, & Orlinsky, 1986).
Next, these results underscore the importance of comprehensive assessments of both mental health and substance use problems and of integrating treatment resources. Specifically, prior comorbid mental disorders, prior mental disorder symptoms and concurrent substance use were related to increased odds of negative outcomes. Information such as this implies a need for integrated interventions designed to address complex comorbid problems, including funding for the concurrent treatment of substance use and mental disorder.
Finally, the remediation of comorbid conditions is an important outcome to assess when evaluating overall treatment cost. This is relevant as rates of treatment failure remain high when those with comorbid problems are provided services that neglect to examine the nexus of all relevant conditions, including concurrent alcohol or illicit drug use (Drake, et al., 2001; Ford, Snowden, & Walser, 1991).
The present results are consistent with prior research emphasizing the multifaceted nature of mental disorder (Mason, et al., 2004; Silver, Mulvey, & Swanson, 2002; Van Dorn, et al., 2009). For example, mental disorder and symptoms were positively associated with poverty, community disorganization, and household antisocial behavior, whereas the same outcomes were negatively related to social support. These factors have not traditionally received attention in intervention development for young adult populations. Nonetheless, as this research indicates, these factors significantly affected mental health outcomes for this population, suggesting that their role in intervention development requires serious thought.
These results also point to a troubling dynamic between treatment and mental health symptoms for non-Whites. Specifically, non-Whites were significantly less likely than Whites to receive outpatient treatment. However, in the bivariate relationship, non-Whites were significantly more likely to report mental health symptoms. (This relationship approached significance in the symptom-based multivariable models in Table 3 and Table 4.) Lack of minority involvement in outpatient services has been identified as a significant problem (Alegria, et al., 2002). However, the present results underscore the importance of better understanding barriers to outpatient treatment for non-Whites. It is also worthwhile to note that substance use was negatively related to service use at age 21 in the mental disorder sample. Substance use often needs to be treated as a concurrent problem, however, substance use’s role in discouraging help-seeking or facilitating treatment-dropout should be explored in future research for young adults.
With respect to study limitations, it is possible that outpatient treatment resulted in short-term improvements in mental disorder and symptoms; however, these short-term improvements would not have been captured, given the three-year interview schedule. Nonetheless, prior research indicates that short-term improvements are sometimes also associated with clients remaining symptomatic at termination. When this is the case, vulnerability to illness relapse within two years is increased (Westen, Novotny, & Thompson-Brenner, 2004).
Second, the data did not contain information on specific types of treatment; the inclusion of this information might have allowed for the conceptualization of treatment as something more specific than “average” treatment or “treatment as usual.” (While it is likely that some of this treatment was of high quality, it also seems appropriate to assume that the majority of this treatment was TAU given that most practitioners provide TAU or “typical” services as opposed to those with an evidence base (Aarons, 2004).) Also, with regards to our analysis of “mental disorder”, we examined five mental disorders and they were examined in combination. Future research should examine outcomes for young adults with other disorders, including schizophrenia and bipolar disorder, with sufficiently large samples. However, our consistent findings for treatment outcomes related to both the global symptom count and disorder-specific symptom counts strengthen the validity of our findings related to “mental disorder”.
Third, it is possible that outpatient treatment served as a proxy for severity of illness; that is, maybe only participants with the most severe mental disorders or symptoms sought services. If this were the case, then finding positive treatment effects would be less likely. However, these data do not support this limitation. Specifically, at Wave 1 (age 21) there were 108 participants with a self-evaluated “severe” mental disorder. Of those participants, 85 (79%) did not have any corresponding outpatient treatment, 16 (15%) had between one and seven visits, and 6 (6%) had eight or more visits. This same pattern is present for Waves 2 (age 24) and 3 (age 27), indicating that the majority of those who self-evaluated their mental disorder as severe did not seek treatment and when they did they were less likely to stay engaged for more than seven sessions. We also included multiple proxies for illness severity in the regression models beyond the specific measure of severity. Controls were included for receipt of psychiatric medications and the presence of a unitary or comorbid mental disorder (or prior symptoms for the symptom-based models), both concurrent to treatment, but lagged in relationship to the outcome. These controls, in addition to our propensity-based weighting of the data would seem to mitigate this limitation.
Fourth, all of the data are based on self-report. However, concerns about the validity of self-report data in this case are diminished given the ongoing and successful nature of this longitudinal sample, which has been followed since 1985. In addition to the sample being followed for many years, we have established good rapport and demonstrated consistent protection of confidentiality concerning many sensitive topics like crime, violence and drug abuse, which people report freely during the interviews. Research comparing different measurement strategies has indicated that self-report surveys administered privately and confidentially, as in this study, provide reliable and valid data (Hindelang, Hirschi, & Weis, 1981).
Fifth, and finally, we were limited to evaluating changes in mental disorder and mental disorder symptoms. The use of multiple outcome measures, including for example, quality of life or treatment satisfaction may have resulted in a more nuanced picture of participants’ perceived treatment experience.
Conclusions
Findings from this longitudinal sample of young adults showed that community-based outpatient treatment did not reduce subsequent mental disorder or mental disorder symptoms. Short-term outpatient treatment was associated with worse outcomes than no outpatient treatment at all. Furthermore, treatment initiators with a mental disorder were more likely to report a subsequent mental disorder compared to those with a mental disorder who remained untreated; and those with a mental disorder who continued treatment across waves were more likely to report a subsequent mental disorder compared to those with a mental disorder who discontinued treatment prior to the next wave. These results are particularly troubling, given the existence of empirically supported treatments for the outcomes examined. Still, it is important to note that we are not implying that outpatient treatment in these data is representative of an iatrogenic effect. This claim cannot be made as we are unable to control for all of the treatment selection effects likely to affect the outcomes that we assessed. However, given our statistical adjustments and assessments of various treatment categories, including intensity and initiation and continuation, we believe that there is no evidence in these data to support the notion that outpatient treatment is improving the lives of these young adults.
The responsibility for successful treatment outcomes, however, is not the sole burden of the practitioner. Much of the onus for implementing effective treatments should be borne by the agencies in which the practitioners operate (McMillen, Zayas, Books, & Lee, 2008). Further, insurance providers and managed care organizations should both support and require the use of empirically validated interventions that address comorbid conditions, including substance use problems, and should ensure that clinicians continuously evaluate their treatment. This level of support should be commensurate with the high expectations placed on clinicians, given that “treatment as usual” or “typical” treatment in community-based mental health settings is often associated with overworked and sometimes undereducated clinicians, most likely operating in underfunded settings (Westen, 2005).
Even so, individual practitioners must assume responsibility, as well. For example, establishing therapeutic rapport early (Barrett, et al., 2008) and employing motivation- or compliance-enhancing interventions (Drake, et al., 2001; Elbogen, Mustillo, Van Dorn, Swanson, & Swartz, 2007) in the treatment encounter are vital to improved outcomes. The nexus of these factors cannot be ignored. Ineffective or incomplete treatment decreases the likelihood of someone reengaging with treatment later in life, setting up a potentially costly cascade of long-term personal, social, and economic problems.
Acknowledgements
This research was supported by grants #1R01DA09679-11 and #9R01DA021426-08 from the National Institute on Drug Abuse, #R24MH56587-06 from the National Institute of Mental Health, and #21548 from the Robert Wood Johnson Foundation. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding agencies.
References
- Aarons GA. Mental Health Provider Attitudes Toward Adoption of Evidence-Based Practice: The Evidence-Based Practice Attitude Scale (EBPAS) Mental Health Services Research. 2004;6(2):61–74. doi: 10.1023/b:mhsr.0000024351.12294.65. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Agresti A. Categorical data analysis. (2nd ed.) Hoboken, NJ: John Wiley & Sons; 2002. [Google Scholar]
- Alegria M, Canino G, Rios R, Vera M, Calderon J, Rusch D, et al. Inequalities in use of specialty mental health services among Latinos, African Americans, and non-Latino whites. Psychiatric Services. 2002;53(12):1547–1555. doi: 10.1176/appi.ps.53.12.1547. [DOI] [PubMed] [Google Scholar]
- American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th Ed. Washington, DC: American Psychiatric Association; 1994. [Google Scholar]
- Andrews JA, Tildesley E, Hops H, Fuzhong L. The Influence of Peers on Young Adult Substance Use. Health Psychology. 2002;21(4):349–357. doi: 10.1037//0278-6133.21.4.349. [DOI] [PubMed] [Google Scholar]
- Antony MM, Rowa K. Evidence-Based Assessment of Anxiety Disorders in Adults. Psychological Assessment. 2005;17(3):256–266. doi: 10.1037/1040-3590.17.3.256. [DOI] [PubMed] [Google Scholar]
- Barrett MS, Chua W-J, Crits-Christoph P, Gibbons MB, Thompson D. Early withdrawal from mental health treatment: Implications for psychotherapy practice. Psychotherapy: Theory, Research, Practice, Training. 2008;45(2):247–267. doi: 10.1037/0033-3204.45.2.247. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickman L. A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of the American Academy of Child and Adolescent Psychiatry. 2008a;47(10):1114–1119. doi: 10.1097/CHI.0b013e3181825af8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bickman L. Why don’t we have effective mental health services? Administration and Policy in Mental Health Services and Mental Health Services Research. 2008b;35:437–439. doi: 10.1007/s10488-008-0192-9. [DOI] [PubMed] [Google Scholar]
- Brown TA, Barlow DH. A Proposal for a Dimensional Classification System Based on the Shared Features of the DSM-IV Anxiety and Mood Disorders: Implications for Assessment and Treatment. Psychological Assessment. 2009;21(3):256–271. doi: 10.1037/a0016608. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Chambless DL, Ollendick TH. Empirically Supported Psychological Interventions: Controversies and Evidence. Annual Review of Psychology. 2001;52:685–716. doi: 10.1146/annurev.psych.52.1.685. [DOI] [PubMed] [Google Scholar]
- Clark DM, Ehlers A, McManus F, Hackmann A, Fennell M, Campbell H, et al. Cognitive Therapy Versus Fluoxetine in Generalized Social Phobia: A Randomized Placebo-Controlled Trial. Journal of Consulting and Clinical Psychology. 2003;71(6):1058–1067. doi: 10.1037/0022-006X.71.6.1058. [DOI] [PubMed] [Google Scholar]
- Daleiden EL, Chorpita BF, Donkervoet C, Arensdorf AM, Brogan M. Getting Better at Getting Them Better: Health Outcomes and Evidence-Based Practice Within a System of Care. Journal of the American Academy of Child and Adolescent Psychiatry. 2006;45(6):749–756. doi: 10.1097/01.chi.0000215154.07142.63. [DOI] [PubMed] [Google Scholar]
- David-Ferdon C, Kaslov NJ. Evidence-Based Psychosocial Treatments for Child and Adolescent Depression. Journal of Clincial Child and Adolescent Psychiatry. 2008;37(1):62–104. doi: 10.1080/15374410701817865. [DOI] [PubMed] [Google Scholar]
- Dickey B, Azeni H. Persons with dual diagnosis of substance abuse and major mental illness: Their excess costs of psychiatric care. American Journal of Public Health. 1996;86:973–977. doi: 10.2105/ajph.86.7.973. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Drake RE, Essock SM, Shaner A, Carey KB, Minkoff K, Kola L, et al. Implementing dual diagnosis services for clients with severe mental illness. Psychiatric Services. 2001;52(4):469–476. doi: 10.1176/appi.ps.52.4.469. [DOI] [PubMed] [Google Scholar]
- Ehlers A, Clark DM, Hackmann A, McManus F, Fennell M, Herbert C, et al. A Randomized Controlled Trial of Cognitive Therapy, a Self-help Booklet, and Repeated Assessments as Early Interventions for Posttraumatic Stress Disorder. Archives of General Psychiatry. 2003;60(10):1024–1032. doi: 10.1001/archpsyc.60.10.1024. [DOI] [PubMed] [Google Scholar]
- Eisenberg D, Gollust SE, Golberstein E, Hefner JL. Prevalence and Correlates of Depression, Anxiety and Suicidality Among University Students. American Journal of Orthopsychiatry. 2007;77(4):534–542. doi: 10.1037/0002-9432.77.4.534. [DOI] [PubMed] [Google Scholar]
- Elbogen EB, Mustillo SM, Van Dorn RA, Swanson JW, Swartz MS. The impact of perceived need for treatment on risk of arrest and violence among people with severe mental illness. Criminal Justice and Behavior. 2007;34(2):197–210. [Google Scholar]
- Fleiss JL, Williams JB, Dubro AF. The logistic regression analysis of psychiatric data. Journal of Psychiatric Research. 1986;20:145–209. doi: 10.1016/0022-3956(86)90003-8. [DOI] [PubMed] [Google Scholar]
- Ford L, Snowden LR, Walser EJ. Outpatient mental health and the dual-diagnosis patient: Utilization of services and community adjustment. Evaluation and Program Planning. 1991;14:291–298. [Google Scholar]
- Garland AF, Kruse M, Aarons GA. Clinicians and outcome measurement: What's the use? Journal of Behavioral Health Services & Research. 2003;30:393–405. doi: 10.1007/BF02287427. [DOI] [PubMed] [Google Scholar]
- Hawkins JD, Catalano RF, Miller JY. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin. 1992;112(1):64–105. doi: 10.1037/0033-2909.112.1.64. [DOI] [PubMed] [Google Scholar]
- Hindelang MJ, Hirschi T, Weis JG. Measuring delinquency. Beverly Hills: Sage; 1981. [Google Scholar]
- Hosmer D, Lemeshow S. Applied logistic regression. 2nd Edition. New York: John Wiley and Sons; 2000. [Google Scholar]
- Howard KI, Kopta SM, Krause MS, Orlinsky DE. The dose-effect relationship in psychotherapy. American Psychologist [Special Issue] 1986;41(2):159–164. [PubMed] [Google Scholar]
- Jacobson NS, Martell CR, Dimidjian S. Behavioral Activation Treatment for Depression: Returning to Contextual Roots. Clinical Psychology: Science and Practice. 2001;8(3):255–270. [Google Scholar]
- Jerrell JM, Wilson JL, Hiller DC. Issues and outcomes in integrated treatment programs for dual disorders. The Journal of Behavioral Health Services & Research. 2000;27(3):303–313. doi: 10.1007/BF02291741. [DOI] [PubMed] [Google Scholar]
- Lawyer SR, Smitherman TA. Trends in Anxiety Assessment. Journal of Psychopathology and Behavior Assessment. 2004;26:101–106. [Google Scholar]
- Lee L-F. Self-selection. In: Baltagi BH, editor. A Companion to Theoretical Econometrics. Malden, MA: Blackwell Publishing; 2000. pp. 383–409. [Google Scholar]
- Madden RG. Legal issues in social work, counseling, and mental health: Guidelines for clinical practice in psychotherapy. Thousand Oaks, CA: Sage; 1998. [Google Scholar]
- Mason WA, Kosterman R, Hawkins JD, Herrenkohl TI, Lengua LJ, McCauley E. Predicting depression, social phobia, and violence in early adulthood from childhood behavior problems. Journal of the American Academy of Child and Adolescent Psychiatry. 2004;43(3):307–315. doi: 10.1097/00004583-200403000-00012. [DOI] [PubMed] [Google Scholar]
- McMillen C, Zayas LE, Books S, Lee M. Quality assurance and improvement practice in mental health agencies: Roles, activities, targets and contributions. Administration and Policy in Mental Health and Mental Health Services Research. 2008;35:458–467. doi: 10.1007/s10488-008-0189-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Oesterle S, Hill KG, Hawkins JD, Abbott RD. Positive functioning and alcohol-use disorders from adolescence to young adulthood. Journal of Studies on Alcohol and Drugs. 2008;69:100–111. doi: 10.15288/jsad.2008.69.100. [DOI] [PubMed] [Google Scholar]
- Rice DP, Kelman S, Miller LS. Estimates of economic costs of alcohol and drug abuse and mental illness, 1985 and 1988. Public Health Reports. 1991;106:280–292. [PMC free article] [PubMed] [Google Scholar]
- Robins JM, Hernan MA, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology. 2000;11:550–560. doi: 10.1097/00001648-200009000-00011. [DOI] [PubMed] [Google Scholar]
- Robins LN, Helzer JE, Croughan J, Williams JBW, Spitzer RL. NIMH Diagnostic Interview Schedule: Version III. Rockville, MD: National Institute of Mental Health; 1981. May, [Google Scholar]
- Rosenbaum P, Rubin D. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70:41–55. [Google Scholar]
- Rosenbaum PR, Rubin DB. Reducing Bias in Observational Studies Using Subclassification on the Propensity Score. Journal of the American Statistical Association. 1984;79(387):516–524. [Google Scholar]
- SAS. SAS/STAT Software: Changes and Enhancements, Release 8.2. Cary, NC: SAS Publishing; 2000. [Google Scholar]
- Schafer JL, Olsen MK. Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behavioral Research. 1999;33(4):545–571. doi: 10.1207/s15327906mbr3304_5. [DOI] [PubMed] [Google Scholar]
- Silver E, Mulvey EP, Swanson JW. Neighborhood structural characteristics and mental disorder: Faris and Dunham revisited. Social Science & Medicine. 2002;55(8):1457–1470. doi: 10.1016/s0277-9536(01)00266-0. [DOI] [PubMed] [Google Scholar]
- Steadman HJ, Mulvey EP, Monahan J, Robbins PC, Appelbaum PS, Grisso T, et al. Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Archives of General Psychiatry. 1998;55(5):393–401. doi: 10.1001/archpsyc.55.5.393. [DOI] [PubMed] [Google Scholar]
- Swanson J, Estroff S, Swartz M, Borum R, Lachicotte W, Zimmer C, et al. Violence and severe mental disorder in clinical and community populations: The effects of psychotic symptoms, comorbidity, and lack of treatment. Psychiatry. 1997;60(1):1–22. doi: 10.1080/00332747.1997.11024781. [DOI] [PubMed] [Google Scholar]
- Swanson JW, Swartz MS, Elbogen EB, Van Dorn RA, Wagner HR, Moser LA, et al. Psychiatric advance directives and reduction of coercive crisis interventions. Journal of Mental Health. 2008;17(3):255–267. doi: 10.1080/09638230802052195. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Swanson JW, Van Dorn RA, Swartz MS. Effectiveness of atypical antipsychotics for substance use in schizophrenia patients. Schizophrenia Research. 2007 doi: 10.1016/j.schres.2007.05.007. [DOI] [PubMed] [Google Scholar]
- Swartz JA, Lurigio AJ. Serious Mental Illness and Arrest: The Generalized Mediating Effect of Substance Use. Crime & Delinquency. 2007;53(4):581–604. [Google Scholar]
- Swartz MS, Wilder CM, Swanson JW, Van Dorn RA, Robbins PC, Steadman HJ, et al. Assessing outcomes for service recipients under New York’s Assisted Outpatient Treatment Program. Psychiatric Services. doi: 10.1176/ps.2010.61.10.976. (Under Review) [DOI] [PubMed] [Google Scholar]
- Tubman JG, Gil AG, Wagner EF, Artigues H. Patterns of Sexual Risk Behaviors and Psychiatric Disorders in a Community Sample of Young Adults. Journal of Behavioral Medicine. 2003;26(5):473–500. doi: 10.1023/a:1025776102574. [DOI] [PubMed] [Google Scholar]
- Van Dorn RA, Williams JH, Del-Colle M, Hawkins JD. The co-occurrence of substance use, mental illness and violence among young adults. Journal of Behavioral Health Services & Research. 2009;36(4):465–477. doi: 10.1007/s11414-008-9159-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Westen D, Novotny C, Thompson-Brenner H. The empirical status of empirically supported therapies: Assumptions, methods, and findings. Psychological Bulletin. 2004;130:631–663. doi: 10.1037/0033-2909.130.4.631. [DOI] [PubMed] [Google Scholar]
- Westen DI. Are research patients and clinical trials representative of clinical practice? In: Norcross JC, Beutler LE, Levant RF, editors. Evidence-based practices in mental health: Debate and dialogue on the fundemental questions. Washington, DC: American Psychological Association; 2005. pp. 161–189. [Google Scholar]
- Wittchen HU, Nelson CB, Lachner G. Prevalence of mental disorders and psychosocial impairments in adolescents and young adults. Psychological Medicine. 1998;28:109–126. doi: 10.1017/s0033291797005928. [DOI] [PubMed] [Google Scholar]