Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2014 Dec 30.
Published in final edited form as: Psychiatry Res. 2013 Sep 5;210(3):684–689. doi: 10.1016/j.psychres.2013.08.008

Measuring participation in an evidence-based practice: Illness management and recovery group attendance

Alan B McGuire a,b,*, Kelsey A Bonfils b, Marina Kukla a, Laura Myers a, Michelle P Salyers b
PMCID: PMC3971633  NIHMSID: NIHMS523055  PMID: 24011850

Abstract

Given the important role of treatment attendance as an indicator of program implementation and as a potential moderator of program effectiveness, this study sought to develop useful indicators of attendance for evidence-based practices. The current study examined consumer attendance patterns in a randomized controlled trial comparing illness management and recovery (n = 60) to a problem solving control condition (n = 58). Associations were examined between consumer clinical indicators, demographics, and level of recovery and indices of attendance. Attendance was poor, but comparable to rates found in many other studies. Four indicators of attendance (percent sessions attended, time enrolled, periods of attendance, and longest period of attendance) were highly inter-related and were more sensitive to baseline differences than a traditional approach of dichotomizing participants into “attenders” and “non-attenders.” Older age, lower hostility, fewer psychotic symptoms, and more education were associated with higher group attendance in both treatment conditions; the client-reported illness management and recovery scale was associated with attendance in the control group. Indicators of attendance were an advancement over dichotomous classification. Strategies to increase attendance are still needed, particularly for younger consumers with greater positive symptoms.

Keywords: Illness management and recovery, Attendance, Schizophrenia, Service intensity, Implementation

1. Introduction

Research and practice emphasize implementation of evidence-based practices (EBPs) in mental health care for consumers with severe mental illness (SMI); consumer attendance is a key indicator of feasibility of a practice (Proctor et al., 2011), or the extent to which a practice can be successfully carried out within a given setting (Karsh, 2004). If a program can be established, but consumers do not attend services or drop-out early, the program is of little use to consumers. To this end, consumers who terminate services early demonstrate worse outcomes than treatment completers (Pekarik, 1983, 1985a; Masi et al., 2003). Additionally, missed sessions drain system resources and may exact an emotional toll on treatment providers (Salmoiraghi and Sambhi, 2010).

Substantial research has explored predictors of attendance, including four systematic reviews (Chen, 1991; Wierzbicki and Pekarik, 1993; Nose et al., 2003; O'Brien et al., 2009), which have produced confusing and inconsistent results. There is some consensus, with most authors agreeing that greater dropout is associated with younger age, minority race, and substance abuse. But individual reviews have concluded factors such as insight (O'Brien et al., 2009), social functioning (Nose et al., 2003), and education and income (Wierzbicki and Pekarik, 1993) are “consistently” associated with attendance, without confirmation from other reviews. At the same time, many theoretically relevant constructs are currently neglected in the empirical literature (e.g., stigma (Corrigan and Rüsch, 2002), although see Fung et al. (2008) for an excellent counterexample).

The confusing and inconclusive literature has led many scholars (Baekeland and Lundwall, 1975; Wierzbicki and Pekarik, 1993; Nose et al., 2003; O'Brien et al., 2009) to conclude that methodological issues are a major issue. In particular, “definitional inexplicitness has been the hallmark of most studies of dropping out of treatment” (Baekeland and Lundwall, 1975, p. 740). For instance, participants are often dichotomized arbitrarily (Sue et al., 1976; Beard et al., 1978; Cohen et al., 1995; King and Canada, 2004; Latimer et al., 2006; Harding et al., 2008). In coding participants to one of only two attendance groups, untold variance in attendance patterns is lost, thus limiting both statistical power and the clinical utility of the results. Further, there may be different aspects of attendance that are important. One study found that variables correlated with attendance could manifest differing relationships when the attendance outcome was operationalized differently (Pekarik, 1985b).

Another potential explanation for the variation in predictors of attendance is the heterogeneity of mental health services studied. A dearth of literature has focused on predictors of attendance in EBPs for people with SMI. To demonstrate in three sample EBPs, only four studies have examined predictors of attendance in participants with SMI: illness management and recovery (two studies) (Bullock et al., 2007; Salyers et al., 2011), cognitive-behavioral therapy for psychosis (one study; Tarrier et al., 1998), and supported employment (one study; Harding et al., 2008). Results from these studies are not conclusive regarding predictors of attendance. In a recent study examining hospital and emergency room use in illness management and recovery participants, fewer hospitalizations, older age, absence of substance abuse, more education, white race, and being single were associated with lower drop-out rates (Salyers et al., 2011). However, Bullock et al. (2007) found no significant predictors of illness management and recovery attendance, (although this may have been due to lack of statistical power). Tarrier et al. (1998) found dropouts from cognitive-behavioral therapy for psychosis had lower premorbid IQ and a trend toward higher baseline depression. In a study comparing two vocational programs, drop-outs (within 6-months) were likely to be more educated, never married, have a schizophrenia-spectrum diagnosis, and be assigned to the stepwise program (as opposed to the supported employment condition) (Harding et al., 2008). Some studies of other, non-SMI populations have found equally inconclusive results (c.f., Eskildsen et al., 2010).

We sought to further the literature by describing attendance within the context of a randomized-controlled trial of an EBP – illness management and recovery (Mueser et al., 2002; Hasson-Ohayon et al., 2007; Levitt et al., 2009; Färdig et al., 2011) using more nuanced indicators of attendance. Illness management and recovery is a 10-month curriculum designed to help consumers with SMI develop goals and be more active and informed about their mental health treatment. We contend that illness management and recovery provides a reasonable example of a clinically complex, recovery-oriented EBP and therefore results may be generalizable to other important practices. We sought to explore the utility of various indicators of attendance by examining sensitivity to differences in consumer characteristics. These indicators are compared to the conventional standard of dichotomization by “attenders” versus “non-attenders.” Moreover, the inclusion of a control condition allowed us to compare differential attendance and predictors between participants assigned to a recovery-oriented EBP and a non-specific service.

2. Methods

2.1. Sample

Participants were enrolled in a randomized-controlled trial comparing illness management and recovery to an active control condition (unstructured problem solving groups). Participants were recruited from a VA medical center and a community mental health center (CMHC) serving consumers with SMI in an urban area. Inclusion criteria were: receiving services at either location, age ≥ 18, diagnosed with schizophrenia or schizoaffective disorder confirmed by the psychotic modules of the Structured Clinical Interview for DSM-IV (First et al., 1996), no severe cognitive dysfunction according to a cognitive screen (Callahan et al., 2002), and no physical health condition that would limit participation in an 18-month study. Participants were randomized into treatment condition (illness management and recovery vs. problem solving control). Participants from both groups are included in these analyses.

Baseline characteristics for the sample are shown in Tables 1 and 2. Participants in the illness management and recovery condition (n = 60) did not differ from the controls (n = 58) on any baseline characteristics. Participants receiving services at the CMHC (M = 18.3, S.D. = 3.68) had more hope at baseline than VA participants (M = 16.6, S.D. = 4.09, t = –2.30, d.f. = 114, p = 0.02) but did not differ on any other measure. Power Analyses: our sample of n = 118 allows us to detect medium effects (effect size ≈ 0.35) at alpha = 0.05, Power = 0.80 (Lipsey, 1990).

Table 1.

Baseline characteristics and association with indicators of participation.

Mean S.D. Median Mode % Available sessions attended Time enrolled Periods of attendance Longest period of attendance > 50% Sessions
Indicators of attendance
    % Available sessions attended 22.5% 27.33% 6.8% 0.0% – 0.88** 0.63* 0.94** 0.88**
    Time enrolled 39.6% 41.69% 18.2% 0% – 0.75** 0.83** 0.71**
    Periods of attendance 0.7 0.82 1.0 0.0 – 0.52* 0.42**
    Longest period of attendance 9.3 12.45 2.5 0.0 – 0.83**
Baseline characteristics
    Age 47.6 8.85 0.20* 0.26** 0.26** 0.16 0.10
    Hope 17.6 3.94 –0.00 –0.06 –0.06 –0.03 0.01
    IMRS-client 3.4 0.52 0.17 0.12 0.06 0.15 0.18
    IMRS-clinician 3.3 0.55 0.23* 0.26* 0.21* 0.16 0.18
    RAS 124.4 15.69 0.04 0.02 0.05 0.02 0.07
    MARS 3.2 2.34 –0.16 –0.13 –0.13 –0.17 –0.15
    Patient activation 37.7 6.29 –0.07 –0.11 –0.09 –0.10 0.01
    PANSS-total 75.6 15.66 –0.20* –0.10 –0.14 –0.15 –0.18
    PANSS– + 15.8 4.90 –0.19* –0.09 –0.13 –0.14 –0.24**
    PANSS– – 19.1 5.55 –0.12 0.07 –0.01 0.00 0.01
    PANSS-emotional 12.6 4.61 –0.17 –0.11 –0.06 –0.12 –0.17
    PANSS-cognitive 17.2 5.54 –0.05 0.02 –0.05 0.01 –0.01
    PANSS-hostility 8.6 3.19 –0.23* –0.27** –0.26** –0.25** –0.15
    Alcohol use 1.7 0.97 –0.21 –0.24* –0.18 –0.19 –0.20
    Drug use 1.5 0.87 –0.08 –0.12 –0.14 –0.07 –0.05
Open in a new tab

Notes: IMRS-Client and IMRS-Clinician are the Illness management and recovery scales – Client and Clinician version, RAS = recovery assessment scale, MARS = medication adherence rating scale; PANSS = positive and negative syndrome scale.

*

p < 0.05

**

p < 0.01

Table 2.

Point-biserial correlations between categorical characteristics and indicators of attendance.

N (%) % Available sessions attended Time enrolled Periods of attendance Longest period of attendance
Female 24 (20.3%) –0.12 –0.14 –0.08 –0.12
White 40 (33.9%) 0.17 0.12 0.10 0.14
Married 16 (13.9%) –0.10 –0.09 –0.11 –0.07
Independent housing 82 (71.3%) 0.16 0.11 0.10 0.12
≥High school/GED 76 (65.5%) 0.18 0.19* 0.27** 0.14
≥ Some college 37 (31.9%) 0.10 0.07 0.28** 0.02
Formal employment 9 (7.8%) 0.06 0.07 0.02 0.06
VA patient 52 (44.1%) 0.00 0.07 0.10 –0.02
Income > $10,000 36 (32.7%) 0.10 0.13 0.24* 0.03
Open in a new tab
*

p < 0.05

**

p < 0.01

2.2. Measures

2.2.1. Attendance

Participants signed in at each session; participants’ attendance was confirmed via medical records and session audio-recordings when needed. Attendance was measured in several ways. Percent sessions attended: because the number of sessions available varied for each consumer's study phase (due to length of months and holiday cancellations), we used the number of sessions attended divided by the number of sessions occurring during their active treatment phase. This indicator is roughly analogous to “dose” of the intervention received. Time enrolled (reported as a percent of total available sessions) is the time from the participant's first eligible session until the last session the participant attended, regardless of whether a participant missed sessions between the first and last. For instance, the last session Consumer 2 (Fig. 1) attended was session seven; therefore, this consumer's time enrolled would be 7/13 sessions = 54%. Periods of attendance were operationally defined as starting when a participant attended at least two sessions (the minimum number that can indicate a pattern of attendance) in a four-session period and ending when the participant missed at least four consecutive sessions. These cut-points roughly correspond to the length of one module in illness management and recovery. In Fig. 1, Consumers 1 and 2 both had one distinct period of attendance, while Consumer 3 had two. We also calculated the longest period of attendance, which was considered meaningful because continuous exposure could be more potent than exposure to an equal number of sessions spread over a large period of time. Finally, for a typical indicator of attendance, we calculated whether participants attended at least 50% of sessions (Corrigan, 1995; Okpaku et al., 1997).

Fig. 1.

Fig. 1

Open in a new tab

Example attendance patterns. Notes: shaded squares represent sessions attended.

2.2.2. Demographics included age, race, gender, marital status, educational level, and housing status

2.2.2.1. Clinical consumer characteristics

Symptoms were measured using the positive and negative syndrome scale (PANSS; Kay et al., 1987; a semi-structured interview with previously demonstrated satisfactory alpha levels (between 0.73 and 0.83) and good reliability and validity (Peralta and Cuesta, 1994). We used the five-factor (positive, negative, cognitive, hostility, and emotional) scoring model (Bell et al., 1994).

The illness management and recovery scales (IMRS) were developed as outcome measures for illness management and recovery (Mueser and Gingerich, 2005). There are parallel client and clinician versions, each containing 15 items. The possible score on each item ranges from 1 to 5, with higher scores indicating better outcomes. The IMRS have shown adequate internal consistency (Cronbach's alpha > = 0.71), strong test-retest correlations over a two-week period (both versions, r = 0.81, p < 0.001), and correlations with other indices of functioning, symptoms, and recovery (Salyers et al., 2007; Hasson-Ohayon et al., 2008; Fardig et al., 2011).

The alcohol use scale-revised (AUR) and drug use scale-revised (DUS-R; Drake et al., 1990) are 5-point rating scales assessing use within the past 6 months. Both instruments have been shown to have high inter-rater reliability (kappa coefficient between 0.80 and 0.95) and the AUS-R has demonstrated high sensitivity (95%) and specificity (100%; Drake et al., 1996b).

The medication adherence rating scale (MARS) is a 10-item scale scored from 0 to 10, with lower scores indicating better adherence. The MARS has been shown to have adequate internal consistency (Cronbach's alpha = 0.75), test-retest reliability (ICC = 0.72), and construct validity, correlating with the Medication Adherence Questionnaire (r = 0.79) and the Drug Attitude Inventory (r = 0.82; Thompson et al., 2000).

The Patient Activation Measure (Hibbard et al., 2010) assesses patient's knowledge, skill, and confidence health self-management. We used the 13-item mental health version (PAM-MH; Green et al., 2010), which focuses on the management of mental illness. The PAM-MH has a 0–100 theoretical scale with higher scores indicating better patient activation. The measure has been found to have comparable reliability (person-item reliability = 0.84 and item-reliability = 0.97) and construct validity as the original (Hibbard et al., 2004).

The recovery assessment scale (RAS; Corrigan et al., 1999) is a 41-item scale designed to assess perceptions of recovery for people with SMI. RAS total has shown good test-retest reliability, internal consistency (alpha = 0.93), and correlations with measures of self-esteem, empowerment, and quality of life.

2.2.2.2. Treatment model integrity

Was confirmed using the illness management and recovery treatment integrity scale (IT-IS; McGuire et al., 2012). IT-IS ratings are made based on audio-recordings of a session. It a 16-item scale which has demonstrated good internal consistency (alpha = 0.90), inter-rater reliability, and the ability to distinguish between illness management and recovery and control sessions.

2.3. Procedures

Participants were randomized to illness management and recovery or problem solving and asked to attend their assigned group for a 9-month active treatment phase (which was followed by a 9-month non-active phase in which they did not receive study interventions). Baseline interviews were conducted by trained research staff. Treatment groups were facilitated by mental health professionals, including a master's-level social worker and doctoral-level psychologists (authors #1 & #3) who had previous experience providing illness management and recovery. Groups were co-facilitated by clinical psychology graduate students with a range of clinical experience.

Problem solving groups were unstructured process groups. Participants were encouraged to use the time to discuss problems and lend mutual support. Group leaders were prohibited from using illness management and recovery materials or techniques. The same group leaders facilitated the experimental and control conditions.

2.4. Analyses

We first calculated descriptive statistics for each indicator of attendance. We compared illness management and recovery and problem solving groups on each indicator of attendance using independent samples t-tests. We examined interrelationships between indicators of attendance by first calculating Pearson's correlations. We explored the relationship between baseline consumer characteristics (demographics, baseline symptoms, and recovery-related variables) and indicators of attendance using Person's r (for continuous variables) and point-biserial correlations for dichotomous variables. Zero-order correlations were used because of the exploratory nature of identifying which characteristics may be related to each indicator of attendance.

3. Results

3.1. Group attendance patterns

The number of possible sessions (for all participants) available for attendance ranged from 38 to 42. On average, participants attended less than a quarter of available sessions and stopped prior to half of potential sessions that had been offered; see Table 1 for descriptive statistics of indicators of attendance. Twenty-seven (22.9%) participants attended at least 50% of sessions. In terms of discrete periods of attendance, most participants (56; 47.5%) had no periods of attendance (i.e., did not attend ≥ 2 sessions), followed by one period of attendance (n = 45, 38.1%), two periods (n = 12, 10.2%), and three periods (n= 5, 4.2%). Participants’ longest period of attendance averaged about nine sessions; considering only participants with at least one period of attendance (n = 62), participants’ longest period of attendance was about eighteen sessions (M = 17.8; median = 16.5; S.D. = 12.04). Taken together, the average participant did not have any periods of active attendance. For participants who did have active periods of attendance, it was generally just one period that lasted about eighteen sessions.

Participants in illness management and recovery attended more sessions (M = 47.0%, S.D. = 41.60) than controls (M = 31.9%, S.D. = 41.60; t(116) = –2.0, p = 0.05), but did not differ on any other measure of attendance. Examining session-by-session attendance patterns (Fig. 2), both conditions experienced sharp declines in the number of participants during the first month of treatment, followed by slow decline for the duration of the study period.

Fig. 2.

Fig. 2

Open in a new tab

Attendance of illness management and recovery and problem solving groups. Notes: moving averages include the average of the target session and three previous sessions.

3.2. Inter-relation of indicators of attendance

Indicators of attendance were strongly correlated (Table 1). However, the correlation amongst percentage of sessions attended, time enrolled, and longest period of attendance (r ranges from 0.83 to 0.94) were stronger than the correlations between the number of periods of attendance and the other indicators (r ranges from 0.52 to 0.75).

Finally, we explored the relationship between a conventional indicator of attendance – at least 50% sessions attended – with our indicators of attendance. The dichotomization was strongly correlated with the three inter-correlated indicators of attendance above (r = 0.70–0.87) and moderately correlated with the number of periods of attendance (r = 0.42).

3.3. Prediction of attendance

Associations between participant characteristics at baseline and attendance indicators are shown in Tables 1 and 2. Three variables were consistently associated with indicators of attendance: age, IMR Scale-Clinician version, and hostility. Higher hostility was associated with significantly poorer attendance on all indicators. Older age and higher scores on the IMRS-Clinician version were associated with significantly better attendance on all indicators except longest period of attendance. In addition, less severe overall symptoms and positive symptoms, higher educational levels, and absence of alcohol abuse were associated with significantly better attendance on at least one of the four indicators of attendance. The dichotomized attendance measure was significantly correlated to only one consumer baseline characteristic – positive symptoms (r = –0.24, p < 0.01).

We also examined the association between participant characteristics and indicators of attendance within illness management and recovery and problem solving conditions. Results were similar for all predictors except education and both IMRSs. Although more education was associated with significantly better attendance in the illness management and recovery condition (as it was in the full sample), no such relationship was observed in the problem solving condition. There was no association between baseline IMRS-Client version and indicators of attendance in illness management and recovery (as it was in the full sample). However, in the problem-solving condition, the IMRS-Client version was associated with percent of sessions attended (r = 0.27, p = 0.04, n = 57) and time enrolled (r = 0.27, p = 0.04, n = 57). The IMRS-Clinician version was significantly associated with attendance in the problem solving condition (like the full sample), but there was no association between the IMRS-Clinician version and indicators of attendance in the illness management and recovery condition.

4. Discussion

Attendance in illness management and recovery and the problem solving control condition varied widely, but was poor overall. The majority of participants had little treatment exposure – median percentage of attendance was 7% of sessions and median time enrolled was only 18% of the active period. This is consistent with drop-out generally occurring early in the treatment process (Bados et al., 2007). It is notable that illness management and recovery participants attended more sessions and dropped-out later than controls.

The level of attendance in the current study is consistent with attendance found in some other studies of EBPs for individuals with SMI (e.g., cognitive-behavioral therapy; Tarrier et al., 1998). In the absence of outreach efforts, attendance appears to be a challenge for many EBPs for people with SMI and merits further attention and active steps to ameliorate this problem. Potential solutions include additional supports like assertive outreach (e.g., Carroll et al., 2006; Smelson et al., 2010) and tailoring of services to increase consumer acceptability. For example, wellness management and recovery, a program adapted from illness management and recovery based on agency and consumer feedback, is 10 sessions long (Bullock et al., 2009). Notably, we found that the average longest period of attendance was nine sessions–this length of attendance may speak to the feasible length of interventions for this population.

We developed several indicators of attendance representing potential gauges of a consumer's attendance in the intervention. These indicators were highly inter-related and no one indicator appeared to better respond to differences in baseline characteristics than other indicators. Number of periods of attendance correlated more weakly with the other indicators; these lower correlations are expected as a low number of periods of attendance could indicate either one long period of attendance in which the participant attended frequently or one short period of attendance in which the participant attended only four sessions. Our attendance indicators were more sensitive than the traditional categorization of “attenders” and “non-attenders.” Although the traditional indicator of attendance of 50% was correlated with the other indicators and may roughly measure attendance, the restriction of variance masks potentially meaningful associations with baseline consumer characteristics. For example, we found the dichotomized indicator only correlated with one baseline characteristic whereas the more nuanced indicators were related to several. Moreover, the data used to create the dichotomy (i.e., the percent of sessions attended) is a more powerful indicator of attendance in itself.

The successful prediction of consumer attendance is valuable to service delivery systems. Significant resources are invested in training staff in EBPs; missed appointments represent a significant missed opportunity (Burgoyne et al., 1983; Matas et al., 1992; VA Office of Inspector General, 2008). Moreover, consumer characteristics associated with EBP attendance may elucidate necessary adaptations for certain populations. For example, educational level was associated with attendance in illness management and recovery – a curriculum-based intervention with a workbook and homework. Consumers with less formal education may feel uncomfortable or struggle with the program. Future research could develop adaptations to the program to make it more acceptable to participants with lower education.

Other predictors of attendance were equally impactful in both groups, perhaps reflecting general predictors of attendance in group therapy. Previous literature frequently has found older consumers (Sue et al., 1976; Young et al., 2000; Coodin et al., 2004; Bados et al., 2007; Salyers et al., 2011) and those with less severe symptoms (Young et al., 2000; Bados et al., 2007) to be more likely to participate in treatment. Younger, more severely disabled populations represent a general challenge to mental health delivery systems. Furthermore, ample literature has displayed the importance of community-based services with assertive outreach for consumers with more severe symptoms (Burns et al., 2007).

The current study had several limitations. The study was exploratory and descriptive, and no causal inferences should be drawn from our results. Predictor variables were those available through the larger RCT and not selected on theoretical grounds for attendance. In addition, the strengths of identified predictors were relatively weak. Subsequent work aimed at examining this complex relationship will require more advanced statistical and theoretical underpinnings; as stated by O'Brien et al. (1972), the “associations [between baseline variables and attendance] are complex and multifaceted” (p. 558). Relevant theories of human behavior (e.g., Goal Setting Theory (Latham and Locke, 2007) or Social Cognitive Learning Theory (Bandura, 1991)) would provide meaningful predictors of attendance (e.g., self-efficacy, importance, and difficulty). Moreover, there are important predictors that may be more difficult to quantify (e.g., group member interactions). Finally, we were unable to take into consideration factors unrelated to the consumer that might also impact attendance. Future research should investigate non-consumer factors in addition to theoretically relevant consumer variables.

The indicators of attendance in the current study represent our best attempt at creating meaningful gauges of attendance. These indicators, although promising, will require further validation in future research. Additionally, we conducted this study in the context of a controlled research trial; therefore, consumer attendance may not be representative of that of consumers in routine treatment. However, this limitation is attenuated by at least three factors. First, although not integrated with consumers’ regular services, the study interventions took place in community clinics already providing services to the consumers (rather than a research-specific setting). Though we may have lost opportunities to increase attendance had the interventions been integrated, our study has enhanced generalizability to service systems which continue to provide brokered and non-integrated services. Second, although participants received incentives for completing study interviews, as in most settings, participants were not incentivized for attending treatment groups. Finally, unlike many RCTs of psychosocial interventions, this study did not require potential participants to attend multiple informational sessions in order to participate (e.g., Drake et al., 1996a, 1999; Bond et al., 2007; Harding et al., 2008). Such requirements artificially select for participants who may be more committed to attending and less likely to dropout. Nonetheless, future studies of consumer attendance in EBPs should take into account attendance of other services in which they are enrolled.

In summary, we developed several potential indicators of attendance in evidence-based psychosocial interventions. These indicators were more sensitive than the traditional dichotomous measure of attendance. However, no one indicator rises above the others and they are highly correlated. Age, education, clinician rated IMR scale, and symptoms (particularly hostility) were associated with attendance generally, whereas education was differentially associated with attendance in illness management and recovery. The attendance indicators developed serve as a useful starting point for further research.

Acknowledgment

This research was funded by Illness Management and Recovery for Veterans with Severe Mental Illness. VA HSR&D: IAC 05-254-3. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.

References

  1. Bados A, Balaguer G, Saldaña C. The efficacy of cognitive–behavioral therapy and the problem of drop-out. Journal of Clinical Psychology. 2007;63:585–592. doi: 10.1002/jclp.20368. [DOI] [PubMed] [Google Scholar]
  2. Baekeland F, Lundwall L. Dropping out of treatment: a critical review. Psychological Bulletin. 1975;32:738–783. doi: 10.1037/h0077132. [DOI] [PubMed] [Google Scholar]
  3. Bandura A. Social cognitive theory of self-regulation. Organizational Behavior and Human Decision Processes. 1991;50:248–287. [Google Scholar]
  4. Beard JH, Malamud TJ, Rossman E. Psychiatric rehabilitation and long-term rehospitalization rates: the findings of two research studies. Schizophrenia Bulletin. 1978;4:622–635. doi: 10.1093/schbul/4.4.622. [DOI] [PubMed] [Google Scholar]
  5. Bell MD, Lysaker PH, Milstein RM, Beam-Goulet JL. Concurrent validity of the cognitive component of schizophrenia: relationship of PANSS scores to neuropsychological assessments. Psychiatry Research. 1994;54:51–58. doi: 10.1016/0165-1781(94)90064-7. [DOI] [PubMed] [Google Scholar]
  6. Bond GR, Salyers MP, Dincin J, Drake R, Becker DR, Fraser VV, Haines M. A randomized controlled trial comparing two vocational models for persons with severe mental illness. Journal of Consulting and Clinical Psychology. 2007;75:968–982. doi: 10.1037/0022-006X.75.6.968. [DOI] [PubMed] [Google Scholar]
  7. Bullock WA, O'Rourke M, Breedlove A, Farrer E, Smith MK. Effectiveness of the illness management and recovery program in promoting recovery: preliminary results. New Research in Mental Health. 2007;17:282–291. [Google Scholar]
  8. Bullock WA, Sage J, Hupp D, Ozbey T, O'Rourke M, Smith MK, Wesp K, Wilcox MSD, Rich PDS, Pieples LSWG. From illness to wellness: an evaluation of Ohio's Wellness Management and Recovery (WMR) program in community health and consumer-operated service agencies. New Research in Mental Health. 2009:312–321. [Google Scholar]
  9. Burgoyne RW, Acosta FX, Yamamoto J. Telephone prompting to increase attendance at a psychiatric outpatient clinic. American Journal of Psychiatry. 1983;140:345–347. doi: 10.1176/ajp.140.3.345. [DOI] [PubMed] [Google Scholar]
  10. Burns T, Catty J, Dash M, Roberts C, Lockwood A, Marshall M. Use of intensive case management to reduce time in hospital in people with severe mental illness: systematic review and meta-regression. BMJ. 2007;335:336–342. doi: 10.1136/bmj.39251.599259.55. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Callahan CM, Unverzagt FW, Hui SL, Perkins AJ, Hendrie HC. Six-item screener to identify cognitive impairment among potential subjects for clinical research. Medical Care. 2002;40:771–781. doi: 10.1097/00005650-200209000-00007. [DOI] [PubMed] [Google Scholar]
  12. Carroll KM, Ball SA, Nich C, Martino S, Frankforter TL, Farentinos C, Kunkel LE, Mikulich-Gilbertson SK, Morgenstern J, Obert JL. Motivational interviewing to improve treatment engagement and outcome in individuals seeking treatment for substance abuse: a multisite effectiveness study. Drug and Alcohol Dependence. 2006;81:301–312. doi: 10.1016/j.drugalcdep.2005.08.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Chen A. Noncompliance in community psychiatry: a review of clinical interventions. Hospital and Community Psychiatry. 1991;42:282–287. doi: 10.1176/ps.42.3.282. [DOI] [PubMed] [Google Scholar]
  14. Cohen K, Edstrom K, Smith-Papke L. Identifying early dropouts from a rehabilitation program for psychiatric outpatients. Psychiatric Services. 1995;46:1076–1078. doi: 10.1176/ps.46.10.1076. [DOI] [PubMed] [Google Scholar]
  15. Coodin S, Staley D, Cortens B, Desrochers R, McLandress S. Patient factors associated with missed appointments in persons with schizophrenia. Canadian Journal of Psychiatry. 2004;49:145–148. doi: 10.1177/070674370404900210. [DOI] [PubMed] [Google Scholar]
  16. Corrigan PW. Correlates of participation and completion in a job club for clients with psychiatric disability. Rehabilitation Counseling Bulletin. 1995;39:42–53. [Google Scholar]
  17. Corrigan PW, Rüsch N. Mental illness stereotypes and clinical care: do people avoid treatment because of stigma? Psychiatric Rehabilitation Skills. 2002;6:312–334. [Google Scholar]
  18. Corrigan PW, Giffort D, Rashid F, Leary M, Okeke I. Recovery as a psychological construct. Community Mental Health Journal. 1999;35:231–239. doi: 10.1023/a:1018741302682. [DOI] [PubMed] [Google Scholar]
  19. Drake RE, Osher FC, Noordsy DL, Hurlbut SC, Teague GB, Beaudett MS. Diagnosis of alcohol use disorders in schizophrenia. Schizophrenia Bulletin. 1990;16:57–67. doi: 10.1093/schbul/16.1.57. [DOI] [PubMed] [Google Scholar]
  20. Drake RE, McHugo GJ, Becker DR, Anthony WA, Clark RE. The New Hampshire study of supported employment for people with severe mental illness. Journal of Consulting and Clinical Psychology. 1996a;64:391–399. doi: 10.1037//0022-006x.64.2.391. [DOI] [PubMed] [Google Scholar]
  21. Drake RE, Mueser KT, McHugo GJ. Clinician Rating Scales: Alcohol Use Scale (AUS), Drug Use Scale (DUS), and Substance Abuse Treatment Scale (SATS) Williams & Wilkins; Baltimore: 1996b. [Google Scholar]
  22. Drake RE, McHugo GJ, Bebout RR, Becker DR, Harris M, Bond GR, Quimby E. A randomized clinical trial of supported employment for inner-city patients with severe mental disorders. Archives of General Psychiatry. 1999;56:627–633. doi: 10.1001/archpsyc.56.7.627. [DOI] [PubMed] [Google Scholar]
  23. Eskildsen A, Hougaard E, Rosenberg NK. Pre-treatment patient variables as predictors of drop-out and treatment outcome in cognitive behavioural therapy for social phobia: a systematic review. Nordic Journal of Psychiatry. 2010;64:94–105. doi: 10.3109/08039480903426929. [DOI] [PubMed] [Google Scholar]
  24. Fardig R, Lewander T, Fredriksson A, Melin L. Evaluation of the Illness Management and Recovery Scale in schizophrenia and schizoaffective disorder. Schizophrenia Research. 2011;132:157–164. doi: 10.1016/j.schres.2011.07.001. [DOI] [PubMed] [Google Scholar]
  25. Färdig R, Lewander T, Melin L, Folke F, Fredriksson A. A randomized controlled trial of the illness management and recovery program for persons with schizophrenia. Psychiatric Services. 2011;62:606–612. doi: 10.1176/ps.62.6.pss6206_0606. [DOI] [PubMed] [Google Scholar]
  26. First MB, Spitzer RL, Gibbon M, Williams JB. Structured Clinical Interview for Axis I DSM-IV. Biometric Research Department. New York State Psychiatric Institute; New York: 1996. [Google Scholar]
  27. Fung KMT, Tsang HWH, Corrigan PW. Self-stigma of people with schizophrenia as predictor of their adherence to psychosocial treatment. Psychiatric Rehabilitation Journal. 2008;32:95–104. doi: 10.2975/32.2.2008.95.104. [DOI] [PubMed] [Google Scholar]
  28. Green C, Perrin N, Polen M, Leo M, Hibbard J, Tusler M. Development of the patient activation measure for mental health. Administration and Policy in Mental Health and Mental Health Services Research. 2010;37:327–333. doi: 10.1007/s10488-009-0239-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Harding B, Torres-Harding S, Bond GR, Salyers MP, Rollins AL, Hardin T. Factors associated with early attrition from psychosocial rehabilitation programs. Community Mental Health Journal. 2008;44:283–288. doi: 10.1007/s10597-008-9128-9. [DOI] [PubMed] [Google Scholar]
  30. Hasson-Ohayon I, Roe D, Kravetz S. A randomized controlled trial of the effectiveness of the illness management and recovery program. Psychiatric Services. 2007;58:1461–1466. doi: 10.1176/ps.2007.58.11.1461. [DOI] [PubMed] [Google Scholar]
  31. Hasson-Ohayon I, Roe D, Kravetz S. The psychometric properties of the Illness Management and Recovery scale: client and clinician versions. Psychiatry Research. 2008;160:228–235. doi: 10.1016/j.psychres.2007.06.013. [DOI] [PubMed] [Google Scholar]
  32. Hibbard JH, Stockard J, Mahoney ER, Tusler M. Development of the Patient Activation Measure (PAM): conceptualizing and measuring activation in patients and consumers. Health Services Research. 2004;39:1005–1026. doi: 10.1111/j.1475-6773.2004.00269.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Hibbard JH, Collins PA, Mahoney ER, Baker LH. The development and testing of a measure assessing clinician beliefs about patient self-management. Health Expectations: An International Journal of Public Participation in Health Care and Health Policy. 2010;13:65–72. doi: 10.1111/j.1369-7625.2009.00571.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Karsh BT. Beyond usability: designing effective technology implementation systems to promote patient safety. Quality and Safety in Health Care. 2004;13:388–394. doi: 10.1136/qshc.2004.010322. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Kay S, Fiszbein A, Opler L. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin. 1987;13:261–276. doi: 10.1093/schbul/13.2.261. [DOI] [PubMed] [Google Scholar]
  36. King AC, Canada SA. Client-related predictors of early treatment drop-out in a substance abuse clinic exclusively employing individual therapy. Journal of Substance Abuse Treatment. 2004;26:189–195. doi: 10.1016/S0740-5472(03)00210-1. [DOI] [PubMed] [Google Scholar]
  37. Latham GP, Locke EA. New developments in and directions for goal-setting research. European Psychologist. 2007;12:290–300. [Google Scholar]
  38. Latimer EA, Lecomte T, Becker DR, Drake RE, Duclos I, Piat M, Lahaie N, St-Pierre MS, Therrien C, Xie H. Generalisability of the individual placement and support model of supported employment: results of a Canadian randomised controlled trial. The British Journal of Psychiatry. 2006;189:65–73. doi: 10.1192/bjp.bp.105.012641. [DOI] [PubMed] [Google Scholar]
  39. Levitt AJ, Mueser KT, DeGenova J, Lorenzo J, Bradford-Watt D, Barbosa A, Karlin M, Chernick M. Randomized controlled trial of illness management and recovery in multiple-unit supportive housing. Psychiatric Services. 2009;60:1629–1636. doi: 10.1176/ps.2009.60.12.1629. [DOI] [PubMed] [Google Scholar]
  40. Lipsey MW. Design Sensitivity. Sage; Newbury Park, CA: 1990. [Google Scholar]
  41. Masi MV, Miller RB, Olson MM. Differences in dropout rates among individual, couple, and family therapy clients. Contemporary Family Therapy. 2003;25:63–75. [Google Scholar]
  42. Matas M, Staley D, Griffin W. A profile of the noncompliant patient: a thirty-month review of outpatient psychiatry referrals. General Hospital Psychiatry. 1992;14:124–130. doi: 10.1016/0163-8343(92)90037-b. [DOI] [PubMed] [Google Scholar]
  43. McGuire AB, Stull LG, Mueser KT, Santos M, Mook A, Rose N, Tunze C, White LM, Salyers MP. Development and reliability of a measure of clinician competence in providing illness management and recovery. Psychiatric Services. 2012;63:772–778. doi: 10.1176/appi.ps.201100144. [DOI] [PubMed] [Google Scholar]
  44. Mueser KT, Gingerich S. Illness Management and Recovery (IMR) scales. In: Campbell-Orde T, Chamberlin J, Carpenter J, Leff HS, editors. Measuring the Promise: A Compendium of Recovery Measures. Evaluation Center @ Human Services Research Institute; Cambridge, MA: 2005. pp. 124–132. [Google Scholar]
  45. Mueser KT, Corrigan PW, Hilton DW, Tanzman B, Schaub A, Gingerich S, Copeland ME, Essock SM, Tarrier N, Morey B, Vogel-Scibilia S, Herz MI. Illness management and recovery: a review of the research. Psychiatric Services. 2002;53:1272–1284. doi: 10.1176/appi.ps.53.10.1272. [DOI] [PubMed] [Google Scholar]
  46. Nose M, Barbui C, Tansella M. How often do patients with psychosis fail to adhere to treatment programmes? a systematic review. Psychological Medicine. 2003;33:1149–1160. doi: 10.1017/s0033291703008328. [DOI] [PubMed] [Google Scholar]
  47. O'Brien A, Fahmy R, Singh SP. Disengagement from mental health services: a literature review. Social Psychiatry and Psychiatric Epidemiology. 2009;44:558–568. doi: 10.1007/s00127-008-0476-0. [DOI] [PubMed] [Google Scholar]
  48. O'Brien CP, Hamm KB, Ray BA, Pierce JF, Luborskey L, Mintz J. Group vs. individual psychotherapy with schizophrenics: a controlled outcome study. Archives of General Psychiatry. 1972;27:474–478. doi: 10.1001/archpsyc.1972.01750280042008. [DOI] [PubMed] [Google Scholar]
  49. Okpaku SO, Anderson KH, Sibulkin AE, Butler JS, Bickman L. The effectiveness of a multidisciplinary case management intervention on the employment of SSDI applicants and beneficiaries. Psychiatric Rehabilitation Journal. 1997;20:34–41. [Google Scholar]
  50. Pekarik G. Improvement in clients who have given different reasons for dropping out of treatment. Journal of Clinical Psychology. 1983;39:909–913. doi: 10.1002/1097-4679(198311)39:6<909::aid-jclp2270390614>3.0.co;2-4. [DOI] [PubMed] [Google Scholar]
  51. Pekarik G. Coping with dropouts. Professional Psychology: Research and Practice. 1985a;16:114. [Google Scholar]
  52. Pekarik G. The effects of employing different termination classification criteria in dropout research. Psychotherapy. 1985b;22:86–91. [Google Scholar]
  53. Peralta V, Cuesta MJ. Psychometric properties of the positive and negative syndrome scale (PANSS) in schizophrenia. Psychiatry Research. 1994;53:31–40. doi: 10.1016/0165-1781(94)90093-0. [DOI] [PubMed] [Google Scholar]
  54. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Administration and Policy in Mental Health and Mental Health Services Research. 2011;38:65–76. doi: 10.1007/s10488-010-0319-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Salmoiraghi A, Sambhi R. Early termination of cognitive-behavioural interventions: literature review. The Psychiatrist. 2010;34:529–532. [Google Scholar]
  56. Salyers MP, Godfrey JL, Mueser KT, Labriola S. Measuring illness management outcomes: a psychometric study of clinician and consumer rating scales for illness self management and recovery. Community Mental Health Journal. 2007;43:459–480. doi: 10.1007/s10597-007-9087-6. [DOI] [PubMed] [Google Scholar]
  57. Salyers MP, Rollins AL, Clendenning D, McGuire AB, Kim E. Impact of illness management and recovery programs on hospital and emergency room use by Medicaid enrollees. Psychiatric Services. 2011;62:509–515. doi: 10.1176/appi.ps.62.5.509. [DOI] [PMC free article] [PubMed] [Google Scholar]
  58. Smelson D, Kalman D, Losonczy MF, Kline A, Sambamoorthi U, Hill LS, Castles-Fonseca K, Ziedonis D. A brief treatment engagement intervention for individuals with co-occurring mental illness and substance use disorders: results of a randomized clinical trial. Community Mental Health Journal. 2010:1–6. doi: 10.1007/s10597-010-9346-9. [DOI] [PubMed] [Google Scholar]
  59. Sue S, McKinney HL, Allen DB. Predictors of the duration of therapy for clients in the community mental health system. Community Mental Health Journal. 1976;12:365–375. doi: 10.1007/BF01411075. [DOI] [PubMed] [Google Scholar]
  60. Tarrier N, Yusupoff L, McCarthy E, Kinney C, Wittkowski A. Some reasons why patients suffering from chronic schizophrenia fail to continue in psychological treatment. Behavioural and Cognitive Psychotherapy. 1998;26:177–181. [Google Scholar]
  61. Thompson K, Kulkarni J, Sergejew AA. Reliability and validity of a new Medication Adherence Rating Scale (MARS) for the psychoses. Schizophrenia Research. 2000;42:241–247. doi: 10.1016/s0920-9964(99)00130-9. [DOI] [PubMed] [Google Scholar]
  62. VA Office of Inspector General . Audit of Veterans Health Administration Efforts to Reduce Unused Outpatient Appointments Report number: 08-00879-36. U.S. Government Printing Office; Washington, D.C.: 2008. [Google Scholar]
  63. Wierzbicki M, Pekarik G. A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice. 1993;24:190–195. [Google Scholar]
  64. Young AS, Forquer SL, Tran A, Starzynski M, Shatkin J. Identifying clinical competencies that support rehabilitation and empowerment in individuals with severe mental illness. Journal of Behavioral Health Services and Research. 2000;27:321–333. doi: 10.1007/BF02291743. [DOI] [PubMed] [Google Scholar]

RESOURCES