Implementing Illness Management and Recovery Within Assertive Community Treatment: A Pilot Trial of Feasibility and Effectiveness

Abstract

Objective:

In this pilot feasibility and effectiveness study, Illness Management and Recovery (IMR) was integrated into Assertive Community Treatment (ACT) to improve recovery and functioning for people with serious mental illness.

Method:

A preliminary, small-scale cluster randomized controlled design tested implementation of IMR within ACT teams in two states. Eight high-fidelity ACT teams were randomly assigned to provide IMR (ACT+IMR; four teams), or standard ACT services (ACT-only; four teams). Clinical outcomes from 101 individuals with schizophrenia-spectrum or bipolar disorders were assessed at baseline, six months, and one year.

Results:

Exposure to IMR (i.e., session attendance, module completion) varied between the ACT+IMR teams, with participants on one team having significantly less exposure. Results from intent-to-treat analyses showed that participants in ACT+IMR demonstrated significantly better outcomes with a medium effect size at follow-up on clinician-rated illness self-management. Although non-significant, a medium effect size was found for one measure of functioning and small effect sizes were observed in client-rated illness self-management, another measure of functioning, and community integration. Secondary analyses showed session and module completion predicted better outcomes on four of the 12-month outcome measures.

Conclusions:

Findings support the feasibility of implementing IMR within ACT teams. Although there were few significant findings, effect sizes on other variables in this small-scale study and the dose-response relationships within ACT+IMR teams suggest this novel approach could be promising for improving recovery for people with serious mental illness. Further large-scale studies utilizing a hybrid effectiveness-implementation design could provide a promising direction in this area.

Despite advances in pharmacological treatment, many individuals with serious mental illness experience significant functional impairments, severe psychiatric symptoms, and frequent re-hospitalizations. Effective psychosocial treatments can improve functioning; however, only a small percentage of people receive those treatments (1-3).

Assertive Community Treatment (ACT) teams serve individuals with the most severe symptoms who are often difficult to engage in services (4). Controlled studies show ACT improves outcomes including re-hospitalization, housing, and treatment retention but is less effective at improving psychiatric symptoms, social functioning, and other functional outcomes (5-15). Further, ACT has been criticized for not being recovery-oriented (16, 17).

Illness Management and Recovery (IMR), a curriculum-based program designed to help individuals pursue personal recovery goals (30), seems well-suited for enhancing outcomes in areas where ACT is less effective. A review (2014) of experimental studies reported that IMR implemented in community-based clinics (18, 19) and supported housing (20) showed significant positive effects on illness self-management, clinician-rated symptom severity, and psychosocial functioning compared to treatment as usual (18, 19) and wait-list control groups (20). A more recent RCT comparing IMR to an active control group showed no significant differences, but participation rates in both treatments were low (21).

Several characteristics of ACT suggest it may be a promising platform for IMR: the flexible nature of ACT allows IMR to be delivered in both individual and group modalities; the community-based approach provides more opportunities for practicing IMR skills in natural settings; and the focus on working with natural supports lends itself to martialing extra support to help individuals achieve recovery goals.

While some have implemented IMR within ACT, several implementation and methodological issues prevent strong conclusions about its effectiveness. Two quasiexperimental studies of ACT+IMR found significant reductions in hospitalizations (22, 23); one also showed significant reductions in substance use (22) and the other in emergency room visits (23). However, IMR was implemented in both studies only by specialists (one peer, one or two clinicians) rather than training the whole team, and both studies lacked well-developed guidelines for implementing IMR. These and similar studies (24, 25) suggest that IMR can be successfully integrated into ACT services if numerous adaptations are made.

Our team undertook a series of research and development activities for implementing IMR within ACT, including developing a manual for implementing IMR within ACT teams (26), conducting a small-scale, open pilot test of this manualized approach, and conducting a qualitative process evaluation to identify barriers, facilitators, and advantages of implementing IMR within ACT (manuscript in preparation). Further, in this article, we report on a pilot evaluation of ACT+IMR in a small-scale cluster randomized clinical trial that aims to provide data about the feasibility of implementing the program and preliminary data on its effectiveness (27).

Methods

A pilot, cluster randomized controlled trial was conducted in which ACT teams were randomized to provide IMR within ACT (ACT+IMR) or standard ACT treatment (ACT-only). The impact of ACT+IMR vs. ACT-only on illness management and recovery outcomes was based on assessments conducted with subsets of randomly selected clients from each team at baseline, six months, and one year follow-up. The study was IRB-approved by participating research organizations [names deleted for blinded review].

Study teams and randomization.

Eight ACT teams in two states were recruited, four teams per state. Selection criteria included no prior IMR training and good fidelity to ACT (scoring ≥ 3.5 out of 5.0) on the Tool for Measurement of Assertive Community Treatment (28) during state-sponsored fidelity assessments in 2012 (29). The mean±SD fidelity score for teams assigned to ACT+IMR was 4.11±.26, compared to 4.02±.35 for ACT-only.

Four of the teams served 80 to 100 clients each; four served 45 to 50 clients each. Randomization to ACT+IMR or ACT-only was stratified by state and team size, resulting in one large team and one small team assigned to each condition in each state. ACT+IMR teams were compensated financially for lost service reimbursement due to staff training time.

Participants.

Twelve to 15 clients were randomly selected for recruitment from each ACT team, based on the following criteria: chart diagnosis of schizophrenia, schizoaffective disorder, or bipolar disorder; ACT admission at least 60 days prior to the study; and projected length of stay on ACT for at least 12 months. Researchers met potential participants to explain the study, inquire about participation interest, and obtain written informed consent from those interested. No clients refused, although one was replaced because of a program transfer.

The study enrolled 101 participants (53 in ACT+IMR, 48 in ACT-only), with a mean age of 43.9±11.6; this sample size would require effect sizes of ∣d∣ ≥56 to achieve power of .80, before controlling for baseline covariates and random effects of treatment site.

Treatment Conditions.

ACT is a multidisciplinary, team-based approach to providing treatment, rehabilitation, and support to high-need, high-risk people with serious mental illness; most services are provided in the person’s home or community, and services are available 24/7 (26). IMR follows a manualized 11-module curriculum to help individuals pursue personal recovery goals and to teach them information, strategies, and skills via group or individual format to manage their psychiatric illness.

ACT+IMR was developed and manualized for this study (26). This model involved training all ACT team members in the ACT+IMR condition in IMR; ACT+IMR specialists provided individual and group-based IMR, and all staff provided community follow-up assistance (e.g., role plays) to assist individuals with practicing IMR skills and pursuing recovery goals. ACT+IMR teams communicated regularly (e.g., during daily meetings) regarding participants’ IMR goals, progress, and follow-up interventions. The ACT team leader, also trained as an ACT+IMR specialist, provided regular IMR supervision.

ACT+IMR teams received training that included written and video materials on IMR, the ACT+IMR treatment manual, a two-day training provided by IMR and ACT experts, a one-day booster training 6-8 months after start-up, and consultation by an IMR expert twice a month for the first six months and monthly for the second six months of implementation (30).

ACT-only teams provided usual ACT services, receiving no IMR training during the study period.

Outcome Measures.

Masters-level interviewers were trained to administer standardized outcome measures. Interviewers conducted face-to-face interviews and were not blinded to treatment conditions. Participants were paid $15 at baseline, $20 at 6 months, and $25 at 12 months.

Illness Self-Management.

The Clinician- and Client IMR Scales evaluated illness self-management across 15 items rated on 5-point behaviorally anchored scales, with higher scores indicating better illness management. Overall scores are sums of the 15 items (ranging from 1 to 5). Clinician ratings were completed by the ACT team member who had the most knowledge about the research participant, excluding those who were primary IMR providers to minimize rater bias. Client ratings were completed by each participant. The IMR Scales have strong psychometric properties (20, 31-33).

Mental Health Symptoms.

The expanded Brief Psychiatric Rating Scale (BPRS) (34) is a semi-structured interview with 24 items rated on 7-point Likert-type scales with higher scores indicating greater symptom severity. The measure is reliable (35) and sensitive to change following IMR (20).

Psychosocial Functioning.

Research interviewers rated participants’ functioning using the Daily Living Activities Scale (DLA-20), the Global Assessment of Functioning (GAF), and the Quality of Life Scale-Abbreviated (QLS-A) (36-38). The DLA is a functional assessment consisting of 20 items measured on 7-point Likert-type scales. The DLA has adequate internal consistency and inter-rater reliability (39). The GAF is a widely-used measure of psychological, social, and occupational functioning with good reliability and validity (40, 41). Scores range from 0 to 100; higher scores indicate better functioning. The seven-item QLS-A (36-38) has similar predictive validity to the longer version (37). Items are rated 0 (virtually absent/low) to 6 (adequate/high), with questions focusing on social functioning, motivation, and positive emotions.

Recovery.

The Recovery Assessment Scale (RAS) includes 41 items rated on 5-point Likert-type scales. The RAS has good psychometric properties and is sensitive to change after IMR (19, 31); the measure’s total score was used as a primary outcome.

Community Integration.

Participants’ involvement in community activities was rated using the Community Integration Measure (CIM; 10 items, rated on five-point Likert-type scales; higher scores indicate poorer community integration). The CIM has shown good internal consistency (42).

Emergency mental health services.

We examined emergency room and hospital admissions for mental health reasons. Research staff collected these data from ACT staff for the 12-month study period.

Statistical Analyses.

We used t-tests for continuous measures and Fisher’s Exact Tests for categorical measures to compare the two conditions on baseline demographic, clinical, and outcome measures. We used repeated measures analysis of covariance (RM-ANCOVA) in a mixed-effects regression context to test differences between groups at 6- and 12-months. Baseline scores on each outcome were entered as covariates; ACT team (i.e., site) was specified as a random effect to control for heterogeneity between teams. (Whether to control for site in small-scale clustered RCT is a matter of debate; thus, we analyzed the data both ways. Results were similar, except the analysis that did not control for site showed one additional significant finding: improved QLS-A scores for the ACT+IMR condition.) Analyses tested both main effects for condition and condition-by-time interactions for differential change between conditions from 6 to 12 months. The between-groups effect size (and 95% confidence interval) was calculated as Cohen’s d, based on the adjusted means at endpoint.

Secondary RM-ANCOVA analyses were conducted on the subscale scores of the BPRS, RAS, and CIM. Given the low utilization of emergency rooms and psychiatric hospitalizations during the study period, these data were dichotomized (no admission vs. any admission). Associations of IMR session attendance and module completion with baseline and 12-month follow-up variables were evaluated to identify participant subgroups related to degree of exposure to IMR, and to explore whether degree of exposure was associated with 12-month outcomes. Session attendance was categorized as “low” (<10 sessions), “medium” (10-24), or “high” (>=25), and module completion was categorized as “low” (<5) or “high” (>=5). Differences between these subgroups were then evaluated via ANOVA (for baseline values) or ANCOVA that controlled for baseline values of the same outcome variable (12-month outcomes). Secondary analyses used p<.01 to determine statistical significance.

Results

Table 1 shows the characteristics of the overall study group and comparisons between the treatment conditions on baseline demographic, clinical, and outcome measures. There were significant differences (p<.05) between conditions in ethnicity, living situation, primary psychiatric diagnosis, and Client IMR Scale scores. ACT+IMR participants were more likely to be housed, non-Latino, to have a mood disorder, and to have lower Client IMR Scale scores. The two groups were similar on the other measures.

TABLE 1.

Characteristics and outcome measures of the study group and differences between the ACT+IMR and ACT-only groups at baseline.

Measure	Overall		ACT + IMR		ACT-only		Group Difference
	N	%	N	%	N	%	Test statistic	P
Race
White	53	53	31	60	22	46
Black	40	40	16	31	24	50
Other	7	7	5	9	2	4	χ2 = 4.26	.119
Ethnicity
Latino	16	16	4	8	12	25
Non-Latino	84	84	48	92	36	75	χ2 = 5.56	.018
Gender
Male	58	59	30	59	28	58
Female	41	41	21	41	20	42	χ2 = .002	.961
Marital Status
Never Married	71	71	34	65	37	77
Married	5	5	2	4	3	6
Widowed/Divorced	24	24	16	31	8	17	χ2 = 2.84	.242
Education Level
< High School	32	33	17	33	15	33
High School/GED	38	39	21	40	17	37
> High School	28	28	14	27	14	30	χ2 = .18	.914
Employment Status
Working	17	17	9	18	8	17
Not Working	82	83	42	82	40	83	χ2 = .02	.897
Living situation
Shelter/Institution	5	5	0	0	5	5
Housed	96	95	53	100	43	95	χ2 = 5.81	.016
Psychiatric Diagnosis
Psychotic Disorder	82	81	39	74	43	90
Mood Disorder	19	19	14	26	5	10	χ2 = 4.22	.040
Substance Use Disorder
Yes	61	60	33	62	28	58
No	40	40	20	38	20	42	χ2 .16	.687
Age (M±SD)	43.9±11.6		43.7±11.7		44.2±11.6		t = .21	.832
Lifetime Hospitalizations (M±SD)	11.0±9.3		10.9±10.3		11.0±8.1		t = .07	.941
Client IMR Scale (M±SD) a	50.4±8.7		48.6±9.6		52.6±7.1		t = 2.34	.021
Clinician IMR Scale (M±SD) a	45.4±7.6		44.6±7.0		46.2±8.2		t = 1.07	.285
BPRS (M±SD)b	51.2±12.1		51.9±12.3		50.5±12.0		t = .56	.574
GAF (M±SD)c	39.8±10.0		40.6±10.4		39.8±10.0		t = .08	.428
DLA-20 (M±SD)d	4.0±0.9		4.0±1.0		4.0±0.9		t = .02	.983
QLS-A (M±SD)e	2.4±1.0		2.3±1.1		2.4±0.9		t = .62	.538
RAS (M±SD)f	159.3±25.5		159.1±23.1		159.6±28.2		t = .08	.934
CIM (M±SD)g	3.9±0.8		3.9±0.7		3.9±0.9		t = .28	.783

Cell sizes vary due to missing data or uncommon responses (e.g., Gender = transsexual)

^aIllness Management and Recovery Scales (client- and clinician-rated). Possible scores range from 1 to 5, with higher scores indicating better perceived illness management.

^bBrief Psychiatric Rating Scale. Possible scores range from 0 to 144, with higher scores indicating greater symptom severity.

^cGlobal Assessment of Functioning. Possible scores range from 1 to 100, with higher scores indicating greater levels of global functioning.

^dDaily Living Activities Scale-20. Possible scores range from 1 to 7 for each item, with higher scores indicating greater levels of independent functioning.

^eQuality of Life Scale-Abbreviated. Possible scores range from 0 to 6, with higher scores indicating higher quality of life.

^fRecovery Assessment Scale. Possible scores range from 41 to 205, with higher scores indicating greater levels of improvement and recovery.

^gCommunity Integration Measure. Possible scores range from 1 to 5 for each item, with higher scores indicating greater perceived integration.

Table 2 presents an overview of each of the 11 IMR modules. ACT+IMR participants completed a mean±SD of 21.3±13.3 IMR sessions (range 0 to 42) and 4.5±3.4 of the 11 IMR modules. We found significant differences between ACT+IMR teams on sessions attended and modules completed. Participants on one team had significantly lower exposure to IMR (see Table 3).

TABLE 2.

Illness Management and Recovery (IMR) modules and content

	Module Title	Module Topics
1	Recovery Strategies	Defining recovery and learning what helps in the recovery process; exploring areas of life client wants to improve; identifying personal recovery goal; breaking down goal and taking the first step toward achieving it; following up on goals and solving problems
2	Practical Facts about Mental Illnessa	Understanding the disorder and its diagnosis; learning what happens after people develop symptoms; taking positive steps to manage the disorder; dealing with negative attitudes and beliefs about mental illnesses (stigma)
3	The Stress Vulnerability Model	Understanding the causes of mental illness; learning what improves symptoms and reduces relapses; understanding treatment options; reducing relapses
4	Building Social Support	Recognizing the importance of social support; connecting with people; having enjoyable conversations; sharing personal information; understanding other people; developing closer relationships
5	Using Medication Effectively	Learning about the role of medication in managing symptoms; recognizing and responding to side effects; making informed decisions; getting the best results from medication
6	Drug and Alcohol Use	Identifying common reasons people use alcohol and drugs; recognizing the problems that alcohol and drugs can cause; weighing the pros and cons of sobriety; identifying personal reasons for sobriety and planning for high-risk situations; finding new ways of getting needs met; making a personal sobriety plan
7	Reducing Relapses	Identifying triggers of relapse; recognizing and monitoring early warning signs of relapse; developing a relapse prevention plan; putting the relapse prevention plan into practice
8	Coping with Stress	Learning what causes stress; identifying the signs of stress; prevention and coping with stress; using relaxation techniques; making a plan for preventing and coping with stress
9	Coping with Persistent Symptoms	Identifying persistent symptoms; coping with depression; coping with anxiety; coping with hallucinations and coping with delusions (false beliefs); coping with sleep problems, low stamina, and low energy; coping with angry feelings and concentration problems; making a plan for continuing to use coping strategies
10	Getting Your Needs Met in the Mental Health System	Overview of community mental health services; financial and health benefits; advocating for yourself in the mental health system
11	Healthy Lifestyles	Diet, part I; diet, part II; exercise; personal hygiene; sleep

^aSpecific handouts available for schizophrenia (2A), schizoaffective (2B), bipolar disorder (2C), major depression (2D), and multiple diagnoses (used in groups; 2E)

TABLE 3.

ACT participant exposure to IMR, by ACT+IMR team, and one-way ANOVAs of the number of IMR sessions attended and IMR modules completed.

ACT+IMR Team	N	M	SD	F(3,52)	p
Number of completed IMR modules				4.99	.004
Team 1	12	4.8	2.2
Team 2	12	5.6	3.3
Team 3	15	2.1	2.4
Team 4	14	6.1	4.0
Total number of IMR sessions attended				5.02	.004
Team 1	12	27.3	8.7
Team 2	12	29.3	13.6
Team 3	15	14.8	13.9
Team 4	14	16.4	10.7

Table 4 displays the statistical results at follow-up for the eight primary outcome measures. A significant difference was found for the Clinician IMR Scale, favoring the ACT+IMR condition with a medium effect size (d =.51). There were no significant group differences on the other seven primary outcomes; however, a medium effect size was observed in the QLS-A (d =.64) in a direction that favored ACT+IMR. Other effect sizes for continuous measures were small and are reported in Table 4. Differences between conditions for binary outcomes were also small: 15.1% ACT+IMR vs. 10.4% of ACT-only participants had at least one ER visit (p = .561), 20.8% of ACT+IMR participants vs. 25.0% of ACT-only participants had at least one psychiatric hospitalization (p = .642). RM-ANCOVA analyses of the five BPRS subscales, the five RAS subscales, and the three CIM subscales revealed similar patterns as the full-scale scores (i.e., no significant differences between conditions).

TABLE 4.

Means and standard deviations at 6 and 12 months and results of repeated-measures analyses of covariance with significance levels and effect sizes for the difference between ACT-only and ACT+IMR groups.

	Condition	6 months		12 months
		N	%	N	%
	ACT+IMR	47	50	42	48	Adjusted	Significance
	ACT-only	47	50	46	52	Group Differencea
Domain/Measure	Condition	M	SD	M	SD	F	df p	ESb	CI
Illness Management
Client IMR Scale c	ACT+IMR	50.6	10.0	52.3	9.2	2.11	.15	.36	−.33 to 1.05
	ACT-only	51.6	7.2	50.0	10.7
Clinician IMR Scale c	ACT+IMR	50.8	8.1	51.0	8.9	6.35	.01	.51	−.16 to 1.18
	ACT-only	48.1	9.0	47.8	10.7
Symptoms
BPRSd	ACT+IMR	46.8	13.5	47.5	21.2	.32	.57	.22	−.39 to .83
	ACT-only	47.6	15.3	44.8	11.8
Psychosocial Functioning
GAFe	ACT+IMR	41.6	9.8	46.2	13.1	.24	.63	.24	−.67 to 1.14
	ACT-only	39.2	9.2	44.0	11.4
DLA-20f	ACT+IMR	4.2	1.0	4.2	0.1	.17	.68	.11	−.53 to .75
	ACT-only	3.9	1.0	4.1	0.9
QLS-Ag	ACT+IMR	2.7	1.1	3.1	1.2	1.15	.29	.64	.02 to 1.24
	ACT-only	2.5	1.0	2.5	1.1
Recovery
RASh	ACT+IMR	168.1	21.0	163.2	25.4	<.01	.95	.03	−.59 to .65
	ACT-only	167.7	20.3	162.7	20.8
Integration
CIMi	ACT+IMR	4.0	0.7	3.9	0.8	.63	.43	−.31	−.91 to .29
	ACT-only	4.0	0.6	4.1	0.7

^aThe Adjusted Group Difference is the treatment effect, adjusted by the baseline value of the outcome, with PACT team specified as a random effect to account for clustering of observations within teams.

^bES, effect size; CI, 95% confidence interval. Effect size is Cohen’s D for the difference between adjusted means at 12-month follow-up.

^cIllness Management and Recovery Scales (Client- and Clinician-rated). Possible scores range from 1 to 5, with higher scores indicating better perceived illness management.

^dBrief Psychiatric Rating Scale. Possible scores range from 0 to 144, with higher scores indicating greater symptom severity.

^eGlobal Assessment of Functioning. Possible scores range from 1 to 100, with higher scores indicating greater levels of global functioning.

^fDaily Living Activities Scale-20. Possible scores range from 1 to 7 for each item, with higher scores indicating greater levels of independent functioning.

^gQuality of Life Scale-Abbreviated. Possible scores range from 0 to 6, with higher scores indicating higher quality of life.

^hRecovery Assessment Scale. Possible scores range from 41 to 205, with higher scores indicating greater levels of improvement and recovery.

ⁱCommunity Integration Measure. Possible scores range from 1 to 5 for each item, with higher scores indicating greater perceived integration.

Baseline variables were evaluated as predictors of IMR session and module completion to identify which participants were more likely to receive greater IMR exposure (see Table 5). Participants were more likely to complete 10-24 sessions (“medium” exposure) or 25 or more sessions (“high”) compared to fewer than 10 sessions (“low”) if they completed high school/GED, did not have a co-occurring axis-II disorder, or had higher baseline DLA-20 or QLS-A scores. Participants were also more likely to complete a high number of IMR modules (≥5) compared to a low number (<5) if they completed high school/GED, did not have a co-occurring substance use or axis-II disorder, or had higher baseline QLS-A scores.

TABLE 5.

Associations of baseline and 12-month outcome variables with treatment attendance and module completion.

	IMR Sessions Completed						IMR Modules Completed
	Low (<10) (N=13)		Medium (10-24) (N=16)		High (≥25) (N=24)		Low (<5) (N=23)		High (≥5) (N=30)
	N	%	N	%	N	%	N	%	N	%
Baseline Variablesa
White Non-Hispanic	7	54	10	63	12	50	12	52	17	57
Male	10	77	8	50	13	54	15	65	16	53
Divorced or Widowed	3	25	6	38	7	29	7	32	9	30
At Least High School/GED	5	42	10	63	20	83*	10	46	25	83**
Beyond High School	2	17	5	31	7	29	4	18	10	33
Employed or Volunteering	1	8	4	25	4	17	2	9	7	24
Mood Disorder	2	15	6	38	6	25	4	17	10	33
Substance Use Disorder	9	69	10	63	14	58	18	78	15	50*
Any Axis-II Disorder	8	62	11	69	6	25*	18	78	7	23***
Hospitalized >10 times	3	25	8	50	10	42	8	36	13	43
Age (M±SD)	41.8±12.9		44.0±12.1		44.6±10.8		42.4±12.3		44.8±11.0
Client IMR Scalec (M±SD)	46.2±10.9		52.2±9.9		47.4±8.2		48.3±10.7		48.8±8.8
Clinician IMR Scalec (M±SD)	42.1±6.1		47.3±6.7		44.1±7.2		43.9±6.6		45.1±7.3
BPRSd (M±SD)	55.1±12.4		51.6±13.7		50.3±11.4		53.8±12.2		50.4±12.4
GAFe (M±SD)	36.2±7.9		43.0±10.1		41.3±11.4		38.3±8.0		42.3±11.8
DLA-20f (M±SD)	3.4±.9		4.3±1.0*		4.0±.9		3.7±.9		4.1±1.0
QLS-Ag (M±SD)	1.7±.9		2.4±1.1		2.6±1.0**		1.9±.9		2.6±1.1**
RASh (M±SD)	166.3±26.0		155.9±28.2		158.1±19.0		162.5±23.8		156.7±22.7
CIMi (M±SD)	3.8±.8		3.8±.9		4.0±.5		3.7±.8		4.0±.6
12-Month Outcome Variablesb
Client IMR Scalec (M±SD)	45.0±10.9		53.4±10.6		53.3±7.5*		48.1±10.1		54.4±8.1**
Clinician IMR Scalec (M±SD)	43.6±3.6		52.1±10.0		51.96±8.5		45.4±7.5		53.3±8.5*
BPRSd (M±SD)	57.5±13.4		44.0±12.5		46.9±26.0		51.7±13.5		45.4±24.1
GAFe (M±SD)	32.8±9.5		50.9±11.0		48.2±13.3		38.3±9.4		52.0±12.6
DLA-20f (M±SD)	3.0±.8		4.5±1.0**		4.3±.8*		3.6±.9		4.4±.9*
QLS-Ag (M±SD)	2.1±.6		3.4±1.6		3.1±.9		2.5±1.2		3.3±1.1
RASh (M±SD)	167.5±35.4		167.8±30.4		159.7±21.0		167.3±30.0		161.2±23.4
CIMi (M±SD)	4.0±1.0		3.6±.9		4.1±.6		3.4±.8		4.1±.6**

^aFor baseline variables, significance tests reflect differences in percentages (tested via Fisher's exact tests) or means (tested via ANOVA) between subgroups based on the extent of their IMR attendance or module completion.

^bFor 12-month outcome variables, significance tests reflect differences between subgroups controlling for baseline variables of the same measure at baseline (tested via ANCOVA).

^cIllness Management and Recovery Scales (Client- and Clinician-rated). Possible scores range from 1 to 5, with higher scores indicating better perceived illness management.

^dBrief Psychiatric Rating Scale. Possible scores range from 0 to 144, with higher scores indicating greater symptom severity.

^eGlobal Assessment of Functioning. Possible scores range from 1 to 100, with higher scores indicating greater levels of global functioning.

^fDaily Living Activities Scale-20. Possible scores range from 1 to 7 for each item, with higher scores indicating greater levels of independent functioning.

^gQuality of Life Scale-Abbreviated. Possible scores range from 0 to 6, with higher scores indicating higher quality of life.

^hRecovery Assessment Scale. Possible scores range from 41 to 205, with higher scores indicating greater levels of improvement and recovery.

ⁱCommunity Integration Measure. Possible scores range from 1 to 5 for each item, with higher scores indicating greater perceived integration.

^*p<.05,

^**p<.01 ,

^***p<.001

Associations of IMR session and module completion with 12-month outcomes, controlling for baseline measures of each outcome, are also presented in Table 5. Completing more IMR sessions was associated with more improvement on the 12-month Client IMR Scale and the DLA-20. Completing more IMR modules was associated with more improvement on the 12-month Client and Clinician IMR scales, the DLA-20, and the CIM. Although non-significant, participants completing more IMR sessions and modules trended toward having lower BPRS scores, and higher GAF and QLS-A scores.

Discussion

These pilot results provide support for the feasibility of implementing IMR within ACT teams. Many participants were able to achieve moderate to high levels of IMR exposure within 12 months, although the variable exposure to IMR across participants and teams suggests some implementation challenges occurred which should be addressed in future studies.

This study found superior outcomes for ACT+IMR participants on only one of eight primary outcomes, clinician-rated illness self-management; however, the effect sizes on other variables and evidence for a potential dose-response relationship with four outcome measures provide some support for the potential effectiveness of ACT+IMR. Consistent with prior IMR research, we found significant improvement on clinician-rated illness self-management with a medium effect size (43). It should be noted, however, there is possible bias in the clinician ratings because the rater (i.e., clinician) was not blind to intervention. Inconsistent with prior research, this study did not find a main effect of treatment on client-rated illness self-management, psychiatric symptoms, or psychosocial functioning as measured by the QLS-A (43); however, a medium effect size was found in the QLS-A and a small effect size was found for client-rated illness self-management, as is consistent with prior research (21). A small effect size for community integration was observed, though this has not been measured in other studies but could be interpreted as an extension of functioning. The lack of significant treatment effect on other measures of functioning and other distal outcomes is consistent with the literature on IMR (43).

There are several potential reasons for the lack of significant treatment effects. First, this small-scale trial had relatively low power due to the use of an active treatment comparison condition and the relatively small sample size. Measures with low frequencies of endorsement—notably psychiatric hospitalizations and ER use—likely suffered from especially low power. In addition, the low and variable rates of exposure to IMR within the ACT+IMR condition, especially within one team experiencing high staff turnover, likely contributed to the lack of significant effects. Consistent with at least one IMR study, future ACT+IMR work may benefit from a longer period of evaluation (i.e., 21 months) (19), as participants may have experienced delayed benefits that occurred beyond the study period, and enhanced implementation strategies to address follow-up IMR training when staff turnover occurs. Further, it should be noted that most participants were still receiving IMR at the end of the study, suggesting that a longer interval is needed both to evaluate the effects and to effectively deliver IMR to this challenging population.

Our standardized measures may not have been sensitive to the benefits that participants experienced from the ACT+IMR intervention, given subjective reports from participants and clinicians that indicated progress towards achieving personal recovery goals across several behavioral domains. Future research should explore idiographic improvements through qualitative research methods, examination of individual goals (44), or statistical methods that accommodate individual changes across a range of outcomes (45). Finally, it is possible that the lack of significant differences reflects the fact that ACT staff were in the relatively early stages of learning IMR, and that greater practice using the model would yield stronger treatment effects.

Future work should also explore alternative ways of increasing the effectiveness and cost-effectiveness of recovery-oriented interventions within ACT. A qualitative process evaluation conducted across three small pilot studies, including this trial (manuscript in preparation), indicates that implementing ACT+IMR can be a time-intensive and complex process with many barriers (e.g., symptom severity, staff workload, communication problems); however many of these barriers can be overcome by specific consultation and implementation strategies (e.g., tailoring IMR consultation to ACT specifically, focusing on client engagement, providing peer support, being flexible with order and number of IMR modules, improving team communication and service integration). Future larger-scale research efforts on ACT+IMR should build on this knowledge of barriers and implementation strategies, which may then lead to better, more cost-effective outcomes; a hybrid effectiveness-implementation design would provide the mechanism for such study (46). Future work may also target ways to increase IMR exposure to clients who are likely to receive fewer sessions, including participants with less education, substance use or axis-II disorders, and lower baseline psychosocial functioning.

A different research direction concerns the duration of IMR. Completing the eleven-module IMR curriculum required about one year of weekly sessions. A more “targeted” approach, delivered individually, that focuses only on IMR topics related to specific goals of each client, could potentially reach more clients in a cost-effective manner. Similarly, other personalized approaches to illness management, especially those that incorporate technology in the delivery of interventions (47-51), also have potential for cost-effectively helping people manage their illness and pursue recovery goals.

Conclusions

The present study provides support for the feasibility of implementing IMR within ACT teams. Although many results were not statistically significant, this study provides initial evidence of a potential dose-response relationship and some medium (but non-significant) effect sizes favoring ACT+IMR. Further, larger-scale efforts using a hybrid effectiveness-implementation design would help to directly test more rigorous consultation and implementation strategies to maximize IMR exposure, and the effectiveness of IMR for improving recovery and functioning outcomes for people served by ACT teams.