Computerized continuing care support for alcohol and drug dependence: a preliminary analysis of usage and outcomes

Abstract

The central aim of this administrative data analysis was to examine usage of a web-based disease management program designed to provide continuing recovery support to patients discharged from residential drug and alcohol treatment. Tailored clinical content was delivered in a multi-media format over the course of 18 months post-treatment. The program also included access to a recovery coach across the 18 months. Consistent with other disease management programs, program usage decreased over time. A small subsample of patients accessed a large number of program modules in the year following treatment; these patients had significantly higher abstinence rates and consumed less alcohol than patients accessing few or no modules. Regression analyses revealed a significant relationship between the number of modules accessed and substance use outcomes in the year following treatment when controlling for motivation, self-efficacy, and pre-treatment substance use. Limiting the analyses to only the more compliant patients did not reduce the magnitude of these effects. These preliminary results suggest that computerized support programs may be beneficial to patients recently treated for drug and alcohol issues. Methods to increase program engagement need additional study.

1. Introduction

The use of computerized disease management programs and other e-health interventions to treat and manage chronic health conditions has increased over recent years. Several research studies have examined participant usage of these programs and the extent to which usage is related to positive health behavior outcomes across a number of domains, including cancer (Gustafson et al., 2005), diabetes (Williams et al., 2007; Glasgow et al., 2005), heart disease (Verheijden et al., 2004) and mood disorders (Farvolden et al., 2005). Overall, these studies suggest that computerized disease management programs can be effective, though rates of engagement are often low.

A number of computerized programs have been developed to address substance use and addictive disorders, with many focusing on alcohol abuse. The majority of these programs provide screening and brief intervention, wherein participants provide information about alcohol and drug use and then receive clinical information, advice, and guidance on how to reduce use. A number of studies suggest these programs are effective. For example, Hester and colleagues (2005) developed the Drinker’s Check-Up (DCU), a brief motivational intervention delivered over the computer in a series of modules. Modules focus on assessing drinking and alcohol-related problems, providing personalized feedback, and helping the participant resolve ambivalence about changing drinking. A recent study found that participants receiving DCU substantially reduced their alcohol use and other alcohol-related problems over a 12 month follow up period (Hester et al., 2005). Studies of other computerized screening and brief interventions have reported similar results (Kypri et al., 2008; Cunningham et al., 2006).

Other studies have shown that computerized versions of popular psychosocial treatment approaches are effective when administered to individuals having a substance use disorder. For example, Carroll and colleagues developed CBT4CBT, which supplemented therapist delivered treatment with 8 weeks of biweekly access to a multimedia computer-based version of cognitive behavior therapy. The first randomized controlled trial of this program focused on individuals with substance dependence attending outpatient treatment. Substance use outcomes were compared for two groups: one group received outpatient treatment plus the computerized version of CBT, the other group received outpatient treatment as usual. The group receiving computerized CBT reported a greater number of abstinent days across the 8-week treatment period and had fewer positive drug screens compared to the treatment as usual group (Carroll et al., 2008). A second report from this study examined substance use over a longer follow up period (6 months) and produced similar findings (see Carroll et al., 2009).

In a randomized controlled trial of opioid-dependent outpatients, Bickel et al. (2008) examined the efficacy of an interactive computerized behavioral treatment intervention based on contingency management and community reinforcement approaches (CRA). Patients receiving the computerized CRA treatment along with vouchers showed roughly the same number of weeks of continuous cocaine and opioid abstinence as patients receiving therapist-delivered CRA with vouchers, and both groups showed significantly longer periods of abstinence than a treatment-as-usual group. In a recent of review of studies utilizing computer-based treatment with individuals dependent on one or more illicit drugs, Moore et al. (2011) reported that most studies found better drug use outcomes for patients receiving computerized treatment than for control group patients not receiving the intervention. Patients also reported being highly satisfied with the computerized interventions and showed levels of treatment engagement and retention that were similar to therapist-provided treatments.

Though research generally supports the effectiveness of computerized prevention programs, such as screening and brief intervention, and computer delivered treatment, very little is known about whether computer technology can be used to deliver continuing care and recovery support to individuals who have already received formalized treatment for a substance use disorder. To our knowledge, in contrast to other disorders like heart disease and diabetes, no computerized continuing care programs have been developed for long term management of substance use disorders, despite a large body of research suggesting that traditional (non-computerized) continuing care programs are associated with better post-treatment outcomes (McKay, 2009a). A recent review of controlled continuing care studies found that interventions that were at least 12 months long or that included more active efforts to deliver the intervention components to patients were more likely to produce positive effects (McKay, 2009b).

Most continuing care programs for substance use disorders, such as those discussed above, require participants to attend structured support groups and/or engage with counselors and other clinical staff over the telephone. It is currently unclear to what extent computerized programs can be utilized to deliver continuing care and recovery support to individuals treated for alcohol/drug dependence. Several years ago, Hazelden staff created My Ongoing Recovery Experience (MORE™), a computerized recovery support program developed for patients receiving residential treatment for alcohol/drug dependence. Unlike other continuing care programs, the program delivers assessments, clinical content, resources and activities over the computer. The program also involves periodic telephone and email contact between patients and recovery coaches. The present study had two goals: to examine usage of the MORE™ program after discharge from residential treatment and to assess the relationship between program usage and substance use outcomes during the year following treatment.

2. Materials and Methods

2.1. The Hazelden MORE program

In 2006, Hazelden, a private non-profit addiction treatment center, launched an innovative, first-of-its-kind computerized recovery support program called My Ongoing Recovery Experience (MORE™). MORE™ was developed by licensed alcohol and drug counselors and mental health professionals for use with adult patients completing residential treatment for alcohol or drug dependence at Hazelden sites. Prior to implementation, patient input regarding program development was obtained through a number of focus groups. Feasibility of the program was tested by administering a sample of the program to several groups of patients who volunteered to be end users. Patients provided a number of recommendations regarding clinical content and program functionality that were incorporated into the final version of the program.

The program incorporates concepts and findings from a number of evidence-based treatment approaches, such as Twelve Step Facilitation, Motivational Interviewing and Cognitive Behavioral Therapy. Through the program patients have access to recovery-related information and services for a period of 18 months following discharge from treatment. MORE™ is an iteratively-tailored, web-based, mixed-media program that provides interactive recovery-related activities, videos, an extensive electronic library of content, opportunities for fellowship with other recovering individuals, and other sources of support. The videos cover a wide range of content, including patient testimonials and information on topics such as how to manage a co-occurring disorder, recognizing factors that may lead to relapse, how to protect oneself from developing an addiction to prescription medications, and how to effectively manage stress. Other computerized resources include a patient journal, workbook activities for practicing and applying recently learned information, and an electronic library of articles pertaining to issues such as relapse prevention, dealing with cravings, and maintaining emotional health. Participants also have access to a network of Hazelden alumni and a number of resources for staying in contact with other individuals in recovery, such as recovery blogs, forums, podcasts, and online 12 step meetings.

The program is delivered in 7 recovery “modules”, each of which begins with an electronically administered assessment where patients provide detailed information about substance use and other aspects of life functioning. Based on the responses, the module then delivers content tailored to the individual needs of the patient. Patients receive modules in a sequential manner over an 18 month period, and modules have to be completed in order. In other words, content from later modules is only accessible if the participant accessed prior modules. In addition, the availability of the modules is timed across the 18 month period. The assessment for Module 1 is completed while patients are still in residential treatment, usually 1 week prior to discharge.

Each module focuses on specific issues that are likely to be experienced at that particular time in recovery. Table 1 provides a summary of each module and its key activities. Modules varied somewhat in length and ranged from 19 pages of content (Module 7) to 36 pages of content (Module 2). The total number of pages across all 7 modules was 201.

Table 1.

MORE™ module content.

Module	Module focus	Sample of key activities
Module 1	Mind, Body, and Spirit	Identifying people, places and things that are triggers to relapse and developing coping strategies to deal with them. Creating a safe home environment free of triggers and cues. Engaging in AA, finding groups, obtaining a home group, finding a sponsor, and using the Twelve Steps.
Module 2	Getting Connected	Developing a supportive network of healthy people, finding friends in recovery, re-connecting with family, and asking for help.
Module 3	Balancing Work and Play	Addressing workplace issues and establishing sober fun. Achieving lifestyle balance.
Module 4	Healthy and Emotional Living	Coping with normal feelings in recovery including anger, anxiety, sadness, and loneliness. Addressing unhealthy thought processes or compulsive behaviors that may emerge.
Module 5	Healthy Connections	Developing healthy intimate relationships, communicating effectively with others, establishing trust.
Module 6	Sober Identity	Establishing an identity as a sober individual, identifying and abandoning old thought patterns and beliefs, and exploring ways to continuously improve emotional and physical health.
Module 7	Life Goals	Re-examining career, education, and life goals for long-term recovery.

In addition to the recovery modules and other computerized program components, another aspect of MORE™ pertains to recovery coaching. Each participant has a personal recovery coach, a licensed drug and alcohol counselor who provides ongoing support and guidance for a period of 18 months following treatment. Coaches maintain periodic contact with the patient both electronically and by telephone through a series of system “flags”, or alert messages that indicate the patient has reached a milestone or is experiencing problems. When coaches receive a flag, they call the patient and discuss his or her current concerns. Patients who cannot be reached by telephone are sent an email message and asked to contact their coach. The frequency of coach to patient contact varies considerably, ranging from no contact at all to several times per month.

2.2. Participants

We examined a sample of residential patients who enrolled in the MORE™ program shortly before discharge; patients were discharged in the period May 2006 – December 2007 (n = 1124). Only participants who met criteria for dependence on at least one substance were included in the sample, those who were classified as having abuse but not dependence were excluded. No other criteria were used to exclude patients from the sample. Table 2 summarizes demographic and baseline clinical characteristics of the sample. The average length of stay was 25.72 days (SD = 5.77) with 92% (n = 1034) successfully completing treatment. Treatment completion was defined as attending the residential program for the period of time prescribed by clinical staff. The sample was 45% female and the average age was 42.12 (SD = 11.70). Forty six percent (n = 516) of the sample was married at the time of treatment admission and 42% (n = 470) was employed full time. Ninety six percent (n = 1079) of patients were Caucasian. Please note that in this section and in later sections, the “n” represents the number of participants with a positive value for the relevant variable, not the total number in the sample.

Table 2.

Demographic and clinical characteristics at baseline.

Variable	% or mean (SD)
Gender
Male	55%
Female	45
Age (in years)	42.12 (11.70)
Education
Didn’t complete HS	3%
HS diploma/GED	24
Associate degree	9
College graduate	36
Some college	13
Graduate degree	15
Marital status
Married	46%
Single, never married	28
Divorced	16
Separated	6
Widowed	2
Cohabitating	2
Employment status
Full time	42%
Part time	5
Unemployed	23
Self employed	10
Retired	6
Student	3
Homemaker	5
Disabled	3
Military	3
Dependence diagnoses
Alcohol	87%
Marijuana	20
Cocaine	22
Amphetamines	9
Opiates	16
BASIS-32
Average score	1.45 (0.73)
Co-occurring disorder	65%
Anxiety disorder	28
Depressive disorder	35
Bipolar disorder	4

Substance dependence and mental health diagnoses were made in the course of routine clinical operations through a comprehensive baseline assessment conducted by an interdisciplinary team of licensed alcohol and drug counselors, psychiatrists, and doctoral-level clinical psychologists. All diagnoses were based on DSM-IV criteria (American Psychiatric Association [DSM-IV-TR], 2000). Eighty seven percent (n = 976) of the sample received a diagnosis of alcohol dependence. The percentage diagnosed as dependent on both alcohol and at least 1 other drug was 35% (n = 392). Sixty five percent (n = 730) had a co-occurring mental disorder in addition to alcohol and/or drug dependence, with 35% (n = 393) having a depressive disorder. The average overall score on the BASIS-32 indicated a moderate level of impairment (1.45; SD = 0.73).

In addition to baseline data, patients completed follow up surveys administered at roughly 6 and 12 months after discharge. Seventy percent of the sample (n = 786) completed the 6 month survey, 54% (n = 611) completed the 12 month survey, and 48% (n = 541) completed both surveys.

2.2. Measures

2.2.1. Measures collected at baseline

In the course of routine clinical operations, several surveys were administered to patients by counseling staff within several days of treatment admission. These surveys captured demographic information, frequency of substance use before treatment, and mental health characteristics.

Pretreatment alcohol and drug use.

Several questions from the modified Form 90 (Project MATCH Research Group, 1993) were used to assess alcohol and drug use during the 90 days prior to treatment admission. Alcohol use items included the number of days the patient had at least one alcoholic drink (drinking days), the average number of drinks consumed on drinking days (drinks per drinking day), and the number of days the patient used each of several recreational drugs, as a function of drug type (e.g., marijuana, cocaine, heroin, etc.). The Form 90 has been used with adolescents and adults with substance use disorders and has demonstrated good validity and test-retest reliability (Tonigan, Miller and Brown, 1997).

Pretreatment 12 step involvement.

Several questions from the Alcoholics Anonymous Affiliation Scale (AAAS; Humphreys et al. 1998) measured 12 step group involvement during the year prior to treatment admission. Patients were asked how many 12 step meetings they had attended during the period and how many of the steps they had “worked”. They were also asked the following yes/no questions: whether they had attended AA (or a related group), whether they considered themselves to be a member of AA, whether they had celebrated a sobriety birthday, whether they had a sponsor, whether they had ever been a sponsor, and whether they had ever experienced a spiritual awakening. The number of “yes” responses to these questions was summed to comprise the“AA composite” score, which served as a quantitative measure of AA involvement separate from meeting attendance (see Witbrodt & Kaskutas, 2005). The AAAS has been shown to have high reliability and validity (Humphreys et al., 1998).

Behavior and Symptom Identification Scale (Basis-32).

The BASIS-32 is a global measure of psychological functioning and was administered within the first week of treatment. It consists of 32 questions that assess five aspects of mental health: relation to self and others, daily living and role functioning, depression and anxiety, impulsiveness, and psychosis. Participants are given 32 items that list various domains of functioning (e.g., “adjusting to major life stresses”) and are asked to rate how much difficulty they experienced in that area during the past week. Higher scores represent greater problem severity. The BASIS-32 has been used with adolescents and adults and has demonstrated acceptable levels of internal consistency, test-retest reliability and concurrent validity (Eisen et al., 1994).

2.2.2. Administrative measures related to the treatment episode

Several variables, including demographic information, were obtained from the administrative data set. Length of stay was captured as the number of days each patient attended residential treatment.

2.2.3. Measures collected within the MORE™ program

Several questions were administered within the assessment for Module 1 of the MORE™ program. This assessment was completed by patients over the computer roughly one week before treatment discharge.

Number of people in the support network.

Module 1 asks a question regarding the patient’s support network: “Currently, who are the main people you rely on for support?” It then lists a number of possible responses (e.g., wife, husband, father, daughter, etc.). A variable was created for each patient by summing together the total number of people identified.

Self efficacy and motivation.

The question for self efficacy asked “How confident do you feel in your ability to work an active recovery program at this time?” (response scale: 1 = not confident at all to 7 = extremely confident). The question for motivation asked “How motivated do you feel in your ability to work an active recovery program at this time?” (response scale: 1 = not motivated at all to 7 = extremely motivated).

2.2.4. Measures collected at follow up

A follow up survey, used in the course of routine healthcare operations, contained questions about behaviors and events that occurred in the time period since discharge; only those questions germane to the present evaluation will be discussed here. Patients completed the follow up survey at 3 different times post-discharge:1 month (25-44 days), 6 months (173-210 days) and 12 months (358-395 days). Questions referred to the time period since treatment discharge or since the last follow up survey date if one was completed. It is important to note we did not analyze any 1 month follow up data because these data showed extremely low variability; with over 85% of patients reporting no substance use during the period. However, any substance use data reported at the 1 month follow up was accounted for in later follow ups.

The substance use questions came from the modified Form 90 (Project MATCH Research Group, 1993) and were the same as those asked at baseline (discussed above). These questions and all others were asked to the patient over the phone by research department outcomes evaluation staff extensively trained to conduct follow up interviews. The 12 step involvement questions were also the same as those asked at baseline except they referred to behaviors during the follow up period. Another question asked the patient if he or she had attended any other formal alcohol/drug treatment during the follow up period.

Each follow up also consisted of several questions designed to measure satisfaction with the MORE program. These questions were asked to each patient who indicated he or she had used MORE at least once since discharge from treatment. One question asked patients to rate the overall quality of the MORE program (on a scale of excellent, very good, good, fair, poor). Other questions asked patients to indicate their perception of how useful the program was to their recovery, for example, “The content of the MORE program has been relevant to my recovery” and “I have begun working a recovery program because of MORE” (both questions used a scale of strongly agree, agree, neutral, disagree and strongly disagree).

2.2.5. Outcomes measures

MORE™ usage: number of modules accessed.

Each of the 7 modules within MORE™ has a tailor date, or the date that a participant first accesses the module assessment. The total number of module tailor dates for each participant was summed together to create a variable called “number of modules accessed”. This variable served as the primary measure of program use. In addition to serving as a secondary outcomes measure, number of modules touched was also used as a predictor in regression analyses for substance use outcomes.

Continuous abstinence.

A patient was counted as continuously abstinent at each follow up if he or she reported no alcohol or illicit drug use during the period since discharge. The percentage of patients continuously abstinent was calculated for the 6 and 12 month follow ups.

Percent days abstinent from alcohol.

At each follow up assessment, patients indicated how many days they consumed alcohol during the period of time referenced by the assessment. This number was then subtracted from the total number of days in the assessment period to get the number of abstinent days from alcohol. This number was then divided by the total number of days in the period and multiplied by 100 to obtain the percent days abstinent from alcohol (PDA). PDA from alcohol at the 6- and 12-month follow-ups took into account the number of abstinent days reported at previously completed evaluations. Though it was possible to calculate PDA for each illicit drug, we could not calculate overall PDA from both alcohol and drugs because use days for different substances often overlapped. Therefore, given that 87% of patients were alcohol dependent, we focused our analyses on PDA from alcohol.

2.3. Analysis strategy

The first set of analyses examined module usage within the MORE™ program, as indicated by the percentage of patients accessing a given module at least one time. We also created a variable that represented the overall level of module use for each participant.

In the second series of analyses, we classified participants into one of two groups based on module usage. The first group was called the “highly adherent” group and consisted of participants who accessed at least 6 of the 7 program modules at least once during the year following discharge. The second group was called the “less adherent” group and consisted of participants who accessed 5 or fewer modules in the year following discharge. A cut off of 5 modules was chosen for several reasons. Dividing the sample using 7 modules as the criterion resulted in a very small number of highly adherent users (less than 6% of patients). However, it was still deemed clinically important to use a fairly stringent definition of highly adherent use, as the MORE™ program was designed to deliver 6 of the 7 modules within 1 year of leaving treatment. Highly adherent and less adherent module users were compared on substance use outcomes at 6 and 12 months postreatment via a series of bivariate tests.

Generalized estimating equations (GEE) and mixed-model regression analyses were then conducted to further examine the relationship between module use and substance use outcomes at 6 and 12 months post-discharge. GEE and mixed effects models have advantages over conventional repeated measures methods in that they allow for missing observations, accommodate measurements made at different time points, provide greater flexibility in modeling the variance-covariance matrix, and permit the estimation of both group and random subject-specific effects (Diggle et al., 2002). The current analysis assumes a compound symmetry structure as we only have two levels of the repeated measures factor (i.e., month 6 and month12). The compound symmetry assumption can only be violated when the repeated measures factor has more than two levels. A separate model was produced for each of two outcomes: the % of patients who were continuously abstinent from alcohol and drugs at follow up and the percent of days abstinent from alcohol at follow up. The number of modules accessed and the time of follow up (6 vs. 12 months) were entered as predictors. In addition, all other variables listed in the Measures section were included in the models as covariates. Even though there was a substantial number of missing assessments, we ran the models on the existing data and did not impute missing data. Continuous abstinence status was only calculated if the appropriate substance use data were available; we did not count participants who were lost to follow up as relapsed.

As a last step, the regression analyses outlined above were conducted only on patients who were deemed to be in compliance with continuing care recommendations. These analyses explored the possibility that patients accessing a large number of MORE™ modules may also be engaging in other prescribed recovery behaviors, such as following through with their continuing care plans and engaging with 12 step fellowships. These patients would be expected to have better substance use outcomes, therefore making it unclear whether outcomes are due to compliance in general rather than to module use specifically. To address this possibility, we identified a subsample of compliant patients and reran the regression models on this group. A patient was labeled compliant if he or she met all of the following criteria at the time of the 1 month follow up: 1). reported following most, some or a few of their continuing care plan recommendations; 2). had an AA composite score of 2 or greater; 3). attended at least 2 AA or 12 step meetings a week.

3. Results

3.1. Module engagement

Table 3 shows the percentage of patients in the sample who accessed each of the 7 program modules. As expected, engagement was highest for Module 1, which was accessed while patients were in treatment. Eighty-four percent (n = 943) of discharges accessed this module at least once. It is important to note that although Module 1 was available to all patients and they were encouraged by clinical staff to log onto the module, some patients opted to not log on. All patients received an introduction to MORE and were briefly trained on how to use the program, but logging onto Module 1 was not required as a part of training. It is unknown why some patients opted to not log onto Module 1 while in treatment. Another key finding was that module engagement steadily decreased over time (post-discharge), with 25% (n = 285) accessing Module 3 and 5% (n = 58) accessing Module 7 (the final module).

Table 3.

Percent of participants accessing each MORE™ program module at least one time.

Module	# of days since discharge	% of May 2006 – December 2007 discharges
1	0	84
2	31-60	38
3	61-90	25
4	91-166	19
5	167-242	12
6	243-365	9
7	366-545	5

Another way to examine module engagement is to sum up the number of modules accessed by each patient and determine how many patients accessed a given number of modules. The mean number of modules accessed was 1.75 (SD = 2.09). Forty four percent (n = 489) of the sample accessed module 1 only (i.e., they did not access any modules beyond module 1) and 5% (n = 58) of patients accessed all 7 modules.

3.2. Substance use outcomes of highly adherent vs. less adherent module users: results of bivariate tests

Regarding continuous abstinence rates, highly adherent users had significantly higher rates than less adherent users at both 6 (81% vs. 54%; p < .001) and 12 months (78% vs. 49%; p < .001). Regarding PDA from alcohol, highly adherent users had significantly higher PDA than less adherent users at both 6 (99%; n = 102 vs. 95%; n = 747, p < .01) and 12 months (99%; n = 94; vs. 95%; n = 584, p < .01).

Another way to examine the relationship between module use and outcomes is to identify patients who accessed one or more modules after treatment discharge and compare their outcomes to patients who do not access any modules after treatment. Because module 1 was the only module accessed during the treatment stay; post-treatment module use was defined as access of module 2 or beyond. Forty percent (n = 454) of the sample had post-treatment module use; the remaining 60% (n = 668) only accessed module 1 (or no modules at all).

Regarding continuous abstinence at 6 months, the post-treatment module use group had a significantly higher abstinence rate (62%; n = 239) than the group that did not access the modules after treatment (53%; n = 248), p < .01. For continuous abstinence at 12 months, the difference between the two groups approached significance (56% vs. 49%, p = .08). For PDA from alcohol at 6 months, the post-treatment use group was significantly higher (97%; n = 381) than the other group (94%; n = 468), p < .01. The difference between the 2 groups for PDA at 12 months was also significant (97% vs. 95%), p < .05.

3.3. Mixed-model regression analyses

Mixed model regression analyses examined the relationship between number of modules accessed and two outcome variables: (1) continuous abstinence (binary) and (2) percent days abstinent (PDA) from alcohol (continuous). The outcome measures were assessed at month 6 and month 12. Generalized estimating equations (GEE; Diggle, Heagerty, Liang, and Zeger, 2002) analyses were used for the binary outcome, and linear mixed effects models (Littell, Milliken, Stroup, & Wolfinger, 1996) were used for the continuous outcome. The models included terms for total number of modules accessed, time of follow up (6 vs. 12 months) and the module by time interaction and specified an exchangeable (GEE) or compound symmetry (linear mixed effects) covariance structure. In addition, the models included the following covariates: self efficacy and motivation at the module 1 assessment, attended formal alcohol/drug treatment during the 1 month follow up, total number of people in the support network, number of pretreatment drinking days, the AA composite score at the 1 month follow up, and presence of a co-occurring disorder. A total of 1,122 participants were included in the analyses.

3.3.1. Results of generalized estimating equations for continuous abstinence

The GEE analysis revealed a significant effect of the number of modules accessed, χ² (1) = 18.17, p < .001, with the likelihood of abstinence increasing as the number of modules increased (OR = 1.18, 95% CI = 1.10 – 1.28). In addition, there was a significant effect of time, χ² (1) = 17.07, p < .001, with higher rates of abstinence in month 6 than month 12 (OR = 1.50, 95% CI = 1.25 – 1.80). The interaction did not approach statistical significance, p = .24. Significant covariates in the model included self efficacy, χ² (1) = 6.44 p = .01, and co-occurring disorder, χ² (1) = 5.81, p = .01, with a higher likelihood of abstinence associated with increases in self efficacy (OR = 1.31, 95% CI = 1.07 – 1.61) and a higher likelihood of not having a co-occurring disorder (OR = 1.52, 95% CI = 1.08 – 2.13).

3.3.2. Results of mixed effect models for PDA from alcohol

The results from the mixed effects model for PDA indicated that the number of modules accessed was a significant predictor of PDA, F(1, 602) = 10.25, p = .001. Time and the module*time interaction were not statistically significant (p < .70 and p < .35 respectively). Significant covariates in the model included self efficacy, F(1, 602) = 4.02, p = .045, number of people in the support network, F(1, 602) = 4.44, p = .035, and number of pre-treatment drinking days, F(1, 602) = 18.32, p < .001. Self efficacy and number in the support network were positively related to PDA from alcohol and number of pre-treatment drinking days was negatively related to PDA from alcohol.

3.4. Regression analyses of compliant patients

The same regression models reported above were conducted on 483 patients who met the definition of being compliant (as defined in the Methods section) with one modification: if a predictor in the original model was one of the variables used to determine compliance, it was dropped from the compliant patient model.

These analyses were conducted to determine if the number of modules accessed continued to predict 6 and 12 month outcomes in the compliant patient models. The GEE for continuous abstinence revealed a significant effect of modules accessed, χ² (1) = 6.38, p = .01, with the likelihood of abstinence increasing with the number of modules (OR = 1.16, 95% CI = 1.04-1.29). There was also a significant effect of time, χ² (1) = 10.09, p = .001, with a greater likelihood of abstinence at month 6 than at month 12 (OR = 1.57, 95% CI = 1.21 – 2.03). The module by time interaction was not statistically significant (p < .27). The mixed effects model for PDA from alcohol indicated the number of modules accessed was a significant predictor of PDA, F(1, 324) = 4.39, p = .04. There was also a significant effect of time, F(1, 211) = 4.61, p = .03, and the module*time interaction approached significance F(1, 211) = 3.56, p = .06. In addition, the covariate of pre-treatment drinking days was a significant predictor, F(1, 324) = 8.61, p = .004 with a lower number of pre-treatment drinking days associated with a higher PDA.

3.5. Satisfaction with the MORE program

On each follow up survey several questions asked patients who indicated using the MORE program after treatment discharge to rate their satisfaction with various aspects of the program. Of the 454 patients (40% of the sample) who were using the program during the follow up period, over 70% rated the program as very good or excellent. Over 80% stated they agreed or strongly agreed with the statement “The content of the MORE program has been relevant to my recovery” and 64% stated they agreed or strongly agreed with the statement “I have begun working a recovery program because of MORE”.

4. Discussion

The present analysis examined usage of the MORE™ program, an innovative, computerized recovery support program for patients who recently completed residential treatment for alcohol or drug dependence. The program is available during the first 18 months after discharge from treatment. The first component of the program involves a series of modules and other resources and activities delivered in a mixed media format over the computer. In addition, each patient has contact with a recovery coach, a licensed drug and alcohol counselor. The results of the present analysis suggest a positive relationship between use of the program modules and post-treatment outcomes. Patients who accessed a large number of modules during the 12 month follow up period were significantly more likely to be abstinent and had a higher percent of days abstinent from alcohol than patients who accessed few or no modules. In addition, regression analyses revealed number of modules accessed was a significant predictor of substance use outcomes during the 12 months following treatment, even when accounting for other covariates such as motivation, self efficacy and pretreatment substance use.

Though a relationship was found between MORE™ use and outcomes, it is unclear whether the relationship is causal. It is possible that patients who are very compliant with all of their continuing care recommendations (only one of which is using MORE™) have better outcomes than noncompliant patients. Overall, the regression analyses conducted on “compliant” patients revealed an effect for MORE program use that was about the same magnitude as that observed in the full sample, but findings should be considered preliminary until they are replicated with a stronger design such as a randomized controlled trial. It is also important to note that this analysis was based on administrative data collected in the course of standard clinical operations. A randomized controlled trial comparing outcomes of MORE™ participants to outcomes of participants not using the program is the only way to determine program efficacy.

This analysis also revealed that a substantial number of patients were not using the program as recommended. Over 80% of patients accessed Module 1 at least once, but this is not surprising given that patients access this module while still in treatment. In contrast, less than half of patients accessed Module 2, which is available roughly 1 month following discharge, with access decreasing steadily across subsequent modules. However, it is important to note that many patients did use MORE™ after leaving treatment and a small number of these patients accessed at least 5 of the 7 modules, indicating some long term use of the program during the year following treatment. Regarding satisfaction with the MORE program, the majority of patients rated the overall quality of the program as excellent, though 30% of patients did not give it a high rating. This result combined with the relatively low levels of engagement suggests that modifications to implementation efforts or to the program itself may be in order. Future research should focus on increasing the acceptability and feasibility of the program. Methods to increase engagement, such as providing patients with incentives, incorporating module use directly into the residential treatment plan, or increasing the frequency and duration of contacts between each patient and his or her recovery coach, should be further explored.

It is important to note that low levels of patient engagement are not unique to MORE™ and several studies have reported low engagement with other web-based addiction programs. Strecher et al. (2008) examined a behavioral web-based smoking cessation program in two samples of HMO members who were attempting to quit smoking. Similar to MORE, the program consisted of distinct sections available over a period of time that delivered tailored messaging and feedback to participants based on their responses. On average participants accessed 2.5 sections during the time period. These numbers are similar to the mean number of program modules accessed in the present study, which was 1.97 for May 2006 – April 2007 discharges and 2.09 for May – December 2007 discharges. Another study examined Quit Smoking Now, a web-based program providing smoking cessation information and guidance on behavior change strategies (McKay, Danaher, Seeley, Lichtenstein & Gau, 2008). The average number of program visits during a 6 month period was 2.14 and over 90% of participants had completely stopped using the program by 6 months post-enrollment. In summary, low levels of patient engagement with web-based, computerized health behavior change programs continue to present clinical and methodological challenges.

Though no published studies to date have examined computerized continuing care programs, a large number of studies have examined patient engagement with more traditional programs. Similar to the engagement results with MORE reported here, these studies indicate that client engagement and retention are an issue with traditional continuing care programs. Cacciola et al. (2008) conducted a study of a phone-based continuing care protocol recently implemented at the Betty Ford Center. This program, called Focused Continuing Care, is offered to all patients who have completed residential treatment and involves several brief telephone-counseling sessions between patients and clinical staff. These calls are scheduled to take place at predetermined time periods after discharge from treatment. A key finding was that although 71% of patients in the sample completed the first call, only 28% completed the last call (which took place roughly a year after discharge). By 6 months post-discharge, less than 40% of patients completed any given call and less than 1% completed all 14 calls of the program. The 1% call completion rate is similar to the module completion rate found for the MORE program, which was 5%. Lash et al. (2007) conducted a randomized clinical trial to determine the effectiveness of a continuing care program for patients attending residential treatment through the VA. The program consisted of 3 processes: 1). clients signed a written contract prior to leaving treatment whereby they agreed to attend continuing care sessions, 2). clients received attendance prompts prior to each scheduled session, and 3). clients were positively reinforced for attendance. Even when patients were explicitly reminded to attend their continuing care sessions and were positively reinforced for doing so, attendance decreased substantially over time, with only 40% attending sessions on a monthly basis throughout the follow up period. These studies as a whole suggest a number of challenges in keeping patients engaged in continuing care programs over time.

A large number of studies have also examined the extent to which traditional continuing care programs enhance treatment outcomes. Similar to our finding that patients who were highly engaged with MORE had better substance use outcomes, several studies have found continuing care programs to be effective for patients who engage with these programs. It is important to note that these studies provide much stronger support than the present study because many were randomized controlled trials that compared patients receiving the program to a control group of patients who did not receive the program. In the study by Lash et al. (2007) described above, clients who participated in the continuing care program were significantly more likely to be abstinent 1 year following treatment than clients in the “treatment as usual” condition. McKay (2005) examined clients attending a 4 week intensive outpatient program through either the VA or their community. Clients in the continuing care condition received brief telephone counseling sessions combined with several group support sessions; these patients were compared to those who received relapse prevention or 12-step oriented group counseling. Continuing care clients had significantly higher abstinence rates in the two years following treatment than the other 2 groups. Several other studies have reported similar findings (for a review, see McKay, 2009).

Several limitations should be considered when interpreting the administrative data reported here. Though the follow up rate at 6 months was 70%, only 54% of the sample was contacted at12 months. The low endpoint follow-up rate raises concerns about potential sample bias. Another issue pertaining to generalizability concerns findings for percent days abstinent from alcohol during the 6 and 12 month follow up periods. PDA was extremely high (i.e., greater than 93%) even at 12 months. These data suggest that patients who are relapsing to alcohol use are not doing so for a long period of time. Potential explanations for this include underreporting of substance use or that a disproportionate number of patients with highly favorable outcomes were reached at follow up. In any case, the relative lack of variability in this outcome reduced the likelihood of finding significant relationships between module use and frequency of abstinent days.

The analyses reported here focused on one aspect of MORE™ usage, which was the total number of modules accessed by each participant. This measure was the only one available at the time, so we could not report other measures such as number of program logins or number of visits to components other than the modules. On a further methodological note, the measure of self efficacy used here differed somewhat from measures used in other studies. Self efficacy is typically referred to as confidence in one’s “ability to stay sober”; in the MORE™ program it is phrased in terms of confidence in one’s “ability to work a recovery program”.

Despite limitations inherent with administrative data sets, these analyses have several strengths. This is the first evaluation we are aware of to examine usage of a unique and innovative web-based continuing care program for patients with alcohol or drug dependence. The results provide support for a positive relationship between use of the MORE™ program and substance use outcomes after treatment and suggest that computerized programs may hold promise as continuing care interventions. This study also utilized both bivariate tests and mixed model regression analyses to explore the relationship between program use and outcomes, and conducted analyses on compliant patients to account for the possibility that the relationship between MORE use and outcomes merely reflected a relationship between outcomes and compliant behavior in general. Future evaluations will further examine how usage impacts outcomes and how program engagement can be increased.