Can Community Clinicians Learn an Evidence-based Psychotherapy on the Web? Pilot Study of an E-Learning Training and Implementation Intervention

Abstract

Objective

A feasibility assessment of on-line training coupled with an on-line learning collaborative to support implementation of evidence-based psychosocial treatments in community mental health systems.

Methods

Five community mental health centers were randomly allocated to receive face-to-face training with local supervision or on-line training plus a learning collaborative supported by expert clinicians. Study participants (n=36) were clinicians interested in learning interpersonal and social rhythm therapy (IPSRT), an evidence-based psychotherapy for individuals with bipolar disorder. After training, participants treated patients (n=136), who reported monthly on the treatment strategies clinicians used.

Results

Clinicians from both clinic groups increased use of IPSRT-specific strategies. Patients of clinicians who received on-line training and patients of those who received face-to-face training reported comparable use of IPSRT techniques by their clinicians.

Conclusions

On-line training of community clinicians is feasible and produces uptake of an evidence-based psychotherapy comparable to uptake by clinicians who participated in face-to-face training.

Many fields facing demands to improve professionals’ skills have increased e-learning use. Indeed, web-based approaches may soon comprise 50% of continuing education across all professions.⁽¹⁾ Improved technology has made accessing training programs faster, lowered costs, and enabled greater participant interaction.

Given the mental health field’s challenges in effectively disseminating and implementing evidence-based practices,⁽²⁾ it is unsurprising that the field has turned to e-learning to cost-effectively train clinicians.⁽³⁾ Evidence-based psychotherapies are often unavailable in community settings, few clinicians enter the workforce trained to implement them,⁽⁴⁾ and therapy developers have limited training capacity. Studies suggest that e-learning programs and in-person training have comparable effects on clinicians’ evidence-based practice knowledge;^{(3, 5)} however, there is a little information about whether e-learning can enhance clinician use of these skills.

To enhance our understanding of how e-learning might affect clinician use of evidence-based psychotherapy, we report on a pilot study of clinicians’ use of Interpersonal and Social Rhythm Therapy (IPSRT), an evidence-based psychotherapy for bipolar disorder, after an e-learning program. We compare patient-reported clinician use of IPSRT techniques by clinicians who participated in a two-day in-person training with use by clinicians who participated in an on-line IPSRT training course supported by a Learning Collaborative.

Methods

We randomly assigned five community mental health outpatient clinics to an intervention arm (3 clinics; e-learning), or a comparison arm (2 clinics, in-person training). We invited all clinicians at the clinics to participate in IPSRT training as part of a study examining alternative approaches to implementing IPSRT. Ninety-eight percent (n=36) of the clinicians agreed to participate, including social workers (53%), licensed professional counselors (42%), clinical psychologists (6%), and nurses (3%). The clinicians averaged 12.9 (sd=9.9) years’ experience as therapists. The clinics were not routinely implementing any evidence-based treatment for bipolar disorder; however, many clinicians had previously been independently trained in other evidence-based psychotherapies, such as cognitive-behavioral therapy.⁽⁶⁾

We invited clinicians and clinical supervisors in the e-learning arm to receive IPSRT training, at their convenience, via an interactive 12-hour on-line program. We considered clinicians to have finished the training when they completed all training modules and passed a final exam. For the first 3–6 months after starting to use IPSRT with clients, the majority of e-learning clinicians also participated monthly in a 60-minute group telephone supervision with an IPSRT expert.

Each e-learning clinic also formed a 3–4 person implementation team consisting of both clinicians and administrators. These small teams participated in a Learning Collaborative comprised of all e-learning clinics. The Learning Collaborative included the fundamental parts of collaborative quality improvement efforts,⁽⁷⁾ as conveyed through Webinars conducted by research team faculty. Implementation teams set aims for their clinics related to implementing IPSRT, made changes designed to achieve aims, and chose simple metrics to determine if changes helped to achieve the aims. Implementation teams collected data for their chosen metrics (e.g. percent of individuals with bipolar disorder who were receiving IPSRT) and shared results with the other teams.

Clinicians and clinical supervisors from comparison condition clinics participated in a traditional, 2-day 12-hour workshop, conducted by IPSRT developers. Clinicians also received on-site supervision of IPSRT cases as part of local routine weekly supervision. They were encouraged to contact IPRST experts by phone with questions. The University of Pittsburgh and RAND Corporation IRBs approved the study.

We subsequently obtained information about clinician use of various therapeutic techniques from 136 individuals with bipolar disorder receiving treatment from IPSRT-trained clinicians (mean=3.8 patients/clinician). Patients completed the Psychotherapy Practice Scale-IPSRT version (PPS-IPSRT) adaptation of the Psychotherapy Practice Scale (PPS) on a monthly basis following therapy sessions with IPSRT-trained clinicians. The PPS is a validated measure of clinician use of core psychotherapy elements.⁽⁸⁾ Patients rate the frequency of therapist techniques using a Likert scale ranging from 1 (never) to 4 (very often). For each IPSRT-trained clinician, we obtained an average of 13.0 ± 15.9 therapy session reports from patients completing the PPS-IPSRT (mean=4.1 ± 3.8 session reports completed per patient.)

The IPSRT scale includes subscales for interpersonal therapy (IPT) (14 items) and social rhythm therapy (SRT) (5 items). Subscale internal consistency was computed using Cronbach’s alpha. (Coefficients IPSRT=92.1, IPT=90.7, SRT=82.8). Our outcome variable, use of IPSRT, was the percentage of 19 IPSRT techniques that patients reported clinicians using “often” or “very often” in a session.

To characterize clinician use of IPSRT techniques following training, we compared mean scale scores of patient-reported clinician behavior by arm. To investigate differences in mean scale scores over time, we estimated multi-variable models of clinician behavior by arm using time elapsed since a clinician completed IPSRT training. Because clinicians completed IPSRT training at different times, we estimated these models as a function of IPSRT training method, time from IPSRT training completion to patient report (zero to 180 days, 181 to 365 days, more than 365 days), and an interaction term of training method and time since completing training. Models included patient level random effects via random intercepts. Using the Breusch and Pagan Lagrangian multiplier test for random effects, we found significant effects for all models (Prob > chibar2 <0.001).

Results

Sixteen clinicians were trained in e-learning clinics and 20 in in-person training clinics. In the first 6 months following IPSRT training, patients reported that trained clinicians were using 61% of the IPSRT techniques often/very often. Use of IPSRT techniques increased over time. Patients reported that in the second 6 months after training (days 181–365) clinicians were using 66% of the techniques often/very often. Twelve months after training, clinicians were using 71% of the IPSRT techniques often/very often, a significantly greater use of IPSRT techniques than immediately after training. (p<0.01) Clinician use of IPSRT techniques was comparable for the in-person training and e-learning groups in all 3 time periods. (Fig 1) In the multivariate regression, we found no significant difference in use of IPSRT techniques between the e-learning and in-person training groups, overall or in any time period.

Figure 1.

Use of IPSRT Techniques by Arm

Techniques such as exploring ways to improve relationships and how moods might be related to those relationships were among those most commonly used by clinicians; techniques such as keeping a record of sleep, wake, and meal times, and a schedule of activities were among the least commonly used. A greater percentage of IPT techniques were used in session often/very often in the first 6 months (64%), second 6 months (69%), and more than a year (75%) after training compared with SRT techniques used often/very often in sessions the first 6 months (49%), second 6 months (56%), and more than a year (59%) after training. Both IPT (p<0.01) and SRT (p<0.01) techniques were used substantially more often after a year than in the 6 months immediately after training. Across the entire post-training period, patients reported that clinicians were more likely to use IPT techniques (71% of the techniques were used often/very often), compared to SRT techniques (56% used often/very often)

Discussion

In this pilot study, patients reported increasing use of IPSRT techniques over time among trained clinicians, with no statistically significant differences between clinicians receiving online vs. in-person training. This finding suggests that relatively efficient training and implementation support efforts can significantly change mental health clinician behavior.

Efficiently disseminating evidence-based psychotherapies to the front line workforce is a key challenge facing the mental health field. Few clinical training program graduates are trained in evidence-based psychotherapies.⁽⁴⁾ Training barriers include high rates of workforce turnover,⁽⁹⁾ clinical demands preventing staff from attending in-person trainings, and poor access to intervention experts and other implementation supports.^{(10, 11)} Our finding that increased clinician use of IPSRT techniques was comparable between the e-learning and traditional training arms suggests ways to reduce some of these barriers. E-learning can provide a more scalable and efficient approach to training large numbers of clinicians, give busy community mental health clinicians greater flexibility in when they participate in training, and ensure newly hired clinicians can receive the same quality training as originally trained clinicians.

We also found that clinicians made less use of patient homework, consistent with what has been documented with cognitive behavioral therapy.⁽¹²⁾ Clinicians may find assigning patients homework challenging, as it deviates from commonly used in-session therapy techniques. Still, we observed a tendency for homework use to increase (albeit from a relatively low baseline) as clinicians became more comfortable with IPSRT, and further research is needed to understand how to enhance clinician and patient use of homework in evidence-based psychotherapies.

Learning Collaboratives can support the effective implementation of health care interventions to improve quality of care,⁽¹³⁾ and we found that clinicians in clinics that participated in the Learning Collaborative as part of the e-learning intervention increased IPSRT technique use comparable to clinicians who participated in in-person training. However, our design does not allow us to assess to what extent e-learning clinicians would have increased skills in the absence of a Learning Collaborative, nor to what extent different components of the Learning Collaborative contributed to enhanced use of IPSRT techniques.

This study has several limitations. The clinics are in a state with a relatively robust public mental health system; in addition, by agreeing to participate in the study, the organizations, clinics, and clinicians evidenced a heightened level of interest in using IPSRT. We believe IPSRT use would be lower in settings where interest was lower; we do not know to what extent our findings would generalize to clinicians in different treatment settings or regions. Nor do we know to what extent our findings would generalize to other psychotherapies; however, we note that the study’s www.ipsrt.org website and associated Learning Collaborative activities were modeled closely on other commonly used psychotherapy e-learning websites ⁽¹⁴⁾ and learning collaborative approaches.⁽¹⁵⁾

This study was not designed to test if the two training approaches were equivalent. However, the relatively tight confidence intervals around the differences at all 3 time points provide some reassurance that our negative finding does not stem primarily from a lack of statistical power. Further research is needed to test whether the two approaches produce comparable clinician use of specific therapeutic techniques, as well as whether they produce comparable patient outcomes.

We cannot assess the association between the training component approaches and increased use of IPSRT techniques or whether clinicians participating in e-learning would have increased skills in the absence of a Learning Collaborative. Nor do we know how variations in the amount or quality of supervision were associated with use of IPSRT techniques. We do not have information to assess if clinician use of the techniques improved patient outcomes. In the absence of videotaped recordings of sessions, we cannot be sure that patient reports of session content accurately reflect the interventions delivered. Finally, absence of an inactive comparison condition raises the possibility that increase in clinician use of techniques is unrelated to the training received.

Despite these limitations, the results of our pilot study could have compelling implications for the mental health field, which has long struggled to efficiently and effectively increase clinician use of evidence-based practices. Our finding of no significant difference between e-learning and in-person training relative to patient reports of IPSRT techniques is an important step towards this goal. Future studies must examine whether e-training of clinicians can achieve equivalent patient outcomes, and explore how emerging interactive technologies, such as virtual simulations and game-based role plays, might also be used to enhance clinician skills and patient outcomes.