A Web-Based Platform to Support an Evidence-Based Mental Health Intervention: Lessons From the CBITS Web Site

Abstract

Objective

To explore the role of Web-based platforms in behavioral health, the study examined usage of a Web site for supporting training and implementation of an evidence-based intervention.

Methods

Using data from an online registration survey and Google Analytics, the investigators examined user characteristics and Web site utilization.

Results

Site engagement was substantial across user groups. Visit duration differed by registrants’ characteristics. Less experienced clinicians spent more time on the Web site. The training section accounted for most page views across user groups. Individuals previously trained in the Cognitive-Behavioral Intervention for Trauma in Schools intervention viewed more implementation assistance and online community pages than did other user groups.

Conclusions

Web-based platforms have the potential to support training and implementation of evidence-based interventions for clinicians of varying levels of experience and may facilitate more rapid dissemination. Web-based platforms may be promising for trauma-related interventions, because training and implementation support should be readily available after a traumatic event.

The tremendous progress in developing evidence-based practices to improve mental health care quality has not been matched by successful implementation of such interventions in community settings (1). Few clinicians enter the workforce trained to implement evidence-based practices (2), and training the current workforce is challenged by limited infrastructure, funding, and support (3). Moreover, the “train and hope” (4) approach (training clinicians without continued implementation support) has resulted in low use of evidence-based practices among community clinicians (3,5). The demand for effective mental health services will likely increase substantially because of recent policies such as the Affordable Care Act (ACA). However, the existing mental health workforce and training infrastructure is unlikely to meet this increased demand, requiring new mechanisms to expand and enhance successful training in and implementation of evidence-based practices (1).

Many clinical fields have increased the use of distance learning via Web sites to enhance workforce skills (6). Recently, there has been a comparable increase in behavioral health care, with Web-based training for substance abuse treatment (7), dialectical behavior therapy (8), and cognitive-behavioral therapy (9). Such efforts have been demonstrated to be as effective as in-person training in improving professionals’ knowledge and skills (7,8,10). However, little is known about who uses such Web sites or how users interact with them.

To increase understanding of the rapidly growing area of distance learning in behavioral health, we examined user characteristics and usage of a Web site that supports training and implementation of an evidence-based intervention for traumatized youths. Findings may inform future efforts to enhance training and implementation support through such Web sites and may begin to inform how Web-based platforms can be used to rapidly disseminate knowledge after traumatic events.

Methods

The Cognitive Behavioral Intervention for Trauma in Schools (CBITS) Web site (cbitsprogram.org) is an online training and implementation support Web site for CBITS, a widely recognized (11), effective treatment for trauma-exposed children (12). The Web site was developed in response to high demand for CBITS training and requests for ongoing implementation support. The site comprises a “public” side that contains general information about CBITS, biographies of the intervention developers, and answers to frequently asked questions and a “private” side that provides registrants with a training course, implementation assistance resources, and a community forum.

To better understand use of the CBITS Web site, we used Google Analytics to examine Web site utilization among all registrants between July 1, 2011, and June 30, 2012 (N=1,406). Average site visit duration, or the average length of time a user spends on the Web site per visit, was used to evaluate Web site engagement. Page views, defined as the total number of pages a registrant viewed during all Web site visits, was the measure used to examine usage of the three sections of the site. Web site user characteristics, including demographic information, training background, clinical experience, purpose for visiting the site, and prior participation in CBITS training, were obtained from the Web site registration survey.

Using SPSS version 21.0, we described registrants’ characteristics and calculated overall Web site engagement and usage of Web site sections. We conducted bivariate analyses, using t tests and one-way analysis of variance (ANOVA), as appropriate, to examine the relationship between user characteristics, Web site engagement, and content usage. The RAND Corporation’s institutional review board (IRB) exempted the study from IRB review.

Results

Registrants were predominantly female (86%; N=1,186 of 1,374) and Caucasian (68%; N=959 of 1,406) and had a master’s degree (68%, N=939 of 1,376) and had not been trained in CBITS (79%; N=1,085 of 1,374). [Further information about user characteristics is available in an online data supplement to this report.] Registrants averaged 13:59 minutes per visit, with registrants not previously trained in CBITS spending significantly longer on the Web site each visit than previously trained registrants (15:11 versus 10:08 minutes, p<.001; Table 1). Registrants with less than four years of CBT experience had significantly longer visits than registrants with four to six years of CBT experience (16:07 versus 11:11 minutes, p<.01), as did registrants with four to seven years of clinical experience compared with those with eight or more years of experience (16:53 versus 11:05 minutes, p<.001). Registrants whose reason for visiting was to work toward becoming a CBITS clinician spent significantly more time per visit than registrants who did not visit the site for this reason (15:29 versus 12:38 minutes, p=.009). We found no other significant differences in visit duration when examining registrants’ purpose for visiting the Web site, including administrators who indicated when registering that they were visiting the site to better understand CBITS.

Table 1.

CBITS Web site usage, by user characteristics, July 2011 through June 2012^a

				Number of pages viewed by section
	Average visit duration (minutes:seconds)			Course			Implementation assistance			Online community
Characteristic or purpose	M	SD	p	M	SD	p	M	SD	p	M	SD	p
Total sample	13:59	20:09		21.94	44.81		6.90	16.49		.71	2.84
Prior training in CBITS			<.001			.001			<.001			<.001
No	15:11	21:51		24.26	47.24		6.16	15.11		.50	2.33
Yes	10:08	11:42		14.05	34.67		10.25	21.18		1.44	4.25
Experience with CBTb			.002			<.001			.730			.835
None	14:48	20.07		13.94	43.44		6.02	10.62		.66	2.66
In school	15:18	19.37		27.14	48.49		7.60	16.97		.66	2.66
In practice
1–3 years	16:07	25:04		27.28	49.97		7.35	17.50		.82	3.33
4–6 years	11:11	14:59		16.15	38.90		6.51	15.53		.54	2.25
>6 years	10:30	15:06		12.43	30.58		6.10	16.87		.79	2.86
Years of mental health experience			<.001			<.001			.694			.635
0	16:25	19:52		21.71	40.77		6.38	15.18		.58	2.23
<1	16:57	20:01		30.53	44.48		8.84	18.86		.42	1.28
1–3	15:37	21:19		29.68	51.43		7.77	17.24		.88	3.06
4–7	16:53	26:57		25.47	49.09		6.75	13.76		.74	2.77
≥8	11:05	15:19		16.30	41.18		6.80	17.84		.77	3.27
Purpose of visit
Become CBITS clinician			.009			<.001			.963			.451
Yes	15:29	22:20		26.79	49.88		6.91	15.60		.65	2.71
No	12.38	17:54		17.63	39.29		6.95	17.25		.76	3.00
Support work as clinician			.578			.364			.089			.063
Yes	13:30	18:33		23.87	49.88		8.25	18.20		.96	3.19
No	14:09	20:43		21.24	42.81		6.45	15.80		.62	2.70
Supervise clinicians			.997			.824			.088			.081
Yes	13:58	18:36		22.68	48.69		9.43	21.86		1.22	4.74
No	13:59	20:23		21.83	44.21		6.56	15.50		.63	2.50
Understand CBITS as administrator			.153			.006			.112			.686
Yes	12:28	18:04		15.52	39.47		5.59	14.40		.64	3.28
No	14:19	20:35		23.36	45.80		7.22	16.90		.72	2.73
Support CBITS implementation			.315			.805			.121			.277
Yes	13:23	17:39		21.61	45.63		7.70	17.00		.80	3.09
No	14:27	21:56		22.21	44.16		6.30	16.05		.63	2.67

^aCBITS, Cognitive Behavioral Intervention for Trauma in Schools

^bCognitive-behavioral therapy

Registrants’ interaction with Web site content, as assessed by page views, was greatest for the training course (mean of 21.94 pages viewed), followed by the implementation assistance section (6.90) and the online community forum (.71). Registrants without prior training in CBITS viewed significantly more pages in the course section than those with prior training (24.26 versus 14.05 pages, p<.001), as did registrants with less than four years of CBT experience compared with registrants with four to six years of experience (27.28 versus 16.15 pages; p<.001). Clinicians with less than eight years of experience as a mental health clinician viewed significantly more pages compared with those with eight or more years of experience (16.30 versus 25.47 pages; p<.001). Individuals whose purpose for visiting was to become a CBITS clinician viewed significantly more pages in the course section than those who did not indicate this as a reason for visiting (26.79 versus 17.63 pages, p<.001). Registrants whose purpose for visiting was to understand CBITS as an administrator viewed significantly fewer pages in the course section than did those who did not indicate this as a reason for visiting (15.52 versus 23.36 pages, p=.006). In the implementation assistance and online community sections, those previously trained in CBITS viewed more pages in both the implementation assistance section (10.25 versus 6.16 pages; p<.001) and the online community section (1.44 versus .50 pages visited; p<.001).

Discussion

Evidence-based practices can “languish” 15–20 years before being routinely used (13), but Web sites may facilitate more rapid dissemination. Our examination may begin to inform how Web sites can be used to support training and implementation of evidence-based practices. We found high levels of Web site engagement overall, with the training section getting the most usage across all user types. The substantial use by untrained individuals was unsurprising, given that many clinicians enter the workforce untrained in effective treatments (2). With the need to increase the availability of training in effective interventions (14) and findings showing that distance learning is an effective and cost-efficient approach to enhancing clinical skills (7), initial and repeated exposure to course material through Web sites is one potentially scalable approach to addressing this implementation barrier.

Although not attracting as much traffic as the training course, the implementation support section had substantial usage. The greater use of this section, particularly among individuals previously trained in CBITS, suggests that once trained, clinicians may seek information designed to support implementation. The availability of implementation support materials is likely to result in more successful implementation because ongoing implementation support is essential for establishing evidence-based practices within a system (3).

There was relatively little use of the online community section, likely resulting from a lack of Web site prompts and new material, features that bring individuals back to Web sites on an ongoing basis (15). Research is needed to identify how online communities can enhance implementation and provide real-time feedback regarding implementation barriers or clinical situations as they arise.

Although little is known about how a nonclinical audience interacts with Web-based platforms primarily designed to train clinicians in evidence-based interventions, our findings reveal that administrators had high rates of engagement with the CBITS Web site, with visit duration times comparable with those of clinicians. However, compared with their clinical counterparts, administrators viewed fewer pages in the course section. Implementation is facilitated when administrators are committed to providing evidence-based practices and are knowledgeable about their use (5). These findings may help us understand how Web sites can be used to facilitate administrator support for the intervention. More broadly, research clarifying the needs of specific user groups is necessary to understand how Web-based platforms can more effectively support each user type.

Our study had several limitations. We had no information on whether Web site usage enhanced adoption, implementation, or sustainability of CBITS. Our study did not assess differences in the quality of online versus in-person CBITS training, nor differences in knowledge and skills gained by clinicians trained online versus in person. However, prior research would likely indicate that they are relatively equivalent (7,8,10). Finally, Web analytics data were gathered with a combination of hypertext transfer protocol access logs, sessions, persistent cookies, and Google Analytics user tracking tools. How information obtained through Google Analytics differs from self-reported usage is not known.

Conclusions

Our findings suggest that Web-based platforms have the potential to support the training and implementation of evidence-based practices for clinicians with varying levels of experience and familiarity with the intervention. These findings are particularly relevant at a time when there are greater incentives to disseminate evidence-based practices and a need to train and support a growing mental health workforce. In addition, Web-based platforms may be promising for supporting trauma-related interventions, for which training in or access to implementation assistance should be available rapidly after a traumatic event.