Dissemination of an evidence-based treatment for cancer patients: training is the necessary first step

Abstract

Evidence-based psychological treatments (EBTs) for cancer patients have not been disseminated in part due to lack of available training. The biobehavioral intervention (BBI) is an EBT designed to alleviate cancer stress and enhance coping. The current study evaluates a training program and uses the Theory of Planned Behavior (TpB) to analyze factors related to intentions to implement BBI. Mental health providers (n = 62) attended a training for BBI. Attendees’ supervisors (n = 40) were later surveyed. Repeated measure ANOVAs assessed change over time in knowledge gains, attitudes towards EBTs/BBI, and self-efficacy. Linear multiple regression analyses assessed relationships between these factors and implementation intentions. BBI knowledge and attitude scores increased from pre- to post-training (ps < 0.01). Significant predictors in the final model were BBI-specific attitudes and self-efficacy (ps < 0.05). The BBI training program was an effective dissemination vehicle. Intervention-specific attitudes and self-efficacy were key factors in predicting providers’ implementation intentions.

INTRODUCTION

Thousands of studies conducted worldwide document the stress associated with cancer diagnosis and treatment. If left untreated, this early stress contributes to a consistently low quality of life, slower recovery, and less meaning in patients’ lives [1–4]. The needs of patients go unaddressed [5] despite the fact that 30 years of randomized clinical trials (RCTs) designed for cancer patients show that we can consistently reduce patients’ stress and enhance their moods [6–9], coping [7, 10], and quality of life [6]. Dissemination is defined as the targeted distribution of information and intervention materials to a specific public health or clinical practice audience to spread knowledge of the evidence-based intervention [11]. Implementation is defined as the use of strategies to adopt evidence-based interventions [11]. Unfortunately, few of these successful evidence-based treatments (EBTs) have been disseminated or implemented.

Continuing education (CE) is the primary vehicle for cancer care providers to learn about EBTs. To achieve the implementation of an EBT, CE programs must be successful in imparting knowledge and motivating attendees to adopt the new intervention techniques post-training. The most commonly reported metrics for the success of CE programs are providers’ knowledge of the EBT and evaluations of the training or instructors. Rarely included are measures of attitudes toward the EBT, self-efficacy (or belief in one’s capability to use the EBT), or providers’ intentions to use it post-training. Training programs that do report these outcomes demonstrate significant improvements from pre- to post-training in EBT knowledge, attitudes, self-efficacy, and intentions, with reporting occurring in programs of substantial variation in content and training modalities with both relatively small (<100) [12–14] and large (>300) [15] number of attendees. This limited data suggest that providers should come away from CEs with knowledge, positive attitudes toward the EBT, belief in their own capability to implement the new treatment (or self-efficacy), and the intention to implement [16].

While current studies examine changes over time in knowledge, attitudes, and self-efficacy, examinations of how these factors are related to providers’ intentions to use the EBT post-training have not been conducted. The Theory of Planned Behavior (TpB) [17] is a classic conceptualization for examining this process (see Fig. 1). As per the model, intentions are preceded by the individual’s attitudes and perceived behavioral control, defined as the perception of the ease or difficulty in performing the behavior and sub-divided into self-efficacy and controllability. Controllability refers to the extent to which the execution of the behavior is driven by internal (vs. external) factors. Indeed, barriers to implementation are considerations not just in theories of behavior but are central factors explored in dissemination and implementation (D&I) research frameworks (e.g., see RE-AIM model [18] for an example). Supervisors’ negative attitudes towards EBTs are cited as a significant barrier to D&I of EBTs [18] and can be conceptualized as a proxy for the degree of control of implementation. To our knowledge, assessment of supervisors’ attitudes has never been done in CE program effectiveness studies despite its importance. Data describing barriers for EBT implementation come from solicitation of providers in general rather than those who have recently attended a CE program [19]. In this context, knowledge should also be added as a variable in the TpB model as trainees must have at least a working knowledge of the EBT before implementation.

Fig. 1.

Theory of Planned Behavior model adapted to include knowledge, in addition to attitudes and perceived behavioral control, as a precedent of intentions to use an evidence-based treatment (EBT)

The TpB model is used to test the outcomes of a CE program for mental health professionals learning the concepts and strategies for conducting an empirically supported intervention for cancer patients. Data show the Biobehavioral Intervention (BBI) produced robust effects, making it a viable candidate for dissemination. A biobehavioral model for understanding the stress of cancer diagnosis and treatment and subsequent disease progression risk was proposed [20], and it informed the development of the manualized BBI. As manualized, the intensive portion of the intervention when delivered to groups is 18 weekly 90-min sessions (approximately 4 months), with an additional eight monthly 90-min sessions for the maintenance portion (see [21] for a detailed description). A randomized clinical trial (RCT: BBI plus assessment vs. assessment only) accrued consecutive patients (n = 227) with newly diagnosed regional (stage II or III) breast cancer. Tests of the efficacy [22–25] demonstrated significant reductions in distress and improvements in social adjustment, health behaviors (diet, smoking cessation), treatment adherence, health, and enhanced T cell immunity for participants in the BBI arm. Tests of the disease endpoint of the trial—recurrence—found that patients in the BBI arm had reduced risk of breast cancer recurrence [hazard ratio (HR) = 0.55, p = 0.034] [23]. In addition to its empirical support, the BBI has multiple characteristics suitable for dissemination, including incorporation of health behavior and adherence components (now termed a “multi-level” intervention [26]), ability to be delivered to individuals or groups, therapist and patient treatment manuals, and others.

Training institutes (described below) were offered for mental health professionals (trainees) to learn the BBI, renamed From Cancer to Health™: A Stress Management and Coping Intervention, and to study this dissemination. The BBI Training Institutes have three goals for mental health professionals (trainees): (1) to learn the conceptualization for the BBI and understand its empirical support; (2) to obtain knowledge, clinical skills, and confidence to use the BBI; and (3) to formulate a plan to implement BBI in their setting or practice. We anticipated that the trainees would evidence positive global evaluations of the Institute and education gains would be found on measures of knowledge and clinical skill. Additional data obtained from trainees and their supervisors operationalized constructs within the TpB model and tested the following hypotheses: (1) trainees’ attitudes toward and self-efficacy to deliver the BBI would improve and (2) knowledge, attitudes, and perceived behavioral control would predict intent to implement the BBI.

METHODS

Participants

Two types of participants were accrued for this study: trainees and their supervisors.

Trainees

Trainees (n = 62) were mental health professionals from social work (44 %), psychology (37 %), mental health nursing (5 %), or other specialties (10 %). The majority (89 %) were licensed, 12.5 years since licensure (median; range = 0–33), had spent 5 years in their current position (median; range = 0–27), and engaged in clinical service provision (median 80 %, range = 15–100 %). Employment sites were 42 % academic medical centers, 26 % community hospitals/centers, 11 % private practice, 10 % supportive care facilities, 8 % Veterans Affairs medical centers, or 3 % other. They were predominantly female (n = 54) and the racial/ethnic distribution was 88 % Caucasian, 7 % Asian, 3 % Latino, and 2 % African-American. Trainees came from 25 states and the District of Columbia with one international (Malaysia) attendee.

Information was obtained from the trainee about the characteristics of the patients they usually treated. As reported, the demographic characteristics of the patients they served were as follows: 75 % Caucasian (median; range = 0–100 %), 12.5 % African-American (median; range = 0–60 %), 9 % Latino (median; range = 0–90 %), 2 % Asian (median; range = 0–25 %), and 1.5 % other (median; range = 0–20 %); 36 % rural (median; range = 0–100 %) vs. urban, and 37 % low income (median; range = 0–90 %), defined as ≤US$35,000 for a family of four.

Supervisors

Supervisors of the trainees were accrued with 56 individuals eligible (6 had more than one trainee in attendance). All were solicited for participation (see description below). Of the 56, 16 (29 %) did not respond for a final n = 40. Supervisors could be from any educational background (not just mental health). Disciplines of the supervisors were 42 % medicine, 40 % mental health, and 18 % business/administration.

Procedures

Announcements to apply to the institute were placed on relevant listservs (e.g., American Psychosocial Oncology Society, Society of Behavioral Medicine’s Cancer Special Interest Group, American Psychological Association’s Divisions 38-Health Psychology and Division 12-Clinical Psychology, Association of Oncology Social Work) as well as distributed to interested individuals who had previously contacted the study PI.

Applications were sought from licensed psychologists, social workers, or mental health nurses employed full-time in clinical service provision to cancer patients. [Applications from individuals from other disciplines or those unlicensed (e.g., post-doctoral fellows) were considered late and only as space was available.] Eligibility criteria were posted and applications were made via the institute’s website http://http://cancertohealth.osu.edu.

The website application portion provided information regarding the NIH funding of the institute, the research participation requirement, and the reimbursement for expenses. In addition to an application, a resume and letter of institutional support from the individual’s clinical or administrative supervisor (i.e., the person who could authorize BBI implementation at the institution) and the supervisors’ contact information were required. When applying, individuals gave informed consent for research participation.

Applicants were screened for eligibility and fit with the aims of Institute and the research. A total of 110 individuals initiated applications for institutes 1 (2012) and 2 (2013). Of these, 77 (70 %) individuals were accepted. Non-acceptance (n = 33, 30 %) was due to an incomplete application (n = 26; 79 %) or ineligibility [not providing services to cancer patients (n = 5; 15 %); no post-graduate degree (n = 2; 6 %)]. After admittance, 12 (14 %) deferred attendance due to scheduling conflicts and three (4 %) withdrew for personal reasons, with 62 individuals attending. Prior to attending, individuals completed an assessment via the website and paper/pencil assessments were completed during the institute. The following were provided to attendees for research study participation: CE credit up to 18 h, manuals and instructional materials, US$250 travel stipend if >75 miles from the institute site, and three luncheons.

Using the contact information provided by attendees, clinical supervisors, and/or institutional administrators were informed that their applicant had been accepted to the BBI Institute. At that time, supervisors learned of their own eligibility and were invited to participate in a brief research study. Consent was completed on the website, along with a brief self-report measure of attitudes towards EBTs and the implementation of the BBI; US$5 was provided for the one-time data completion. Supervisors completed the measures an average of 6 weeks after trainee’s institute attendance.

BBI Institute Description

The BBI Institute was 3 days (18 training hours) conducted in a Department of Psychology at a large university in the Midwest. Upon arrival, trainees received copies of the BBI Therapist Manual and Patient Guidebook, a progressive muscle relaxation CD, and a binder with copies of the slides and other training materials. There were six experts/trainers, all Ph.D. clinical psychologists with expertise in the BBI conceptualization, methods, and skills. The training was delivered using a combination of lecture-style presentations (40 %; hereafter referred to as “didactics”), role play and group discussions (35 %; hereafter referred to as “experientials”), and small group practice sessions (25 %; hereafter called “practices”). Didactics provided a conceptual rationale, review of empirical support, and a detailing of a specific component. Experientials provided the practice of components between trainee pairs (e.g., “Teach your partner the Rhythmic Walking exercise.”) and discussions of implementation (e.g., “Write down patients’ common negative thoughts about his/her body image. Next, discuss with each other how to use cognitive reappraisal with these thoughts.”). During the latter, trainers monitored and assisted individuals/groups as needed. Practices with one trainer and 6–8 trainees were conducted in small rooms in the psychological clinic located in the same building and provided specific guidance in conducting a BBI session (e.g., Using the Therapist Manual, two “therapist” trainees lead the remaining “patient” trainees in a group discussion of how to identify sources and types of social support.).

Measures

Institute evaluation (BBI-Eval)

Trainees anonymously rated each of the 17 institute sessions and provided an overall rating. Using a student evaluation of instruction (SEI) form from the university as a template, eight questions, each with a 5-point Likert scale (1 = strongly agree to 5 = strongly disagree), assessed how well the session was taught (e.g., “The subject matter was well organized”, “I learned a great deal from this instructor”). Using the SEI questions enabled subsequent comparison to the university’s database of graduate psychology course respondents (hundreds) to the same items. An additional 17 questions, also with a 5-point scale (1 = strongly agree to 5 = strongly disagree), assessed if the institute’s program objectives were met (12 items) and overall learning achieved and perceived impact on practice (5 items).

Education gains

Two measures were used. A 32-item BBI Knowledge (BBI-Know) measure was developed and consisted of multiple choice (M-C) and true/false items assessing content from didactic, experiential, and practice sessions. Items were for the following topics: Introduction to the BBI (1 item), Stress conceptualization (4 items), Information seeking (3 items), Problem solving (4 items), Social support and assertive communication (7 items), Sexuality (5 items), Health behaviors (5 items), and Maintenance (3 items). Items were scored (0 = incorrect, 1 = correct), summed, and converted to percentage totals calculated for each area and for the total scale (ranging from 0–100 %). The BBI-Know was administered pre- and post-institute.

A Clinical Analogue Assessment (BBI-CLIN) evaluated trainees’ conceptual and clinical skills. Trainees received five written vignette topics, one each for stress conceptualization, problem solving, social support, sexuality, and health behaviors. A vignette described a specific patient and detailed her concern or difficulty and the task was to use BBI techniques to aid her (see Appendix for an example). Trainees were allotted a minimum of 3 h to prepare for each topic. The assessment occurred in individual, 6-min meetings with a confederate in private rooms with audio/video capabilities. Confederates were research assistants (n = 6) trained to respond to the conceptual and strategy content relevant for each topic. They memorized standard responses to anticipated queries from the trainees. With 62 trainees, 310 (n = 62 × 5) recordings were anticipated with 275 (89 %) available. Thirty-five (11 %) were not scored due to equipment difficulties (19 taped sessions) or non-compliance (14 sessions).

Trainees’ recordings were rated for mastery on four key contents or skills that each vignette was designed to elicit from the trainee (a sample scoring item is available in the appendix). Each item had a 0 to 6 rating scale with definitions at each point (e.g., 0 = only says brainstorming is used to generate solutions; 6 = 3 of 3 principles stated with a rationale for each). The four items for each vignette are summed for a score ranging from 0 to 24. BBI-Clin total scores were obtained by summing all five vignettes with a possible range of 0 to 120. The measure was based on the 0 to 6 rating scales and format of the cognitive therapy rating scale (CTRS) [27], a measure of therapist competency in cognitive therapy. Guidelines for the CTRS suggest therapists should score a minimum of 3.5 (on average) to demonstrate competence [28]. Therefore, BBI competence ratings for each vignette were set at 14 (3.5/ea. ×4 items) and 70 for the total score (14 × 5 vignettes), or approximately 60 % of the total possible score.

To determine scores, two raters first independently scored six recordings for a topic and a single measure two-way mixed effects intraclass correlation was calculated. Criterion was set at >0.80. Raters then discussed their scoring on each item of each recording. If the correlation coefficient was ρ < 0.80, the raters independently scored six additional recordings from the topic, calculated the correlation coefficient, and discussed again. Four of the five topics had acceptable reliability after the first round, and all five were acceptable after the second round. Interrater reliability scores were: stress conceptualization, ρ = 0.804; problem solving, ρ = 0.913; social support, ρ = 0.952; sexuality, ρ = 0.946; health behaviors, ρ = 0.949. After criterion was achieved for a topic, the remaining tapes were divided and scored independently by one of the raters.

Attitudes

Two measures were used. The 15-item Evidence-Based Practice Attitudes Scale (EBT-Att) [29] includes items such as “I am willing to try new types of therapy/interventions even if I have to follow a treatment manual.” Items are scored using a 5-point Likert scale (0 = not at all to 4 = to a very great extent). Four items are reverse-scored, and then all items are summed for a possible range of 0–60. The EBT-Att was administered pre- and post-institute with an internal consistency reliability at pre-institute of α = 0.831.

To assess Attitudes Toward BBI (BBI-Att) specifically, a 30-item semantic differential scale [30] was created. Thirty pairs of bipolar adjectives chosen for their potential descriptiveness of the BBI intervention assessed the dimensions of evaluation (11 items; e.g., worthless/valuable); potency (13 items; e.g., simple/complex); and activity (6 items; dull/stimulating). A 7-point scale, ranging from −3 to +3, was used for each item, anchored at the two poles (e.g., −3 = worthless, +3 = valuable). Positive and negative poles were counter balanced to mitigate the effect of response sets. Items were summed for a possible range of −90 to +90. The measure was administered at post-Institute with an internal consistency of α = 0.858.

Perceived behavioral control: self-efficacy

Two measures were used. The 25-item Counselor Activity Self-Efficacy Scale (general SE) [31] assesses self-efficacy for general (e.g., attending, listening) and advanced (e.g., reflection of feeling, handling silences) counseling skills. A 10-point Likert scale (0 = not at all confident to 9 = totally confident) is used. The items are summed for a possible range of 0–225. The general SE was administered pre- and post-institute with an internal consistency reliability of α = 0.973 pre-institute.

Eight additional questions assessed self-efficacy for conducting the BBI techniques (BBI Self-Efficacy Scale; BBI-SE). Seven items assessed BBI therapeutic skills (progressive muscle relaxation, problem-solving, etc.) using a 10-point Likert scale (0 = not at all confident to 9 = totally confident) and one item assessed confidence in conducting treatment in a group format. The items were summed for a total score ranging from 0–72. The BBI-SE was administered pre- and post-institute with an internal consistency reliability of α = 0.851 pre-institute.

Perceived behavioral control: controllability

A supervisor measure was used. The EBT-Att was adapted to reflect the supervisor’s attitude towards the trainee’s use of EBTs. For example, the item, “I like to use new types of therapy/interventions to help my clients” was revised to read, “I like for the trainee to use new types of therapy/interventions to help his/her clients.” One item could not be modified (“How likely would you be willing to adopt it at your institution if it was required by your supervisor?”) and was eliminated. For the 14 items, a 5-point Likert scale (0 = not at all to 4 = to a very great extent was used for a total score ranging 0–56. Internal consistency reliability was α = 0.774.

For additional description, a visual analogue scale was used for supervisors to rate the balance between challenges versus benefits of BBI implementation. The anchors were “The challenges will always outweigh the benefits” (left side) and “The benefits will always outweigh the challenges” (right side). Markings were subsequently scored from 0 to 100 with scores >50 indicating benefits outweighing the challenges.

Intentions

A 10-item measure (BBI-Intent) assessed trainees’ intentions to use each of the 10 BBI components (e.g., assertive communication, physical activity). Items were rated on a 5-point Likert scale (0 = never to 4 = always). Items were summed for a total score ranging from 0–40. The BBI-intent was administered post-institute with internal consistency of α = 0.88.

Analytic strategy

Descriptive analyses were conducted for each measure by time point. Preliminary analyses were conducted to determine (1) variation, if any, in trainee characteristics by institute year and (2) associations between trainee demographic characteristics (including institute year) and outcomes. Analyses of change from pre- to post-Institute used repeated measures analysis of variance (rANOVA), with time as a within-subjects factor and relevant control variables selected empirically as covariates. An ANOVA compared supervisors’ attitudes toward evidence-based practices (EBT-Att-Sup) to the trainees’ attitudes.

For regression analyses, correlations between predictors [education gains, attitudes, self-efficacy, controllability (supervisors’ attitudes)] and the intent to use outcome were examined. Four linear regression analyses (one each for knowledge, attitudes, self-efficacy, and controllability) examined relationships between predictors and outcomes. When two measures were used (e.g., self-efficacy), the general measure was entered first and the BBI-specific measure entered second. A final linear regression analysis was conducted with all significant predictors from the four models to determine best model fit and variance contributed by each predictor.

RESULTS

Preliminary analyses

See Table 1 for descriptive statistics by assessment time point. Analyses showed some trainee characteristics to differ by institute year. Trainees attending year 1 were older (M age = 49 vs. 42; F_(1,52) = 6.116, p < 0.05) and thus had graduated with their clinical degree earlier (M years since degree = 18 vs. 11; F_(1,53) = 5.176, p < 0.05), spent more time in their current position (M in years = 9 vs. 5; F_(1,52) = 6.514, p < 0.05), and were more likely to be licensed (97 vs. 81 % licensed; Χ²_{(1, n = 62)} = 4.026, p < 0.05) than those attending in year 2. Pre-institute, year 1 trainees reported more confidence in both general clinical skills [general SE; M = 192 vs. 175; F_(1,60) = 6.099, p < 0.05] and BBI-specific clinical skills [BBI-SE; M = 58 vs. 52; F_(1,60) = 7.859, p < 0.01]. Analyses comparing trainees whose supervisors did versus did not participate indicated no differences on sociodemographics (ps > 0.18) or education gains, attitudes, self-efficacy, or intent (ps > 0.12).

Table 1.

Descriptive statistics for variables included in the Theory of Planned Behavior regression models

	Pre-institute		Post-institute
Scale	Mean (SD)	Range	Mean (SD)	Range
Education gains
BBI knowledge (BBI-K now)	63.3 % (9.8)	38–88 %	83.8 % (8.2)	63–97 %
Standardized patient assessments (BBI-CLIN)	–a		69.9 (14.5)	36–94
Attitudes
Attitudes towards evidence-based treatments (EBT-Att)	45.9 (6.9)	29–58	47.6 (7.0)	34–59
Attitudes toward BBI (BBI-Att)	–a		56.2 (14.3)	24–85
Self-efficacy
Counselor activity self-efficacy scale (general SE)	183.6 (27.8)	103–225	196.1 (18.2)	156–225
Self-efficacy for BBI techniques (BBI-SE)	54.9 (9.8)	34–72	62.1 (6.1)	47–72
Controllability (supervisors’ attitudes)
Supervisors’ EBT-Att	–a		42.7 (5.8)	31–54
BBI implementation challenges vs. benefits scale	–a		68.1 (19.4)	2–95
Intentions
Intent to use BBI techniques (BBI-Intent)	–a		30.8 (5.3)	19–40

^aPost measures only

Several professional characteristics were associated with explanatory variables. Post-institute, psychologists scored higher on the BBI-Know [M = 86.5 % vs. 82.2 %; F_(1,60) = 4.232, p < 0.05] and reported higher levels of general SE [M = 206 vs. 190; F_(1,57) = 12.119, p = 0.001] and BBI-SE [M = 65 vs. 61; F_(1,57) = 6.972, p < 0.05] than non-psychologists. Individuals working in hospitals reported lower general SE than those working in other settings [M = 193 vs. 207; F_(1,57) = 7.388, p < 0.01] and less positive attitudes towards evidence-based practices [EBT-Att; M = 46 vs. 51; F_(1,57) = 4.923, p < 0.05]. Therefore, clinician and facility types were included as control variables in the relevant rANOVAs examining change over time.

Descriptive analyses and analyses of change

Institute evaluation

Trainees rated the BBI Institutes highly. On a 1–5 scale, the overall mean for the eight items assessing quality of the instruction for the 17 sessions was 4.66 (SD = 0.26; range = 4.28 to 4.89), higher than the 5-year overall rating (M = 4.41, SD = 0.74) of the instructors (i.e., professors, n = 18) of graduate courses in psychology. Improvements from years 1 to 2 were noted for the majority (94 %) of the sessions. Trainees agreed that the program objectives were met (overall M = 4.31, SD = 0.41) and that content and instruction was appropriate for post-degree training (overall M = 4.32, SD = 0.62).

Education outcomes

Controlling for clinician type, a main effect of time was found for the BBI-Know [F_(1,60) = 162.193, p < 0.001, partial η² = 0.73], indicating improvements in knowledge from pre- to post-institute. The interaction effect was non-significant.

For the BBI-Clin, mean scores for each of the five vignettes ranged from 10.8–17.0 out of 24. Scores were higher for year 2 versus year 1 with significant differences noted on health behaviors [F_(1,47) = 28.924, p < 0.001] and the total score [F_(1,53) = 11.243, p = 0.001; see Fig. 2]. A rating of 70 (60 %) or higher indicates competence and was achieved by 53 % (n = 33) of trainees on the total score. The percent of trainees achieving competency on the five vignettes ranged from 37 % (problem solving) to 53 % (stress conceptualization).

Fig. 2.

Mean scores of the BBI clinical analogue assessment by institute year for each of the vignettes and total score. Scores presented as a percentage of the total possible with 60 % indicative of competence achievement

Attitudes and perceived behavioral control

Controlling for facility type, a main effect of time was found for the EBT-Att [F_(1,57) = 7.323, p < 0.01, partial η² = 0.11], suggesting trainees’ had more positive attitudes toward evidence-based treatments from pre- to post-institute. The interaction was non-significant.

Controlling for clinician and facility type, no main effect for time was found for general SE [F_(1,55) = 1.293, p = 0.26, partial η² = 0.02] but an interaction effect of time × institute was significant [F_(1,55) = 7.710, p < 0.01, partial η² = 0.12]. Trainees from year 2 reported lower general SE pre-institute but similar levels to year 1 trainees post-institute. The interactions between time and the control variables were non-significant. Controlling for clinician type, similar results were seen with the BBI-SE scale, with no main effect for time [F_(1,56) = 0.832, p = 0.366, partial η² = 0.02] but a significant interaction effect of time × institute [F_(1,56) = 12.334, p = 0.001, partial η² = 0.18]. Again, year 2 trainees had lower BBI-SE pre-institute but similar levels as year 1 trainees post-institute. The interaction of time × clinician type was not significant.

Supervisors had positive attitudes towards their trainee’s use of evidence-based treatments (Supervisors’ EBT-Att; M = 42.7, SD = 5.8). The mean is similar to available population norms [30] and not significantly different from the trainees’ EBT-Att scores [M = 45.2 (SD = 5.8); F_(1,80) = 3.902, p = 0.052; Note that the 15-item EBT-Att that trainees completed was rescored using the relevant 14 items to compare to supervisors].

Descriptive data of the implementation balance rating showed the majority of supervisors (78 %) reported the benefits of BBI implementation outweighed the challenges (Md = 70/100). In regression analyses, supervisors’ EBT-Att was a significant explanatory variable of the implementation balance rating [adjusted R² = 0.12; F_(1,35) = 6.058, p < 0.05]. That is, supervisors with more positive attitudes toward trainees’ EBT use also reported more benefits than challenges to BBI implementation.

Regression analyses predicting intentions

As seen in Table 2, the trainees’ report of BBI-Intent was significantly correlated with the BBI-Att, General SE, and BBI-SE measures (ps < 0.05). As hypothesized, model one (attitudes predicting intentions) was significant [adjusted R² = 0.11, F_(2,54) = 4.365, p < 0.05] with the BBI-Att remaining as the only significant explanatory variable (β = 0.357, p < 0.01). Model two (education gains predicting intentions) was not significant [adjusted R² = −0.03, F_(2,52) = 0.119, p = 0.89]. Model three (self-efficacy predicting intentions) was significant [adjusted R² = 0.12, F_(2,56) = 5.007, p = 0.01] with the BBI-SE remaining as the only significant explanatory variable (β = 0.379, p < 0.05). Model four [controllability (supervisors’ attitudes) predicting intentions] was not significant [adjusted R² = −0.02, F_(2,37) = 0.581, p = 0.57].

Table 2.

Correlations for variables included in the Theory of Planned Behavior regression models

	Variable	1	2	3	4	5	6	7	8
Education gains
1	BBI knowledge (BBI-Know)
2	Standardized patient assessments (BBI-CLIN)	0.12
Attitudes
3	Attitudes towards evidence-based treatment (EBT-Att)	0.03	0.18
4	Attitudes towards BBI (BBI-Att)	−0.13	0.01	0.28
Self-efficacy
5	Counselor activity self-efficacy scale (general SE)	0.11	−0.17	0.19	0.32
6	Self-efficacy for BBI techniques (BBI-SE)	0.30	−0.12	0.25	0.29	0.72
Controllability (supervisors’ attitudes)
7	Supervisors’ EBT-Att	−0.16	0.19	0.40	−0.04	0.05	0.13
8	BBI implementation challenges/benefits scale	0.19	0.39	0.03	0.10	0.04	0.10	0.33
Intentions
9	Intent to use BBI techniques (-intent)	0.00	0.04	0.15	0.36	0.29	0.39	−0.04	0.15

Data in bold = p < 0.05

In the final model, significant explanatory variables from the above models were entered, with BBI-SE as step 1 and BBI-Att as step 2. The final model was significant [adjusted R² = 0.22, F_(2,54) = 8.902, p < 0.001] with BBI-SE explaining 11 % of the variance in BBI-Intent (β = 0.348, p < 0.01) and BBI-Att explaining 7 % (β = 0.268, p < 0.05).

DISCUSSION

The BBI training institute was successful in achieving the stated goals and in so doing it provided a feasible and comprehensive prototype for CE delivery and evaluation. The most effective CE instructional method (i.e., multicomponent) was used and found to be of high quality, even rated higher than graduate level instruction in a highly ranked psychology department [32]. As expected, demonstrable improvements occurred not only in trainees’ knowledge, but also in their clinical facility with the treatment strategies as shown with a novel clinical analogue assessment. The training generated positive attitudes toward and high self-efficacy to use BBI, both of which predicted trainees’ high intentions to use it upon returning to their own institution.

Supervisors reported that the benefits of BBI would outweigh the challenges to its implementation, which may bode well for the trainees’ implementation of BBI. The institutes included sessions discussing intervention adaptation, another departure from typical CE programs. Adaptation was included as it was anticipated that trainees would have different patient populations than the newly diagnosed breast cancer patients with which the BBI was validated [22]. Moreover, implementation of an innovation (such as an EBT) typically requires some adaptation of the innovation, the setting, or both [33].

These data are in a context of the general paucity of research on EBT training. In fact, EBT training has been described “as not evidence based” ([34], p. 496). Moreover, a search for results from EBT trainings of interventions for cancer patients found only one offering, that of Clark and colleagues’ cognitive behavior therapy skills for cancer survivors (CBT-CS) [35]. The CBT-CS was also multicomponent and assessed trainees pre- to post-workshop on CBT knowledge and self-efficacy and found improvements in both. If there is to be implementation of EBTs for cancer patients, dissemination through substantive training opportunities are first needed. An impressive example of such is the Veterans Health Administration’s (VA) multi-year effort [36] to disseminate and subsequently have their clinicians implement EBTs (e.g., acceptance and commitment therapy [15], CBT for depression [37]). Their comprehensive “gold standard” dissemination, also 3 days, shows education gains, positive attitudes toward the EBTs, and increased self-efficacy for the therapists participating in the training.

It is rare for a CE program for mental health professionals to assess clinical skill via a simulation despite frequent use in medical education programs [38]. To our knowledge, only one training program has used confederate patients (CP) in assessing skills learned by mental health professionals [39]. In that study, ratings of therapists’ use of motivational interviewing skills for addiction with CPs were similar to those with “live” patients, indicating that CPs could be a valid strategy for assessing therapist skills. In the current study, in addition to its value in evaluating clinical competence, the inclusion of the vignette recording appeared to be a prompt for trainees’ self-study of the material, as many reported wanting to perform well during the taping. Results showed that 53 % of trainees achieved a rating of competent. We did not anticipate that any trainee would be rated at the expert level of 85 % or above, as only 2–3 h of training was spent on any one topic. Indeed, trainees were given the expectation for additional review of their notes and the manuals after returning to their institution. Thus, conducting a clinical assessment actually provided another learning opportunity for trainees in addition to providing unique behavioral data of education gains.

Another distinguishing aspect of this research is its theoretical import, using the Theory of Planned Behavior (TpB) [17] to study the relevant proximal outcome, trainees’ intent to use the BBI. Support for the TpB model was found, with trainees’ attitudes and self-efficacy remaining as significant explanatory variables of intent to use. The constructs of attitudes and self-efficacy were assessed with a general measure and one BBI-specific, and it was the BBI-specific versions of both that predicted intentions rather than the general measures. This underscores the value of treatment-specific measures in evaluations of EBT trainings.

Education gains are not specified as a precedent in the TpB model but were logically included, reasoning that clinicians must be knowledgeable about an EBT before any implementation. The existing literature did not provide examples of trainees’ knowledge in relationship to intentions. In the current study, education gains were unrelated to intent. This suggests that having knowledge about an EBT is necessary, but not sufficient, to influence intent to use the EBT.

Controllability in this TpB model was studied from a relevant organizational perspective for implementation of an EBT, operationalized here as supervisors’ attitudes towards EBTs. This seemed an appropriate (and meaningful) measure in the context of the therapists’ EBT training, as supervisor support is not under the providers’ control. Data point to the importance of such factors. Cook and colleagues [40] found common setting/institution barriers to include lack of support for CE training when surveying over 1,600 psychotherapists (social workers, psychologists, marriage and family therapists, etc.). Similar data exist regarding the dissemination/implementation of CBT [41], with barriers including limited availability of intensive training for therapists and lack of organizational support [37]. In this study, trainees and supervisors had similar (positive) attitudes towards EBTs, and as the trainee’s EBT attitudes did not predict intent to use, neither did their supervisors’. Had attitudes not been similar, one or the other may have been a significant predictor. In future research, a direct measure of control as perceived by the trainee might be considered. It could be readily constructed as described by the TpB authors [42] to assess an individual’s perceptions about his/her capability to perform the behavior (e.g., “How much control do you think you have over …?”). Assessing supervisors’ attitudes towards EBTs remains relevant, however, because of their attitudes and opinions are important to implementation [18].

Trainees attending the BBI Institute received incentives for research study participation, i.e., no-cost training, continuing education credits, and a travel stipend. It is unclear what effect, if any, the incentives had on the data obtained. Perhaps individuals were more apt to provide favorable evaluations than otherwise because of these incentives. We feel this is unlikely for several reasons. Travel expenses remained considerable for most, with US$250 covering approximately one third of current air fares and the 3-day hotel stays averaging to a total of  US$900. Other costs (e.g., time off from work, child care, etc.) were also incurred. These expenses are the same as or similar to those incurred for CE hours obtained during attendance to professional meetings. In sum, the US$250 did not appear to be a coercive amount. Discussed during the institute and noted on the application website, the incentive was provided for research participation, not institute attendance.

CONCLUSION

As noted by the Institute of Medicine report and prior ones [5, 43, 44], psychosocial care for cancer patients has poor availability and when offered, it is usually not evidence-based. The BBI intervention is the one of the first EBTs for cancer patients to be disseminated to community mental health professionals. We trained community mental health professionals in the intervention itself and encouraged them to make a change to their behavior post-institute, in the form of adding the BBI intervention to their clinical practice. The BBI Institutes offered effective, multicomponent education as evidenced by knowledge, clinical competence, attitudes, and self-efficacy gains. The components of the multimodal instruction were mirrored in the assessments used and, in combination with theory to predict EBT usage, this was a novel measurement model in comparison to the strategies found in the CE program evaluation literature [45]. From our experience, this is an effective model for intervention dissemination. It is also the case, however, that dissemination education must assist trainees with adaptation and navigating barriers to implementation. EBT implementation is complex multi-stage effort [46], but the first step is EBT information provision and education that is comprehensive and effective.

APPENDIX

Sample Standardized Patient Assessment (BBI-CLIN)

The problem-solving vignette was as follows:

Jane has been experiencing memory and concentration difficulties, which her nurses have told her, are ‘chemo brain.’ She seems distressed as she describes comparing her pre-cancer functioning (managing an office, keeping her household in order, organizing family activities such as birthday parties) to her current functioning (having difficulty remembering appointments, forgetting questions she wanted to ask the doctor, struggling to understand treatment options). Currently, she is most worried about the ‘chemo brain’ problems at work. You have already spent some time defining the problem with her and the specific difficulties identified are poor memory, concentration difficulties, and forgetfulness (see attached Problem Analysis worksheet).

Your task is twofold:

1. Explain “brainstorming” as applied in problem solving.

2. Work with Jane to generate solution(s)

When there are enough solutions, STOP. Do not proceed to other stages of problem solving.

Sample BBI-CLIN scoring item

An example of one item from problem solving and the relevant content for the answer is the following:

• Item: To what extent did the therapist explain the process of “brainstorming?”

• Answer: (1) Generating multiple solutions. (Rationale: the more solutions you generate, the more likely you are to find one that works well/those who are successful at problem solving are the ones who generate more potential solutions.) (2) Deferring judgment about feasibility. (Rationale: ideas that first seem impossible may be combined with others; any idea can generate another new idea.) (3) Thinking of the “big picture” solutions. (Rationale: getting stuck in details makes it harder to think of solutions.)