Implementation of an evidence-based biobehavioral treatment for cancer patients

Abstract

One aim of dissemination and implementation (DI) research is to study the translation of evidence-based treatments (EBTs) from the research environments of their development and testing to broader communities where they are needed. There are few behavioral medicine DI studies and none in cancer survivorship. A determinant model (Setting, Therapist, Education, imPlementation, and Sustainability (STEPS)) was used to conceptualize DI of mental health treatment and frame a longitudinal study of implementation of a behavioral medicine EBT—a biobehavioral intervention (BBI) for cancer patients. Using effective dissemination strategies, therapists were trained in the BBI and followed to determine if implementation occurred. Participants (N = 108) were psychologists, social workers, and other oncology mental health providers from diverse settings to whom the BBI had been disseminated. BBI trainers then provided 6 months of support for implementation (e.g., monthly conference calls). Therapists reported number of patients treated, with or without the BBI, at 2, 4, and 6 months; use of support strategies was tracked. Generalized linear mixed models show that the proportion of patients treated with BBI ranged from 58 to 68%, with a 2% increase across follow-ups. Therapist and setting characteristics did not predict usage. Implementation of a behavioral medicine EBT provides a “real-world” demonstration of a BBI moved from the research setting to diverse communities. As the first study in cancer, it is an encouraging example of training and supporting mental health providers to deliver evidence-based psychological treatment and finding their success in doing so.

Research supports the efficacy of psychological interventions, but for most, there is a failure of translation from the research environments of their development and testing to clinical settings where they are needed. Addressing this longstanding gap (“implementation cliff”) is complex [1, 2] and is the focus of dissemination and implementation (DI) science. Rabin et al. [3] define dissemination as an active approach to spread evidence-based treatments (EBTs) to target audiences. Implementation, however, is the process of using and integrating the EBT into the setting [4].

We were interested in studying the implementation of a health psychology EBT—a biobehavioral intervention (BBI) for cancer patients. We looked to other areas of health psychology for examples as there has been little DI research in cancer control [5], with few studies of dissemination [6] and none of implementation. Dissemination studies in pain management, cardiovascular disease, diabetes, HIV/AIDS, smoking cessation, and others [7–11] were identified. As the research focus is implementation, we provide exemplars from health psychology and mental health literatures.

The Center for Disease Control and Prevention (CDC) High Impact HIV/AIDS Prevention Project (HIP) [12] has provided trainings (workshops, materials, webinars, etc.) and offered support to health providers. As of 2009, more than 12,000 providers were trained to use 18 risk reduction interventions [13], but rates of actual usage were low. After 12 months, Kelly et al. [10] reported that providers offered interventions to an average of only 1.2 patients, whereas providers who received consultation support intervened with an average of six patients. There is more DI research in mental health [14]. The UK’s program, “Improving Access to Psychological Therapies (IAPT),” is a push for providers to use EBT guidelines for the treatment of mood and anxiety disorders. After providing support and consultation, Clark et al. [15] reported that 40% of patients received guided treatment. Another example, the National Child Traumatic Stress Network (NCTSN) trained 280 clinicians within 3 years who then provided EBTs to 1078 clients [16]. A similar effort in Hawaii to have multisystemic therapy [MST; 17] used with adolescents with antisocial behavior problems [18] resulted in approximately 300 new patients treated with MST annually [19]. In summary, the available health psychology and mental health implementations are uncontrolled studies, but they have gone on to report provider treatment usage or number of patients served. They suggest that implementation of a new treatment is achievable, albeit with varying degrees of success, suggesting that implementation support may be facilitative.

The implementation of an EBT by mental health providers in oncology following dissemination and training was studied. The EBT used was formulated from a biobehavioral model for understanding the stress of cancer and its impact on disease progression [20]. A biobehavioral intervention was devel-oped and tested in a randomized trial with breast cancer patients, and it produced robust psychological, behavioral, and health effects [21–25]. Components of the intervention have also been used successfully for tailored treatments [e.g., 26].

This DI effort began with a multilevel framework, Setting, Therapist, Education, imPlementation, and Sustainability (STEPS) (see Fig. 1) [27]. As is represented visually, STEPS is a model of increasing effort at successive levels. We first considered setting (see S in STEPS) and therapist (see T in STEPS) factors when designing the dissemination effort. An earlier report [28] provided data on the effectiveness of BBI dissemination via training institutes for oncology mental health providers. Using multimodal education strategies (see E in STEPS), the institutes were effective in producing knowledge gains and clinical facility with the BBI, enhanced self-efficacy and positive attitudes towards EBTs, and positive intentions to implement BBI.

Fig 1.

Multidimensional model of Setting, Therapist, Education, imPlementation, and Sustainability (STEPS) factors of dissemination and implementation of mental health EBTs. Implementation is emphasized for the present analysis

This research follows the therapists to discover if BBI implementation actually occurred and, if so, at what level (see emphasis of P in STEPS). Upon return to their home facilities, therapists’ BBI usage was tracked for 6 months, and during this time, they were provided with clinical and implementation support, as has been suggested [29–31], along with materials to aid in BBI adoption within their setting. Therapists’ engagement in support activities was monitored, but the primary outcome was BBI usage. A secondary aim explored provider (therapist)- and system (setting)-level predictors of BBI usage, as some see implementation failures as largely due to multilevel contextual factors [32, 33]. Early-career mental health providers [34, 35], for example, are generally more motivated to use EBTs, as are ones with a higher level of education or more experience [36, 37]. The list of potential organizational factors is extensive [38–40], but we focus on two. One is the size of the organization [41], and the other is the organizational leader, i.e., his/her knowledge of and attitudes about EBTs and the level of their support for implementation [42]. Taken together, the study provides the first demonstration of EBT implementation in cancer control and, further, explores factors that may impact success.

METHODS

Participants

Therapists

Providers (N = 108) from four BBI Institutes [28] were studied. The sample was predominantly female (n = 95; 88%) and middle aged (M = 42.93 years; SD = 10.55). Racial/ethnic distribution was 88% Caucasian, 7% Asian, 3% Latino, and 2% African-American. The majority (92%) was licensed, coming from the disciplines of clinical psychology (36%), social work (44%), mental health nursing (3%), postdoctoral positions in psychology (6%), or others (11%). Excluding postdocs, providers had been licensed an average of 12.23 years (range = 1–36), worked 6.07 years (range = 0–27) in their current position, and spent 74% (range = 5–100%) of their time providing clinical services. They came from 31 states in the USA, Puerto Rico, and four foreign countries (Brazil, Israel, Kenya, and Malaysia).

For the interest of the reader, therapists’ information about the patients that they typically served were as follows: 56% of patients were female (median; range = 5–100%); 70% Caucasian (range = 0–90%), 13% African-American (range = 0–80%), 8% Latino (range = 0–100%), 2% Asian (range = 0–100%), and 3.9% other (range = 0–42%); 34% rural (median; range = 0–100%) vs. urban; and with 35% being of low income (i.e., annual net of <$35,000 for family of four). Across providers, the most common site of cancer treated was breast (mean = 15%; range = 0–100%), but nine other sites were also reported.

Managers/supervisors

Leaders (N = 97) in the therapists’ organization that had a supervisory role participated. Their professions were as follows: 43% mental health, 33% medicine, and 21% business/administration from the following settings: 51% academic medical centers or VA hospitals, 23% community hospitals/centers, 14% supportive care facilities, 8% private practice, or 4% other.

Procedures

Therapist application, selection, and dissemination education

Complete descriptions are available [28]. Briefly, announcements were posted on listservs and the institute website (www.cancertohealth.osu.edu), which provided a description of content and teaching methods to be used, application materials, and note regarding research participation in the program evaluation. A total of 183 individuals completed informed consent and initiated applications for four institutes (2012–2014). Of them, 119 (65%) were accepted. Non-acceptance (n = 64) was due to an incomplete application (n = 50; 78%) or ineligibility (n = 14; 22%). Eleven (9%) deferred to a later session, with 108 attending.

A letter of endorsement from the therapist’s supervisor was required for the application. One hundred four individuals (ten supervised more than one trainee) were notified of the respective therapist’s acceptance to the institute, and when training was completed, they also learned of their eligibility to participate in the research. Of the 104, 7 declined participation for a final N = 97. Consent and assessments were completed on the website. To enhance compliance, participants received a small incentive ($5).

During the 3-day institutes, therapists received a BBI therapist manual and a patient guidebook and didactic (40%), experiential (35%), and small group (25%) education from five Ph.D. trainers on the eight components of BBI [28]. Later, therapists were prompted by an e-mail to log usage of BBI on the website (each trainee had a personal link) at 2, 4, and 6 months. To enhance compliance, therapists received a small incentive ($15) for each assessment.

Implementation support

Support (see in the following) was provided for 6 months. Compilations of implementation strategies exist (e.g., N = 68 in [43]), but research has not yet identified specific ones to promote uptake and adoption of EBTs. Of the existing strategies, five were chosen to assist with clinical and organizational aspects of implementation.

Manuals

The trainee-only website had pdfs of the manuals, worksheets from the patient manual, and mp3s of progressive muscle relaxation. Additional “hard copies” of manuals were provided at cost ($30 ea.).

Planning

During the final session of the institute, therapists were prompted to begin implementation planning by completing an adaptation plan. Using a plan template, they detailed aspects of BBI delivery (e.g., individual vs. group format) and anticipated usage of the eight BBI components (e.g., stress conceptualization, social support; [44]). Therapists were to complete and send (e-mail) the plan in the next month, and within 2 weeks of receipt, a telephone meeting with a trainer provided feedback, with the goal being to assist in therapists’ programmatic BBI adherence.

Quality monitoring

There were two areas of focus. First, to aid clinical implementation, six monthly, hour-long conference calls were offered. Moderated by one or two trainers, groups were formed with five to eight therapists from the same institute. Call topics were as follows: BBI implementation challenges; adapting BBI while maintaining fidelity; BBI component review and usage discussion; and lastly, review of implementation successes, challenges, and lessons learned. Call summaries were posted within 1 week on the website. Secondly, data-driven clinical decision-making was taught. During the institute, the principle of evaluating one’s clinical work was presented and brief patient outcome measures were discussed. Measures of mood, health behaviors, and others were posted on the website and trainers prompted usage on the second conference call.

Marketing

Materials were provided (www.cancertohealth.osu.edu) to assist therapists to inform/educate local stakeholders of the efficacy, characteristics, and the new availability of BBI. A fact sheet tailored for organizational leaders or referral sources (MDs, RNs, community organizations), a PowerPoint describing BBI empirical support, patient brochures, and other materials were available. PDF reprints on topics timely to oncology mental health providers (e.g., anxiety and depression treatment guidelines) were posted.

Finance

The successful implementation of EBT depends in part on the availability and allocation of organizational resources [45, 46]. This task is challenging enough in large organizations, but it is more challenging in small organizations [39, 47] that typically have few unused resources. The majority of the therapists were in settings in which funding of psychosocial services was limited, and for some, it was essential to seek internal or external funds to offer anything new/additional. Templates were provided for the common elements (e.g., descriptions of clinical need, budget justification) of small grant proposals.

Measures

Therapist engagement in implementation support

Data came from three sources: (1) participation in conference calls; (2) completion of adaptation plans; (3) website downloads (total counts) for grant templates, marketing materials, and manuals).

Outcome: BBI usage

A web-based (Qualtrics) log was used for therapists to report service provision (treatment) to individuals and groups within the previous month. Therapists primarily spent their time in service provision. Thus, we anticipated that therapists could, in most cases, readily report the total number of patients treated, corresponding to billing hours for some. In another portion of the survey, therapists then reported the total number treated with BBI. Reports were obtained at 2, 4, and 6 months.

Predictors

Therapist

Characteristics. Coded variables were age, years since terminal degree, years licensed, time in current position, and discipline (psychology, social work, nursing, others).

Percent effort. Therapists reported the percent of weekly effort (40 h equivalent) in clinical, teaching, administrative, or other tasks.

Setting

Type. Organizations were coded as community settings (49%; i.e., community hospital, community oncology or primary care practice, cancer support community centers, sole proprietor practice) versus non-community settings (51%; cancer centers and Veterans’ Administration Hospitals).

Manager/supervisor discipline. Coded were mental health, medical, or business administration.

Manager/supervisor attitudes. Two measures were used. (1) The Evidence-Based Practice Attitudes Scale (EBPAS) [48] was adapted for use. For example, the item “I am willing to try new types of therapy/interventions even if I have to follow a treatment manual” was adapted to “I am willing to support the therapist in trying new types of therapy/interventions even if s/he has to follow a treatment manual.” The 14 EBPAS items were scored on a five-point Likert scale (0 = not at all to 4 = to a very great extent); four items are reversed-scored. Items are summed for a possible range of 0–56. Internal consistency was 0.76. (2) Supervisors rated the perceived ratio of challenges to benefits of BBI implementation. A visual analogue scale was used, with the anchors of 0 = challenges will always outweigh the benefits and 100 = benefits will always outweigh the challenges.

Analytic strategy

Data describe therapists’ engagement in implementation support. Tallies of events (e.g., conference calls) and tasks (adaptation plan completion) are reported along with website downloads of BBI materials.

For usage outcome analyses, all available data from 108 therapists were used. Using the previous month as the frame of reference, therapists reported the total number of cancer patients treated and the total number of cancer patients treated with the BBI at 2, 4, and 6 months. A proportion with a possible range of 0 (BBI never used with a patient) to 100 (BBI used with all patients) was calculated. Missing data were assumed to be missing at random, were conditional on measured covariates, and were ignored. There were 67 missing or partial response observations. If therapists reported zero patients (i.e., no opportunity to use BBI), being true for approximately 9% of observations, then the observation was removed. In total, 232 observations were used.

A generalized linear mixed model with a logistic link function was used. A logistic transformation of the proportion was done, and a linear equation predicted the log odds that a therapist would use the BBI to treat a patient. To estimate the change in the log odds of usage, a logistic regression model with a random intercept for each therapist and an extra error term to account for over dispersal residuals was used. This model is similar in logistic regression, but the true response variable here is a proportion, rather than a binary one. Each therapist has his or her own intercept in order to account for the observations being correlated. Time since completion of the BBI Institute and percent effort were treated as continuous variables. The institute attended was a nominal variable with the first institute being the reference level. The therapist’s setting type was treated as a binary variable with non-community settings being the reference level.

RESULTS

Engagement in implementation support

Regarding involvement in planning, of the 108 providers, 92 (85%) completed the adaption plan and 80 (74%) participated in the individual follow-up call. Regarding quality monitoring, 86% participated in the conference calls, with a median of three calls (range 0–6). Therapists could also post questions and participate in discussion on a message board; 80% of therapists accessed the “trainee-only” portion of the website. There were 1509 downloads (est. 18 per trainee) of materials. Of them, 54% were of BBI education/marketing materials, 21% of quality monitoring (call notes and patient measures), and 3% of finance (grant templates). Also, 22% of downloads were of BBI manuals and materials. Additional hard copies of manuals were requested; 508 patient (est. 4.7 per trainee) and 41 therapist manuals were shipped.

BBI usage

No trainees used the BBI prior to attending the institute. Thereafter, the percentage of patients (with 95% confidence intervals) treated with BBI was high and increased with successive assessments (see Fig. 2). The mean was 58.92 (SD = 35.00) at 2 months, 65.10 (SD = 34.88) at 4 months, and 68.46 (SD = 34.01) at 6 months postinstitute. The final model showed that time was significant ($\widehat{\beta }$ = 0.09, SE = .05, p < .05); at each subsequent follow-up, there would be, on average, a 2% increase in the proportion of BBI usage for each therapist. Institute (1–4) was a factor in these analyses and was found to be not significant, showing that usage levels were replicated across the four therapist samples.

Fig 2.

Therapists’ EBT usage, measured as the proportion of the patients treated with the BBI, increased with each subsequent assessment following the training institute

Predictors of usage

Therapist factors—professional characteristics and service provision—were examined. Age, time since degree, time since licensure, and therapists’ discipline did not predict usage. Only the amount of time (%) that therapists spent teaching yielded this finding, $\widehat{\beta }$ = 0.04, SE = .02, p = .054, suggesting that therapists who spent more time per week on teaching may have had higher BBI usage rates.

Setting factors were studied. Descriptively, supervisor scores on the EBPAS indicated positive attitudes towards their therapist’s use of EBTs (M = 43.5, SD = 5.7). Also, the majority of supervisors (78%) reported that the benefits of BBI implementation outweighed the challenges (M = 70/100). Analyses showed that supervisors’ EBPAS attitudes, their benefit rating, and their professional discipline were not significant predictors of therapists’ BBI usage. Analyses regarding the organization type suggested therapists at community facilities to possibly be more likely to use the BBI ($\widehat{\beta }$ = 0.59, SE = .35, p = .089) in comparison to those at large medical centers/hospitals.

DISCUSSION

Bridging the gap between research and practice, implementation of a behavioral medicine EBT was studied. Much of the focus within the DI literature, particularly in health psychology, is dissemination rather than implementation [49–52], with few subsequent reports of actual EBT usage. Marty, Rapp, McHugo, and Whitley [53] have proposed and operationalized more than 25 implementation outcomes (e.g., evidence for leadership, evidence for support staff), though interestingly, EBT usage is not among them. Considering the implementation demonstrations discussed previously, each used a different outcome, i.e., ratios of therapists trained to therapists using the treatment [54] or the number of patients treated, ratio of patients referred and patients treated [15], and non-ratio counts, such as number of patients treated [19, 16] or number of times that the treatment was used [10]. We used raw numbers, and therapists reported that objectively high proportions of patients, 58–68%, were treated with BBI, a finding replicated across time and with four institute cohorts.

All therapists received effective BBI training [28], and this was likely important in the high usage rates as level of expertise can be a predictor of level of implementation [36, 37]. Otherwise, the therapists were diverse in age, profession, and experience; came from settings varying in size, economic models of service delivery, and types of patients served; and resided in every region in the USA. All of the latter are different from the context for BBI’s design and testing, i.e., a randomized control trial (RCT) conducted at large university with a NCI-designated comprehensive cancer center in the Midwest with the BBI delivered in a Department of Psychology by clinical psychologists to predominately Caucasian breast cancer patients. The differences between the implementation settings and that of the RCT speak to the generalizability of the BBI.

Moreover, the high usage rates point to the scalability of the BBI. Many implementation trials are done in the context of systemwide implementations, such as the Improving Access to Psychological Therapies (IPAT) program in the UK or cognitive behavior therapy rollouts in the VAH [14], which include major efforts to achieve visibility of the EBT within the organization, multiple supports to supervisors and therapists, and others. For BBI implementation, therapists were provided with a “toolbox” of resources (e.g., adaptation plans, marketing materials) that they could use, but they alone were responsible for making BBI happen in their setting. This context makes the usage data all the more impressive.

Implementation support has been recommended to achieve usage. We do not know if the support “package” was influential, though a convergence of data suggests that it may have been. For example, 80–85% of the trainees were engaged with and sought guidance from the institute staff and its resources. The number of downloads of the educational materials suggests that therapists worked to achieve organizational and/or community awareness of the availability of BBI (e.g., Cancer Update News, 2015) and garner support for implementation, and they described such on the calls. Accumulating data suggest that clinical support and materials may be key to EBT implementation [55].

Health services are largely delivered through organizations which influence adoption and sustainability of innovations [56, 57]. Much of the organizational EBT literature chronicles negative aspects that hinder implementation [54, 58]. Even when organizational support is not barrier ridden but absent or weak, implementation suffers [59, 38]. Setting variables were examined and the managers/supervisors’ (positive) view of EBTs and their view of the effort needed to implement BBI were likely important for the respective therapists, but they were not influential in predicting usage. The absence of effect may have been due to the observed restricted ranges on the measures, as the managers as a group had favorable attitudes towards EBTs. Therapists’ verbal reports, however, suggested that organization type was important. Therapists from cancer centers, for example, spoke of the bureaucracy and the numbers of people from whom approval for implementation was needed. Other studies [60, 33] indicate that community settings may provide more flexible structures and climates; organizations open to innovations are more apt to consider using EBTs [61, 33].

Ultimately responsible for the delivery of EBTs, however, are therapists. Unlike findings from other studies [36, 37], therapist characteristics, including level of education and years in practice, were not associated with BBI usage, but there is a suggestion that the odds of BBI usage may have increased for therapists reporting spending more time teaching. The reason for this is not obvious, but there may be a relationship between those who teach and the appeal of an EBT like BBI, as therapists’ difficulty with understanding research support has been a barrier to some EBT implementations [62]. Taken together, the data shed some light on factors of potential influence, but additional research is needed to further test the role of setting and therapist factors in EBT implementations.

The outcome—proportion of patients receiving the EBT—does not illuminate the nature of its delivery; i.e., What were therapists’ adaptations versus “out-of-the-box” use of the BBI? There is a tension between delivery of an EBT as manualized and delivery that is achievable in a particular setting [63, 64]. With the diversity of therapists and organizations represented, some degree of adaptation was anticipated. To aid therapists’ familiarity with the treatment as tested, the manuals were used extensively during the institutes. Further, the adaptation plan was an effort to provide fidelity feedback before implementation proceeded. The percentage of manual downloads (20%), the hundreds of patient manuals subsequently provided, and anecdotal data from the conference calls suggest that when BBI was delivered, much of it was manualized, if only because therapists found it easier to do so. Lastly, the usage rates reported here occurred in the period of implementation support, and some [60, 65] note that sustainability begins when support ends.

We offer key messages for implementation and recommendations for future research. We note that clinical effectiveness research is concerned with external validity, generalizability, and patient outcomes, whereas implementation research assumes the effectiveness of the intervention and focuses on uptake of interventions by providers or systems of care. “Pure forms” of each differ in the unit of randomization and analysis, the nature of the intervention, and the measured outcomes, and thus, the contribution of one is relevant to the other, but their contributions are, in the main, non-overlapping. Alternatively, hybrid designs [66] are ones with a “built-in” dual focus. Such designs offer the means to achieve the outcomes enumerated in the final “S” level of STEPS and would be an important direction for DI of behavioral medicine EBTs. Regarding the BBI, these data confirm its scalability and transportability to diverse healthcare organizations. In need of further development is a tailoring of the BBI components to match levels of patient need, thereby achieving greater efficiency (and reduced costs) of psychological care and expansion to other modes of delivery, such as internet-assisted BBI.

In conclusion, the transfer of psychological EBTs to the community has proven challenging. To help, the National Institutes of Health and the US Department of Veterans Affairs [4] have both developed training programs in DI science. More DI demonstrations are needed as are studies determining if implementation actually occurs and, if so, do particular support strategies enable it [67]. For the present, the STEPS conceptualization, the dissemination model of the BBI Institutes, and the implementation support strategies employed stand as exemplars for future DI research in health psychology.

Compliance with ethical standards

All procedures, including the informed consent process, were conducted in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000.

Conflict of interest

The authors declare that they have no conflicts of interest.