Ensuring Treatment Fidelity in a Multi-site Behavioral Intervention Study: Implementing NIH Behavior Change Consortium Recommendations in the SMART Trial

Abstract

The Stories and Music for Adolescent/Young Adult Resilience during Transplant (SMART) study (R01NR008583; U10CA098543; U10CA095861) is an ongoing multi-site Children’s Oncology Group randomized clinical trial testing the efficacy of a therapeutic music video intervention for adolescents/young adults (11–24 years of age) with cancer undergoing stem cell transplant. Treatment fidelity strategies from our trial are consistent with the NIH Behavior Change Consortium Treatment Fidelity Workgroup (BCC) recommendations and provide a successful working model for treatment fidelity implementation in a large, multi-site behavioral intervention study. In this paper we summarize 20 specific treatment fidelity strategies used in the SMART trial and how these strategies correspond with NIH BCC recommendations in 5 specific areas: 1) study design, 2) training providers, 3) delivery of treatment, 4) receipt of treatment, and 5) enactment of treatment skills. Increased use and reporting of treatment fidelity procedures is essential in advancing the reliability and validity of behavioral intervention research. The SMART trial provides a strong model for the application of fidelity strategies to improve scientific findings and addresses the absence of published literature illustrating the application of BCC recommendations in behavioral intervention studies.

Introduction

The primary goal of treatment fidelity is to increase scientific confidence that changes in targeted outcomes are due to the intervention under investigation. Treatment fidelity includes methodological strategies that enhance reliable and consistent behavioral intervention delivery during clinical trials [1, 2]. Historically, the concept of treatment fidelity was conceptualized as “treatment integrity,” which examined whether an intervention was delivered as designed [3]. Over the years, the definition of treatment fidelity has expanded to incorporate three additional concepts including “treatment differentiation,” whether study conditions differed from one another in the intended manner; “treatment receipt,” whether participants understood or used the intervention skills during sessions; and “treatment enactment,” whether study participants used skills learned during the intervention in their daily life outside the study environment [3–5].

Treatment fidelity is vitally important to the design and implementation of behavioral intervention research because it influences investigators’ ability to address a wide range of study issues, including the ability to: 1) draw accurate conclusions about intervention efficacy, 2) replicate studies, 3) identify essential features of an intervention, 4) reduce random and unintended intervention variability to improve statistical power, 5) test theoretical questions, and 6) disseminate and translate clinical findings [1–3, 6]. Despite their importance, treatment fidelity strategies are inconsistently used and rarely reported [2, 3]. Reasons for limited reporting are difficult to determine; however, Borelli and colleagues speculate that reasons may include poor implementation by the researchers, lack of reporting despite satisfactory implementation, or journal editorial policies [2].

In 2004, the Treatment Fidelity Workgroup of the National Institutes of Health (NIH) Behavior Change Consortium (BCC) published recommendations to encourage more consistent incorporation of treatment fidelity strategies into behavioral intervention research. The recommendations describe 5 components of treatment fidelity including: 1) study design, 2) provider training, 3) treatment delivery, 4) treatment receipt, and 5) enactment of treatment skills. Published manuscripts describing successful implementation of BCC treatment fidelity recommendations will help investigators interpret the guidelines and design strategies for their own studies, encouraging greater adoption of the proposed guidelines. However, published examples of BCC implementation are currently limited [6–8]. The purpose of this paper is to define and describe treatment fidelity strategies being used in a multi-site phase II behavioral intervention study. Treatment fidelity strategies from our trial are consistent with the NIH BCC Treatment Fidelity Workgroup recommendations and provide a working model for successful treatment fidelity implementation in a large, multi-site behavioral intervention study.

SMART Trial Study Overview

The Stories and Music for Adolescent/Young Adult Resilience during Transplant (SMART) study (R01NR008583; U10CA098543; U10CA095861) is an ongoing multi-site Children’s Oncology Group randomized clinical trial testing the efficacy of a therapeutic music video (TMV) intervention against a low-dose audio-book control condition for adolescents/young adults (AYA) undergoing stem cell transplant. The Resilience in Illness Model (RIM) provides the theoretical framework that guides evaluation of the intervention (see Table 1 for dependent variables). Participants complete measures at baseline, immediately post-intervention, and 100-days post-transplant. Throughout the study period, investigators at 6 children’s hospitals and 3 adult hospitals across the United States have accrued participants and delivered the intervention and comparison protocols.

Table 1.

Resilience in Illness Model Latent Variables and Components

	RIM Latent Variables	Components
1	Illness-related Distress	Uncertainty in Illness Symptom-related Distress (pain, anxiety, fatigue, mood)
2	Family Environment	Family Adaptability/Cohesion, Family Communication
3	Perceived Social Support	Perceived Social Support from Friends Perceived Social Support from Health Care Providers
4	Defensive Coping	Evasive, Emotive, Fatalistic
5	Positive Coping	Confrontive, Optimistic, Supportant
6	Derived Meaning	Hope and Spiritual Perspective
7	Resilience	Confidence, Self-Esteem, Self-Transcendence
8	Quality of Life	Sense of Well-being

The TMV is a 6-session intervention (2 sessions/week over three weeks) that uses songwriting and video production to encourage self-reflection and communication about what is important to the AYA. The intervention is structured to accommodate the anticipated occurrence of treatment-related symptoms, such as fatigue, pain, and mucositis, by clustering active intervention activities such as brainstorming, songwriting, and voice recording during the first 3 sessions and more passive activities such as picture selection and storyboarding during the last 3 sessions. The low-dose audio-book control condition was structured to parallel the timing, duration, and individual therapist contact of the TMV intervention. During the low dose audio-books condition, AYA choose audio-recorded books from a standardized list of selections. Participants then listen to the recording and have the opportunity to discuss their impressions about the book content with a trained intervener. The intervention team is comprised of 14 board-certified music therapists (referred to as interveners), who deliver the 6-session manualized intervention and a low dose control conditions to adolescents/young adults, 11 – 24 years of age, undergoing stem cell transplant for an oncology condition. Prior to study implementation, the core investigative team established comprehensive treatment fidelity procedures.

Procedures

In this paper we describe SMART trial treatment fidelity strategies, organized through the framework of the NIH BCC Treatment Fidelity recommendations. Table 2 provides definitions for each of the 5 treatment fidelity components proposed by the NIH BCC. Subsequently, we describe specific strategies employed in the SMART trial for each treatment fidelity component.

Table 2.

Treatment Fidelity Components and Definitions^*

Treatment Fidelity Component	Definition
Study Design	Study design procedures ensure that a study can adequately test its hypotheses in relation to underlying theory and clinical practices.
Training Providers	Standardized provider training includes procedures to ensure that interventionists have been satisfactorily trained to deliver the intervention to study participants.
Delivery of Treatment	Treatment delivery procedures monitor and improve delivery of intervention and comparison conditions; ensuring that treatment is delivered as intended.
Receipt of Treatment	Treatment receipt focuses on the participant and includes procedures to assure that the treatment was both received and understood.
Enactment of Treatment Skills	Enactment of treatment skills includes processes to monitor and improve participant ability to perform treatment-related behavioral skills and cognitive strategies in relevant real-life settings as intended.

^*Bellg et al. (2004). Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations for the NIH Behavior Change Consortium. Health Psychology, 23(5), 443 – 451.

1. Study Design

According to the BCC, study designs play an essential role in treatment fidelity. Three study design features that specifically enhance treatment fidelity include: 1) the use of theory to guide intervention design and the selection of variables, 2) specifications about treatment dose within and across study conditions, and 3) advanced planning for implementation setbacks. Table 3 summarizes study design goals, with corresponding strategies employed in the SMART Trial.

Table 3.

Treatment Fidelity Strategies for Study Design

Goal	NIH BCC Description	SMART Strategies
Ensure that interventions are congruent with relevant theory and clinical experience.	Operationalize interventions to reflect theory; define independent and dependent variables most relevant to the “active ingredient” of the intervention.	Intervention Theory (Robb, 2000) “Essential” intervention elements: Structure Autonomy support Supportive Relationships Theory of Problem/Measurement Model (Haase, 2004)
Ensure same treatment dose within and across conditions.	Ensure that treatment “dose” (measured by number, frequency, and length of contact) is adequately described and is the same for each subject within and across treatment and control conditions.	Dose specifications: 6 sessions 2 sessions/week for 3 weeks 60 min/session Opportunities to engage in condition-related activities outside sessions Dose monitoring: Web-based intervener field note: Session date Session begin/end time Patient-reported time spent on condition-related activities outside sessions Dose confirmation: External quality assurance monitoring Audio-recorded sessions
Plan for implementation setbacks.	Address possible setbacks in implementation (e.g., intervener turn-over).	Train minimum of 2 interveners per study site to guarantee intervener availability and minimize impact of turnover, vacations, and illness.

Ensure interventions are congruent with relevant theory and clinical experience

The SMART trial integrates two theoretical models - one model to inform intervention design, and a second model to describe the problem and inform outcome measures. Robb’s Contextual Support Model of Music Therapy provides a model for the intervention [9, 10]. Based on motivational coping literature and clinical experience, Robb’s model specifies “essential” elements of the music-based video intervention and how they function to improve resilience outcomes identified in Haase’s Resilience in Illness Model [11–14]. Similarly, Robb’s model guided design of the low dose condition, which controls for 3 nonspecific treatment effects including personal attention, audio stimulation, and structured time. The Haase Resilience in Illness Model (RIM) describes the mechanisms for the phenomena we are targeting – how to increase protective factors and decrease risk factors associated with positive health outcomes in adolescents and young adults (AYA) undergoing stem cell transplant for cancer. The RIM describes possible mechanisms that explain improvement in positive health outcomes for AYA during cancer treatment and provides a validated measurement model to evaluate study outcomes [11–15].

Ensure same treatment dose within and across conditions

There are four treatment dose specifications in the SMART trial that ensure consistency within and across the intervention and low dose conditions (see Table 3). Dosing is monitored using secure, web-based field notes that interveners complete on desk-top computers after completing each session. Interveners record the session date, time the session began and ended, and the amount of time the patient reported working on study-related activities between sessions (See Figure 1).

Figure 1.

Intervener Field Note

Plan for implementation setbacks

When a study site has only one trained intervener, it is vulnerable to lapses in recruitment and/or intervention delivery due to intervener illness, vacation, schedule conflicts, or changes in job status. In order to avoid potential disruptions we train and maintain a minimum of two interveners for each study site.

2. Training Providers (Interveners)

A second component of treatment fidelity is the use of standardized training programs to ensure consistent delivery of study protocols across interveners and sites. The necessity for standardized training is obvious in large behavioral intervention studies requiring multiple interveners who have different educational and clinical backgrounds. Table 4 summarizes intervener training goals and corresponding SMART strategies. (Note: in the SMART trial we refer to providers as interveners and use this term in the subsequent discussion).

Table 4.

Treatment Fidelity Strategies for Training Providers

Goal	NIH BCC Description	SMART Strategies
Standardize training.	Ensure that training is conducted similarly with different interveners.	Standardized treatment protocols/training manuals. Standardized training session content: Study overview Theories underlying the problem and intervention Clinical overview of SCT and AYA development Intervention protocol training Low dose condition training Intervener responsibilities & workflow/communication Quality assurance procedures Online database and study management website Strategies to minimize communication bias and unblinding Participant evaluation of training sessions.
Ensure provider skill acquisition.	Train interveners to well-defined performance criteria.	Role playing an essential feature of training. External quality assurance monitoring until interveners meet set criteria for adherence.
Minimize “drift” in provider skills.	Ensure that intervener skills do not decay over time (e.g., show that intervener skills demonstrated halfway through the intervention period are not different than skills immediately after initial training).	External quality assurance monitoring includes periodic checks once intervener reaches competency. Bi-weekly intervention team conference calls.
Accommodate provider differences.	Ensure adequate level of training in interveners of differing skill level, experience, or professional background.	Individualized supervision and monitoring. Position description. Minimum competencies.

Standardize training

Many behavioral researchers standardize treatment and control condition protocols, but rarely describe standardization of training [2]. Behavioral intervention trials can take several years to complete, creating an elevated risk for intervener turnover and repeated training for new personnel. Standardizing protocol training assures that interveners have equivalent training experiences and are equipped to consistently deliver study protocols regardless of when they join the study team. In the SMART trial all interveners receive a study manual and attend a 2-day, 12-hour training session. The study manual includes two categories of information: 1) generic study-related information, and 2) intervener-specific information. Table 5 summarizes study manual content. Training session content is also standardized and delivered by members of the study’s executive team. SMART intervener training is comprehensive and includes nine major topics (see Table 4).

Table 5.

SMART Intervener Study Manual

Generic Study-Related Information*

Standard Operating Policies/Procedures

Study Overview

Reporting/Documentation Guidelines

Communication Flowchart

Overview Adolescent/Young Adult Development

SCT treatment progression/side effects

Intervener-Specific Information

Job Description/Responsibilities

Intervention Protocol

Low Dose Control Protocol

Related Literature (theory/clinical implementation)

Quality Assurance Monitoring Forms/Procedures

^*Generic information is provided to all study personnel (i.e., project managers, co-investigators, evaluators).

Ensure provider skills acquisition and minimize drift

Role playing music video and low dose protocols is an essential training feature that helps trainers evaluate intervener competence, readiness to deliver study conditions, and the need for individualized instruction. After training, interveners participate in bimonthly intervention team conference calls to discuss on-going cases, answer questions, and review study protocol procedures.

To minimize “drift” over time, our study team implemented quality assurance monitoring procedures that include criteria for evaluating intervener adherence in delivering both intervention and low dose control protocols. Quality assurance procedures include self-monitoring and external monitoring. All study sessions are audio-recorded, encrypted, and uploaded to a secure, web-based server that can only be accessed by the intervener and external monitor. Self-monitoring procedures require that interveners listen to their audio-recorded session and simultaneously complete a session-specific checklist that specifies essential elements of each study session (See Figure 2). Similarly, an external monitor listens to an intervener’s audio-recorded session and completes the same session-specific checklist. External QA monitoring is conducted by two primary investigators who are PhD-prepared, board-certified music therapists.

Figure 2.

Intervener Quality Assurance Checklist

Self- and external monitoring are completed for the first 5 participants randomized to each condition. Individual phone meetings are scheduled to discuss any quality assurance discrepancies. If there are minimal to no discrepancies during initial monitoring, self- and external monitoring are completed for every 5^th participant enrolled in each condition. If there continue to be discrepancies or if an intervener has not conducted sessions for 2 months, self- and external monitoring is completed for the next enrolled participant.

Accommodate provider differences

This ongoing, individualized form of training and professional supervision is one way we accomplish the fourth goal of intervener training, which is to accommodate intervener differences in experience, skill, and background. Intervener differences have also been minimized by developing a clear position description that specifies intervener qualifications. All interveners must hold a bachelor’s degree (or its equivalent) in music therapy from an American Music Therapy Association-accredited program and hold board certification through the Certification Board for Music Therapists. In addition, interveners must have training and experience in delivering music-based interventions that are grounded in a cognitive-behavioral approach to therapy.

3. Delivery of Treatment

The third component of treatment fidelity is the use of strategies to ensure that treatment is delivered as intended. Primary strategies include standardization of treatment protocols and monitoring intervention delivery. Table 6 summarizes treatment delivery goals, with corresponding SMART strategies.

Table 6.

Treatment Fidelity Strategies for Treatment Delivery

Goal	NIH BCC Description	SMART Strategies
Control for provider differences.	Monitor and control for subject perceptions of nonspecific treatment effects (e.g., perceived warmth, credibility, etc. of intervener) across intervention and control conditions.	Audio-recorded sessions to allow supervisor feedback to interveners. Monitor intervener interaction styles and attributes. Hire interveners that demonstrate supportive interaction style and have experience working with AYA. Interveners work with both intervention and low dose condition participants. Conduct qualitative interviews at the end of the study.
Reduce differences within treatment.	Ensure that interveners in the same condition are delivering the same intervention.	Manualized intervention and low dose protocols. Audio-recorded sessions for self and external quality assurance monitoring. Standardized quality assurance checklists for intervention and low dose control conditions.
Ensure adherence to treatment protocol.	Ensure that treatments are being delivered in the way in which they were conceived with regard to content and treatment dose.	See quality assurance strategies listed above. Review omissions and/or protocol deviations with interveners on an individual basis. Computerized checklists allow protocol deviation tracking across interveners and conditions.
Minimize contamination between conditions.	Minimize contamination across treatment/control conditions, especially when implemented by same intervener.	Specific protocols for intervention and low dose conditions. Train interveners on theory/rationale underlying structure/delivery of low dose condition. Train interveners to address participant questions about randomization and their assigned condition using non-biased explanations. Audio monitoring of protocol adherence and intervener responses to patient/family questions about the study.

Control for provider differences

The first goal of treatment delivery is to monitor and control for nonspecific treatment effects that might result from intervener-specific traits or common factors, such as personality or interaction style. We use five strategies to control for intervener-specific traits that are listed in Table 6. As previously described, all intervener sessions are audio-recorded to monitor protocol adherence, including monitoring therapist attributes that have been shown to contribute to positive outcomes such as warmth, empathy, and respect for the patient [16]. Our second strategy is to observe the interaction style of potential interveners during job position interviews. We hired individuals who demonstrated desired attributes such as warmth and active listening. In addition, we hired individuals who had previous clinical work experience with adolescents and young adults. The goal of our third strategy is to minimize therapist effects across study conditions by training interveners to deliver both the intervention and low dose conditions. Although this introduces a potential threat for therapist bias it controls for differences in discipline-specific background and training. We also have a minimum of two interveners per study site, which allows us to rotate interveners across study participants. Finally, we use post-study qualitative interviews to monitor non-specific therapist effects reported by patients and parents post-intervention. During these interviews, parents and patients are asked to describe their experience with their assigned study condition.

Reduce differences within treatment/ensure adherence to protocol

Strategies to reduce differences within treatment and ensure intervener adherence to protocols are similar and include the use of manualized protocols for the intervention and low dose control conditions, audio-recorded sessions to facilitate quality assurance (QA) monitoring, and standardized QA checklists to monitor protocol adherence. The “gold standard” for ensuring adequate treatment delivery is to evaluate sessions based on a priori criteria using direct observation or recorded sessions. In the SMART trial all sessions are audio-recorded and then evaluated for adherence using a standardized quality assurance monitoring form. As previously described, we use two types of QA monitoring: self-evaluation and external monitoring. Requiring interveners to listen to their own sessions and complete session-specific QA checklists helps interveners advance their own skills by heightening their awareness of protocol deviations and more subtle aspects of intervention delivery including timing, word choice, and interaction style. Self-evaluation also serves to reinforce essential content that needs to be delivered in each session.

Minimize contamination between conditions

The fourth goal of treatment monitoring is to minimize contamination between conditions, especially when treatment and control conditions are delivered by the same provider. Cross-contamination occurs when a study participant in the control condition receives components from the treatment condition. Five of the previously described SMART strategies also help control for cross-contamination; these strategies are summarized in Table 6.

4. Receipt of Treatment

Unlike the previous aspects of fidelity focused on how the treatment is delivered, receipt of treatment focuses on patients and their ability to perform protocol-related skills. If a patient does not understand or is unable to perform skills presented in the intervention protocol, an otherwise effective intervention may be incorrectly identified as ineffective [1]. Similarly, if patients are unable to perform low dose condition skills, the study is at risk for inadequate controls and may lead to erroneous findings. Table 7 summarizes treatment receipt goals, with corresponding strategies from the SMART trial.

Table 7.

Treatment Fidelity Strategies for Receipt of Treatment

Goal	NIH BCC Description	SMART Strategies
Ensure participant comprehension.	Ensure that participants understand the information provided in intervention, especially when participants may be cognitively compromised, have a low level of literacy/education, or not be proficient in English.	Intervener uses active questioning and behavioral observation to assess patient comprehension.
Ensure participant ability to use cognitive skills.	Make sure that participants are able to use the cognitive skills taught in the intervention (e.g., reframing, problem solving, preparing for high-risk situations, etc.).	We use the same strategies to assess both cognitive & behavioral skills. Standardized documentation about patient involvement. Narrative summarizing patient response to protocol and potential external factors influencing response.
Ensure participant ability to perform behavioral skills.	Make sure that participants are able to use the behavioral skills taught in the intervention (e.g., relaxation techniques, food diaries, cigarette refusal skills, etc.).

Ensure participant comprehension and ability to use/perform skills

Interveners are trained to assess patient comprehension through active questioning and performance of study-related activities through behavioral observation. Interveners document their observations and assessments using a secure, web-based field note entry. The field note includes a 9-item, Likert-type scale to assess behavioral indicators of patient engagement and space for a narrative entry (See Figure 1). Behavioral engagement includes patient participation (i.e., attentive), comprehension (i.e., understood directions), and effort (i.e., tried hard). Narrative notes summarize patient response to study conditions, social or environmental factors that may influence treatment receipt (e.g., family or friends present in the room; number of interruptions), and patient or family member comments. Narrative notes augment information from the behavioral engagement items and provide information that can be used to interpret response to treatment.

5. Enactment of Treatment Skills

Fidelity strategies for enactment of treatment focus on patients’ ability to perform treatment-related behavioral and cognitive skills in relevant real-life settings. This is not to be confused with treatment adherence or treatment efficacy; rather, it is specific to patients implementing study-related skills outside of sessions. Table 8 summarizes treatment enactment goals with corresponding strategies.

Table 8.

Treatment Fidelity Strategies for Enactment of Treatment Skills

Goal	NIH BCC Description	SMART Strategies
Ensure participant use of cognitive skills.	Ensure that participants actually use the cognitive skills provided in the intervention in appropriate life settings.	Interveners encourage participants to engage in independent “homework” between sessions, but it is not required. Documentation of out-of-session study-related activities. Individual qualitative interviews are conducted with study participants and their parents 100 days post-transplant.
Ensure participant use of behavioral skills.	Ensure that participants actually use the behavioral skills provided in the intervention in appropriate life settings.

Ensure participant use of cognitive/behavioral skills

In the SMART trial we are testing an intervention that is implemented during acute treatment for cancer. Enactment of cognitive/behavioral skills beyond treatment sessions is not a primary aim of the intervention, nor is it required for successful participation in study conditions. However, opportunities for treatment skill enactment do exist and occur at two time points: 1) between scheduled sessions, and 2) post-treatment.

Between scheduled sessions

The music video intervention relies on the participant’s willingness and ability to engage in the creative processes of lyric writing and video design. Interveners encourage participants to engage in independent lyric writing and video design “homework”, but it is not required. For the low dose condition, the intervener provides instruction and assesses the participant’s understanding of the steps needed to listen in the intervener’s absence, but between-session listening is not required.

Post-treatment

We are conducting qualitative interviews with study participants 100 days post-transplant. Interviews will help us determine how study participants experienced both the intervention and low dose study conditions and whether or not they continued to use study-related skills once they completed sessions.

To illustrate further, during the final session for our music video condition, participants view their completed music DVD project and are given an opportunity to share their video with family, friends, and/or healthcare providers. Enactment of study-related skills post-treatment is not considered an essential element of the intervention we are testing, so participants are not directed to use the DVD or study-related skills in a specified manner. Rather, participants choose how they use their music DVD post-intervention. It is possible that enactment of study-related skills may occur post-intervention for the intervention and low dose control groups. Our investigative team is very interested in learning more about how patients may be using study-related skills post-intervention; qualitative interviews will allow us to answer these questions.

Discussion

The NIH BCC recommends that behavioral intervention researchers use strategies to address five components of treatment fidelity. Although the BCC provides useful conceptual descriptions of these components, the responsibility for demonstrating their successful employment belongs to investigators who, through thoughtful design, implementation, and evaluation strategies, work to enhance the causal inferences of their intervention studies.

The successful implementation of the guidelines requires the extensive knowledge and creative endeavors of a multidisciplinary investigative team that can design strategies that will create a fit with the ideographic nature of each intervention - taking into consideration the nature of the intervention, study design, and desired outcomes. Despite the need, there have been only a small number of published accounts that describe implementation of BCC recommendations in a behavioral intervention trial [6–8]. In addition to this paper, each of the aforementioned studies has defined components of treatment fidelity within the context of the trial and described a variety of intervention and study-specific strategies used and corresponding challenges.

Four primary challenges encountered by our study team while implementing the BCC guidelines were: 1) clearly describing the theoretical link between the identified problem and the intervention – explaining how the TMV would affect change in RIM latent variables, 2) using the intervention theory to guide decisions about low-dose condition design - identifying intervention features that needed to be controlled for through the low-dose condition, 3) managing extended time periods between participant accrual within sites, which resulted in additional intervener quality assurance monitoring – interveners did not have enough opportunities to establish adherence within the timeframes specified in the QA monitoring protocol to reduce the requisite number of monitored sessions, and 4) addressing high intervener turn-over that resulted in more intervener training and QA monitoring. A majority of the challenges were mitigated by identifying the challenges and problem solving strategies during our pilot studies, which helped to decrease study start-up time, study set-backs, and related costs. Quality assurance consistently required time and attention throughout the study that resulted in increased study costs. However, our study team found the vigilance was essential for overall scientific and clinical integrity of the study. Early and consistent attention to QA monitoring helped establish a routine that was not viewed by the team as burdensome, but as a source of professional growth and development. Our intervention team found the QA monitoring process to be something that helped strengthen professional skills both in and outside the study context. We found that the application and design of strategies for some components of treatment fidelity are more ambiguous than others. For example, “training providers” and monitoring “treatment delivery” are fairly clear concepts, so identifying appropriate strategies is straightforward. In contrast, components such as “treatment enactment” are less clear and may prove to be the most challenging aspect of fidelity for investigative teams to define and monitor.

Treatment enactment, by BCC definition, refers to the intervention skills used outside of the study context. This is distinct from the identification of study outcomes (efficacy) or treatment adherence. Resnick and colleagues (2005) defined it as skills required to achieve study outcomes[6]. Radziewcz and colleagues (2009) did not use strategies to monitor treatment enactment, stating that findings about patient preferences and engagement in the intervention over time would ultimately characterize fidelity for enactment [8]. In our trial, we monitored patient-initiated engagement in study-related activities between scheduled sessions as an indicator of treatment enactment. These variations demonstrate the importance of having investigative teams interpret, define, and identify strategies within the context of their own study.

In summary, ensuring treatment fidelity through the BCC recommendations requires thoughtful application – balancing realities of the clinical world with rigorous study design. However, the effort dedicated to the design of effective strategies will assist in protecting the internal validity of the study. We have presented one model for the application of fidelity strategies to improve the scientific findings.