Abstract
Purpose
A growing waitlist for Mindfulness in Motion (MIM), an evidence-based worksite mindfulness-based intervention, necessitated a training system with built in fidelity assurance to meet program demand. MIM was delivered as part of an organizational strategy in a large academic health center to enhance Health Care Professional (HCP) well-being. In order to ensure that the intervention was being delivered the same way to each cohort, a process to ensure intervention fidelity was developed for MIM.
Method
The core components of MIM informed the development of a detailed fidelity monitoring system to ensure consistent intervention delivery. Each MIM cohort was conducted with both trained facilitators and trained intervention fidelity monitors.
Results
Across 11 cohort offerings of MIM, each 8 weeks in length, there was a mean adherence rate of 0.9886, SD = 0.0012.
Conclusion
The fidelity monitoring system allowed for a reliable expansion of MIM offerings to HCPs and for a seamless pivot to fully virtual MIM delivery, necessitated by COVID-19.
Keywords: Intervention Fidelity, Mindfulness, Health Care Professional, Organizational Strategies, Well Being
Introduction
Health Care professionals (HCPs) experience high levels of stress on the job, predisposing them to burnout, and its host of negative consequences. 1 This epidemic is now widely recognized with an urgent call to reduce HCP occupational distress. 1 In 2017, the first author and her research team set out to address this through organizationally sponsored delivery of Mindfulness in Motion (MIM), an 8 week, 1 hour/week resiliency building intervention designed for HCPs to prevent burnout.2,3 The 300+ person waitlist for MIM grew at a rate faster than the creator of MIM (first author) could manage; thus, a training program for MIM facilitators was developed to ensure program fidelity and to address this need. Taking direction from literature authored by the National Institute of Health (NIH) Behavior Change Consortium, 4 the research team wanted to ensure replicability of MIM content so that it could be delivered in a reliable way by multiple facilitators.
To ensure consistency of program content, eight 15 minute videos were created to deliver the didactic scientific background of mindfulness alongside eight 15 minute experiential practice videos. This allowed for content to remain the same, but for a facilitator to be trained to lead the 30 minute discussion in between the didactic and experiential video delivery. As participants who were particularly impacted by MIM began requesting to be trained as MIM facilitators, it became apparent that it would be possible to expand the reach of this program organizationally through a train-the-facilitator model that utilized uniform videos to deliver content, a standardized facilitator training, and manual, with touch points during the 8 week session. This approach followed the recommendations for enhancing treatment fidelity in health behavior change studies. 4
A common criticism of mind–body interventions is the lack of validity and reliability. 5 Fidelity monitoring is essential for successful mind–body intervention research to move forward, and for this study, it was essential to collect data to determine that all HCPs receiving MIM received the same intervention. The training program for facilitators taught them how to actively lead the discussion portion of the weekly intervention with standardized reflection questions. Facilitators did not teach the program, rather they led the program based on a detailed manual therefore ensuring consistency across cohorts.
In a previous study on MIM, participant outcome data showed no significant difference between creator-led MIM (n = 137) and facilitator-led MIM (n = 83) across the 4 major outcomes of burnout, perceived stress, resilience, and work engagement. 6 Up to this point, creator-led and facilitator-led MIM were conducted in-person. This was an important step in ensuring intervention integrity, as it was necessary to confirm that the MIM training program for facilitators delivered similar outcome results. Fidelity assurance also supported the organizational goal of large-scale dissemination of the MIM program. Due to the advent of COVID-19, fully virtual cohorts of MIM were necessitated; the existing fidelity monitoring system made this possible.
Fidelity literature is sparse regarding mind–body interventions, further warranting the need for a concrete and detailed fidelity checking system for MIM expansion. 5 More recent research shows that there are four key components crucial to intervention fidelity: design, training, delivery, and intervention receipt. 7 Design acts as a treatment manual outline, describing the intervention theory, strategies, and goals. 7 Training allows for the proper intervention skill levels, such as education, experience, and implementation style. 7 Monitoring of delivery is the most important to ensure competence and understanding of intervention by the facilitator. 7 Intervention receipt includes measures to test validity and reliability. 7 The MIM facilitator training program was designed with intervention fidelity in mind, and the intervention fidelity monitors were included in each of the MIM facilitator training sessions so that both fidelity monitors and facilitators understood program logistics, goals, and delivery details.
In addition to the MIM-specific fidelity monitoring system, the Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC) system, developed in 2012, offers a broad set of guidelines that can be used to evaluate fidelity of mindfulness-based interventions. 8 When fidelity monitoring was developed for MIM, researchers were unaware of MBI:TAC, yet similarities exist. Many of the key domains of MBI:TAC are reflected in the MIM fidelity monitoring system: session organization and pace, facilitator embodiment of mindfulness, guiding mindfulness practices, didactic teaching, and community in the group learning environment. The didactic teaching is delivered via the prerecorded videos. The MBI:TAC is a broad system; the MIM fidelity monitoring system embodies similar domains in a fashion tailored specifically to the MIM intervention. This similarity further supports the validity of the MIM fidelity monitoring system. This report details the facilitator training program and monitoring system utilized to ensure MIM intervention integrity.
Methods
The fidelity monitoring system was developed as part of a larger study on the password protected website portal specific to MIM which includes pre/post intervention assessment (IRB Approval #: 2017B0321).
Facilitator Selection
Potential MIM facilitators (n = 7) were recruited during their time as participants in the MIM program. Facilitators come from various backgrounds in the health care field, including but not limited to physician, clinical psychologist, public health professional, nurse, physical therapist, and medical resident. Qualifications for an individual to be invited to be a facilitator included having an embodied mindfulness practice and displaying confidence in one’s ability to lead a group. If interested in becoming a facilitator after an invitation was extended, each person completed the MIM program as a full participant as well as attended an 8 week session shadowing another more experienced facilitator. One of the sessions they attend (whether as a participant or in a shadowing role) must be led by the program creator and then one must be led by a facilitator using the prerecorded videos.
Facilitator Training
Following initial participation and shadowing, the future facilitator then attends a 2 ½ day training. The first day consists of an overview of the 8 weekly themes and logistics of program delivery. The second day consists of each facilitator in training presenting one week of MIM to the creator and MIM program managers to receive feedback and verify competency. Each facilitator is given a facilitator manual that details how to deliver each part of the program, including showing didactic and experiential videos as well as facilitating discussion questions. At the end of the 2 ½ day facilitator training, fidelity monitors (student research assistants) are included in the training in order to understand the program structure and the fidelity monitoring system. Each facilitator is assigned one fidelity monitor per each MIM cohort they facilitate. To address any problems that arise and to learn from other facilitators and fidelity monitors, three half-hour meetings are conducted with the facilitators, fidelity monitors, and first author throughout the 8 weeks of the MIM program.
After this training sequence is completed, a facilitator is deemed ready to lead their own MIM cohort. Each facilitator had previously participated in MIM before being trained to lead MIM sessions themselves and was encouraged to draw upon their own mindfulness experience as a participant to further enhance the cohorts they led.
Fidelity Monitoring Checklist
To determine which items to include in the fidelity monitoring checklist, MIM program managers and the first author discerned core elements of MIM: reflective writing, didactic instruction (via prerecorded video), community building discussion, experiential yoga/mindfulness practice (via prerecorded video), and closing meditation. Using this core structure as a framework, the essential items were incorporated into a monitoring checklist (Table 1). The purpose of the checklist was to ensure that facilitators felt comfortable teaching MIM in the same structured manner each time, yet allowed facilitators to use their own expertise in leading the discussion portion of MIM. This process ensured that all program participants received the program in the same way, yet allowed for facilitator and participant interaction to be authentic and relational. 9
Table 1.
Mindfulness in Motion fidelity monitoring checklist example:Week 3.
| Fidelity check assessment | ||||
| Core | MIM for health care professionals delivered in-person | MIM for health care professionals delivered virtually | Meets Expectation Yes/No | Ideas for Improvement |
| Be present 15 minutes early | Arrives 15 minutes early | Starts meeting 15 minutes early, ready for early virtual joiners | ||
| Have music playing before start time | Sets up music and has it playing before others arrive | Sets up music and has it playing before others join | ||
| Video prepared | Has video ready to go (downloaded to computer or from thumb drive) and materials set out | Has video ready to go (downloaded to computer or from thumb drive) | ||
| Reflection | As participants enter, tells them what reflection page they need to be working on when they get seated (PG 22-23 of workbook) | As participants join, tells them to work on reflection page that they received via email: What is something that you care about, and because you value this, it causes you to feel stressed? | ||
| Pre breath count | Breath count—enter on PG 21 | Breath count—participants write on reflection page and enter into Qualtrics link sent by fidelity checker in zoom chat box | ||
| Reflection question discussion | Discusses the reasons for thinking about the answers on PG 22 | Discusses the reasons for thinking about the answers on reflection question | ||
| Video on the evidence base of mindfulness | Shows week 3 didactic video | Shows week 3 didactic video | ||
| Facilitates discussion | Facilitates discussion on a habit that you do that helps you be who you want to be (either at home or at work!) w/o inserting self and keeping discussion moving and within time frame | Facilitates discussion on a habit that you do that helps you be who you want to be (either at home or at work!) w/o inserting self and keeping discussion moving and within time frame | ||
| Discusses breath work and nervous system connection as detailed in training manual | Discusses how breath work is tied with our nervous system | Discusses how breath work is tied with our nervous system | ||
| Experiential practice | Shows experiential video | Shows experiential video | ||
| Post breath count | Conducts 2nd breath count—enter on PG 21 | Conducts 2nd breath count—enter on Qualtrics link | ||
| Class closure via facilitator | Nice closure to class, offering a closing reflection on what emerged during discussion | Nice closure to class, offering a closing reflection on what emerged during discussion | ||
| Home practice | Home practice for week emphasized by facilitator | Home practice for week emphasized by facilitator | ||
| Ending class | Ending class on time | Ending class on time | ||
Results
There were 11 cohorts led by 7 trained facilitators, with a student research assistant conducting the fidelity monitoring checklist each time. In the 11 MIM cohorts, there were a total of 128 participants (average of 12 participants per cohort, with a maximum of 18 participants per cohort so that every participant has the opportunity to participate in the discussion while keeping to the one-hour session length). In order to determine adherence rate to the monitoring checklist, each week was considered one unit. Adherence for that week was determined by meeting every item on that week’s fidelity monitoring checklist. If an item was not met, it was considered nonadherence to protocol for that week. Therefore, throughout the 11 cohorts, there were 88 total weeks upon which to check adherence (8 weeks per cohort × 11 cohorts = 88 total weeks). Of these 88 weeks, there was only one week in a single cohort in which adherence was not met. Therefore, the number of weeks in which adherence was 100% was 87 of the 88 total weeks analyzed, yielding a mean adherence rate of 0.9886, SD = 0.0012.
Discussion
This study detailed the process of ensuring intervention integrity and fidelity of the MIM program regardless of which facilitator delivered the intervention. This was accomplished through creating video content that was delivered to all facilitator-led cohorts, the facilitator training program, through continuous evaluation via a fidelity monitoring checklist, and lastly, through check-ins three times during each of the MIM 8 week sessions. Check-ins with the creator and program manager allowed facilitators to reflect on their experience and make iterative adjustments when needed. These meetings proved to be a critical step as they enabled real-time adjustments to be made during the 8 week program, as needed.
By ensuring that an intervention is delivered the same way each time, confounding variables are minimized, thus enhancing internal validity. Reliability and external validity are increased when the program fidelity is ensured and similar results are shown through various trials.10,11 Because reliability and external validity have been illustrated, this supports the overall content validity of MIM. The fidelity monitoring checklist system developed for MIM enabled the reliable expansion of MIM for HCPs. It also allowed for a seamless pivot for adapting MIM to virtual delivery when necessitated by the advent of COVID-19. The core elements of the program had already been identified and incorporated into a checklist for each week. Transitioning to virtual delivery was facilitated by ensuring that each item on the fidelity monitoring checklist was converted to allow for consistent virtual delivery, across multiple facilitators.
An area for future inquiry would be to assess the facilitator/participant relationship that occurs during each cohort, as other studies have explicated the importance of the relationship between participants with trained trainers.8,9 Although session content is standardized, the individual facilitator brings their embodied mindfulness practice to each session. Future research exploring the impact of facilitator on participant experience may prove to be a valuable study.
Conclusion
With the addition of a standardized facilitator training and manual, followed by intervention monitoring, MIM fulfills the recently outlined blueprint for organizational strategies addressing HCP well-being, 12 which suggests foundational programming, cultural transformation, iterative experimentation, and sustainability.13-15 This study demonstrated the replicability of MIM content between multiple facilitators delivering MIM following a rigorous facilitator training program. Without ensuring intervention integrity, it would not have been possible to scale and disseminate the MIM to the extent it has throughout our health system, for both in-person and virtual delivery. It was the development of fidelity assurance that was the necessary thread that has allowed for embodiment of an organizational strategy, scaling MIM for dissemination across the health system, to help promote health care professional well-being.
Footnotes
Declaration of Conflicting Interests: The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) received no financial support for the research, authorship, and/or publication of this article.
ORCID iDs
Maryanna D. Klatt https://orcid.org/0000-0001-6932-8221
Rani Bawa https://orcid.org/0000-0002-4215-170X
References
- 1.National Academies of Sciences EaM . Taking Action against Clinician Burnout: A Systems Approach to Professional Well-Being. The National Academies Press; 2019. [PubMed] [Google Scholar]
- 2.Klatt M, Steinberg B, Duchemin AM. Mindfulness in motion (MIM): An onsite mindfulness based intervention (MBI) for chronically high stress work environments to increase resiliency and work engagement. J Vis Exp 2015;(101):e52359. doi: 10.3791/52359. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Moffatt-Bruce SD, Nguyen MC, Steinberg B, Holliday S, Klatt M. Interventions to reduce burnout and improve resilience: Impact on a health system’s outcomes. Clin Obstet Gynecol. 2019;62(3):432-443. [DOI] [PubMed] [Google Scholar]
- 4.Bellg AJ, Borrelli B, Resnick B, et al. Enhancing treatment fidelity in health behavior change studies: Best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004;23(5):443. [DOI] [PubMed] [Google Scholar]
- 5.Stone JA. Why does treatment fidelity matter. Altern Ther Health Med. 2015;21(4):24-25. [PubMed] [Google Scholar]
- 6.Klatt M, Blake A, Bawa R, Gabram O. In-person vs. partial-video delivery for mindfulness programming in interprofessional healthcare professionals: Does it influence the outcomes? Global Advances in Health and Medicine. 2020;9(ICIMH 2020 Abstracts):P08.26LB. doi: 10.1177/2164956120912849 [DOI] [Google Scholar]
- 7.Gearing RE, El-Bassel N, Ghesquiere A, Baldwin S, Gillies J, Ngeow E. Major ingredients of fidelity: A review and scientific guide to improving quality of intervention research implementation. Clin Psychol Rev. 2011;31(1):79-88. [DOI] [PubMed] [Google Scholar]
- 8.Crane RS, Eames C, Kuyken W, et al. Development and validation of the mindfulness-based interventions - teaching assessment criteria (MBI:TAC). Assessment Dec 2013;20(6):681-688. doi: 10.1177/1073191113490790 [DOI] [PubMed] [Google Scholar]
- 9.Monteiro LM. Mindfulness as relational: Participants’ experience of mindfulness-based programs are critical to fidelity assessments. Global Advances in Health and Medicine. 2020;9:2164956120940280. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Robb SL, Burns DS, Docherty SL, Haase JE. Ensuring treatment fidelity in a multi‐site behavioral intervention study: Implementing NIH behavior change consortium recommendations in the SMART trial. Psycho Oncol 2011;20(11):1193-1201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Wyatt G, Sikorskii A, Rahbar MH, Victorson D, Adams L. Intervention fidelity: Aspects of complementary and alternative medicine (CAM) research. Cancer Nursing. 2010;33(5):331. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Shanafelt T, Stolz S, Springer J, Murphy D, Bohman B, Trockel M. A blueprint for organizational strategies to promote the well-being of health care professionals. NEJM Catalyst Innovations in Care Delivery 2020;1(6):1. [Google Scholar]
- 13.Klatt MD, Bawa R, Gabram O, et al. Embracing change: A mindful medical center meets COVID-19. Global advances in health and medicine. 2020;9:2164956120975369. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Klatt M, Westrick A, Bawa R, Gabram O, Blake A, Emerson B. Sustained resiliency building and burnout reduction for healthcare professionals via organizational sponsored mindfulness programming. EXPLORE. 2021;S1550-S8307(21):00082-00083. [DOI] [PubMed] [Google Scholar]
- 15.Mehta LS, Murphy DJ. Strategies to prevent burnout in the cardiovascular health-care workforce. Nat Rev Cardiol. 2021:18(7):455-456. [DOI] [PMC free article] [PubMed] [Google Scholar]