Rapid Cycle Evaluation and Adaptation of an Inpatient Tobacco Treatment Service at a U.S. Safety-Net Hospital

Abstract

Background

To address disparities in smoking rates, our safety-net hospital implemented an inpatient tobacco treatment intervention: an “opt-out” electronic health record (EHR)-based Best Practice Alert + order-set, which triggers consultation to a Tobacco Treatment Consult (TTC) service for all hospitalized patients who smoke cigarettes. We report on development, implementation, and adaptation of the intervention, informed by a pre-implementation needs assessment and two rapid-cycle evaluations guided by the Consolidated Framework for Implementation Research (CFIR) and Expert Recommendations for Implementing Change (ERIC) compilation.

Methods

We identified stakeholders affected by implementation and conducted a local needs assessment starting 6 months-pre-launch. We then conducted two rapid-cycle evaluations during the first 6 months post-implementation. The CFIR informed survey and interview guide development, data collection, assessment of barriers and facilitators, and selection of ERIC strategies to implement and adapt the intervention.

Results

Key themes were: (1) Understanding the hospital's priority to improving tobacco performance metrics was critical in gaining leadership buy-in (CFIR Domain: Outer setting; Construct: External Policy and Incentives). (2) CFIR-based rapid-cycle evaluations allowed us to recognize implementation challenges early and select ERIC strategies clustering into 3 broad categories (conducting needs assessment; developing stakeholder relationships; training and educating stakeholders) to make real-time adaptations, creating an acceptable clinical workflow. (3) Minimizing clinician burden allowed the successful implementation of the TTC service. (4) Demonstrating improved 6-month quit rates and tobacco performance metrics were key to sustaining the program.

Conclusions

Rapid-cycle evaluations to gather pre-implementation and early-implementation data, focusing on modifiable barriers and facilitators, allowed us to develop and refine the intervention to improve acceptability, adoption, and sustainability, enabling us to improve tobacco performance metrics in a short timeline. Future directions include spreading rapid-cycle evaluations to promote implementation of inpatient tobacco treatment programs to other settings and assessing long-term sustainability and return on investment of these programs.

Plain language summary

Hospital-based tobacco treatment programs are highly effective in increasing smoking cessation rates, yet few hospitals serving low-income patients have implemented such programs. Guided by the Consolidated Framework for Implementation Research (CFIR) and the Expert Recommendations for Implementing Change (ERIC) compilation, we conducted a pre-implementation local needs assessment and two rapid-cycle evaluations to design and implement an inpatient Tobacco Treatment Consult service at Boston Medical Center, the largest urban safety-net hospital in New England. This manuscript expands knowledge on the role of rapid-cycle evaluations using the CFIR-ERIC based evaluations, via a case study of implementation and adaptation of a hospital-based tobacco treatment program. Pre- and early-implementation theory-based process evaluations were critical in gaining insight into key factors influencing implementation, and adapting our intervention and selecting implementation strategies in response, thus improving acceptability, adoption, and sustainability. This approach has broad applicability and should be studied in other settings to guide, adapt, and improve the implementation of guideline-recommended tobacco treatment into hospital-based practices to improve quality of care and adherence to public reporting programs. To elevate and maintain relative priority of inpatient tobacco treatment programs, future directions should include spreading rapid-cycle-based evaluations to assess long-term sustainability and return on investment of these programs.

Introduction

Cigarette smoking is the leading cause of preventable death in the United States, and disproportionately affects populations with low socioeconomic status (Babb et al., 2017; Jamal et al., 2018). While several effective, guideline-recommended tobacco treatments exist (2008 PHS Guideline Update Panel, 2008), uptake has been disappointing in the United States due to limited reach to individuals who smoke, sluggish adoption by clinicians, and other implementation challenges. There is a clear need to optimize interventions and implementation to enhance the effectiveness and reach of tobacco treatment.

Hospitalization offers an opportunity to engage individuals who may not otherwise seek tobacco treatment. Hospital-based interventions, particularly with combined pharmacotherapy, counseling, and post-discharge support, are effective in increasing smoking cessation (Cartmell, Dismuke, et al., 2018; Cartmell, Dooley, et al., 2018; Fu et al., 2014; Haas et al., 2015; Nahhas et al., 2016; Ylioja et al., 2017). Recognizing this opportunity, Joint Commission (JC), the largest standard-setting and accrediting body in U.S. healthcare, recommends tobacco performance measures (screening and treatment for tobacco use) for all hospitalized patients who smoke cigarettes (Fiore & Adsit, 2016). Few hospitals serving low-income patients (Anderson et al., 2009; https://www.medicaid.gov/medicaid/index.html; Institute of Medicine (US) Committee on the Changing Market, 2000; Waitzkin, 2005), however, have implemented JC quality standards. To further stimulate hospital-based tobacco treatment for socioeconomically disadvantaged individuals, MassHealth (Massachusetts’ Medicaid program that provides Medicaid public insurance for low-income Massachusetts residents) rewarded hospitals through pay-for-performance mechanisms for improving tobacco treatment during hospitalization and at discharge.

As the largest urban safety-net hospital (U.S. designation for hospitals providing healthcare regardless of patients’ insurance status or ability to pay) in New England, Boston Medical Center (BMC) serves a socioeconomically disadvantaged population with high smoking rates. In response to JC standards and MassHealth incentive programs for tobacco treatment, we designed and implemented an inpatient Tobacco Treatment Consult (TTC) service. We created a smoking cessation Best Practice Advisory (BPA) and order-set in BMC's electronic health record (EHR), designed to trigger a referral to the TTC service for all hospitalized patients with “current smoking” status in the EHR. The goal was to increase individuals receiving tobacco treatment regardless of clinical condition or motivation to quit. The TTC service provides consultation to approximately 25% of the 4,500 patients who smoke admitted to BMC each year; we have demonstrated the effectiveness of the TTC service at improving patient-level outcomes (6-month smoking abstinence) and tobacco performance measures (Herbst et al., 2020).

We now report how we tailored the TTC service and its implementation to meet the needs of our community. We performed a pre-implementation local needs assessment, followed by two rapid-cycle evaluations during early implementation, all guided by the Consolidated Framework for Implementation Research (CFIR) and designed to gather perspectives of stakeholders on critical barriers and facilitators to implementation. In response we adapted the intervention and applied targeted Expert Recommendations for Implementing Change (ERIC) strategies to address identified barriers. Rapid-cycle evaluations provide actionable information in real-time to stakeholders, allowing continuous evaluation for intervention improvement (Anker et al., 1993; Gale et al., 2019; Keith et al., 2017; McMullen et al., 2011; Shrank, 2013). The CFIR provides a menu of 39 constructs organized into five domains (intervention characteristics, outer setting, inner setting, characteristics of individuals, implementation process) describing the organizational and contextual settings associated with effective implementation (Figure 1 (Damschroder et al., 2009)). When used proactively, the CFIR guides the identification of modifiable barriers and facilitators to implementation success and selection of implementation strategies from the ERIC taxonomy to promote implementation and maintenance of interventions (King et al., 2020; Kirk et al., 2016; Powell et al., 2015). We illustrate how gathering pre-implementation and early-implementation data in consecutive rapid-cycle evaluations allowed us to develop and refine the TTC intervention to improve implementation, patient-level outcomes, and adherence to public reporting programs.

Figure 1.

CFIR domains and constructs.

Methods

The Boston University Medical Campus Institutional Review Board approved this study.

Approach

The overarching framework guiding rapid-cycle evaluations was the CFIR (Damschroder et al., 2009; Damschroder & Lowery, 2013; Kirk et al., 2016). The CFIR informed questionnaire, survey and interview guide development, data collection, analysis, and tailored implementation based on ERIC strategies (Figure 2). The content of the questionnaires, surveys, and interview guide, along with frequencies of patient responses, appear in Supplementary Materials 1A–D.

Figure 2.

Timeline of development, implementation, adaptation, and evaluation activities informed by CFIR domains.

We convened clinical, analytic, and research teams (Appendix A) and a purposive sample of stakeholders (patients, clinicians, leadership, and TTC team members) to achieve broad representation. We sought stakeholder input at multiple time points leading up to and during early implementation. Pre-implementation needs assessment focused on assessing the current state of inpatient tobacco treatment from the perspective of multiple stakeholders to inform design and implementation of an intervention suited to the local context. Based on this assessment, we developed the first iteration of the tobacco treatment intervention. We then conducted two rapid-cycle evaluations to identify modifiable barriers to implementation, which informed adaptation and selection of ERIC strategies to overcome barriers in real-time, while maintaining fidelity to the core components of the intervention (Aarons et al., 2012; Stirman et al., 2013).

Pre-implementation needs assessment

In the 6 months prior to implementation, we conducted a local needs assessment with leadership, patients, and clinicians to inform intervention development. Surveys and interview guides were designed to prompt stakeholders to discuss perceptions of inpatient tobacco treatment, how inpatient tobacco treatment was currently conducted, and potential influential factors and expectations of an inpatient TTC service.

Specifically, we conducted formal meetings, telephone interviews, and email correspondence (10 meetings; Figure 2) with hospital leadership (n = 3), the EHR support team (n = 2), and data analytics team (n = 2). To understand how patients perceived inpatient tobacco treatment and how hospitalization influenced motivation to quit smoking, we conducted interviews and surveys (Supplementary Material 1A) after receiving informed consent with a convenience sample of 21 adults hospitalized in the 6 months prior to initiation of our intervention who smoked at the time of admission. The questionnaire focused on patients’ needs, facilitators, and barriers to quitting smoking. At BMC, resident physicians are frontline clinicians for hospitalized patients (placing orders, following through with TTC recommendations); accordingly, we deemed them to be key stakeholders likely to influence implementation. To inform design of our intervention and implementation plan, a 21-item questionnaire, assessing knowledge of guidelines, practices surrounding tobacco treatment, and barriers to using evidence-based tobacco treatment (Supplementary Material 1B) was administered to a convenience sample of Internal Medicine residents attending a local educational conference; 67.6% (88/130) responded.

We analyzed quantitative and qualitative data from these sources using the CFIR as a template for content analysis and as a guide to interpret findings. We developed a table organized by CFIR domains and relevant constructs which included columns to populate key barriers, facilitators, and actions taken (i.e., ERIC implementation strategies deployed and/or adaptions to the intervention) based on analysis of data (Table 1). We discussed summarized findings with stakeholders and developed the first iteration of the intervention.

Table 1.

Subset of CFIR constructs highlighting barriers, facilitators, and ERIC strategies to improve implementation.

OUTER SETTING The economic, political and social context within which an organization resides
External Policy and Incentives: A broad construct that includes external strategies to spread interventions, including policy and regulations (governmental or other central entity), external mandates, recommendations and guidelines, pay-for-performance, collaboratives, and public or benchmark reporting.
Barriers	Facilitators	Process, Intervention	ERIC Strategies
Leadership (Meetings) Limited resources to support inpatient tobacco treatment.	Leadership (Meetings, Interviews) Leadership prioritized inpatient tobacco treatment, due to MassHealth incentive programs & public reporting requirements.	Created business plan on EHR-based inpatient TTC service. Vetted goals and funding mechanisms of such a service.	Access new funding
Patient Needs & Resources: The extent to which the needs of those served by the organization (e.g., patients), as well as barriers and facilitators to meet those needs, are accurately known and prioritized by the organization.
Barriers	Facilitators	Process, Intervention, or Adaptation	ERIC Strategies
TTC team/Leadership (Meetings) Perception of clinicians not prioritizing smoking counseling. Challenge to consult on 550 “current smoker” admissions each month.	Patients (Surveys and Interviews) 66.7% of patients stated that hospitalization increased their motivation to quit smoking. Several patients expressed in interviews that bedside teaching would be helpful.	Developed brochure to educate patients about tobacco treatment options and engage them in treatment Addressed resource constraints by maximizing efficiency: met with EHR team to simplify referrals, medication order sets, and note templates.	Involve patients/consumers Develop educational materials Use data experts Build a coalition
INNER SETTING: The cultural and structural contexts through which the implementation process occurs
Relative Priority: Individuals’ shared perception of the importance of the implementation within the organization.
Barriers	Facilitators	Adaptation	ERIC Strategies
Leadership (Meetings) Inpatient tobacco treatment has been prioritized, but prior efforts to meet measures had little buy-in from frontline providers Leadership expressed concern of the high cost of a full-time NP.	Leadership (Meetings) Given failed attempts to improve tobacco measures, leadership receptive to intervention. Existing EHR infrastructure to trigger tobacco treatment order set for current smoking status Leadership acknowledged the higher risk of losing MassHealth incentive payments EHR team prioritized Epic-build for TTC service over competing requests given tight timeline.	Capitalizing on prior EHR build, we operationalized the BPA+TTC order set as a potential solution to lack of improvement in meeting tobacco measures. Compromise reached on NP at 0.5 full-time equivalent (FTE) and respiratory therapist (0.5 FTE).	Involve executive board Remind clinicians Use data experts Create new clinical teams
Compatibility: The degree of tangible fit between meaning and values attached to the intervention by involved individuals, how those align with individuals’ own norms, values, and perceived risks and needs, and how the intervention fits with existing workflows and systems.
Barriers	Facilitators	Adaptation	ERIC Strategies
Clinicians (Interviews) BPA fatigue, time constraints, competing priorities, and poor communication with the TTC service Low clinician follow-through of TTC recommendations.	Clinicians (Interviews) Clinicians valued the TTC service and perceived that hospitalized smokers valued TTC services. Clinical Analytic team (data review) Hospitalized smokers were receptive to inpatient tobacco treatment.	Reallocation of resources from outpatient to inpatient tobacco treatment program. Efforts to improve communication with primary team on TTC recommendations Expanded formulary so that tobacco treatment regimens during hospitalization could be continued on discharge.	Change service sites Place innovation on fee for service lists/formularies
Leadership Engagement: Commitment, involvement, and accountability of leaders and managers with the implementation of the innovation.
Barriers	Facilitators	Adaptation	ERIC Strategies
TTC team (meetings) TTC members concerned that leadership support would decline once metrics met.	Leadership (Meetings) Leadership team valued TTC team and ensured continued support for TTC service after completion of incentive programs.	The Chief Medical Officer sponsored our program; TTC team felt supported by leadership.	Build a coalition Involve executive board
Available resources: The level of resources dedicated for implementation and on-going operations, including money, training, education, physical space, and time.
Barrier	Facilitator	Adaptation	ERIC Strategies
TTC team (meetings) The tobacco consult team felt overly busy, stretched thin, and short-staffed.	TTC team (meetings) Resources available for all TTC members to receive formal tobacco training at an ATTUD-certified institution.	Hospital leadership budgeted for tobacco-trained team member (1.0 FTE) to join the team	Access new funding
INTERVENTION CHARCTERISTICS Interventions have core components (essential elements) and an adaptable periphery (adaptable elements, structures and systems related to the intervention and the organization into which it is being implemented)
Relative advantage: Stakeholders’ perception of the advantage of implementing innovation vs an alternative solution
Barriers	Facilitators	Adaptation	ERIC Strategies
Clinicians (Interviews) Dominant theme that tobacco treatment is an outpatient issue and would be extra work in inpatient setting.	Leadership (Meetings, interviews) Leadership highlighted a tight timeline for showing improvement due to risk of losing payment incentive and were “open to other opportunities.”	Decision made to develop dedicated TTC service (rather than primary inpatient team) to deliver smoking cessation counseling.	Create new clinical teams
Adaptability: The degree to which an innovation can be adapted, tailored, refined, or reinvented to meet local needs.
Barrier	Facilitator	Adaptation	ERIC Strategies
Clinicians (Interviews) Clinicians found “non-adaptable” aspects of the BPA+TTC order set frustrating.	Clinicians (Early-implementation Survey): 78.8% (26/33) reported they were very or somewhat satisfied with BPA+order set. 93% (23/25) rated the BPA+order set as very or somewhat easy to use.	Continuous iteration of BPA+order set with EHR team based on feedback from clinicians.	Conduct cyclical small tests of change Use data experts Remind clinicians
CHARACTERISTICS OF INDIVIDUALS The individuals involved with the intervention and/or implementation process
Knowledge & Beliefs: Individuals’ attitudes toward and value placed on the innovation, as well as familiarity with facts, truths, and principles related to the innovation.
Barriers	Facilitators	Adaptation	ERIC Strategies
Clinicians (Surveys) Clinicians had little awareness of tobacco treatment guidelines and had significant knowledge gap on the efficacy of available treatment.	Leadership (Meetings, interviews) Hospital already looking for opportunities to improve tobacco outcomes. Evidence for tobacco treatment during hospitalization well accepted by leadership.	TTC team presented evidence during grand rounds on efficacy of inpatient tobacco treatment. Developed educational materials, which clinical champion distributed to front-line residents Clinical champion met with incoming resident teams to educate about TTC service	Conduct educational meetings Develop educational materials Distribute educational materials Identify and prepare champions Conduct educational outreach visits

Rapid-Cycle evaluations

Following intervention development, we conducted two rapid-cycle evaluations (Gale et al., 2019; Keith et al., 2017; Stetler et al., 2006) during early implementation, each lasting up to 3 months, to tailor key program elements to stakeholder needs. At the end of each rapid-cycle test, we assessed perceptions on the value of the TTC service to leadership and clinicians (acceptability). We noted barriers to adoption and suggestions for improvement from the perspectives of clinicians and the TTC team itself. We also assessed the ability of the TTC intervention to be implemented as designed (feasibility).

To assess these outcomes, the TTC service held weekly meetings. The team was encouraged to share experiences to enable a thorough understanding of implementation from varying perspectives. With informed consent, we complemented these data with surveys (n = 33) and interviews (n = 25) with inpatient clinicians focused on acceptability of intervention components (Figure 2; Supplementary Materials 1C–D). Surveys and interview guides were designed to elicit impressions of the TTC service and context-specific facilitators and barriers to adoption, including how well it integrates into clinical workflow, and suggestions for improvement. To inform feasibility, we worked with the data analytics team (Proctor et al., 2011). Acceptance rates of TTC + order-set were reported monthly (adoption); weekly feedback was provided on progress to meeting hospital tobacco performance measures (effectiveness). Technical issues and BPA concerns were reported as they arose.

Using the CFIR as our template, we summarized findings from our data sources on acceptability and feasibility. We presented findings to our stakeholders and using ERIC implementation strategies, we devised a plan for varying the implementation approach to optimize acceptability and feasibility as needed in the next rapid-cycle test. Lastly, we conducted two retrospective quasi-experimental analyses to examine the effectiveness of the TTC service on 6-month quit rates and hospital-level performance measures (Herbst et al., 2020). We discussed summarized findings with stakeholders to inform planning to sustain implementation over time.

Results

We demonstrate how we used specific CFIR constructs and ERIC strategies during pre-implementation needs assessments and early-implementation rapid-cycle evaluations to guide development, implementation, and adaptation in real-time of the opt-out EHR-based inpatient TTC service (Figure 2). Key themes included: (1) Understanding the hospital's priority to improving tobacco performance metrics and committing to develop an intervention that would meet these needs was the major reason for receiving funding; (2) Minimizing clinician burden allowed the successful implementation of the TTC service; (3) Demonstrating effectiveness was the major driver to sustaining the program. We elaborate on these themes and others below. Table 1 is organized by CFIR constructs and highlights barriers and facilitators to implementing the TTC service, adaptations made to improve implementation outcomes, and targeted ERIC strategies to tailor implementation to our local context. Table 2 shows the ERIC strategies, stratified by pragmatic “ERIC clusters” deployed throughout the study (Perry et al., 2019).

Table 2.

ERIC strategies used in this study.

ERIC Strategy	Specific actions
Develop stakeholder relationships
Build a coalition	Assembled a stakeholder panel of patients, clinicians, department chairs, nurses, IT team, tobacco experts, and hospital leaders, focusing on developing stakeholder interrelationships
Involve executive board
Involve patients/consumers
Identify and prepare champions	Selected a chief resident as our clinical champion to provide bidirectional support
Organize implementation teams and team meetings	Convened clinical, analytic, and research teams Set up weekly TTC team meetings to monitor progress and experiences.
Use evaluative and iterative strategies
Conduct local needs assessment	Conducted pre-implementation meetings with leadership and conducted clinician surveys and interviews to understand relative priority of intervention
Assess for readiness and identify barriers and facilitators	Assessed data from various sources and stakeholders in the period leading up to and during implementation to determine barriers, facilitators, and readiness to implement
Conduct cyclical small tests of change	Conducted first rapid cycle test to obtain feedback on feasibility, compatibility with workflows, and suggestions for improvement on the first iteration of the intervention Conducted second rapid cycle test to understand perceptions of adapted intervention
Train and educate stakeholders
Develop educational materials	Developed and distributed educational materials explaining the importance of inpatient tobacco treatment and the TTC service to incoming residents Provided updates on TTC service to incoming residents
Distribute educational materials
Conduct educational meetings
Conduct educational outreach visits	Educated clinicians on efficacy of guideline-recommended tobacco treatment for hospitalized patients and about the new TTC service during medical and subspecialty grand rounds
Adapt and tailor to context
Use data experts	EHR team created BPA+TTC order set and simplified referrals, medication order sets, and note templates to address TTC time constraints
Change service sites	Reallocation of resources from outpatient to inpatient tobacco treatment program.
Support clinicians
Create new clinical teams	Developed dedicated TTC service (rather than primary inpatient team) to minimize clinician burden increase the likelihood that guideline-concordant tobacco treatment is delivered to hospitalized individuals who smoke cigarettes
Remind clinicians	Operationalized the BPA+TTC order set as a potential solution to lack of improvement in meeting tobacco measures
Financial strategies
Access new funding	Created business plan and vetted funding for EHR-based inpatient TTC service
Place innovation on fee for service lists/formularies	Expanded inpatient formulary so that tobacco treatment regimens during hospitalization could be continued on discharge.

Pre-implementation: Developing stakeholder relationships, local needs assessment, and formulating initial implementation plan

From the outset, we recognized it is critical to develop an intervention that meets stakeholder needs. Hence, using strategies from the ERIC compilation (Powell et al., 2015), we (1) assembled a stakeholder panel of patients, clinicians, hospital leaders, and tobacco specialists, focusing on developing stakeholder relationships, and (2) undertook a series of evaluative and interactive steps to understand local needs of our BMC community (Table 2). We summarize how stakeholder input and the needs assessment facilitated the development of the “opt-out” EHR-based TTC service.

Large discrepancies existed between leadership prioritizing improving tobacco performance measures and the realities of clinician barriers to inpatient tobacco treatment workflow

Hospital Administration: Through pre-implementation meetings with leadership, we learned the hospital had elected to report data to JC and MassHealth incentive programs on tobacco measures. The measures targeted for improvement were TOB-2 (tobacco use treatment provided or offered during hospitalization) and TOB-3 (tobacco use treatment provided or offered at discharge). Leadership shared that BMC was underperforming, with 2015 rates of 2.7% for TOB-2 and 0% for TOB-3. They highlighted a priority of improving performance (Domain: Outer setting; Construct: External Policy and Incentives) and were seeking opportunities to rapidly improve inpatient tobacco treatment delivery.

Clinicians (pre-implementation survey and interviews): By contrast, many clinicians did not see an advantage of delivering inpatient compared to outpatient tobacco treatment (Domain: Characteristics of Individuals; Construct: Knowledge and Beliefs):

“I feel the rapport is much better in the outpatient than the inpatient setting.”

“We’re focused on other things as an inpatient, rather than quitting smoking.”

Residents were unaware of tobacco metrics, guidelines or the importance of implementing the intervention in the hospital.

“I’m aware of tobacco management and medications that we can offer patients. I don’t know if there's specific guidelines for the hospital, or measurements they’re looking at.”

Furthermore, residents had little incentive to provide inpatient tobacco treatment, largely due to perceived barriers including time constraints (78.8%; 67/85) and knowledge gap on treatment efficacy (49.4%; 42/85). Residents additionally had substantial gaps in knowledge of best practices for tobacco treatment: 62.3% (53/85) were unfamiliar with tobacco treatment guidelines; 70.1% (61/87) had received <4 h of tobacco treatment education and 46% (40/87) felt this was inadequate; only 13.7% (12/87) felt “very comfortable” with administering tobacco treatment pharmacotherapy.

Process: Planning

We convened our stakeholder panel, analyzed data, and reviewed pros, cons, and evidence behind various inpatient tobacco treatment strategies. It was clear that any tobacco treatment intervention we established needed to minimize additional work to inpatient clinicians. Given the large volume of hospitalized individuals at BMC who smoke (approximately 550 per month), we also recognized the need for efficiency in delivering tobacco treatment. We discussed with hospital leadership how an EHR-based TTC service might improve efficiency and quality of care to our patients, without adding significant burden to inpatient clinicians. Because the TTC service could simultaneously improve tobacco performance metrics, leadership was receptive to further discussions and suggested we discuss feasibility with the EHR team, develop a business plan, and present our ideas formally at the monthly leadership meeting.

Prior failed efforts to improve tobacco performance metrics with “Optional” approaches to delivering tobacco treatment informed key elements of the tobacco treatment consult intervention

Hospital Administration: We met again with administration and the EHR team and reviewed BMC's previous efforts to improve tobacco performance, namely an EHR-based nursing tobacco treatment education intervention:

For TOB-2: A template that met inpatient tobacco treatment counseling requirements had been created to auto-populate in the nursing section of the EHR when hospitalized patients with “current smoking” status were identified. A question in the social history asking if the patient would like medications for smoking cessation during hospitalization was included; if answered yes, a medication order-set would open. However, this question was optional, representing a barrier to adoption: “Unfortunately there was no way to make this question required and as such was seldom answered.”

For TOB-3: Along with the question regarding inpatient medications, an optional question was added about post-discharge treatment. If answered yes, on discharge a telephone quitline referral would auto-populate. Again, the optional completion of this question represented a barrier to adoption: “Unfortunately providers did not answer this question and as such there was no change in compliance on TOB-3. Also, the thought was that medications prescribed for inpatients would be continued on discharge, but as noted above this workflow did not work as intended.”

Process: Planning

We reconvened our stakeholders and discussed how given the hospital's prior failed attempts to improve performance measures, we had a tight timeline to develop an intervention that demonstrated rapid improvements in these measures. We proposed creating an EHR-based inpatient intervention consisting of a new smoking cessation (Best Practice Alert) BPA + order-set and a TTC service comprised of nurse practitioners (NPs) with pulmonologist oversight. The failed experience with an “optional” approach to bedside nurses offering tobacco treatment directly informed three key elements of our intervention: (1) the requirement that the BPA be addressed (rather than optional); (2) targeting the resident physician who places most orders as the intervention recipient; (3) tasking a separate team of trained tobacco specialists with providing tobacco treatment, rather than relying on frontline clinicians with competing demands.

Minimizing clinician barriers factored into designing the electronic health record-based tobacco treatment consult intervention

From clinician surveys, we recognized that a system that relied entirely on the primary inpatient team to proactively consult the TTC service would likely fail given multiple clinician barriers including time constraints, knowledge gaps about tobacco treatment, and overall skepticism on the utility of inpatient tobacco treatment. Yet, we were reluctant to completely remove residents from the TTC-referral workflow, which would incorrectly imply that treating tobacco dependence is not the responsibility of all clinicians.

We analyzed the pros and cons of three different EHR-based TTC-referral interventions:

1: optional BPA + TTC order-set, referral based entirely on provider decision to consult TTC service.

2: required BPA + TTC order-set, with clinician sign-off needed to trigger automatic referral to TTC service and “opt-out” option.

3: Referral entirely automated based on smoking status, without any provider input or opt-out option.

We chose “Intervention 2” with the provision of a clinical champion (an internal medicine chief resident) obtaining early-implementation feedback from residents on how to modify the BPA + TTC order-set and better integrate the TTC service with the clinical workflow of the primary inpatient teams. We conducted surveys and interviews with hospitalized patients and found that 76% (16/21) thought it was “extremely” important to quit cigarettes and 66.7% (14/21) stated that hospitalization increased their motivation to quit. These results were shared with clinicians in educational lectures, hoping that understanding patients’ preferences might positively influence clinicians’ perceptions of inpatient tobacco treatment.

Process: Planning

We mapped out the first iteration of the BPA + TTC order-set. We designed the BPA + TTC order-set to “pop up” in the EHR for any individual designated with “current smoking” status (a field with 96% accuracy) (Seth et al., 2020) admitted to the hospital, alerting the clinician to (1) order a consult to the inpatient TTC service (TTC order-set), and (2) place a referral to the ambulatory tobacco treatment center (internal resource; Figure 3A). Clinicians could decline acting on the BPA by selecting one of these options: (1) patient declines; (2) will order later; or (3) patient does not smoke cigarettes.

Figure 3.

Screenshots of Best Practice Advisory (BPA) for hospitalized individuals with “Current Smoking Status.”

Upon receiving the referral, we proposed that the TTC service would provide and document: (1) inpatient counseling; (2) recommendations to the medical team for pharmacotherapy during hospitalization and post-discharge; and (3) linkage to outpatient tobacco treatment (electronic referral to the ambulatory tobacco treatment center at discharge and provision of information on the Massachusetts Quitline).

Meeting performance metrics and participating in incentive programs were the major drivers for administration funding the tobacco treatment consult service

Hospital Administration: We created a business plan and formal presentation for hospital leadership and discussed how the “opt-out” EHR-based TTC service could achieve improved quality of care and favorable financial return on investment for the hospital (Leatherman et al., 2003). We anticipated needing 2.0 full-time equivalent (FTE) tobacco trained specialists (TTS) to staff the TTC service (each TTS would see approximately 10 patients per day). We proposed hiring NPs because their salary would be supported by professional fees, instead of relying entirely upon a TTS who cannot bill for services (e.g., TTS-trained respiratory therapist (RT) or social worker).

Leadership reviewed the business plan and responded favorably:

“It sounds like the program being proposed would meet our requirements. The [MassHealth] metrics are identical to the Joint Commission TOB-2 and TOB-3 metrics. In addition, we report on TOB-1, TOB-2, and TOB-3 to MassHealth as part of their Pay-for-Performance Program, so this would also support that initiative.”

Process: Executing

While leadership expressed concern about the high cost of supporting 2.0 FTE tobacco trained NPs, they acknowledged the higher risk of losing incentive payments. The cost of losing incentive payments elevated the relative priority of implementing the TTC service over other initiatives. After negotiations with hospital leadership, acknowledging other priorities, and competing hospital demands on the system, we received funding from the administration for 0.5 FTE NP and 0.5 FTE RT to staff the TTC service. We recognized this compromise meant it would not be feasible for the TTC service to consult on all 550 inpatients with “current smoking” status admitted each month.

The EHR team prioritized the Epic-build for the TTC service over competing requests given the tight timeline for meeting performance measures (Domain: Inner Setting; Construct: Relative Priority). Given the existing infrastructure to trigger a tobacco treatment order-set for inpatients with “current smoking” status from the prior EHR-based nursing intervention, the BPA + TTC order-set was built in a short-timeframe and ready to be trialed in the first rapid-cycle test.

Readiness for implementation

Prior to initiating rapid-cycle testing, we prepared for readiness in several ways. The NP and RT achieved TTS proficiency by attaining certification to Association for the Treatment of Tobacco Use and Dependence standards and practicing delivery of the intervention under the direct supervision of the TTC medical director. We created a checklist for the TTC service to track tasks completed and patient participation and set up weekly meetings to discuss experiences.

To facilitate clinician understanding of the underlying principles that justify tobacco treatment during hospitalization, the TTC team educated clinicians on the efficacy of guideline-recommended tobacco treatment for hospitalized patients during medical and subspecialty grand rounds. Since chief residents provide both support of key missions of the hospital and function as advocates for residents (the primary end-users of the TTC intervention), we trained a chief resident (NH) as our clinical champion to provide bidirectional support to meeting our goals. We worked with our clinical champion to develop brief educational materials reviewing the evidence behind inpatient tobacco treatment and explaining the TTC service, which she then distributed to incoming residents (see sample email in Appendix B) and followed with education sessions. Lastly, the Chief Medical Officer emailed a hospital-wide message with the expectation that all providers consult the TTC service for hospitalized individuals who smoke cigarettes prior to the go-live date.

Early-Implementation: First rapid-cycle evaluation

Each week after the “opt-out” EHR-based TTC service went live, we corresponded with the data analytics team to monitor progress toward meeting metrics and met the TTC team. Our champion attended the TTC team meetings monthly and provided feedback on clinical feasibility, compatibility with workflows, and suggestions for improvement based on clinician input. After the end of the first 2-month rapid-cycle test, we administered a survey to clinicians to assess implementation outcomes. We identified two major concerns: (1) TTC team members were stretched thin; and (2) clinicians were frustrated with certain intervention components. To address these concerns, we sought additional members to deliver tobacco treatment and adapted intervention components. These findings are summarized below.

The tobacco treatment consult team was overly busy, a barrier that was overcome by short-term donation of clinical hours dedicated to tobacco treatment from the respiratory therapist department

Data Analytics team: We received preliminary data on performance for TOB-2 and TOB-3 monthly. The analytics team relayed that with the current level of effort of the TTC team, we would be on track to meeting performance metrics.

TTC team: The availability of resources was a major concern (Domain: Inner Setting; Construct: Readiness for Implementation; Sub-construct: Available Resources). A common tension was finding a balance between providing some level of basic tobacco treatment to everyone versus a more comprehensive treatment plan to selected patients (e.g., individuals hospitalized for a smoking-related health event). The TTC team felt overly busy and short-staffed, raising concerns about sustainability.

Adaptations to TTC service: Concerns were voiced to the administration. Leadership from RT generously offered support in the short term to enable meeting performance metrics, including securing time to have two RTs complete training in tobacco treatment. Each RT dedicated 2–4 h per week (depending on clinical duties) to seeing inpatients who smoke. In addition, the EHR team identified ways to use the EHR more efficiently (build “smart phrases,” TTC note templates, medication order-sets), thus reducing time constraints.

Stakeholder feedback on the tobacco treatment consult service led to rapid adaptations of intervention components

Stakeholder feedback revealed how certain intervention components were barriers to adoption (Domain: Intervention characteristics; Construct: Adaptability). Several problematic aspects were not considered core components and could therefore be adapted, as described below.

Clinicians: Clinician feedback from the 2-month post-implementation survey showed that 90.9% (30/33) of respondents rated the BPA + order-set as very or somewhat easy to use, and 78.8% (26/33) reported they were either very or somewhat satisfied with the BPA + order-set. There were certain “non-adaptable” aspects of the BPA + TTC order-set that clinicians found frustrating, such as the need to address the BPA before viewing other sections of the EHR:

“I don’t like that it (BPA) prevents you from getting to the chart when you need to do something.”

Data Analytics team: The original BPA + order-set had several options for resolving the BPA, including “Patient declines (TTC services).” The analytics team informed us that the majority of clinicians found a work-around to their frustrations and resolved the BPA by selecting “Patient declines.”

TTC team: When clinicians ordered “ambulatory referral to smoking cessation” at the time of the initial consult to the TTC team, our referral coordinator would get the referral (hundreds per week), despite only 24.6% (254/1033) accepting follow-up in the outpatient program at discharge.

Adaptations to BPA ± TTC order-set: To achieve more consistent and sustainable implementation we made adaptations (Figure 3B): (1) The EHR team modified the BPA so that the only way clinicians could decline the BPA + order-set is to click “Will place order later,” at which time the BPA triggered again at 8 h, thus allowing clinicians time to stabilize acutely ill, newly admitted patients. We removed all other options to decline the BPA, including “Patient declines.” (2) To facilitate initial consultation to the TTC team, we added a “No call necessary” feature to the order-set. (3) We removed the order for “Ambulatory referral: Pulmonary Smoking Cessation” from the BPA + order-set. Instead, the TTC team provides post-discharge appointment times in BMC's Tobacco Treatment Clinic, along with an informational flyer, for patients accepting an outpatient referral (Appendix C).

Early-Implementation: Second rapid-cycle test

We tested the adapted BPA + TTC order-set in the next rapid-cycle test. At the end of rapid-cycle testing, we debriefed with stakeholders, conducted clinician interviews to understand perceptions of the adapted intervention, and reviewed bimonthly reports on progress towards meeting metrics. We additionally assessed outcomes that would influence sustainability: (1) patient-level outcomes (receipt of Nicotine Replacement Therapy (NRT) and 6-month quit rates between individuals seen and not seen by the service); and (2) effect of the TTC service on performance measures.

Stakeholders had positive perceptions of the adapted intervention, but barriers to fully integrating inpatient tobacco treatment into clinical workflow persisted

Clinicians: Feedback in interviews to the adapted intervention was positive:

“You just click on a box and put in your pager number and it goes off. That's satisfying.”

“At first you had to put more information into it and then I didn’t always do it, but now that it's easier, I do it pretty much all the time.”

Clinicians expressed they valued the TTC service and perceived that patients were receptive to inpatient tobacco treatment, demonstrating acceptability (Domain: Inner Setting; Construct: Implementation Climate; Sub-construct: Compatibility). Time constraints, competing priorities, and poor communication with the TTC service remained barriers to fully integrating tobacco treatment into existing workflows (Seth et al., 2020).

Data analytics: 96.8% (1000/1033) of patients accepted inpatient counseling, while only 24.6% (254/1033) accepted post-discharge follow-up in the outpatient tobacco treatment program. Despite giving an appointment date and time for the outpatient program prior to discharge, only 13.8% (35/254) of those accepting an outpatient referral attended.

TTC team: The TTC team noted low follow-through of TTC recommendations for pharmacotherapy at discharge. Among patients for whom the TTC recommended NRT, the primary inpatient team ordered inpatient NRT for 82.5% (652/790) but provided prescriptions for NRT at discharge to only 48.8% (351/719). Among patients desiring NRT at hospital discharge, the TTC team noted that clinicians were more likely to order outpatient NRT if patients were already receiving NRT while inpatient, likely because of ease of carrying forward any current prescriptions through discharge processes versus manually ordering new prescriptions at discharge.

Process: Reflecting, Evaluating, and Adaptation

Adaptations to TTC service: Based on clinician feedback, considerable efforts were made to improve communication with primary teams, particularly related to post-discharge planning. For example, in addition to documenting pharmacotherapy recommendations in the EHR, primary teams were text-paged recommendations; clinician feedback on communication efforts was positive. Given low attendance to the outpatient program, BMC reallocated personnel time from the outpatient to the inpatient program. Simultaneously, the TTC team held several meetings to discuss interventions to overcome barriers to low outpatient follow-through (e.g., reminder calls/letters, telephone counseling). We met with pharmacy leadership and succeeded in expanding the inpatient formulary so that tobacco treatment pharmacotherapy initiated during hospitalization could easily be continued post-discharge.

The tobacco treatment consult service improved patient-level outcomes and hospital-level performance metrics, which were major drivers for sustaining our program and securing additional resources

Data analytics: We compared receipt of NRT and 6-month quit rates between individuals seen by the service (n = 505) and those not seen due to time constraints (n = 680) using multivariable logistic regression (Herbst et al., 2020). Individuals seen by the TTC service had higher odds of receiving NRT during hospitalization (51.5% vs. 35.9%, adjusted odds ratios (AOR) = 1.93 [95% CI 1.5–2.45]), receiving NRT at discharge (32.5% vs. 12.4%, AOR = 3.41 [2.54–4.61]), and 6-month smoking abstinence (14.9% vs. 10%, AOR = 1.48 [1.03–2.12]). We conducted an interrupted time series analysis to examine the effect of the TTC service on hospital-level performance measures pre- versus post- implementation of the intervention; the TTC service was associated with an immediate increase in hospital-level performance on the post-discharge tobacco treatment measure (TOB-3) (Herbst et al., 2020).

TTC team: The biggest concern expressed by TTC members was that support from leadership would decline once measures set by MassHealth incentive programs had been met. The TTC team met with leadership, who assured them that their work was valued and would be supported after completion of incentive programs (Domain: Inner Setting; Construct: Readiness for Implementation; Sub-construct: Leadership Engagement).

Process: Reflecting

TTC Reflections: The TTC team felt supported by leadership engagement. Despite leadership initial stating the budget would not allow for additional resources, after negotiations, performance review, and understanding that donated support from the RT department was temporary and helped lead to meeting performance metrics, leadership agreed to an additional member (1.0 FTE) to the TTC team to sustain implementation and maintain performance.

Discussion

We describe how rapid-cycle evaluations using the CFIR helped identify contextual factors that influenced implementation of an “opt-out” EHR-based TTC service for hospitalized adults who smoke cigarettes. We found that pre- and early-implementation evaluations were critical in understanding key factors that influenced implementation processes, identifying barriers to implementation, adapting modifiable components of the intervention, and selecting ERIC implementation strategies in response to improve acceptability and adoption. Rapid-cycle evaluations uncovered unanticipated issues, enabling us to address them and thereby improve tobacco performance metrics in a short timeline (Herbst et al., 2020).

The most influential CFIR constructs in designing the intervention were within the Outer and Inner Domains. Although not initially transparent, understanding the incentive programs the hospital had committed to (Domain: Outer setting; Construct: External Policy and Incentives), analyzing the facilitators and barriers of prior initiatives in reaching these metrics, and addressing how our proposal could overcome shortcomings in a rapid timeline (Domain: Inner Setting; Construct: Relative Priority) were critical in gaining leadership buy-in. The most influential ERIC strategies to address CFIR-identified barriers were grouped into three ERIC clusters: using evaluative and iterative strategies, particularly conducting a local needs assessment; developing stakeholder relationships; and training and educating stakeholders.

During implementation, the most useful CFIR construct was in the Inner Domain (Domain: Inner Setting; Construct: Readiness for Implementation; Sub-construct: Available Resources), which allowed us to identify stakeholder (in particular, clinician and TTC team) barriers. ERIC strategies clustering into “using evaluative and iterative strategies,” especially conducting cyclical small tests of change allowed us to continuously address stakeholder barriers. For example, given resource constraints, the TTC team struggled with finding a balance between “meeting a metric,” which sometimes meant offering rudimentary tobacco counseling to a vast majority rather than comprehensive treatment to selected patients. Efforts to mitigate resource constraints included early engagement with the EHR team to use the EHR more efficiently and donation of additional hours for tobacco treatment from the RT department, thus allowing the team more time for bedside counseling. Another implementation challenge was developing a workflow acceptable to clinicians. The Compatibility construct (Inner Domain) allowed us to identify clinician barriers to fully integrating tobacco treatment into existing workflow and make real-time adaptations regarding prompt timing of the BPA and improved communication strategies with the primary team.

We demonstrate how rapid-cycle approaches allowed us to use interim data in iterative ways to track progress and improve the intervention in real-time. Utilizing the CFIR to gain insights into implementation challenges and ERIC to identify strategies to overcome them, we improved 6-month quit rates and adherence to public reporting programs (Fiore & Adsit, 2016; Herbst et al., 2020; Shelley et al., 2017) which in turn convinced leadership to continue to invest resources in our program, promoting sustainability. Studies from the Medical University of South Carolina, also a U.S. safety-net hospital, showed improved outcomes with a Tobacco Dependence Treatment Service, namely healthcare savings, improved smoking quit rates, and fewer 30-day hospital readmissions (Cartmell et al., 2018; Cartmell, Dooley, et al., 2018; Nahhas et al., 2016). Together our studies add to the growing evidence of why health care administrators, particularly at safety-net hospitals, should invest in hospital-based tobacco treatment services.

Our study has limitations. As a study conducted in a single safety-net hospital, the results of our study may not represent experiences and beliefs of all clinicians. Rapid-cycle evaluations do not allow the scientific rigor of an in-depth analytic approach. Interviews and meetings with some stakeholders (leadership) were conducted by TTC members; our respondents may have tailored answers to socially desirable responses. Our results reflect findings from a sample of clinicians who volunteered to participate, which may not be representative.

Future directions

Significant barriers to integrating tobacco treatment programs into hospital settings exist, most notably overcoming clinician constraints in delivering tobacco treatment due to competing priorities and obtaining funding for initial startup and sustainability. We show how a pre-implementation local needs assessment, followed by early-implementation rapid-cycle evaluations, is critical in overcoming these barriers and gaining an early understanding of key factors that influence implementation processes, as well as success of implementation and clinical outcomes. We believe our approach (1) facilitated successful implementation of an opt-out EHR-based inpatient TTC service, (2) provided leverage for hospital leadership to fund and sustain the program, (3) decreased provider burden in delivering tobacco treatment, (4) increased smoking cessation, and (5) improved adherence to public reporting programs.

While we opted for a dedicated tobacco treatment service with shared responsibility between the primary team and TTC service, these decisions, guided by the CFIR, may be handled differently by other hospitals, depending on local practices, resources, and institutional culture. Rapid-cycle evaluation has broad applicability, and we recommend this approach as a roadmap to guide, tailor, and improve implementation of hospital-based tobacco treatment by other hospitals. New and existing tobacco treatment programs must continue to measure, study, and disseminate data on implementation outcomes, effectiveness, interventions to reduce provider burden, cost-savings, revenue generated, and incentives gained from these programs, as highlighted in a recent review of several successful tobacco treatment models in the United States (Palmer et al., 2020). Rapid-cycle evaluations are a strategy for the Center for Medicare and Medicaid Innovations to accelerate health system innovations (Shrank, 2013), and are increasingly feasible as the availability of EHR data in real-time supports adapting programs in response to interim findings. While we no longer are required to report performance measures, we continue to incorporate iterative processes using rapid-cycle evaluations and performance feedback to promote the sustainability of our program. Lastly, further research is needed on how to elevate and maintain relative priority of inpatient tobacco treatment programs such that they are sustainable even after transient policy-related incentives have ended. Only then will administrators invest in initiating and maintaining funding for tobacco treatment programs in hospital systems.

Declarations

Ethics approval and consent to participate: The Boston University Medical Campus Institutional Review Board approved this study.

Consent for publication: N/A

Availability of data and materials: All data generated or analyzed during this study are included in this published article and its supplementary information files.

Competing interests: Dr. Kathuria is an UpToDate Section Editor, Tobacco Dependence Treatment section. The authors have no other conflicts of interests to disclose.

Disclaimer: The views expressed in this article do not necessarily represent the views of the Department of Veterans Affairs or the United States Government. The funding organizations had no role in the design and conduct of the study; the collection, management, analysis, and interpretation of the data; or the preparation, review, or approval of the manuscript.

Funding: This work was supported by the Boston University Evans Center for Implementation and Improvement Sciences (CIIS) and supported in part by resources from the Edith Nourse Rogers Memorial VA Hospital.

Authors’ contributions: HK had full access to all of the data in the study and takes responsibility for the content of the manuscript, including the data and analysis. Study concept and design: HK and RSW. Acquisition of data: HK, NH, BS, KC, EDH, MZ, CO, CF, ISI, MW, CW, LS, JO, RGM. Analysis and interpretation of data HK, NH, BS, KC, EDH, MZ, CO, CF, ISI, MW, CW, LS, JO, RGM, RSW. Drafting of the manuscript: HK and RSW. Critical revision of the manuscript for important intellectual content: HK and RSW. Obtained funding: HK and RSW. Study supervision: HK and RSW. All authors read and approved the final manuscript.

Acknowledgments: N/A