An analysis of adaptations to multi-level intervention strategies to enhance implementation of clinical practice guidelines for treating tobacco use in dental care settings

Abstract

Introduction

Our team conducted a cluster randomized controlled trial (DUET) that compared the effectiveness of three theory-driven, implementation strategies on dental provider adherence to tobacco dependence treatment guidelines (TDT). In this paper we describe the process of adapting the implementation strategies to the local context of participating dental public health clinics in New York City.

Methods

Eighteen dental clinics were randomized to one of three study arms testing several implementation strategies: Current Best Practices (CBP) (i.e. staff training, clinical reminder system and Quitline referral system); CBP + Performance Feedback (PF) (i.e. feedback reports on provider delivery of TDT); and CBP + PF + Pay-for-Performance (i.e. financial incentives for provision of TDT). Through an iterative process, we used Stirman's modification framework to classify, code and analyze modifications made to the implementation strategies.

Results

We identified examples of six of Stirman's twelve content modification categories and two of the four context modification categories. Content modifications were classified as: tailoring, tweaking or refining (49.8%), adding elements (14.1%), departing from the intervention (9.3%), loosening structure (4.4%), lengthening and extending (4.4%) and substituting elements (4.4%). Context modifications were classified as those related to personnel (7.9%) and to the format/channel (8.8%) of the intervention delivery. Common factors associated with adaptations that arose during the intervention included staff changes, time constraints, changes in leadership preferences and functional limitations of to the Electronic Dental Record.

Conclusions

This study offers guidance on how to capture intervention adaptation in the context of a multi-level intervention aimed at implementing sustainable changes to optimize TDT in varying public health dental settings.

1. Introduction

In the field of practice improvement there is growing consensus for the need to balance intervention fidelity and adaptation in order to optimize the implementation and sustainability of evidence based practices in health care settings [[1], [2], [3]]. An exclusive focus on fidelity may result in poor fit and short-term gains that are not sustainable, however adaptations that change essential components of an intervention may not produce the results achieved in efficacy trials. Balancing this tension requires an understanding of how to deliver interventions with appropriate fidelity while allowing for adaptations to fit local context [1].

Formative evaluations are often used to tailor interventions to different settings prior to program implementation [4,5]. However, there are often a number of unexpected implementation barriers within clinical settings that cannot be predicted prior to the intervention's initiation [1]. Unfortunately, effectiveness and implementation studies rarely capture the rationale, nature and number of implementation adaptations, and the process by which adaptations were made in a systematic or consistent manner [2,6].

Our team conducted a cluster randomized controlled trial that compared the effectiveness of three theory-driven implementation strategies on dental provider adherence to tobacco dependence treatment (TDT) guidelines [7]. In this paper, we describe the process of adapting the implementation strategies to the local context in dental public health clinics in New York City (NYC) [2]. Stirman's framework offers guidance for classifying the types of modifications made when interventions are implemented. The current study provided an opportunity to apply Stirman's model in dental public health care settings in which there is little data on barriers and facilitators to achieving system changes for quality improvement (QI). Findings will contribute to the methodology for measuring adaptations to inform scale-up and sustainability of evidence-based practices for promoting adherence to TDT guidelines.

2. Methods

2.1. Study context and setting

This analysis was conducted in the context of a three-arm cluster randomized controlled trial, the DUET (Dentists United to Extinguish Tobacco) Project, that evaluated system-level strategies for implementing practice guidelines for TDT in 18 dental health clinics from 2013 to 2017. These strategies included: ARM 1) Staff training and current best practices (CBP) which included a chart system to prompt tobacco use screening, brief counseling and cessation pharmacotherapy, and a system to refer patients to cessation counseling resources; ARM 2) CBP + performance feedback (PF); and ARM 3) CBP + PF + pay for performance (P4P) [[8], [9], [10], [11], [12], [13], [14], [15], [16], [17]]. Table 1 describes the characteristics of participating clinics and dental providers. Fifteen of the 18 sites used an electronic record, but these varied across sites. The mean number of dentists per site was 17.5. There was wide variation in the number of DDS/DMDs (SD 10.9), in large part because several sites included residency programs and residents were included in the DDS/DMD category.

Table 1.

Characteristics of participating dental clinics.

Dental Clinics Characteristics (n = 18)
Dental Clinic Type	n	%
Hospital affiliated	5	28%
Federally Qualified Health Center	8	44%
Other	5	28%
Member of a Practice Based Research Network (PBRN)
Yes	8	44%
No	4	22%
Don't know	6	33%
Type of Dental Record Used
Paper	3	17%
Electronic	15	83%
Clinic Volume
Small (100–400 adult patients per week)	11	61%
Medium (401–750 adult patients per week)	4	22%
Large (>750 adult patients per week)	3	17%
Number of Full-Time and Part-Time Dental Care Providers	mean	SD
Dentists (DDS, DMD)	17.5	10.9
Specialists	2.4	4.8
Dental Hygienists	1.4	1.1
Dental Assistants	3.8	5.7

2.2. Intervention (implementation strategies)

Table 2 highlights the core implementation strategies. All study sites were required to make dental clinic staff available for a 1 h training that included an overview of evidence-based approaches for treating tobacco use, a demonstration of how to use their chart system to screen for tobacco use and document cessation assistance (i.e., brief counseling, referral) and how to refer patients to the state Quitline and/or their local cessation program. About midway (4.5 months) through the nine-month intervention, sites received a “booster” training to reinforce the tobacco treatment workflow, and to address questions or barriers experienced since the initial training in CBP. Sites randomized to ARMs 2 and 3 also received quarterly performance feedback reports on provider delivery of cessation services using chart audit procedures. ARM 3 sites additionally received $20 for each patient with chart documentation that a tobacco user received cessation assistance (i.e., brief counseling, referral, and/or prescription). Details of the study design are described in a previous publication [7].

Table 2.

Frequency and type of modifications made to the implementation of the "DUET″ intervention in NYC dental public health clinics.

Type of Modification	Stirman Definition (Stirman et al., 2013)	Modification Frequency			Modification types by intervention component
		Number of modifications across all clinicsa (n = 227)	Average number of modifications per sitea (Range)	% of total modifications	Staff traininga	Chart system and workflowa	Quitline referral systema	Toolkitsa	Performance feedbackb
Content Modifications	Modifications made to the content itself, or that impact how aspects of the treatment are delivered
Tailoring/tweaking/refining	Any minor change to the intervention that leaves all of the major intervention principles and techniques intact while making the intervention more appropriate, applicable or acceptable	113	6.3 (5-7)	49.8%	29 (24.4%)	36 (30.3%)	18 (15.1%)	18 (15.1%)	12 (15.1%)
Adding elements	Additional materials or activities are inserted that are consistent with the fundamentals of the intervention	32	1.8 (0-4)	14.1%	0 (0.0%)	18 (50.7%)	7 (19.7%)	0 (0.0%)	7 (29.6%)
Departing from the intervention (“Drift”)	The intervention is not used in particular situation or the intervention is stopped, whether this stoppage was for part of a session or a decision to discontinue the intervention altogether	21	1.2 (1-2)	9.3%	0 (0.0%)	14 (57.1%)	0 (0.0%)	0 (0.0%)	7 (42.9%)
Loosening structure	Elements intended to structure intervention sessions do not occur as prescribed in the manual/protocol	10	0.8 (0-1)	4.4%	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	10 (100%)
Lengthening/extending	A longer amount of time than prescribed by the manual/protocol is spent to complete intervention or intervention session	10	0.6 (0-1)	4.4%	10 (100%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Substituting elements	A module or activity is replaced with something that is different in substance	3	0.2 (0-1)	1.3%	0 (0.0%)	0 (0.0%)	3 (100%)	0 (0.0%)	0 (0.0%)
Total Content Modifications		189	10.5 (6-14)	83.3%	39 (18.8%)	68 (32.9%)	28 (13.5%)	18 (8.7%)	36 (26.1%)
Contextual Modifications	Modifications made to the way the overall treatment is delivered
Personnel	The intervention is being delivered by personnel with different characteristics	18	1 (1-1)	7.9%	18 (100%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	0 (0.0%)
Format/Channel	Changes are made to format or channel of treatment delivery	20	1.1 (0-3)	8.8%	4 (14.3%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	16 (85.7%)
Total Contextual Modifications		38	2.1 (1-4)	16.7%	22 (47.8%)	0 (0.0%)	0 (0.0%)	0 (0.0%)	16 (52.2%)
Overall Total Modifications (kappa = .92)		227	12.6 (7-17)	100%	61 (24.1%)	68 (26.9%)	28 (11.1%)	18 (7.1%)	52 (30.8%)

^a18 sites (all sites).

^b12 sites (only sites in arms 2 and 3 received performance feedback).

2.3. Data sources

We used a mixed methods approach, drawing from multiple data sources, to categorize and code modifications made to the DUET intervention/implementation strategies.

• (1)Needs Assessment: Upon enrollment, each participating Dental Director completed a baseline survey to capture organizational characteristics. For example, the survey captured staffing structures and whether clinics had a Dental Residency program. Furthermore, the DUET project coordinator met with each clinic's Dental Director, in addition to other relevant clinic staff (i.e. Clinic Manager, Administrator, Information Technology (IT) staff), to conduct a baseline assessment of current workflow (i.e., staff roles and responsibilities related to tobacco use and timing and process of documenting tobacco use and cessation assistance in the chart). The assessment also included a detailed review of the Electronic Dental Record's (EDR) functionality including the capacity to extract TDT quality indicators for performance reports and to evaluate the presence and location of a section that prompted screening and documentation of tobacco use. With input from the Dental Director and other members of the research study team, the Project Coordinator identified the initial modifications that were needed to ensure that sites could implement the prescribed intervention elements and what types of modifications were needed to maximize intervention fidelity.
• (2)Site Observations: Sites visits were conducted by the project coordinator at baseline, 4.5-months and 9-months post-intervention. We used a site observation tool which captured use and implementation of TDT clinical processes and workflows, such as location of smoking status documentation in the EDR, how Quitline referrals were made, and whether patient educational materials were visible and accessible.
• (3)Field Notes: During the intervention period, the DUET Project staff and dental clinic leadership discussed challenges that might have arisen in implementing the intervention components and made shared decisions about any necessary modifications. Extensive notes were taken during these discussions. The weekly DUET research meetings also included review and discussion of proposed modifications to the study intervention protocol. Additionally, we reviewed the training power point slides, feedback reports, and clinical workflow maps that were tailored for each site.

2.4. Coding of modifications

Through an iterative process, we used Stirman's modification framework to classify and code post-hoc modifications made to the DUET intervention. The framework considers the levels of modification delivery, including by and for whom modifications were made, and the context or content of each modification [2]. Two DUET team members (CK, AC) independently reviewed and extracted information from the study data sources to identify and categorize the types of content and context modifications made to the DUET intervention. This initial coding schema was first applied to a subset of six sites, allowing for additional categories to emerge, until saturation of codes was achieved. We created a codebook with detailed definitions of each modification category, including specifications and examples, and shared this with the rest of the study team for review. Based on feedback and discussion among study team members, some modifications were re-classified or refined, and redundant categories were collapsed. After revisions were made to create a finalized comprehensive codebook, CK and AC independently applied the modfication coding scheme to all 18 study sites, using a dichotomous rating scale of 0 = no modification and 1 = yes modification. A Cohen's kappa coefficient was computed to determine inter-rater reliability between the two coders. Discordant coding was discussed with the rest of the study team until consensus was achieved.

2.5. Analyses

We conducted basic descriptive analyses of the type and frequency of modifications made to the DUET intervention across the 18 participating dental clinic sites. We then summarized the proportion of each modification category by each of five study intervention components: (1) Staff training, (2) Chart system and workflow, (3) Quitline referral system, (4) Toolkits, and (5) PF (Table 2). P4P is not included in the analysis because this intervention was delivered as planned, at the site level based on evidence that cessation assistance was provided. Any adaptations made to the P4P protocol were related to adaptations made to the performance reports (i.e., how cessation assistance was defined) and reported in the PF category.

Weighted percentages were used to inflate the observed counts for PF, both in the numerator and denominator, to adjust for only 12 of 18 sites assigned to this intervention component. We also provide descriptive examples of the types of adaptations made to each intervention component and factors influencing adaptations as observed and reported by intervention study staff (Table 3).

Table 3.

DUET intervention components, types of adaptations and factors influencing adaptations.

Intervention Component	Description as intended in protocol	Types of Adaptations (Examples)	Factors Influencing Adaptations
Staff Training	⁃Conduct 1-h training on PHS Guidelines for Treating Tobacco Use and Dependence with dental providers ⁃Conduct booster training at midpoint of intervention period	Content ⁃TailorVed training ⁃Added sessions for new providers ⁃Updated trainings with new research Context ⁃Delivered training via webinar or videoconference ⁃Delivered to varied provider types	⁃Emergence of new research ⁃Provider turnover (i.e. new residents) ⁃Dental director preference ⁃Scheduling restraints ⁃Site staffing structure
Chart System and Workflow	⁃Implement an electronic chart system that allows documentation of tobacco screening and treatment ⁃Create a clinical workflow map for provision of tobacco use treatment (TDT)	Content ⁃Tailored documentation of tobacco screening and treatment in electronic dental record (EDR) ⁃Loosely determined roles and responsibilities instead of workflow map ⁃Added workflow for offering Nicotine Replacement Therapy (NRT) at the Point-of-Care	⁃Varied EDR systems ⁃Lack of IT support ⁃EDR limitations ⁃Site staffing structure ⁃Dental director preference ⁃Availability of funds for distributing NRT
Quitline Referral System	⁃Integrate referral system to link patients to the NYS Smokers' Quitline and other cessation programs	Content ⁃Tailored addition of other referral systems (i.e. Asian Smokers' Quitline, onsite cessation program) ⁃Tailored mode of referral (paper fax, electronic fax, online portal) to Quitline ⁃Substituted other modes of referral to the Quitline (i.e. “Request-a-call” feature on Quitline website)	⁃Lack of fax machine and/or staff to fax paper referrals ⁃Availability of other cessation programs ⁃Dental director preference ⁃Patient population
Toolkits	⁃Provide Smoking and Oral Health Quit Kit developed specifically for dental care providers	Content ⁃Tailored and refined toolkit contents (i.e. added site-specific materials; materials translated in Chinese and Spanish)	⁃Emergence of updated materials ⁃Availability of site-specific resources ⁃Dental director preference ⁃Patient population
Performance Feedback	⁃Distribute quarterly provider performance feedback reports displaying graphs of each individual performance compared to peers and to a benchmark ⁃Dental Director distributes reports to providers no later than 30 days following the end of the quarter ⁃Clinic receives a financial incentive of $20 for each patient with chart documentation of receiving tobacco cessation assistance, capped at $5000	Content ⁃Tailored how data was generated for reports (i.e. manual chart review, automatic) and who generated the data (i.e. clinic staff or research staff) ⁃Loosened structure of reports (i.e. unable to report data on all TUT measures) ⁃Added "dummy codes" to EHR in order to identify and query smokers Context ⁃Changes to the when and how the reports were distributed to providers ⁃Discussed performance feedback reports with dental director prior to distribution	⁃Varied EHR systems ⁃Lack of IT support ⁃EHR limitations ⁃Dental director preference ⁃Time constraints

3. Results

3.1. Classification of modifications

Content modifications are defined by Stirman's framework as changes to the intervention content itself. We identified examples of six of Stirman's twelve content modification categories, including: (a) Tailoring/tweaking/refining, (b) Adding elements, (c) Substituting elements, (d) Departing from the intervention (“drift”), (e) Loosening structure, and (f) Repeating elements [2]. Context modifications, refer to changes made to the delivery of the intervention. We identified examples of two of Stirman's four context modification categories: (a) Format and (b) Personnel. Table 2 shows the definitions for each category, frequencies for each modification and the modifications made by type of intervention components. Inter-rater reliability between the two coders was established (kappa = .92).

3.2. Content modifications

Content modifications accounted for the majority of total modifications (83.3%).

• a.Tailoring, tweaking, or refining was the most common type of adaptation (49.8%). Although this type of modification was observed across all intervention components, the majority of modifications occurred in relation to required changes in the chart system and related workflow. In order to integrate TDT into clinical practice, workflows were individualized according to clinical volume, staffing and leadership preferences. Furthermore, there was wide variation in EDRs, requiring adaptations in terms of the location for where and how tobacco use and treatment plans were documented.
• b.Loosening structure of the intervention (4.4%) predominantly occurred in relation to the provider performance feedback reports (PF). Data availability varied depending on the EDR and what tobacco use and treatment delivery was routinely documented. Therefore, we were unable to consistently retrieve all of the predefined TDT quality indicators. These included the percent of patients: (1) Screened for smoking status, (2) Smokers advised to quit, (3) Smokers prescribed pharmacotherapy, and (4) Smokers referred to the Quitline or other cessation service. This required modification of PF reports in some sites that were not able to provide data needed to report all of the TDT quality indicators. In addition, data retrieval delays and inconsistent support from the clinics' information technology (IT) staff resulted in delivering PF reports later than planned (i.e., more than 30 days after the end of the study quarter).
• c.Departing from the intervention accounted for 9.3% of modifications. For example, creating a smokers' registry was originally a component of current best practice. The purpose of the registry was to facilitate performance reports and referrals to the state Quitline. However most of the sites' EDRs were not designed to allow users to query smoking status or to automatically transfer patient information on smoking status to a backend database. There was also little enthusiasm among dental directors to create registries that they felt the IT staff did not have the capacity to support. Rather they agreed to allow the study team to add an “unbillable“ or “dummy” CDT Code (Code on Dental Procedures and Nomenclature-- standardized codes used for processing dental claims) indicating patients' smoking status. For example, in some sites this code was labelled “TOBUserYes”, and could be selected by dental providers to indicate that the patient was a smoker. The “TOBUserYes” code could later then be queried in the EDR to populate a list of tobacco users, which was then used to facilitate data extraction for performance reports.
• d.Adding elements, which accounted for 14.1% of modifications, also involved changes to the clinical workflow. For instance, some sites affiliated with a large health care system provided free nicotine replacement therapy (NRT) to distribute to patients. Therefore, they were not interested in prescribing cessation medication through their EDR. The research team assisted these sites in developing a system to order NRT distribution at the point-of-care. Another example of adding elements was modification of the referral system which was originally conceived as one that facilitated referrals to the state or national Quitline. However, several sites had an onsite cessation program and preferred to support this service. In these sites we developed site-specific workflows to facilitate these referrals.
• e.Lengthening and extending accounted for a small percentage of modifications (4.4%) and exclusively entailed the need for additional staff trainings in sites with dental residency programs. These training programs are only one year (starting in July) and when the intervention straddled a new cohort of residents we needed to conduct additional trainings for these new staff members.
• f.Lastly, substituting elements occurred within the context of Quitline referrals (1.3%). In sites that lacked a fax machine and/or staff to fax paper referrals to the Quitline or did not have an e-referral system, we provided access to an online referral program that allowed referrals through the Quitline website. All of these options triggered a proactive Quitline call to referred smokers, however, the web-based system did not allow for progress reports to be sent from the Quitline to providers describing their patients' treatment plans.

3.3. Context modifications

Context modifications were classified into modifications made to personnel (7.9%) and to the format/channel (8.8%) of the intervention delivery. The type of personnel who received the intervention varied across dental sites. Dental director preferences and site staffing dictated which type of dental care providers participated in TDT trainings. In some sites, Dental Assistants played an essential role in asking patients about tobacco use, and therefore were trained alongside other dental providers, whereas in others they were neither involved in clinical care nor the delivery of TDT and therefore were not included in the TDT training.

The format of staff trainings was also modified in some sites. The training was intended to be delivered in-person, onsite and in a group setting. Modifications included delivering training via webinar, videoconference or one-on-one. These changes were often driven by scheduling constraints and provider turnover. Additionally, due to dental director preferences and time constraints, changes were also made to how performance reports were distributed to providers (i.e., by dental director vs staff, in group meetings vs email).

3.4. Factors influencing adaptations

Table 3 describes factors that influenced each type of adaptation of the intervention components. Common factors associated with adaptations included staff changes, time constraints and variation across sites in leadership preferences for how TDT would be integrated into routine dental care. Functional limitations of the EDRs accounted for the majority of adaptations. It was difficult for dental providers to navigate through the EDR to document preventive services in general, and tobacco use screening and treatment specifically. Moreover, EDRs were typically not integrated with electronic medical records (EMRs) in sites with both services which also created additional workflow problems for providers. For example, in some sites dental providers were forced to exit the EDR and enter the EMR to prescribe cessation pharmacotherapy. Finally, most EDRs did not have the functionality to generate performance reports from clinical data collected during routine visits. Each of these system limitations were addressed through a range of adaptations that were tailored to each site.

3.5. Discussion

Stirman's Adaptation and Modification Framework provided a useful guide for systematically analyzing intervention component modifications that were made prior to launching the intervention study period. Consistent with best practices for implementing practice change, we engaged dental leadership at each site in a formative assessment intended to tailor planned implementation strategies to the local clinic infrastructure (e.g., EDR features), to current practices related to TDT, and to identify other potential barriers to practice change [18]. However, dental practice settings varied greatly and the multi-component strategies created system-level implementation challenges that required ongoing problem-solving with dental clinic staff and leadership resulting in the additional modifications captured in this analysis.

Through application of the Stirman framework we identified both content and context modifications to the intervention strategies under investigation. The DUET interventions relied heavily on optimizing use of the EDR to monitor and improve TDT guideline adoption. Therefore, it is not surprising that most of the adaptations were driven by variations in the type, functionality, and usability of the EDR that sites were using. In general, limitations in the dental sites' EDRs created inefficient workflows for screening and documenting TDT. Moreover, many EDRs did not have a standardized location for documenting TDT (e.g., counseling/cessation provided), or an automated Quitline referral system. To maximize fidelity to the key intervention components related to EDR functions, we collaborated with sites' IT staff to modify EDR systems. These modifications included programming new documentation systems and creating new workflows within the existing EDR infrastructure to ensure that all EDRs included a consistent location for screening and documenting treatment and referrals.

Compared with medical practices, dental care settings have also been slow to adopt standardized approaches to efficiently access information in EDRs [19]. For example, most dental clinics' EDRs did not have the reporting functionality to generate clinical data for quality improvement (QI) purposes in general, and performance feedback, specifically. Again, we collaborated with site stakeholders (dental directors, IT staff) to make site specific modifications to EDRs and established new workflows for extracting and aggregating data for performance reports.

Involving stakeholders in the intervention implementation and adaptation process is widely recognized as essential to improve implementation effectiveness [3,20]. Throughout the study, we followed a participatory research approach to ensure that intervention adaptations were informed by local organizational culture, policies, procedures and workflow. Whenever possible, we worked collaboratively with dental site IT staff to modify EDR systems to integrate these components into their preventive care screening and treatment processes. This collaborative process, and the need to adapt the intervention components to each study site, required significant resources and start-up time and should be anticipated in staffing and budget considerations for future implementation efforts. No two study sites had the same workflow or procedures for screening and treating tobacco use, even when they were part of the same broad health system. However, effective adaptations made to EDRs, for example, were applied subsequently to other sites using the same EDR, and lessons learned from sites that had similar EDR functionality and staffing arrangements were shared and replicated with other sites.

Technology-driven implementation strategies will likely need further adaptation to the next generation EDRs. During the study, several participating dental practices were already making changes to improve integration of their EDRs with EMRs when co-located with medical sites, and updating EDRs to improve system usability. In this rapidly changing health care environment researchers may need to expand their focus beyond achieving short-term study goals and collaborate with participating sites to plan for future adaptations when designing implementation strategies. As part of this planning process, our experience suggests the need to expand stakeholder engagement, particularly IT support, while designing implementation strategies, with greater emphasis on the goal of building local capacity to adapt to future innovations in health care delivery while maintaining quality [[21], [22], [23]].

With the expectation that integrating pre-specified intervention components into complex, varied local practice contexts will likely require some modifications, it follows that adaptation could be defined as an implementation strategy. In a recent project that used concept mapping to develop terminology for describing implementation strategies, “adapt and tailor to context” emerged as one of the 73 distinct implementation strategies [24]. Consistent with this conceptualization, a few studies have tried to pre-specify “allowable” adaptations at the start, providing a “menu” of intervention alternatives [25,26]. This prospective approach may allow for more standardized and interpretable measurement of intervention fidelity. However, as observed in this study and others, there are often significant variations observed in workflow and systems across study sites and unanticipated barriers and practice changes that emerge throughout the intervention period. Our findings suggest that the challenge of adaptation measurement can be addressed through the consistent application of a framework like Stirman's to guide detailed documentation of modifications made during the formative and implementation phases of research.

Several recent articles have incorporated Stirman's model in their protocols [[27], [28], [29]] and a growing number of studies have used the model to monitor and report modifications [27,30], and link type of adaptations to outcome data [31]. Other models that suggest methods for systematically tracking adaptations include the Dynamic Adaptive Process Model and the Managing and Adapting Practice approach to evidence-based practice [1,32].

As more investigators adopt standard methods for documenting adaptation there is an opportunity to begin to develop a database to catalog modifications across studies and estimate how much modification can made to an intervention in a real world setting and still maintain its effectiveness. Chambers and Norton proposed creating such a database that they refer to as an “adaptome”, conceived as a platform “to house systematically captured information about variations in delivery of evidence-based interventions across multiple populations and contexts” [21]. These data could inform future intervention development by offering examples and scenarios of modifications that may be applicable to similar settings and populations. These data would also create an opportunity to view, longitudinally, how implementation strategies adapt to a rapidly changing health care system (e.g., new technology, new care process, new staffing models) [21,22].

The study had limitations. First, we did not link quantity and type of modifications with outcome data, which is critical to understanding the impact of modifications on intervention effectiveness. With a sample of 18 dental care sites, power limitations precluded us from conducting outcome analyses. Second, rather than having a prospective plan for systematically capturing and categorizing modifications, Stirman's framework was applied retrospectively. While we did not systematically delineate allowable modifications prior to intervention implementation, formative data from our initial needs assessments, and careful documentation of modifications in our field notes, provided a comprehensive record that we were able to draw on for this analysis.

Increasing attention is being paid to the important role dental providers play in providing preventive care services, including TDT [33,34]. Yet, dental practice settings have been largely left out of national initiatives to optimize the use of EDRs to drive QI [35]. This study highlighted the need for dentistry to work with EDR vendors and to integrate medical and dental informatics efforts to create systems that facilitate evidence-based preventive care and QI efforts.

4. Conclusion

This study offers guidance on how to capture intervention adaptations in the context of a multi-level intervention aimed at improving implementation of TDT in public health dental settings. Using a systematic approach to describe necessary adaptations, while maintaining fidelity to key intervention components, may increase potential for sustainability, replication and scale up of evidence-based implementation practices.

Ethics approval

Approval by the institutional review board of the University of New York School of medicine was obtained prior to the study (IRB# i12-01148).

Consent for publication

Not applicable.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare that they have no competing interests.

Funding

This work was supported by a grant from the National Cancer Institute (R01CA162035). The National Cancer Institute had no role in the study design, collection, analysis and interpretation of data, writing of the report or the decision to submit the article for publication.

Authors' contributions

DS conceived the analysis, AC, CK, DK, DS and JO participated in the analysis, CK, DS and JO contributing to writing and editing the manuscript. All authors read and approved the final manuscript.

Abbreviations

TDT

Tobacco dependence treatment guidelines

NYC

New York City

CBP

Current Best Practices

PF

Performance Feedback

QI

Quality Improvement

IT

Information Technology

EDR

Electronic Dental Record

EMR

Electronic Medical Record

Appendix A. Supplementary data

The following is the supplementary data related to this article: