Abstract
Background
Emergency departments are seeing an increase in acute exacerbations of chronic disease in the older-adult population. The delivery of palliative care in the emergency department can increase goal-concordant care at the end-of-life for this population. New interventions in palliative care for emergency medicine require large, pragmatic, complex health interventions due to the heterogeneous and dynamic environment of emergency departments. These complex interventions must balance fidelity with adaptability, while being rooted in theory, to produce an intervention that can be applied in a variety of contexts.
Methods
Primary Palliative Care for Emergency Medicine (PRIM-ER) is a large, pragmatic, complex health intervention. This paper outlines the conceptual theory-based design as well as the study form and functions of PRIM-ER to exemplify how this complex intervention has balanced fidelity with adaptability.
Results
A form and function matrix was created to highlight the key objectives and tailored intervention components of PRIM-ER. Each intervention component was also linked to one or more elements of the Theory of Planned Behavior to support provider behavior change and the delivery of palliative care services and referrals.
Conclusion
The application of theory and delineation of forms and functions, as well prospective adaptation monitoring of large complex interventions can support the balance of fidelity with adaptability to encourage successful interventions among a variety of clinical environments.
Keywords: Palliative care, Emergency medicine, Theory, Research design, Complex interventions, Health services
Highlights
-
•
PRIM-ER is a quality improvement complex intervention to enhance goal-concordant care in the emergency department.
-
•
Complex interventions must balance fidelity with adaptability to enable transferability across various contexts..
-
•
Applying form and function in a theory-based conceptual model supports fidelity and adaptability in complex interventions.
1. Introduction
Emergency Departments (EDs) care for society's most vulnerable older adults and play a pivotal role in the care trajectory of seriously ill patients nearing the end of life. Half of Americans aged 65 years and older will be seen in the ED in their last month of life, and three-quarters will visit the ED in their last 6 months [1]. EDs must balance the potential harms and benefits of hospitalization for seriously ill older adults [2,3], many of whom prefer to receive their end-of-life care at home, outside of the hospital setting [4,5]. The current clinical model continues to focus on treatment of acute illness and favors life-sustaining therapies, which may contradict the wishes of these older adult patients. Little attention has been paid to the delivery of goal-concordant care in the ED for older adults with serious illness. Palliative care interventions in the ED can capture high-risk patients at a time of crisis and can dramatically improve patient-centered outcomes [6,7]. New interventions are required to support the education of providers and implementation of practices to deliver goal-concordant palliative care services in the ED.
The design of novel interventions in palliative emergency medicine (EM) lends itself to the creation of large, pragmatic, complex health interventions due to the heterogeneous and dynamic environment of EDs. Complex interventions are defined by a loose set of criteria, including: 1) the intervention contains numerous interacting components, 2) individuals delivering and receiving the intervention often exhibit complex behaviors, 3) the intervention incorporates numerous organizational levels, 4) numerous and varied outcome measures, and 5) the degree of flexibility permitted in the delivery of the intervention [8,9]. The multicomponent nature of complex interventions, especially those containing complex behavioral aspects of participants, requires these studies be rooted in established theory. Simple randomized trials rely less on theorized design since they often ignore the broader context of real-world settings. However, the effectiveness of complex health system interventions relies on an adjusted view of, “standardization,” to tailor the intervention to local contextual needs, rather than a purely uniform intervention for all [10]. Using theorized design and adapting complex interventions to local needs has shown to increase effectiveness of public health interventions [[11], [12], [13], [14]]. One method of adaptive design includes the constructs of form and function [8,10]. Functions of a complex intervention represent a purpose or goal, while forms are the tools or processes used to achieve a function(s) [8,10]. Identifying forms and functions allows adapted complex interventions to retain a level of standardization and integrity in design [10], and is supported by the Patient-Centered Outcomes Research Institute (PCORI) methodology standards for complex interventions [15].
The Primary Palliative Care for Emergency Medicine (PRIM-ER) study is a complex intervention based on the Theory of Planned Behavior with pre-established form and function components incorporated into the conceptual model. PRIM-ER is a pragmatic example of the interplay of these three key components of research design (complex interventions, theory, and form and function) that can serve as a model for future interventions and outlined herein to exemplify how adaptive design and theory can serve as a foundation for a complex health intervention.
1.1. PRIM-ER description
PRIM-ER is an education, training, and technical support quality improvement intervention for the ED. It utilizes a cluster-randomized stepped wedge design in which 35 EDs within 18 healthcare systems are randomly assigned a sequential order to begin the intervention. The order in which the EDs begin their intervention period is block-randomized and spread over 24 months (from May 2019 to June 2021). Two of the 35 sites were selected to pilot the intervention to optimize feasibility, fidelity, and usability. All sites serve as their own control, and a new site begins the intervention every three weeks. Each site will undergo preliminary work prior to initiating the intervention, including workgroups consisting of palliative care, emergency nursing, social work/case management, informatics, and ED operations representatives to discuss how best to implement the intervention at each site. The main study outcomes that will be measured are: post-intervention ED disposition to an acute setting, healthcare utilization 6 months following an ED visit, and survival following the index ED visit as a result of the intervention. This data will be obtained from the master beneficiary summary file, inpatient, outpatient, home health, hospice and vital status files of Centers for Medicare and Medicaid Services (CMS) claims of the beneficiaries. Estimates of the baseline rate of acute care admission, healthcare utilization, and survival following the index ED visit will be calculated from Medicare claims data in order to evaluate whether there was a change before and after implementation of the intervention. Further details on the PRIM-ER protocol are published elsewhere [16].
The intervention consists of four core components and was designed by the PRIM-ER principal investigator (PI) and eight co-investigators of varying disciplines including but not limited to: implementation science, health services research, geriatric nursing, biostatistics, and informatics. The four components include: 1) evidence-based multidisciplinary primary palliative care education, 2) simulation-based workshops on communication in serious illness, 3) clinical decision support, and 4) provider audit and feedback. The education on palliative care for emergency medicine providers (physicians, nurse practitioners and physician assistants) is accomplished via an online (pre-reading) module and in-person. Full-time EM providers take a 1-h online didactic course, EPEC-EM, targeting primary palliative care knowledge and skills in needs assessment and referral. They also attend a 4-h in-person simulation workshop, EM Talk, focused on end-of-life communication. These sessions include simulated patients/families, role-playing, and small group learning with constructive feedback from master clinicians. The master clinicians are a national group of physicians with experience in palliative care who have previously participated in a 6-day train-the-trainer course in communications training around serious illnesses. Similar to EPEC-EM, full-time EM nurses take a 1-h online course, ELNEC, tailored to nurses focusing on primary palliative care knowledge and skills in needs assessment and referral.
Clinical decision support consists of electronic triggers for palliative care in the electronic health record (EHR). Interruptive best practice alerts will be embedded in the EHR to aid providers in identifying patients who are likely to benefit from palliative care or hospice. Finally, audit and feedback reports to monitor provider and departmental performance will be developed and disseminated to faculty and staff as well as EM leadership. A learning monitoring system will track participation in educational activities and encourage cross-fertilization and learning among sites.
Each site will dedicate a nurse and physician champion to assist with implementing the intervention. The champions will facilitate attendance at didactic and workshop sessions, disseminate information about local resources, work with the informatics team to reinforce protocols, and implement trigger criteria to identify older adults who may benefit from further needs assessment and follow-up.
In order to garner buy-in and support during the pilot year of PRIM-ER, the PRIM-ER principal investigator and NYU study team members will have in-person meetings with each site. During the site visits, the PRIM-ER study team will meet with each site's ED departmental leadership teams (e.g. Department Chair, Site PI, Medical and Nurse Director etc.), to review the intervention details and timeline, as well as better understand the clinical workflow at each enrolled ED.
Additionally, key informant workgroup meetings will occur during the site visit to identify key barriers and facilitators of ED-based palliative care and discern related themes and patterns that may impact the structuring of clinical workflows. Each key informant workgroup will last approximately 60–90 min and a will be audio-recorded and transcribed. Participants will include site level staff from each of the following disciplines: palliative care, emergency medicine nursing, chief medical informatics officer, ED clinical operations, social work/case management, and ED education leadership. Workgroup questions will include but are not limited to the following: 1) What clinical decision support (CDS) tools are currently available within your EHR to address PRIM-ERs clinical goals and objectives? 2) Within this cluster of tools, which have been shown to have the most impact at your institution? 3) How can the PRIM-ER team configure and customize the CDS tools to suit your ED and workflow? During the key informant workgroups the PRIM-ER study team will also review and discuss the sample PRIM-ER CDS and discuss the workflow, ask workgroup participants to collectively complete a CDS mapping document [17], review the mapping document as a team to understand discrepancies and overlap, and come to a consensus on the CDS key features that would be beneficial at each site. The key informant workgroup feedback will be used to ensure that the NYU study team designs and implements a CDS model that meets the unique needs of each site's ED.
1.2. Applying a theory-based conceptual model to the PRIM-ER intervention
Applying theory to research, also termed theory-based design, involves incorporating the components of established theoretical models within the aspects of a research study to help predict and understand complex phenomena, such as human behavior. Theory-based design is key to, “guide and inform research so it can, in turn, guide development efforts and improve professional practice.” [18] The PRIM-ER conceptual model is rooted in the Theory of Planned Behavior, which is part of the social cognitive theory of behavior change first described by Icek Ajzen [19,20]. The theory articulates how an individual's behavioral intentions and behaviors are shaped by, 1) their attitude toward the behavior, 2) subjective norms (perception about the behavior that is influenced by others), and 3) perceived behavioral control (an individual's perceived ease or difficulty of performing the particular behavior). This theory is of importance when used to understand and predict healthcare professionals' intentions and behaviors, as many clinical practice adoption decisions are individual professional decisions [21]. A systematic review by Goden et al. (2008) found the application of the Theory of Planned Behavior outperformed other social cognitive theories in predicting healthcare professionals' behavior [22]. This theory has also been demonstrated in relation to palliative care. Physicians' ratings of their knowledge and attitudes around palliative care are indeed associated with their end-of-life practices [23].
In the context of PRIM-ER, attitudes include physicians’ support for palliative care practice and philosophy, as well as their views on physician-patient communication in serious illness. Subjective norms include whether it is within the emergency medicine scope of practice to discuss goals of care with a seriously ill patient in the ED. Perceived behavioral control is based in self-efficacy theory, and is described in the palliative care context as the comfort or ease with which emergency providers are able to have a goals of care discussion or to discuss hospice services. Each PRIM-ER intervention component was created based on one, or more, of the three main components of the Theory of Planned Behavior. The goal is to change the culture of emergency medicine through behavior change of providers to support the delivery of primary palliative care in the emergency department. Each intervention component is expanded via key functions and adaptable forms to implement the complex intervention through the lens of theory-based behavior change.
2. Methods
2.1. Matrix development process
A PRIM-ER form vs function matrix was developed to capture the interplay between the components of a complex intervention, study forms and functions, and theory-based design. The matrix is a product of the PRIM-ER design, which was developed by PI, author CG, and co-investigators [16]. PRIM-ER was designed using a multidisciplinary approach, experience from prior studies, empirical evidence, multifaceted and highly innovative clinical informatics and technology techniques, and a focus on complex intervention trial design with applied theory, as described above [16]. The matrix was created using an iterative process over the course of two months with bi-weekly meetings among three members of the research team (with oversight by the PI) to extract the core components of the PRIM-ER study to display the study design elements in a matrix format. The inclusion and exclusion of important design aspects in the matrix was determined among the research team using consensus development techniques [24].
3. Results
3.1. Form vs. function matrix
The following matrix (Table 1) details the intervention components and forms and functions of PRIM-ER. The matrix contains four main columns, 1) the intended change rationale rooted in the Theory of Planned Behavior each intervention component is addressing, 2) the intervention component, 3) core functions of each intervention component and, 4) the forms each site can tailor and adapt to their local site context to carry out the deliverables of the pragmatic trial.
Table 1.
Primary Palliative Care for Emergency Medicine (PRIM-ER) Form vs. Function Matrix.
| Motivating Need/Problem | Intervention Component | Core Functions (Standardized) | Forms (Tailored) |
|---|---|---|---|
| 1. Nurse education to change subjective norms, attitude, and behavior. | Delivery of ELNEC training curriculum 1-h course to nurses | Objective: Primary palliative care knowledge and skills in needs assessment and referral for nurses |
|
| 2. EHR modifications to change provider behavior and expectations around management of seriously ill older adults with life-limiting illness. | Implement an EHR modification/alert to identify patients at risk for short-term mortality | Objective: Alerts and new workflow to refer to palliative care, home care, and hospice services |
|
| 3. Learner centered audit and feedback to enhance verbal persuasion. Change subjective norms by implementing sustainable systems that encourage palliative care referrals. | Implement a system to track EHR modification data, make data driven decisions and create a sustainable process of data monitoring and feedback | Objective: Create a sustainable system during the intervention period that incorporates ED-specific continuous quality improvement process around palliative care |
|
| 4. Simulation –based communications workshops targeting perceived behavioral control, attitude, and building providers self-efficacy. | Implement EM Talk, a 4-h in person communication training. Have EM providers complete the online pre-reading material EPEC-EM online. | Objective: Deliver simulation workshop in end-of-life communication to Emergency Medicine providers |
|
ELNEC: End-of-Life Nursing Education Consortium.
EHR: Electronic Health Record.
BPA: Best Practice Alert.
ED: Emergency Department.
ECOG: Eastern Cooperative Oncology Group.
DNR/DNI: Do not resuscitate/Do not intubate.
EPEC-EM: Education in Palliative and End-of-Life Care for Emergency Medicine.
As a specific example, the nurse intervention component (Item 1, Table 1), is designed to increase palliative care knowledge and skills as a core function, but can be modified to be delivered online, in-person, or via a hybrid method. However, for EM-Talk (Item 4, Table 1) the delivery method can only occur in-person due to the interactive and small group nature of the simulation workshop. EHR modification can be tailored in a variety of ways, as they inherently require adaptation to workflows and processes that may differ across sites. Each EHR modification process will use an algorithm for identifying patients at high risk for short-term mortality and an alert process for notifying providers. A clinical reminder to screen for palliative care will trigger when a provider engages with a patient who may benefit from palliative support. This alert can occur via different forms depending on the workflow processes at each ED to ensure fit within a new setting and discourage resistance. The clinical reminder could pop up during the patient assessment by a nurse, a social worker, a case manager, or when the EM physician evaluates the patient. Additionally, access to potential referral resources (e.g. home care, hospice, etc.) varies at each site, thus the content (tailored form) must be adapted to the local context of the site in order to implement the intervention successfully.
3.2. Linking theory with intervention components
The three fundamental elements of the Theory of Planned Behavior, including attitudes, subjective norms, and perceived behavior control, underlay the multiple intervention components of PRIM-ER to support successful behavior change via the intervention. These fundamentals provide the foundation of the motivating needs/problems listed in Table 1.
Attitudes- To change the attitudes of providers to more favorably adopt the delivery of palliative care in the ED, PRIM-ER initiates nurse education, as well as physician, physician assistant, and nurse practitioner education and simulation-based workshops. Addressing behavior change at the nurse and physician level is important for the homogenous communication of palliative care support to patients via a variety of ED providers. The provider education and workshops focus on knowledge of palliative care philosophy and skill building, which subsequently helps form favorable attitudes towards palliative care.
Subjective Norms- The education and simulation-based workshops also contribute to subjective norms by providing the skills required for providers to have goal-orientated discussions with seriously ill, end-of-life patients. Once a team-based approach is developed via skill building, subjective norms are strengthened by the regular and familiar palliative care discussions that occur among colleagues and patients in the ED. The audit and feedback intervention is also an important proponent of developing subjective norms by incorporating continuous data monitoring, feedback, and quality improvement processes. Improving quality and receiving feedback on a regular basis encourages a sense of normalcy and repetition to palliative care practices to support a positive perception of palliative care from a source external to the individual provider.
Perceived Behavioral Control- The ease with which providers conduct palliative care conversations with patients is enhanced by the education and simulation-based workshops. In addition to the more didactic skill building, EHR modifications implemented at each research site will streamline the process of palliative care referrals, alerts, and improves workflow to facilitate palliative care practices. Personal knowledge and skills are clinically enhanced in a real-world setting through the EHR modifications to bring legitimate behavioral, clinical, and philosophical change to the ED.
4. Discussion
4.1. Fidelity versus adaptability: form and function in PRIM-ER
Treatment fidelity is defined as, “the methodological strategies used to monitor and enhance the reliability and validity of behavioral interventions.” [25] Addressing treatment fidelity has numerous benefits, including supporting internal and external validity, greater confidence in treatment results and understanding of how and why an intervention works, improving statistical power, as well as potentially greater research dissemination [25,26]. To ensure the fidelity of the intervention, but permit adaptation to local context, forms and functions were clearly delineated at the time of the PRIM-ER protocol development.
When designing the PRIM-ER intervention components and key functions the research team used preliminary data, available literature in the related field, personal expertise, as well as collective experience from EM and palliative care experts. Safeguards and monitoring systems are incorporated, when applicable, to maintain study fidelity and support the key functions among the 35 sites responsible for implementing all components. For example, to maintain fidelity of the education components (Items 1 & 4, Table 1), the curriculum and materials for both the provider and nurse education components, EPEC-EM and ELNEC, were leveraged and edited by emergency physicians and nurse educators with the goal of addressing issues that are specific to the practice of EM. Since EPEC-EM and ELNEC are standardized curricula that are developed using pre-packaged slides and materials, monitoring the fidelity of their delivery is not required. However, for the provider simulation-based component, EM-Talk, fidelity will be monitored at the level of the EM Talk trainers and emergency provider participants using recommendations of the Treatment Fidelity Workgroup of the National Institutes of Health Behavior Change Consortium to ensure EM Talk's reliability, validity, and fidelity [25]. The delivery plan for EM-Talk includes detailed and standardized training of teachers and actors and review of content against an a priori performance checklist to ensure consistency in EM Talk delivery.
The forms of PRIM-ER are adaptable to increase transferability of the intervention by tailoring the application of the intervention to local settings. The forms are essential to the pragmatic nature of the PRIM-ER study, and the need for adaptability in implementation of complex public health/health services interventions continues to be supported to help reduce the divergence between research evidence and clinical application [[12], [13], [14], [15],27]. It can be challenging to determine what adaptations are permissible and how to approach developing the adaptations. Both systematic and non-systematic approaches exist, but both have pros and cons regarding documenting adaptations, insights learned regarding the adaptations, and flexibility of the adaptations to local settings [28]. Currently, no gold standard exists when developing and implementing intervention adaptations, but implementation barriers must be addressed at the level of each site [29]. Certain studies take the approach of creating a “menu,” of options for implementation activities so sites can tailor the intervention from the options available [29]. PRIM-ER also has a “menu” of implementation options and importantly uses the preliminary workgroups, as well as physician and nurse champions at each site to identify implementation barriers. This allows each site to select which intervention options (forms) will lead to greatest success at their location.
Monitoring the adaptations at each site is also important for the longevity of a successful intervention. Tracking adaptations and implementation drift can help identify successful versus unproductive intervention variations in different contexts [29,30]. PRIM-ER will prospectively implement the Framework for Reporting Adaptations and Modifications Expanded (FRAME) model developed by Stirman et al. (2019) to monitor implementation adaptations at each site to better understand the impact of the modifications on the intervention [30]. While monitoring intervention adaptations is important, its application is not a universal practice [31], and often applied retrospectively [32].
The balance between fidelity and adaptability is a challenge experienced among all complex interventions [14,27,29]. Balancing this tension is necessary in order to ensure the intervention allows the flexibility of adapting to local context while ensuring it is delivered according to the outlined protocol [33]. Within the field of implementation science, many theories, models, and frameworks exist in an effort to translate research into practice and address this balance. A systematic review led by Tabak et al. revealed an inventory of 61 models that can aid in effectively developing interventions [34]. Additionally, other models such as the Consolidated Framework for Implementation Science [35], Dynamic Adaptation Process [33], FRAME [30], Key Functions/Implementation/Context [36], and Form versus Function [8,10], to name a few, take into account the delicate balance between fidelity [37,38] and adaptation. Thus, the use of theoretical models and frameworks to guide the development of PRIM-ER, as outlined here, is not particularly novel. However, developing the pre-identified intervention form and function components within the context of the Theory of Planned Behavior, and aiming to leverage the FRAME model prospectively to document intervention modifications is a more innovative approach that will assist in deeply understanding the PRIM-ER complex intervention in greater detail.
5. Conclusion
Complex interventions present several design, implementation, and analytical challenges. Transferability of the intervention, fidelity, and adaptation must be balanced in order to scale-up, replicate, and sustain an intervention while accounting for local contextual needs. Establishing key functions and adaptable forms in the conceptual model of large complex interventions helps balance the need for fidelity versus adaptation. Behavior change of the local providers and leadership responsible for implementation is also essential and supported by the incorporation of behavioral change theory in the development of complex interventions. PRIM-ER is an example of a complex intervention that incorporates theory-based design, and applies form and function to encourage the delivery of palliative care in the ED.
Funding
This work is supported within the National Institutes of Health (NIH) Health Care Systems Research Collaboratory by the NIH Common Fund through cooperative agreement U24AT009676 from the Office of Strategic Coordination within the Office of the NIH Director and cooperative agreement UG3/UH3 AT009844 from the National Institute on Aging. Support was also provided by the NIH National Center for Complementary and Integrative Health Administrative Supplement for Complementary Health Practitioner Research Experience through cooperative agreement UH3 AT009844. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Declaration of competing interest
The authors have no competing interests to declare.
References
- 1.Smith A.K., McCarthy E., Weber E. Half of older Americans seen in emergency department in last month of life; most admitted to hospital, and many die there. Health Aff. 2012;31(6):1277–1285. doi: 10.1377/hlthaff.2011.0922. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Morganti K.G., Bauhoff S., Blanchard J.C. The evolving role of emergency departments in the United States. Rand Heal Q. 2013;3(2):3. http://www.ncbi.nlm.nih.gov/pubmed/28083290%0Ahttp://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC4945168 [PMC free article] [PubMed] [Google Scholar]
- 3.Hwang U., Shah M.N., Han J.H., Carpenter C.R., Siu A.L., Adams J.G. Transforming emergency care for older adults. Health Aff. 2013;32(12):2116–2121. doi: 10.1377/hlthaff.2013.0670. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Grudzen C.R., Richardson L.D., Major-Monfried H., Kandarian B., Ortiz J.M., Morrison R.S. Hospital administrators' views on barriers and opportunities to delivering palliative care in the emergency department. Ann. Emerg. Med. 2013;61(6):654–660. doi: 10.1016/J.ANNEMERGMED.2012.06.008. [DOI] [PubMed] [Google Scholar]
- 5.Stone S.C., Mohanty S., Grudzen C.R. Emergency medicine physicians' perspectives of providing palliative care in an emergency department. J. Palliat. Med. 2011;14(12):1333–1338. doi: 10.1089/jpm.2011.0106. [DOI] [PubMed] [Google Scholar]
- 6.Wu F.M., Newman J.M., Lasher A., Brody A.A. Effects of initiating palliative care consultation in the emergency department on inpatient length of stay. J. Palliat. Med. 2013;16(11):1362–1367. doi: 10.1089/jpm.2012.0352. [DOI] [PubMed] [Google Scholar]
- 7.Grudzen C.R., Richardson L.D., Johnson P.N. Emergency department-initiated palliative care in advanced cancer a randomized clinical trial. JAMA Oncol. 2016;2(5):591–598. doi: 10.1001/jamaoncol.2015.5252. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Perez Jolles M., Lengnick-Hall R., Mittman B.S. Correction to: core functions and forms of complex health interventions: a patient-centered medical home illustration (journal of general internal medicine, (2019), 10.1007/s11606-018-4818-7) J. Gen. Intern. Med. 2019 doi: 10.1007/s11606-019-04885-z. [DOI] [PubMed] [Google Scholar]
- 9.Craig P., Dieppe P., Macintyre S., Michie S., Nazareth I., Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. Int. J. Nurs. Stud. 2013;50(5):587–592. doi: 10.1016/j.ijnurstu.2012.09.010. [DOI] [PubMed] [Google Scholar]
- 10.Hawe P., Shiell A., Riley T. vol. 328. 2004. https://about.jstor.org/terms (Complex Interventions: How “Out of Control” Can a Randomised Controlled Trial Be?). Accessed October 28, 2019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Mullen P.D., Green L.W., Persinger G.S. Clinical trials of patient education for chronic conditions: a comparative meta-analysis of intervention types. Prev Med (Baltim). 1985;14(6):753–781. doi: 10.1016/0091-7435(85)90070-2. [DOI] [PubMed] [Google Scholar]
- 12.Huntink E., Heijmans N., Wensing M., van Lieshout J. Effectiveness of a tailored intervention to improve cardiovascular risk management in primary care: study protocol for a randomised controlled trial. Trials. 2013;14(1) doi: 10.1186/1745-6215-14-433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Gagliardi A.R. Tailoring interventions: examining the evidence and identifying gaps. J. Continuing Educ. Health Prof. 2011;31(4):276–282. doi: 10.1002/chp.20141. [DOI] [PubMed] [Google Scholar]
- 14.Escoffery C., Lebow-Skelley E., Haardoerfer R. 2018. A Systematic Review of Adaptations of Evidence-Based Public Health Interventions Globally. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Pcori Methodology Standards Standards for studies of complex interventions. https://www.pcori.org/research-results/about-our-research/research-methodology/pcori-methodology-standards#Complex Published 2019. Accessed October 28, 2019.
- 16.Grudzen C.R., Brody A.A., Chung F.R. Primary palliative care for emergency medicine (PRIM-ER): protocol for a pragmatic, cluster-randomised, stepped wedge design to test the effectiveness of primary palliative care education, training and technical support for emergency medicine. BMJ Open. 2019;9(7) doi: 10.1136/bmjopen-2019-030099. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Tan A., Durbin M., Chung F.R. Design and implementation of a clinical decision support tool for primary palliative Care for Emergency Medicine (PRIM-ER) BMC Med. Inf. Decis. Making. 2020;20(1) doi: 10.1186/s12911-020-1021-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Swanson R.A., Holton E.F. Berrett-Koehler; 1997. Human Resource Development Research Handbook : Linking Research and Practice. [Google Scholar]
- 19.Ajzen I., Madden T.J. Prediction of goal-directed behavior: attitudes, intentions, and perceived behavioral control. J. Exp. Soc. Psychol. 1986;22(5):453–474. doi: 10.1016/0022-1031(86)90045-4. [DOI] [Google Scholar]
- 20.Ajzen I. The theory of planned behavior. Organ. Behav. Hum. Decis. Process. 1991;50(2):179–211. doi: 10.1016/0749-5978(91)90020-T. [DOI] [Google Scholar]
- 21.Grol R.P.T.M., Bosch M.C., Hulscher M.E.J.L., Eccles M.P., Wensing M. Planning and studying improvement in patient care: the use of theoretical perspectives. Milbank Q. 2007;85(1):93–138. doi: 10.1111/j.1468-0009.2007.00478.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Godin G., Bélanger-Gravel A., Eccles M., Grimshaw J. Healthcare professionals' intentions and behaviours: a systematic review of studies based on social cognitive theories. Implement. Sci. 2008;3(1):36. doi: 10.1186/1748-5908-3-36. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Bradley E.H., Cramer L.D., Bogardus S.T., Kasl S.V., Johnson-Hurzeler R., Horwitz S.M. Physicians' ratings of their knowledge, attitudes, and end-of-life-care practices. Acad. Med. 2002;77(4):305–311. doi: 10.1097/00001888-200204000-00009. [DOI] [PubMed] [Google Scholar]
- 24.Rothstein A., Butler C.T., Rothstein A. On conflict and consensus: a handbook on formal consensus decisionmaking. 2009. www.ic.org Accessed March 20, 2020.
- 25.Bellg A.J., Resnick B., Minicucci D.S. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium. Health Psychol. 2004;23(5):443–451. doi: 10.1037/0278-6133.23.5.443. [DOI] [PubMed] [Google Scholar]
- 26.Carroll C., Patterson M., Wood S., Booth A., Rick J., Balain S. A conceptual framework for implementation fidelity. Implement. Sci. 2007;2(1):1–9. doi: 10.1186/1748-5908-2-40. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Chambers D.A., Norton W.E. The adaptome: advancing the science of intervention adaptation. Am. J. Prev. Med. 2016;51(4):S124–S131. doi: 10.1016/j.amepre.2016.05.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Leerlooijer J.N., Ruiter R.A.C., Reinders J., Darwisyah W., Kok G., Bartholomew L.K. The World Starts with Me: using intervention mapping for the systematic adaptation and transfer of school-based sexuality education from Uganda to Indonesia. Transl Behav Med. 2011;1(2):331–340. doi: 10.1007/s13142-011-0041-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Moore J.E., Mascarenhas A., Marquez C. Mapping barriers and intervention activities to behaviour change theory for Mobilization of Vulnerable Elders in Ontario (MOVE ON), a multi-site implementation intervention in acute care hospitals. Implement. Sci. 2014;9:160. doi: 10.1186/s13012-014-0160-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Stirman S.W., Baumann A.A., Miller C.J. The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions. Implement. Sci. 2019;14(1) doi: 10.1186/s13012-019-0898-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Von Thiele Schwarz U., Förberg U., Sundell K., Hasson H. Colliding ideals - an interview study of how intervention researchers address adherence and adaptations in replication studies. BMC Med. Res. Methodol. 2018;18(1) doi: 10.1186/s12874-018-0496-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Shelley D.R., Kyriakos C., Campo A., Li Y., Khalife D., Ostroff J. An analysis of adaptations to multi-level intervention strategies to enhance implementation of clinical practice guidelines for treating tobacco use in dental care settings. Contemp Clin Trials Commun. 2018;11(July):142–148. doi: 10.1016/j.conctc.2018.07.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Aarons G.A., Green A.E., Palinkas L.A. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implement. Sci. 2012;7(1):1–9. doi: 10.1186/1748-5908-7-32. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Tabak R.G., Khoong E.C., Chambers D.A., Brownson R.C. Bridging research and practice: models for dissemination and implementation research. Am. J. Prev. Med. 2012;43(3):337–350. doi: 10.1016/j.amepre.2012.05.024. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Damschroder L.J., Aron D.C., Keith R.E., Kirsh S.R., Alexander J.A., Lowery J.C. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement. Sci. 2009;4(1):1–15. doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Villeval M., Gaborit E., Berault F., Lang T., Kelly-Irving M. Do the key functions of an intervention designed from the same specifications vary according to context? Investigating the transferability of a public health intervention in France. Implement. Sci. 2019;14(1):1–13. doi: 10.1186/s13012-019-0880-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Dusenbury L., Brannigan R., Falco M., Hansen W.B. A review of research on fidelity of implementation: implications for drug abuse prevention in school settings. Health Educ. Res. 2003;18(2):237–256. doi: 10.1093/her/18.2.237. [DOI] [PubMed] [Google Scholar]
- 38.Pérez D., Van der Stuyft P., Zabala M.C., Castro M., Lefèvre P. A modified theoretical framework to assess implementation fidelity of adaptive public health interventions. Implement. Sci. 2016;11(1):1–11. doi: 10.1186/s13012-016-0457-8. [DOI] [PMC free article] [PubMed] [Google Scholar]