Routine provision of evidence-based care in clinical practice is an elusive goal. In 2003, adults in the United States were receiving only about half of the care recommendations for 30 common conditions associated with substantial morbidity and mortality.1 Similar findings were reported for 175 indicators of quality of care for children across the United States2 Quality of care for US adults changed little between 2003 and 2016.3
With these sobering statistics as a backdrop, the Agency for Healthcare Research and Quality launched its EvidenceNOW initiative,4 creating and funding seven regional cooperatives to identify and implement effective strategies for improving quality of cardiovascular health care in community-based primary care practices. These cooperatives are testing implementation strategies tailored to hundreds of small- to medium-sized practices. Implementation strategies are “methods or techniques used to enhance the adoption, implementation, and sustainability of a clinical program or practice”5 (eg, practice facilitation, educational outreach visits, shared learning collaboratives, and data feedback and benchmarking).6
Healthy Hearts Northwest (H2N) is an EvidenceNOW cooperative that is a partnership between the MacColl Center for Health Care Innovation at the Kaiser Permanente Washington Health Research Institute, Qualis Health in Washington and Idaho, the Oregon Rural Practice-based Research Network (ORPRN) based at the Oregon Health Sciences University, and the University of Washington’s Institute of Translational Health Sciences. Qualis Health and ORPRN, both of which have worked with clinical practices on numerous practice improvement initiatives and research projects, recruited 259 practices across Washington, Oregon, and Idaho that had 10 full-time or fewer providers and met stage 1 electronic health record meaningful use criteria. Practices in H2N were as far as 829 miles from a research team hub.
The H2N study is a four-arm pragmatic trial, with practices in all arms receiving external practice facilitation that supports quality improvement strategies to increase adoption of the four ABCS of heart disease prevention: Aspirin use by high-risk individuals, Blood pressure control, Cholesterol management, and Smoking cessation.7 Practice facilitators hired by the H2N study provided 15 months of practice improvement support, making at least five face-to-face visits to each practice and monthly phone calls between visits. Each practice facilitator served geographic clusters of 10 to 20 practices. These external practice facilitators were trained in the Model for Improvement approach to quality improvement.8
The H2N study used a two-by-two factorial design to compare the effectiveness of adding educational outreach and shared learning opportunities to practice facilitation (PF) on practice quality improvement capacity and the ABCS clinical quality measures. The four factors (intervention arms) are (1) PF alone, (2) PF and shared learning, (3) PF and educational outreach, and (4) PF combined with both shared learning and educational outreach. Half of the H2N practices were randomly selected to receive the educational outreach intervention; half were randomized to a shared learning opportunity of a site visit to a practice using exemplar quality improvement strategies. The educational outreach intervention focused specifically on improving the “cholesterol management” measure by promoting routine use of a 10-year cardiovascular disease (CVD) risk calculator in a new population of adults—those 40 to 75 years without previous atherosclerotic CVD, diabetes, or a low-density lipoprotein cholesterol over 190, as well as the use of statin medications for primary prevention among individuals with appropriate 10-year risk levels. This intervention was chosen based on evidence-based recommendations from the American College of Cardiology, the American Heart Association, and the US Preventive Services Task Force.9,10
This article focuses on the development of one component of the H2N study’s intervention—the virtual educational out-reach program, which was adapted from the principles, structure, and elements of academic detailing (AD). AD is an effective method of modifying professional behavior in which a trained outside expert delivers evidence-based educational messages in person to a clinician. A Cochrane review of 28 studies found that AD produced a median adjusted risk difference of 5.6% in improving professional performance in a health care setting,11 yet it has practical limitations, especially at scale across a large geography. In developing the H2N educational outreach program, the H2N research team recognized that adaptations to the structure and methods of traditional AD12 would be required to address constraints presented by the H2N study design. For example, geographic spread of practices across three states would make traditional in-person educational outreach visits impractical and costly. Similarly, a focus on delivering the intervention to the clinical care team, rather than the lead clinician only, required a change from the usual one-on-one clinician visits for the H2N intervention. This article describes the development of a virtual educational out-reach program for use across a broad geographic region at scale for the H2N study.
DEVELOPMENT OF THE EDUCATIONAL OUTREACH PROGRAM
Our research team first reviewed the AD literature to identify and better understand the underlying theory, evidence base, principles, structure, and elements of traditional AD programs.12–15 In addition, we engaged leaders from the National Resource Center for Academic Detailing (NaRCAD)13 in discussions about our proposed educational outreach plans. AD applies social marketing theory alongside the best clinical evidence to change health professional behavior.16 Traditional AD programs take the effective communication/behavior change/marketing approach used by pharmaceutical industry sales representatives (“detailers”) and place it in service of providing clinicians with unbiased evidence-based information to optimize clinical decision-making. Traditional AD involves in-person, one-on-one visits with lead clinicians in a medical practice, in which an extensively trained detailer develops a sustained relationship and has a series of highly interactive discussions with individual clinicians. Through this close interaction, the academic detailer elicits a clinician’s needs and knowledge, focuses on the key change messages most relevant to the clinician’s practice, identifies and addresses specific barriers to change, and secures commitment from the clinician to follow through on those changes.12
We engaged stakeholders in designing an adapted educational outreach intervention consistent with traditional AD principles to promote use of a 10-year CVD risk calculator, and use of statins for primary prevention among individuals with appropriate 10-year risk levels. Multiple CVD risk calculators are available,17–20 and our intervention intentionally chose not to recommend use of a specific calculator so as to accommodate the preferences of different clinicians. Our stakeholders included an “Affinity Group” with five clinicians from primary care practices in Washington and Idaho, two of whom were from practices enrolled in the H2N study, five outreach educators, and a content expert in CVD risk reduction. We used a key driver diagram as a design tool to develop a roadmap for our intervention overall. Key driver diagramming is a widely used approach to inform the design of interventions used in quality improvement initiatives.8,21,22 This framework (Fig. 1) delineated the aim of the intervention, what was needed to accomplish this aim (the primary drivers), and the strategies for effecting change (the interventions), as well as a series of principles and underlying assumptions about the intervention based on our understanding of clinical practice, our experience with influencing practice change, our review of the literature, and conversations with NaRCAD representatives (Table 1).
TABLE 1.
Principles and Assumptions | Influence on Intervention Development |
---|---|
Understanding the barriers and challenges to use of the CVD risk calculator was important to tailor key messages and supporting materials for H2N practices | Engagement of an H2N “Affinity Group” of clinician stakeholders comprising primary care physicians in the H2N geographic area |
Guidelines are best communicated by practicing clinician peers to the physicians in the H2N practices | Recruitment and training of primary care physicians to conduct the educational outreach visits |
It is possible to create a virtual educational outreach intervention that can uphold key features of traditional AD | Engagement of NaRCAD leaders to review our intervention and its features, training, and materials for its consistency with traditional AD features |
H2N, Healthy Hearts Northwest.
The research team, which included investigators with expertise in implementation science, quality improvement, and practice redesign, designed the educational outreach program with the Affinity Group, the outreach educators, and the content expert through monthly webinars over an 8-month period. We received feedback on the roadmap as well as the underlying assumptions and principles for the intervention. We reviewed the structure and elements of traditional AD and asked for feedback on our recommended adaptations. We worked to uphold as many elements of traditional AD as possible in building H2N’s educational outreach program. Consistent with traditional AD programs, the research team, with consultation from a pharmacist experienced in AD, developed an extensive training program with role play opportunities to prepare the primary care physicians who would serve as out-reach educators (from hereon called educators) for the visits. We chose primary care physicians as educators given their expertise in clinical medicine and in the use of CVD risk calculators in their own practices. Four of the five educators were members of the H2N study team. We also created and adapted a range of resource materials for the educators, including a script that led the educators through traditional elements of an AD visit—the introduction, needs assessment, key messages, barriers and obstacles to adopting clinical change, and securing commitments to change. To develop these resource materials, the design group met and reviewed identified barriers and facilitators, and worked iteratively with the stakeholder group of clinicians to develop approaches to address the barriers and leverage the facilitators.
THE ADAPTED EDUCATIONAL OUTREACH PROGRAM
The H2N educational outreach program adapted both the structure and elements of a traditional AD program. Table 2 summarizes the structural adaptations. The wide geographic spread of H2N practices in urban and rural locations across several states and the limited number of educators concentrated in two cities made in-person, one-on-one visits time and cost prohibitive, leading to the H2N study’s virtual webinar-based or telephonic educational outreach visits. The H2N study’s level of resources did not support ongoing visits with a peer physician educator. The H2N study’s practice facilitators, who met with their assigned practices’ quality improvement teams at least monthly, could follow up after the educational outreach visit to support changes that the practice chose to promote routine and appropriate use of the CVD risk calculator. Although the research team did not initially plan to involve practice facilitators, the team quickly recognized their importance and included them in recruitment of practices to the outreach education visits and in visit follow-up. Finally, because the process of CVD risk calculation can include different care team members, the research team identified the clinical care team rather than individual clinicians as the target for the intervention and shifted from a one-on-one to a clinical team visit. Each practice identified its own clinical care team, which included various participants such as physicians, nurse practitioners, pharmacists, medical assistants, nurses, care coordinators, and/or clinic administrators. If a physician was unavailable to lead a practice’s participation in this intervention, the practice designated an individual most likely to drive this change in that setting.
TABLE 2.
Traditional AD Program Structure | H2N Constraints/Enablers | Adapted H2N Virtual Outreach Program |
---|---|---|
In-person visits | Broad geographic spread | Make virtual 30-min visits, facilitated by webinar technology (or phone if webinar not possible) |
One-to-one visits | Limited educator resources (a small number of educators with limited time for visits); scheduling with multiple individual clinicians across a practice burdensome to practice and research team | Fit “visits” into meetings and other time slots that multiple clinicians across a practice could attend Developed a short educational video on use of a CVD risk calculator that could be watched by the clinical team asynchronously |
Lead clinician-only audience | CVD risk reduction is well-suited to clinical team-based action | Invited medical staff, quality coordinators, care coordinators, and clinicians, as appropriate for the practice’s context. Asked for a clinician to attend the “visit” |
Extensively trained detailer | Multiple primary care clinicians on the H2N research team enthusiastic to serve as educators | Developed “detailing” training program and resources (NaRCAD-vetted) for primary care clinician educators. Key messages developed in collaboration with practicing clinician advisory group and content expert in CVD risk reduction |
Multiple visits | Limited educator time with practice; practice facilitators who conduct monthly visits or calls | Sent summary of call to practice and copied practice facilitator. Practice facilitators followed up on practice change ideas. Held optional follow-up “office hours” phone calls for all clinical care teams to share successes and barriers to increasing use of CVD risk calculator |
H2N, Healthy Hearts Northwest.
Figure 2 shows adaptations to the elements of a traditional AD visit.13 A traditional academic detailer provides an introduction personalized to the clinician, assesses the clinician’s needs and priorities on the topic, delivers key change messages while eliciting and addressing barriers or objections, then summarizes the discussion and closes the visit with specific practice change recommendations based on the conversation. The H2N virtual webinar-/phone-based format and the fact that the visits included one educator with multiple representatives from the practice posed challenges to providing these visit elements (eg, developing an interactive discussion of barriers, needs, and obstacles to using the CVD risk calculator). Accordingly, adaptations were developed for each visit element (Fig. 2).
Introduction
The introduction to the virtual educational visit was facilitated because practices were pre-enrolled in the H2N program and expected to be offered learning opportunities to reduce patients’ CVD risk. The H2N study’s practice facilitators had developed relationships with multiple providers and staff in each practice and were in a position to introduce the virtual outreach program to the practice and attend the visit if available.
Needs Assessment
To abbreviate the individual needs assessment during the 30-minute virtual outreach visit, we used baseline clinician and practice surveys that the H2N study had gathered for its overall evaluation to help identify practice context as well as provider attitudes and experiences with CVD risk calculation that might elucidate potential barriers. This included practice characteristics such as the mean number of patient visits per day by a full-time clinician, level of burnout among clinicians and staff members, and current level of reported use of a CVD risk calculator across the practice. In addition, the educators took roll call at the beginning of the virtual visit to know which team members and roles were participating. With this information, they could elicit views from different participants and tailor the messaging based on who was attending.
Key Messages, Features, and Benefits
Because the H2N study had limited educator resources (a small number of educators with limited time for visits), and the practices also had limited time for visits, we created a 7-minute educational video demonstrating the use of a CVD risk calculator by a physician and by a member of the clinical care team to guide a patient–clinician conversation about statins for primary prevention. This video “predelivered” key message components of the H2N study’s virtual outreach visit. The clinical team was asked to watch the video before the outreach visit; it also could be watched by clinicians and team members who were unable to attend the visit. The H2N team created a script and a “Detail Aid,” vetted by NaRCAD, with three key change messages for the educators to cover during the visit.
Handling Objections and Barriers
During the outreach program design phase, the research team solicited Affinity Group input to anticipate objections and barriers and to develop possible solutions to address them. The objections, barriers, and solutions were compiled into a reference document and provided as training materials to the educators. Outreach visits illuminated further objections and potential solutions, and the research team distributed additional support materials to educators throughout the 6-month period during which educators were conducting virtual visits. Time constraints did not always allow for eliciting and addressing barriers from clinicians and staff one at a time; having these support materials allowed the educators to efficiently address barriers and objections raised during the visit. Examples of barriers and their solutions include (1) not all clinicians in a group practice will agree to adoption of the CVD calculator and guideline-recommended use of statins; solution: promote early adopters of CVD risk calculation who can discuss their use of the calculator across the practice; (2) patient concerns about statin risks; solution: provision of publicly available patient facing handouts that discuss myths about statins; and (3) lack of trust in the guidelines; solution: provided published articles that addressed clinician concerns.
Summary
In a traditional AD visit, the summary is personalized to an individual clinician and the academic detailer carefully assesses both verbal and nonverbal cues to ensure the messages have been received. The H2N study’s outreach visit summary was delivered telephonically and drew largely from preplanned key messages. Educators provided an e-mail summary of the visit to practice member attendees with their proposed actions and copied the summary to the practice facilitator for further discussion at his or her next contact with the practice.
Close
An in-person “closing” with commitment to action was not possible. Limited educator resources also meant that neither multiple visits over time to develop an ongoing relationship with the practice nor assessment by the academic detailer of whether commitments were kept were possible in the H2N intervention. Instead, the H2N intervention relied on the practice facilitator relationship with the sites to follow-up on commitments made and actions proposed during the visit. The practice facilitators were able to integrate information from the e-mail summary of the visit when they next met with the practice to facilitate their quality initiatives. The degree to which the specific commitments from the visit were further discussed and implemented was at the discretion of the practice and the facilitator.
The H2N study team successfully conducted the adapted educational outreach visits in 44 clinics spread across three states. Consistent with H2N and EvidenceNOW goals, all practices were small- to medium-sized, with 41% having 6 or more providers. The majority (84%) of the practices specialized in family medicine; 39% were in rural locations. Just over 40% of practices were affiliated with health systems or hospital systems, 39% were independent practices, and 18% were federally qualified health centers or Indian Health/Tribal Health centers.
DISCUSSION
The H2N study team developed a tailored virtual educational outreach program using principles of traditional AD and deployed it with small- and medium-sized rural and urban primary care practices across three states, seeking to increase their use of CVD risk calculation and prescription of statins for primary prevention, when appropriate. The H2N study team made adaptations to the structure of these visits, shifting from in-person to virtual visits, one-to-one visits to one-to-many visits, and from multiple visits with a “detailer” to a single visit with an educator supplemented by follow-up with an established practice facilitator.
When moving an effective methodology such as AD from a formal program to individual settings as in the H2N study, it is critical to maintain the general principles of the method while carefully monitoring the interaction between the methodology and the variation across and limitations of reaching a large number of sites across a large geography. AD success relies on securing commitment to change during the clinician encounter, building on elements of a personal conversation. In attempting to replicate this success during a virtual visit, we mapped traditional AD visit elements to the virtual components delivered for H2N. For example, to build a relationship with the practices, we relied on the pre-existing connection between the practice facilitator and the clinicians and staff at the H2N sites. The practice facilitator could introduce the educational out-reach opportunity to the practice and follow-up on the commitments and actions proposed during the visit at a subsequent contact with the practice. We also relied on delivering some content asynchronously with the virtual outreach visit. We provided a short video covering key message information to be viewed ahead of the visit and sent follow-up summaries to both the practice facilitator and clinical care team members.
We have found only one example of this type of outreach program in the literature. Hartung et al.23 developed a distance-learning AD model for use with rural primary care practices in Oregon and compared it with more traditional in-person AD. Two practices received the video conference–based and two in-person detailing strategies in four visits over 8 months. Hartung et al’s pilot project found high rates of provider satisfaction, likelihood to change practice, and likelihood to continue in the program. However, a lower proportion of clinicians who received video conference-based detailing reported that they were highly likely to change practices compared with those who received in-person detailing. This raises questions about the effectiveness of distance-based AD, a question that the H2N study will answer once data collection and analysis of ABCS outcomes by study arm are complete.
There is growing interest in advancing our understanding of how best to ensure that evidence from research is implemented in a sustainable fashion across large populations and in diverse health care settings.24 Early efforts in this area include identifying, developing, and tailoring implementation strategies and testing their effectiveness in real-world settings.25 Powell et al,6 in their refined taxonomy of implementation strategies, identified educational outreach visits as one such strategy. Although described as a “discrete implementation strategy,” others have acknowledged that these strategies must be tailored to the situation, linking specific strategies to the known features of the innovation (here, the CVD risk calculator), the settings and context, and the target of behavior change.26 This article describes how the H2N research team tailored the educational outreach implementation strategy, using principles of AD, to both the intended educational content and the H2N study’s dispersed, small- to medium-sized primary care practice settings. In addition, the H2N research team developed resources and training to ensure that the outreach educators maintained fidelity to the program elements and structure because they were carefully adapted to maintain the core principles of AD.
The adapted virtual outreach visit had an aggressive timeline from design to implementation due to the short, 3-year H2N study time frame. This limited the research team’s ability to iterate the virtual outreach program’s design and materials as we gained experience with the visits; eg, the key messages, barriers, and obstacles became clearer after visits began. Obtaining patient stakeholder input into the educational intervention might have enhanced the design of the educational visit; however, our advisory group of clinicians ensured representation from the intended target audience for the intervention. A longer development timeline would also have allowed the research team to tailor the virtual outreach visits to different constellations of clinic participants (eg, if staff attended without clinicians) or its baseline characteristics (eg, if the practice was already using the CVD risk calculator). Finally, we did not formally assess the quality and consistency of the visits themselves, although we provided the physician educators with scripts and a field notes template to ensure that all visit elements were completed in the limited 30-minute time frame and held an early debriefing session with the educators to discuss consistency across the visits.
The H2N research team has developed a virtual educational outreach program for geographically dispersed practices by making significant adaptations to the structure of traditional AD, while developing intervention strategies that maintain key AD elements such as assessing needs, succinct presentation of key messages by a well-informed detailer, engaged discussion, and follow-up to support planned changes. The research team successfully conducted these virtual visits across a diverse set of 44 practices in three states. Given the design constraint of a single short, virtual visit, bundling the visit with follow-up from an established practice facilitator to ensure that action is taken may be a critical program element. Our adapted virtual outreach program has general application to programs working to implement interventions into geographically dispersed practices and can help overcome the limitations posed by more traditional resource-intensive AD programs. Further study of this educational outreach implementation strategy is needed to evaluate the efficacy and cost-effectiveness of this virtual approach compared with traditional AD.
Lessons for Practice.
Avirtual educational outreach program can be deployed across diverse, geographically dispersed primary care practices by making significant adaptations to the structure of traditional AD, while maintaining key AD visit elements.
It is important to ensure that the core principles of the AD method are maintained because this method is tailored to the needs of diverse clinical practices.
ACKNOWLEDGMENTS
The authors thank Drs. Roger Chou, Christopher Dodd, Adriana Linares, Andrew Roquiz, Seth Cowan, and Anneliese Floyd for their involvement in developing the content for the virtual educational outreach program. They appreciate Dr. James Ralston’s contributions to conducting several virtual outreach visits. They thank other members of the Healthy Hearts Northwest team who assisted in organizing and evaluating the virtual educational outreach program—Elena Kuo, Nicole Van Borkulo, Carly Levitz, Consuelo Norris, and Ben Balderson; NaRCAD staff, Bevin Shagoury and Arielle Mather, who assisted with consultation on the virtual outreach program structure and materials, and helped develop Figure 2 for the manuscript; and Ann Wheeler, who assisted in training the outreach educators. Finally, they appreciate the work of the Kaiser Permanente Washington Health Research Institute staff who produced the virtual outreach educational video—Melissa Parson, Jessica Ridpath, Rebecca Hughes, John Ewing, and Heidi Bornstein.
Disclosures: Fischer serves as a clinical consultant for Alosa Health, an educational nonprofit that provides academic detailing services in several states. The remaining authors declare no conflict of interest. Supported by Grant number R18HS023908 from the Agency for Healthcare Research and Quality. The National Resource Center for Academic Detailing is supported by Grant number R18HS023236 from the Agency for Healthcare Research and Quality. Healthy Hearts Northwest is affiliated with the Institute of Translational Health Sciences, which is supported by the National Center For Advancing Translational Sciences of the National Institutes of Health under Award Number UL1 TR002319. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality or the National Institutes of Health.
Contributor Information
Laura-Mae Baldwin, Department of Family Medicine, Institute of Translational Health Sciences, University of Washington, Seattle, WA, and Director, Community Engagement, Institute of Translational Health Sciences, University of Washington, Seattle, WA..
Michael A. Fischer, Division of Pharmacoepidemiology and Pharmacoeconomics, National Resource Center for Academic Detailing, Brigham and Women’s Hospital, Boston, MA, and Associate Professor of Medicine, Harvard Medical School, Boston, MA..
Jennifer Powell, Powell & Associates, LLC, Asheville, NC..
Erika Holden, MacColl Center for Health Care Innovation, Kaiser Permanente Washington Health Research Institute, Seattle, WA..
Leah Tuzzio, Kaiser Permanente Washington Health Research Institute, Seattle, WA..
Lyle J. Fagnan, Department of Family Medicine, Oregon Health & Science University, Portland, OR..
Jeff Hummel, Qualis Health, Seattle, WA..
Michael L. Parchman, MacColl Center for Health Care Innovation, Kaiser Permanente Washington Health Research Institute, Seattle, WA..
REFERENCES
- 1.McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–2645. [DOI] [PubMed] [Google Scholar]
- 2.Mangione-Smith R, DeCristofaro AH, Setodji CM, et al. The quality of ambulatory care delivered to children in the United States. N Engl J Med. 2007;357:1515–1523. [DOI] [PubMed] [Google Scholar]
- 3.Levine DM, Linder JA, Landon BE. The quality of outpatient care delivered to adults in the United States, 2002–2013. JAMA Intern Med. 2016;176:1778–1790. [DOI] [PubMed] [Google Scholar]
- 4.Agency for Healthcare Research and Quality. EvidenceNOW: advancing heart health in primary care. 2016. Available at: https://www.ahrq.gov/evidencenow/about/index.html. Accessed May 12, 2017.
- 5.Proctor EK, Powell BJ, McMillen JC. Implementation strategies: recommendations for specifying and reporting. Implement Sci. 2013;8: 139. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the expert recommendations for implementing change (ERIC) project. Implement Sci. 2015;10:21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Parchman ML, Fagnan LJ, Dorr DA, et al. Study protocol for “Healthy Hearts Northwest”: a 2 × 2 randomized factorial trial to build quality improvement capacity in primary care. Implement Sci. 2016;11:138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Langley GJ, Moen RD, Nolan KM, et al. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed San Francisco, CA: Jossey-Bass; 2009. [Google Scholar]
- 9.Stone NJ, Robinson JG, Lichtenstein AH, et al. 2013 ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol. 2014;63(25 pt B):2889–2934. [DOI] [PubMed] [Google Scholar]
- 10.Bibbins-Domingo K, Grossman DC, Curry SJ, et al. Statin use for the primary prevention of cardiovascular disease in adults: US Preventive Services Task Force recommendation statement. JAMA. 2016;316:1997–2007. [DOI] [PubMed] [Google Scholar]
- 11.O’Brien MA, Rogers S, Jamtvedt G, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane database Syst Rev. 2007:CD000409. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Soumerai SB, Avorn J. Principles of educational outreach (“academic detailing”) to improve clinical decision making. JAMA. 1990;263:549–556. [PubMed] [Google Scholar]
- 13.National Resource Center for Academic Detailing. NaRCAD. 2017. Available at: https://www.narcad.org/. Accessed May 12, 2017.
- 14.Avorn J, Soumerai SB. Improving drug-therapy decisions through educational outreach. A randomized controlled trial of academically based “detailing”. N Engl J Med. 1983;308:1457–1463. [DOI] [PubMed] [Google Scholar]
- 15.Avorn J, Soumerai SB, Everitt DE, et al. A randomized trial of a program to reduce the use of psychoactive drugs in nursing homes. N Engl J Med. 1992;327:168–173. [DOI] [PubMed] [Google Scholar]
- 16.Evans WD, McCormack L. Applying social marketing in health care: communicating evidence to change consumer behavior. Med Decis Making. 2008;28:781–792. [DOI] [PubMed] [Google Scholar]
- 17.Goff DC Jr, Lloyd-Jones DM, Bennett G, et al. 2013 ACC/AHA guideline on the assessment of cardiovascular risk: a report of the American College of Cardiology/American Heart Association Task Force on practice guidelines. Circulation. 2014;129(25 Suppl 2):S49–S73. [DOI] [PubMed] [Google Scholar]
- 18.D’Agostino RB Sr, Vasan RS, Pencina MJ, et al. General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008;117:743–753. [DOI] [PubMed] [Google Scholar]
- 19.Ridker PM, Buring JE, Rifai N, et al. Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA. 2007;297:611–619. [DOI] [PubMed] [Google Scholar]
- 20.Ridker PM, Paynter NP, Rifai N, et al. C-reactive protein and parental history improve global cardiovascular risk prediction: the Reynolds Risk Score for men. Circulation. 2008;118:2243–2251. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Bailie J, Laycock A, Matthews V, et al. System-level action required for wide-scale improvement in quality of primary health care: synthesis of feedback from an interactive process to promote dissemination and use of aggregated quality of care data. Front Public Health. 2016;4:86. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Perla RJ, Bradbury E, Gunther-Murphy C. Large-scale improvement initiatives in healthcare: a scan of the literature. J Healthc Qual. 2013;35:30–40. [DOI] [PubMed] [Google Scholar]
- 23.Hartung DM, Hamer A, Middleton L, et al. A pilot study evaluating alternative approaches of academic detailing in rural family practice clinics. BMC Fam Pract. 2012;13:129. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Chambers DA, Glasgow RE, Stange KC. The dynamic sustainability framework: addressing the paradox of sustainment amid ongoing change. Implement Sci. 2013;8:117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Glasgow RE, Chambers D. Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clin Transl Sci. 2012;5: 48–55. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Grol R, Wensing M. Effective implementation: a model In: Grol R, Wensing M, Eccles M, eds. The Implementation of Change in Clinical Practice. Edinburgh, United Kingdom: Elsevier; 2005:41–57. [Google Scholar]