Table 2.
Implementation Outcome Assessment Among Active Implementation Studies.
| Author | Year | Evidence-Based Practice? | Implementation Outcomes of Interest | ||||
|---|---|---|---|---|---|---|---|
| Acceptability | Appropriateness | Fidelity | Implementation Cost | Sustainment | |||
| External studies | |||||||
| Bolin JN | 2013 | No | Clinics and pharmacies did not have existing staff person dedicated to monitor kiosks | Leadership lightly involved in MOU process and evaluation interviews | Standardized—everyone used same kiosk | Resource/$ limitations to pay for Wi-Fi, etc | Potential continuation for some sites |
| Braun PA | 2013 | No | Challenge to keep regular hygienists in this new position | Leadership involved in implementation and evaluation | Not mentioned | Resource/$ limitations in getting reimbursed for dental care | Ongoing for 5+ years |
| Breslau ES | 2010; 2015 | Yes | Understaffed—felt that this new initiative was extra work on top of other responsibilities | Engagement of community partners was challenging; leadership was a clear barrier for community organizations/clinics feeling invested in projects | Option to customize—each local partnership chose their own evidence-based practice | Resource/$ limitations for clinics to provide cancer care to more patients in need | Sustained in only a few cases; success stemmed from relationship building |
| Labbok MH | 2013 | Yes | Challenge in relying solely on lactation consultants to do all the work and lack of self-efficacy among other staff members | Leadership buy-in present, but not aligned with existing health system culture | Standardized 10-step breastfeeding program | Expense of baby-friendly designation as a barrier | Not mentioned |
| Novick G | 2015 | Yes | Understaffed to handle the more challenging group scheduling process | Some sites aligned with organizational culture (especially with a great staff champion) while others faced major cultural barriers and apprehension about the program | Standardized group prenatal visit program among intervention sites | Resource/$ limitations, mainly major organizational financial shortfalls | Sustained at only half of sites able to address implementation barriers |
| Waitzkin H | 2011 | Yes | Understaffed in times of staff turnover | Structural bureaucracy/leadership limits in terms of professional roles of promotoras vs other staff; Some leadership buy-in | Standard approach to promotora work with patients identified with depression | Not mentioned | Not mentioned |
| Internal studies | |||||||
| Borkowski N | 2013 | No | Not mentioned | Innovation aligned with the shared health system culture of the coalition; leaders very engaged in establishing a coalition with change management strategies | Option to customize—each primary care site determined their own redesign projects | Not mentioned (beyond context for the program) | Sustainment bolstered by monthly data reports and scheduled meetings over first 2 years of program deployment |
| Clark NM | 2014 | No | Not mentioned | Aligned with organizational culture—greatest success when diabetes programs aligned with other programmatic goals and with substantial leadership buy-in | Option to customize specific diabetes programmatic activities | Not mentioned | Striving toward sustainment with necessary policy changes |
| Dennehy P | 2011 | No | Lack of existing sufficient computer skills themselves | Aligned with health system culture—huge emphasis on the partnership/leadership model to ensure long-term success | Option to customize their EHR implementation plans | Resource/$ limitations—one institution removed IT funds shortly after program got under way | Sustained with another grant as a part of an ongoing community-based research network |
| Drainoni ML | 2016 | Yes | Strong acceptability by all staff | Strong associated appropriateness of intervention at leadership level | Standardized—original policy deployed | Not mentioned | Not mentioned |
| Garg SK | 2016 | No | Lack of existing staff IT expertise | Leadership and staff buy-in for the overall goals of the texting program; structural bureaucracy/leadership limits, particularly around HIPAA regulations | Option to customize, each clinic chose their own texting use case | Not mentioned (beyond grant funding) | Not mentioned |
| Johnson TL | 2015 | No | Appropriate staffing/tech support to build the new algorithm to identify high-utilizer patients | Innovation aligned/meeting organizational needs across multidisciplinary teams; leadership buy-in to create the program | Not mentioned | Not mentioned (beyond grant funding) | Sustained use of algorithm system-wide |
| McMullen CK* | 2013 | No | Not mentioned | Innovation aligned with health system culture; leadership buy-in, especially in the inspiration/motivational phase of the work | Option to customize the specific primary care redesign process | Provided modest financial incentives | Sustained due to “incubator” experience shared by organizational leaders |
| Ramos-Gomez FJ | 2014 | No | Not mentioned | Program worked because dental students were required to do rotations in this community-based dental program, and the community partners already had relationships with underserved patient populations to refer | Standardized trainings and use of standardized forms | Limited funds for program, but free labor because dental students provided the care as a part of training | Sustained by integrating program into covered prenatal care bundle |
| Samaan ZM* | 2016 | Yes | Appropriate staffing/tech support, especially for the QI consultant to manage the day-to-day work and paid protected time for some staff to participate | Innovation aligned with health system culture (multidisciplinary team represented all relevant stakeholders); leadership buy-in on steering committee | Customized over time and then became a standard process | Internal funds provided to launch and sustain program | Sustained by developing standardized orientation and training process |
| Smith MG | 2017 | No | Not mentioned | Appropriate local primary care providers and pharmacy leaders collaborated to build partnership | Standardized protocol requested from all pharmacies and care teams | Not mentioned | 40% had difficulty maintaining consistency of medication reviews month to month |
| Steiner BD* | 2008 | No | Early physician buy-in; case managers and physicians need more time to meet together | Innovation aligned with health system cultures, largely because of the local control of the work; leadership buy-in | Option to customize as each site focuses on their own specific QI projects | Existing payment models not sustaining work for highly complex patients, but Medicaid funding key to overall program coordination | Sustained through ongoing support by state office of rural health |
| Sugarman JR | 2014 | Yes | Understaffed, particularly with high staff and provider turnover and low staff morale in some instances | Innovation aligned with health system culture; leadership buy-in | Standardized the 8 change concepts for all clinics to work on | Barrier to getting full insurance reimbursement for a new model of care | Sustained but facing ongoing financial constraints |
| Wagner EH | 2014 | Yes | Not mentioned | Innovation aligned with health system culture; leadership buy-in as central to mission of organization | Standardized the 8 change concepts for all clinics to work on | Not mentioned | Sustained through established trust in the improvement process |
Abbreviations: EHR, electronic health record; HIPAA, Health Insurance Portability and Accountability Act; IT, information technology; MOU, memorandum of understanding; QI, quality improvement.
*
These studies provided key examples of note, especially in combining various implementation methods.