Addressing Ambulatory Safety and Malpractice: The Massachusetts PROMISES Project

More than half of malpractice claims involve care in the ambulatory setting (Zuccotti and Sato 2011; Saber Tehrani et al. 2013). As the sole project of the seven AHRQ‐funded Patient Safety and Medical Liability Demonstration Program grants to focus on outpatient malpractice safety and risk, the Massachusetts PROMISES (Proactive Reduction of Outpatient Malpractice: Improving Safety, Efficiency, and Satisfaction) project sought to identify and improve safety in what we referred to as “3 + 1” areas of known risk in the outpatient setting: test result, referral, and medication management plus overarching communication issues (Agency for Healthcare Research and Quality 2014). We assembled a coalition of key Massachusetts safety, regulatory, malpractice, and academic groups and recruited 25 small‐ and medium‐sized primary care practices of which 16 were randomly selected to receive a multifaceted improvement intervention. While the quantitative outcomes and evaluation fell short of demonstrating some of the significant improvements we had hoped to show, here we describe how we developed and fielded our intervention, describing some of the lessons learned in the course of this project and implications for future efforts in this field.

We chose to address the “3 + 1” outpatient safety risk areas based on their demonstrated role as both areas of vulnerability as well as potential for improvement. First, failed follow‐up of abnormal tests has consistently been found to be a malpractice safety risk (Schiff 2006; Callen et al. 2012; McDonald et al. 2013). Second, based on earlier work, and data collected during the PROMISES project, problems in referral management also clearly represent problematic processes with “dropped balls” that pose malpractice risks. Baseline surveys of the practices consistently revealed that this was an area where staff and administrators perceived risk, and this finding was in turn used to focus efforts on referral coordination (Gandhi and Lee 2010; Singer et al. 2015ba; Tuot et al. 2015). Third, while medication errors appear relatively infrequently in outpatient malpractice claims, widespread evidence of outpatient medication quality and safety problems suggested that attention to medication processes was also warranted (Thomsen et al. 2007; Taché, Sönnichsen, and Ashcroft 2011). Crosscutting each of these three risk areas is a problem in outpatient communication—both provider to provider and provider to patient. We were particularly interested in communication issues during clinical encounters and around sensitive communication that needs to occur when things go wrong (CRICO Strategies 2016).

To improve processes and practices in the “3 + 1” domains, the PROMISES project team helped intervention practices in a variety of ways that featured educational sessions (both face‐to‐face collaborative learning sessions and online webinars) and materials, and facilitation of Plan‐Do‐Study‐Act (PDSA) cycles by two state‐employed improvement advisors (IAs). Each practice assembled a team for the project, which typically included a lead clinician, practice manager, and representative nursing and/or clerical staff person. The IAs worked with the practices, making regular visits (~1–2/month) and phone calls. At these visits, IAs shared tools and ideas and helped troubleshoot problems, brought problems back to project improvement specialists (from the Institute for Healthcare Improvement, Massachusetts Coalition for the Prevention of Medical Errors, and Brigham and Women's Hospital Safety co‐investigators) for review and consultation, and provided accountability for ongoing progress with an emphasis on the importance of measurement data and regular and small PDSA tests.

PROMISES investigators anticipated that practices would have very limited time for dedicated safety improvement efforts and hence designed program activities to respect this limitation by tailoring components of the intervention to accommodate busy schedules and competing external demands. Accordingly, we customized the scheduling and content of IA visits and held collaborative learning sessions at more convenient evening hours and locations.

We also recognized the importance of helping practices leverage PROMISES activities to simultaneously fulfill requirements such as Patient Centered Medical Home certification, insurers’ quality reporting, or other organizational quality improvement imperatives, such that—to the greatest extent possible—we were easing these burdens rather than adding additional work, by aligning our requests for data with existing reporting requirements. Nonetheless, many of our 16 practices found it challenging to maintain a steady pace of improvement during the 15‐month intervention period and to achieve measurable change in outcomes compared to control practices.

Challenges faced by the practices centered largely around finding time to work on improvement activities, staffing turnover, and challenges in accessing data needed for improvement work and project metrics. Practice leaders and managers were often too busy putting out fires (e.g., related to unexpected problems or staffing shortages, or seasonal volume fluctuations) and simply getting through the day's work to have extra time to work on improvement—a phenomenon others have also observed as critical to quality and safety improvement that has been labeled as a shortage of “adaptive reserve” (Nutting et al. 2009; Singer et al. 2015b). Several practices experienced nearly 100% turnover in staff during their 2‐year relationship with the project.

To measure and evaluate test result and referral follow‐up safety, the PROMISES evaluation team researchers required intervention practices to identify patients with selected abnormal laboratory results or referrals so that charts could be reviewed to measure and evaluate how reliably these test result and referral follow‐up loops were closed. However, most practices were unable to leverage their electronic medical record (EMR) systems for what should have been easily retrievable lists of patients. As a result, the research team had to develop a workaround, going instead to the practices’ laboratory vendors to obtain lists of abnormal results meeting review criteria. However, this additional work for researchers was less concerning than the practices' inability to leverage their electronic systems for their own learning and improvement.

The project's research design and evaluation team also faced significant unanticipated challenges. The first impacted practice recruitment. Our initial intent was to involve the two participating major malpractice insurers (who together covered >85% of the state of MA clinicians) at multiple stages in this research project, particularly in identifying and helping to recruit 16 practices apiece (32 total), for equally sized control and intervention study arms. However, one of the consumer members of the Massachusetts state Institutional Review Board surprisingly and strenuously objected to the insurers playing any role in recruitment (and other aspects of the project that related to working directly with their insured practices that were the insurers’ customers), contending this would be “coercive.” Thus, the insurer's role in the project was more circumscribed than anticipated, making recruitment much more difficult.

Nonetheless, the two insurers did contribute in important ways to the PROMISES Project, particularly in sharing their office practice assessment tools, which we adapted for our evaluation metrics and chart review, as well as sharing closed claim malpractice data. Based on this pooled closed claim data, we uncovered the striking finding that, in primary care, diagnostic errors outnumbered other types by a factor of 6:1 and that these diagnostic error claims were disproportionately “settled”—36.8% of the primary care diagnosis cases versus 21.4% of all other non–general medicine cases (p < .001)—suggesting that many diagnostic errors were less defensible and represented bona fide improvement opportunities (Schiff et al. 2013).

In addition to the aforementioned challenge identifying and reviewing charts with abnormal test results or referrals, the evaluation team faced the challenge of adapting existing validated survey tools (related to outpatient safety, safety communication, culture, and trust) to serve as sensitive instruments to detect potential changes resulting from an intervention that was limited to a 15‐month time frame (Crabtree et al. 2011; PROMISES Project. Staff, Administrators, Patient Survey Instruments).

Based on the learning from these challenges in addressing outpatient malpractice safety liability risk, the project developed a series of webinars and education materials, now freely available on the Internet (PROMISES Project. Educational Materials and Online Webinars). One of these products, a brochure and accompanying video featuring Dr. Lucian Leape, a founder of the patient safety movement, provides rationale and guidance for outpatient apology and disclosure of medical errors. We adapted the historic Harvard hospital 40‐page “When Things Go Wrong: Responding to Adverse Events” consensus statement on inpatient apology and disclosure into a four‐page practical outpatient‐oriented guidance tool (Massachusetts Coalition for the Prevention of Medical Errors 2006; Schiff et al. 2014; PROMISES Project. When Things Go Wrong in the Ambulatory Setting). Other materials and videos in the 14‐module Patient Safety Curriculum developed by the PROMISES team, with input from quality and safety consultants from the Institute for Healthcare Improvement, include background and case study modules such as Getting Started, Improving Communication, Process Improvement for Test Results and Referrals, and Sustaining Change. We have also posted tools that some of the practices developed such as sample letters to encourage patients to reliably follow up with new fecal occult blood screening processes they developed and one‐page Agenda Setting leaflets to engage patients and clinicians in improving communication during clinical encounters (PROMISES Project. Educational Materials and Online Webinars).

In conclusion, the PROMISES project targeted an important but undeveloped area of malpractice risk and safety, one now considered a priority area of research interest by AHRQ (Agency for Healthcare Research and Quality 2013). Achieving higher reliability for management of test results, referrals, and high‐risk medications will require collaborative efforts to both support practices to overcome current challenges in highly stressed and fragmented environments that are barriers to better communication, as well as more research studying what interventions are the most effective, practical, and affordable solutions. The project was predicated on the belief that engaging practice teams and strengthening a culture of process‐minded continuous improvement are an important foundation to delivering our promise to patient‐centered, safe, and malpractice‐free outpatient care (Kerrissey et al. 2016). Future research will need to test this hypothesis more definitively. We believe that lessons derived from the PROMISES intervention can help guide this work.

Supporting information

Appendix SA1: Author Matrix.