Abstract
The potential for health information technology (IT) to improve health care delivery has been appreciated for decades, but “digitizing” health care can also introduce new risks and even harm. As the use of health IT has grown, these risks have become more apparent. The authors of this report evaluated the efforts of 11 hospitals and ambulatory practices to use an improvement strategy and tools developed to promote safe use of health IT and to diagnose, monitor, and mitigate health IT–related safety risks. Through interviews, the authors discovered that some health care organizations (especially hospitals) with expertise in process improvement were able to identify and begin to mitigate health IT–related safety risks, but in most others, awareness of these risks was limited (especially in ambulatory practices). The authors concluded that better tools like the recently released Safety Assurance Factors for EHR Resilience (SAFER) Guides are needed to help organizations optimize the safe use of health IT. However, health care organizations will require a better understanding of the safety risks posed by electronic health record (EHR) use to take full advantage of the SAFER Guides. There may also be a need for additional tools and metrics (and further usability studies of existing tools and metrics) to better support the needs of health care organizations as they increasingly rely on health IT to improve the quality and safety of patient care.
Health information technology (IT) safety has several dimensions: using health IT to make care safer, ensuring that health IT is itself safe, and ensuring that health IT is used safely. The potential for health IT to improve the safety of health care delivery has been appreciated for decades, but the role of health IT in introducing safety risks has been recognized only more recently. As the use of health IT has grown, users have begun also to observe its fallibility. Hardware and software can malfunction. Data can be lost or corrupted during transmission. Deploying complex technologies in a complex organizational environment can introduce new hazards and safety risks. Identifying and mitigating health IT safety risks is a relatively new undertaking for most health care organizations. The introduction of health IT safety improvement initiatives could be expected to face many of the challenges that accompany introduction of any change to clinical practice. Introduction of new tools and practices can require substantial organizational effort.
Acknowledging the need for better information on the experience of organizations attempting to manage the risks posed by health IT, the U.S. Office of the National Coordinator for Health Information Technology (ONC) contracted with a team at the RAND Corporation (RAND), a nonprofit research organization; ECRI Institute (ECRI), a nonprofit research organization and patient safety organization (PSO); and health informatics research experts at Baylor College of Medicine and the University of Texas to develop and evaluate a prototype approach for engaging hospitals and ambulatory practices in health IT safety risk identification and mitigation projects. The project had the following goals:
Explore the challenges organizations face in deciding whether to participate in health IT safety risk identification and mitigation.
Test a simple diagnostic approach that participating organizations could use to identify health IT safety risks.
Assist organizations in developing and carrying out a short-term project intended to identify and reduce health IT safety risks.
Evaluate the results of the projects.
Evaluate the governance and management approaches used by organizations to manage health IT safety risks.
Identify barriers and facilitators to health IT risk identification and mitigation in hospitals and ambulatory practices.
Implementation of Health IT Safety Projects
The recruitment of sites and facilitation of process improvement projects were led by ECRI. The evaluation was led by RAND. Health informatics research experts at Baylor College of Medicine and the University of Texas provided expert input throughout. From a sample of 44 hospitals and ambulatory practices, 12 hospitals and nine ambulatory practices completed a survey of their EHR capabilities, and seven hospitals and four ambulatory sites were invited and agreed to participate. During a nine-month period, each participating site undertook a process improvement strategy led by ECRI that included assembling a project team; selecting a safety risk topic area; prioritizing practices within that area; specifying a work plan, including risk management activities, measures, and a monitoring plan; implementing the work plan; and monitoring progress and adapting the work plan as needed. The hospitals and ambulatory practices had access to several resources, including technical assistance. They also tested reporting safety events related to health IT using the standardized definitions and reporting forms developed by the Agency for Healthcare Research and Quality (AHRQ) Common Formats.
Evaluation
To learn about the sites' experience with the process improvement strategy, including the resources and safety event reporting, an evaluation team from the RAND Corporation conducted in-person and telephone interviews with representatives of six of the hospitals and ambulatory practices. The evaluation team used a semi-structured interview protocol to elicit information about the sites' experiences with identifying risks and implementing new health IT safety practices, as well as their experiences with the AHRQ Common Formats. The evaluation data were analyzed thematically and described in case study reports. A comparative analysis was performed to identify differences and similarities in sites' implementation experiences; to develop a series of lessons learned; and to offer recommendations that may be useful to hospitals and ambulatory practices seeking to manage safety risks posed by health IT, policy makers, electronic health record (EHR) developers, and other stakeholders.
The 11 participating sites were geographically diverse and encompassed both large and small hospitals and ambulatory practices. The sites varied widely in their Health Information Management Systems Society (HIMSS) scores, which reflect health care organizations' level of EHR adoption. Four of the seven participating hospitals already reported to a PSO before beginning the health IT risk mitigation project; none of the four ambulatory practices did so before the project. The interviews revealed the diversity of sites' experiences with and commitment to patient safety and risk management, as well as their allocation of staff and other resources to health IT projects and improvement efforts. In ambulatory sites, IT staff often had non-IT responsibilities, limiting their availability for IT projects.
Most project leaders came from the risk management, quality, and IT departments. All sites selected a topic area (e.g., clinician communication, computerized provider order entry) and specific risk mitigation activities. Some sites drew from material in the draft Safety Assurance Factors for EHR Resilience Guides (SAFER Guides), which were released by ONC in their final forms after the implementation phase of this project. All sites selected or developed metrics for measuring implementation progress, and most sites engaged in adverse event reporting to a PSO using the AHRQ Common Formats.
Most of the participating organizations found it difficult to identify and modify health IT safety risks within the nine-month project period, even with the resources and technical assistance available. Even though several organizations narrowed the focus of their projects, they encountered significant barriers at every stage of the process.
Lessons from the Pilot Project
“Readiness” to Conduct Health IT–Related Risk Identification and Mitigation Projects
Health care organizations may have limited capacity to join an externally initiated health IT risk management initiative and to sustain participation over time. Only a third of the hospitals and ambulatory practices invited to participate in the study agreed to volunteer. Among organizations that were contacted but decided not to join, “[poor] alignment with current and planned projects” was a commonly cited reason for declining to participate.
Organizations with the highest level of readiness to engage in detecting and mitigating health IT risks have in-house expertise and prior experience in conducting organizational quality improvement and risk management projects. In those sites that achieved their project objectives, we observed a preexisting and relatively sophisticated patient safety improvement infrastructure that included an adverse event reporting system and routine monitoring and analysis of patient safety-related events.
Alignment of Health IT Safety Projects with Other Quality, Safety, and Information Technology Initiatives
“Previously known problems” were more likely to be selected as targets of intervention than were problems identified through a diagnostic assessment. Each site completed a standardized diagnostic assessment designed to assist the staff in identifying potential targets for risk mitigation, but most sites selected intervention targets on the basis of known problems with safety, quality, attesting to meaningful use (MU) criteria (the demonstration of “meaningful use” of EHR so as to qualify for an incentive payment), or a combination of these items.
Similarly, projects appeared more likely to progress if they were aligned with the organization's priorities and current initiatives. Most of the sites faced the task of identifying health IT–related risks in the context of competing institutional priorities. Competing (or synergizing) priorities included business growth, meeting MU criteria, and addressing recent adverse event “near misses” or quality of service issues.
Projects also seemed more likely to succeed if they were aligned with current federal policy directives. Federal MU policy was an important driver for organizations in selecting and prioritizing initiatives. Organizations tended to view health IT safety through the lens of their efforts to meet MU standards.
Importance of Organizational Leadership
The case studies made clear the importance of organizational leadership to achieving success. Organizations whose project teams had close involvement of executive leadership were more likely to make progress in identifying and mitigating safety risks. In any organization, executive leadership sets priorities, allocates resources, directs the attention of staff to specific issues, creates accountability structures, and manages competing external demands. Disconnects and miscommunication between hospital or ambulatory managers and front line clinicians seemed to impede several steps in the identification, selection, and conduct of projects.
Challenges in Identifying Health IT Safety Risks
Organizations tended to view health IT as a solution to patient safety problems, while overlooking the potential of health IT to contribute to safety problems or to create new types of safety risks. Organizations installing, expanding, or upgrading EHRs are focused on ensuring that systems are operational and support necessary functions and that staff have sufficient training to use EHRs meaningfully. While these concerns clearly have implications for patient safety, the new safety risks associated with the implementation and use of health IT, especially EHRs, were not perceived in general as requiring focused effort.
Ambulatory practices encountered greater challenges than hospitals in identifying and addressing health IT safety risks. Resource constraints in ambulatory practices, particularly smaller practices, limited the ability of leadership to prioritize (or in some cases even recognize) health IT safety problems. None of the ambulatory practices we studied had full-time risk management staff.
Challenge of Matching Project Scope and Resources to the Demands of a Health IT Safety Project
Perhaps the most important determinant of project success was the availability of resources to commit to the health IT safety project. The most frequently cited barrier to and facilitator of successful implementation of projects was the timely and adequate allocation of staff effort and other resources to the project. Successful conduct of a risk mitigation project frequently required a substantial effort by project leaders, many of whom took on this effort in addition to a full-time job as clinician or practice manager. Risk management staff, quality and safety officers, and IT staff had to redirect attention and resources from current operations and health IT projects with looming deadlines (such as accomplishing meaningful use certification or planning for the International Classification of Diseases, 10th Revision, Clinical Modification transition) to pursue these risk mitigation projects.
Mismatch between the selected scope of the project and the available staffing sometimes led to poor project design (even when substantial expertise was available within the organization). Because health IT risks are sociotechnical in nature, they involve individuals conducting highly complex workflows that interact with complex technologies. This is an inherently challenging analytic problem. Furthermore, organizations may struggle with determining the best approach for engaging front line clinicians who both possess the knowledge of workflow challenges and may have to make changes to workflow in order for a safety risk mitigation project to succeed.
Practical Tools to Identify and Address Health IT Safety Risks
Health care organizations, and in particular small ambulatory practices, need tools to help them identify and address safety risks attributable to health IT. The challenges noted by each of the lessons above suggest the need for practical, easy-to-use tools that can help organizations identify health IT–related risks and set priorities for addressing them. Development of several of these tools (diagnostic assessment, the SAFER Guides, and metrics used by the participating organizations) began during this project, but these are prototypes that need additional refinement.
Staff at the hospitals and ambulatory practices reported that they found navigating the AHRQ Common Formats for reporting patient safety events to be burdensome. The series of steps used to arrive at the reportable risk seemed unnecessarily complex to many. Even when data were drawn from hospital adverse event reporting systems or EHRs, staff had to complete forms manually to submit them to the PSO, in part because of misalignment between the Common Format categories and the categories used in participants' event reporting systems.
Discussion
The challenges and lessons identified in this pilot project point to several opportunities to increase the safe use of health IT systems. We draw several conclusions about the current state of health IT safety risks:
With few exceptions, awareness of the safety risks introduced by health IT is limited.
The traditional departmental “silos” between risk management, IT, and quality and safety management may impede the ability of organizations to recognize and respond to health IT safety risks.
External facilitation appears to be important to hospitals and practices; however, the model for providing consultation and technical assistance requires further elaboration.
Most ambulatory practices lack the risk management, IT, and quality and safety expertise that is available in hospitals.
There is an urgent need for tools and metrics to enable project teams in hospitals and ambulatory practices to detect, mitigate, and monitor health IT safety risks.
The current structure of the EHR marketplace, and the low awareness of the risks introduced by health IT systems, lead to weak incentives for EHR developers and providers to invest in the type of joint effort required to reduce health IT safety risks.
Given the current situation, we saw several opportunities to make progress on safe use of health IT:
Awareness
To raise awareness of the health IT safety issue, two steps are necessary and closely related: to integrate and align the health IT safety agenda with the broader patient safety agenda and to engage front line clinicians in identifying and mitigating risk. A campaign built on the model established by the patient safety movement could very effectively alert front line clinicians to health IT as an important component of patient safety.
Fostering Collaboration Among Departments and Disciplines
Health IT safety is a cross-cutting area that creates an opportunity for risk management staff, safety staff, and IT staff to collaborate. Each disciplinary perspective contributes distinct knowledge to the detection, analysis, and mitigation of health IT safety risks. Several enablers of collaboration could support future initiatives: (1) disseminating best practices (case study examples of organizations that have successfully tackled a particular problem) and project templates (step-by-step project guides for specific problems or checklists); (2) providing staff from distinct disciplines with training in core terminology and methods related to safe use of health IT; and (3) developing a cadre of experts who can provide consultation through regional extension centers (RECs), PSOs, or other organizations and can facilitate training programs. The SAFER Guides provide a valuable tool for multidisciplinary, multifunctional teams to optimize the safety and safe use of health IT, EHRs in particular.
Strengthening External Facilitation and Consultation
Often hospitals and ambulatory practices lack the size and scale to support in-house expertise sufficient to carry out effective detection and mitigation of health IT safety. There will undoubtedly be a need for external facilitation and consultation, especially among rural hospitals and small ambulatory practices. Organizations likely to be engaged in this role include RECs and PSOs. PSOs are obvious candidates to support adverse event reporting; this project demonstrates that adverse event reporting is possible with the right data collection infrastructure but currently is often done manually. Finally, ensuring safe use of health IT will require that staff are trained on a mix of the retrospective methods used to analyze patient safety events, as well as proactive approaches designed to prevent patients safety events that may be introduced by health IT. PSOs, RECs, or other organizations could facilitate this staff training.
Supporting Ambulatory Practices
Ambulatory practices, in particular, may need more outside help if they are to succeed in identifying and mitigating health IT–related risks. Developing a “facilitator” workforce may be an opportunity to improve safety in these types of practices. Generally, facilitators receive specialized training and certification, and then serve multiple practices—providing access to the kinds of expertise and hands-on support that is typically only available to larger medical groups and hospitals.
Developing and Refining Tools and Metrics
The findings from our pilot project suggest that more work is needed to develop effective and usable tools and reporting systems. The prototype diagnostic tool we applied in the pilot was less useful to participants than hoped. An effective diagnostic approach that can be used by hospitals and ambulatory practices to identify and prioritize topics for health IT safety projects could build on and modify the tool we developed. The draft SAFER Guides that informed the implementation of the risk mitigation projects in our pilot are promising and useful. The SAFER Guides were finalized after this research project was largely complete. Further study of the SAFER Guides should evaluate their utility in practice and help to continuously improve the safety of health IT. Most organizations found reporting using the AHRQ Common Formats to be onerous and cumbersome. Revising the AHRQ Common Formats, especially for ambulatory practices, should be a high priority if adverse event reporting of health IT safety events is to be useful and guide further intervention.
Strengthening Incentives for EHR Developers to Optimize the Safety and Safe Use of EHRs
Health IT safety is a shared responsibility of EHR developers and their clients who use EHRs in a complex sociotechnical environment. MU of certified EHR technology has the potential both to improve patient safety, if implemented and used correctly, and to introduce new sources of patient safety hazards. The participants in this research project were motivated to qualify for MU incentives, but often did not appreciate the potential of EHR systems to introduce new safety risks. MU standards and EHR certification could provide incentives for EHR developers to work with their clients to optimize the safety and safe use of their EHR products and services. Surveillance associated with certification of EHRs could be used to identify and address EHR features that may be unsafe (such as poorly constructed CPOE with clinical decision support). Finally, some EHR developer interventions could help managers and clinicians to monitor deviations from intended, safe patterns of EHR use.
Conclusion
The investment that is converting the U.S. health data infrastructure into a 21st century enterprise has the potential to improve care for patients in countless ways. However, “digitizing” the health system also has the potential for harm. In this project, we worked with 11 hospitals and ambulatory practices to evaluate a process improvement strategy and tools developed to help health care organizations diagnose, monitor, and mitigate health IT–related safety risks. While many of the health care organizations (especially the hospitals) had expertise in process improvement, we found a general lack of awareness of health IT–related safety risks (especially in ambulatory practices) and concluded that better tools are needed to help these organizations use health IT to improve care and to optimize the safety and safe use of EHRs. The SAFER Guides provide an excellent beginning, but until health care organizations have a better understanding of the safety risks posed by EHR use, tools like the SAFER Guides may not be used to their full potential. There may also be a need for additional tools and metrics (and further usability study of existing tools and metrics) to better support the needs of health care organizations as they use health IT to improve the quality and safety of patient care.
Footnotes
This study was sponsored by the U.S. Office of the National Coordinator for Health Information Technology, and was conducted within RAND Health, a division of the RAND Corporation.