Organization and Representation of Patient Safety Data: Current Status and Issues around Generalizability and Scalability

Abstract

Recent reports have identified medical errors as a significant cause of morbidity and mortality among patients. A variety of approaches have been implemented to identify errors and their causes. These approaches include retrospective reporting and investigation of errors and adverse events and prospective analyses for identifying hazardous situations. The above approaches, along with other sources, contribute to data that are used to analyze patient safety risks. A variety of data structures and terminologies have been created to represent the information contained in these sources of patient safety data. Whereas many representations may be well suited to the particular safety application for which they were developed, such application-specific and often organization-specific representations limit the sharability of patient safety data. The result is that aggregation and comparison of safety data across organizations, practice domains, and applications is difficult at best. A common reference data model and a broadly applicable terminology for patient safety data are needed to aggregate safety data at the regional and national level and conduct large-scale studies of patient safety risks and interventions.

Errors and adverse events occurring in the course of medical management have been identified as a significant cause of patient morbidity and mortality in a number of different countries. A report published by the Institute of Medicine (IOM) in 1999 estimated that more than one million preventable adverse events occur in the United States each year.¹ A similar prevalence was reported in the United Kingdom, where one pilot study found that than 10% of patients in a hospital setting experienced adverse events, about half of which were preventable.² Data from other countries provide further evidence that preventable medical errors are a significant and widespread problem.^3,4

To gain a better understanding of these phenomena, learn more about the incidence and types of medical errors, and the factors contributing to these errors, we first need to acquire a broad range of raw data relating to patient safety and quality of care and perform a comprehensive analysis of the data. Patient safety data^* are available from many different sources for analysis. The most obvious sources are traditional incident reports^5,6,7,8 that providers (typically nurses⁹), prepared in response to a clinical event or variation in a standard process of care. Traditional paper-based strategies prevail but increasingly are being replaced by computer-based incident reporting systems in an effort to make analysis more efficient.^5,6,7,8 Departmental and institutional case reviews (e.g., root cause analyses,^10,11 monthly morbidity and mortality reviews) performed for deaths or cases with unexpectedly poor outcomes are another source of data. Adverse event data are also generated through routine audits of clinical records and active surveillance of signals and alerts from computerized order-entry or electronic patient records.¹² Some institutions collect data in a “prospective” manner, performing periodic quality assurance audits or rounds in an effort to identify safety issues and hazards before events occur.¹³ Less conventional sources of patient safety data include patient complaints, end-of-shift reports, multidisciplinary rounds in which patient management issues and coordination of care issues are discussed, executive walk rounds summaries, and malpractice allegation abstracts.

Individually, data from each of these sources tend to be used for a specific and somewhat narrow set of purposes including:

1. Care Improvement: Identifying opportunities to implement solutions directly.

2. Benchmarking: Determining the frequency and range of errors for purposes of benchmarking safety-related performance parameters.^3,4

3. Causal Modeling: Identifying or inferring the causes of a specific event, and developing an intervention to reduce the likelihood of recurrence.^14,15

4. Compliance with Regulatory Agencies: Mandatory and voluntary reporting to regulatory agencies of the government, and to accreditation and licensing bodies.^16,17

Because of the narrow purpose for which the data are collected, they may, in fact, be underutilized. If the data from different sources could be combined and subjected to aggregate analysis, collectively they might lead to a better understanding of broad system-based safety issues such as cross-disciplinary coordination, communication, interactions between policy and process of care, and understanding the effects of safety initiatives. The aggregate data could be used for benchmarking, for quality improvement, and for scientific research on types and causes of errors and interventions to reduce errors. Quality assurance managers within departments and institutions then would use the data for operational improvements. Researchers would find aggregated data to be a valuable resource for identifying errors and analyzing their causes. Administrators would analyze the data to identify problems that need further attention and could direct research efforts or new interventions to address the identified problems.

There are several reasons why patient safety data are underutilized despite their potential value. First, data from many of these sources are formatted as free text, and we currently lack the necessary tools to analyze or process complex, unstructured forms of data in a reliable and efficient manner. Even when these data are structured and coded, such as in computerized incident reporting systems, variations in the terminologies (often hierarchically organized as taxonomies of errors and contributing factors) and data models (that describe entities, their attributes, and their relationships) between different reporting systems make aggregate analysis of data difficult. Some of the variation in terminologies and data models reflects a narrow application focus (e.g., pharmacy incident reporting systems). But even for general safety concepts, e.g., patient-based, provider-based, and system-based factors that contribute to or mitigate risk, adverse events, and unsafe acts, there are wide variations in terms, definitions and their hierarchical organizations for describing similar concepts. As the President of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO) noted: “It is no small irony that the progressively expanding national discussions on patient safety over the past several years are not based on a common language. This critical missing element has hindered our collective ability to collect patient safety data in a consistent fashion, analyze process failures, mine data (e.g., trends, pattern analysis), and disseminate new knowledge about patient safety.”¹⁸ In the absence of a common and principled design approach to patient safety terminologies and data models it can be difficult reconciling even minor variations.

A comprehensive and formal representation of adverse clinical events and concepts relating to them would significantly improve our ability to manage and analyze an increasingly large and heterogeneous store of patient safety data. One generalizable approach to integration of heterogeneous patient safety data would be to create a representation that consists of a reference data model and a logically modeled terminology. The reference data model would then be refined to create specialized data models that represent data from various sources. Health Level 7 (HL7) uses an analogous approach to creating representation for various messaging standards.¹⁹ The data models rely on a controlled terminology to provide the values for attributes of the entities in the model. To support comprehensive modeling, the representation must be logically sound and cover a wide range of concepts including the human factors, organizational factors, system processes, information technology, and communication factors that contributed to or compensated for the severity of the event, and also the sequelae of the event itself.

In the sections that follow, we discuss the sources and properties of patient safety data to which institutions have access, how such data are currently managed and analyzed, and how the specific properties of the data from various sources limit the extent to which such data can be utilized. We focus on data models and terminologies currently being used to encode errors, adverse events, risks, and outcomes. We conclude by discussing how a common reference data model and a well-designed, broad, structured and logically sound terminology are needed to provide scalable and generalizable methods for managing and extracting knowledge from increasingly large repositories of patient safety data. Of note, such a representation would not attempt to define “error” per se, but instead, provide a means for representing all of the features of an adverse event along as many dimensions as possible.^†

Sources of Patient Safety Data

From a functional standpoint, patient safety data sources can be divided into two large classes. One class consists of sources that contain data about specific incidents. The other class consists of sources that contain data that describe general issues related to safety and hazards at a particular institution. Different sources of data from both these classes complement each other in the types of incidents and the content of the data. Thus, aggregating data from these sources can result in more comprehensive analyses than by examining data from each source separately.

The representation of data vary considerably even within each of these sources. In some sources, the data largely are semistructured blocks of narrative text that are very difficult to analyze using standard database methods. In other sources, containing structured, coded data, there is variation and inconsistency in the data model and terminology used within and between different reports. The heterogeneity in representation, among these sources of data described in the following subsections, raises challenges for data management and analysis.

Sources of Data on Incidents

Incident Reports

Incident reports are accounts by front-line personnel about specific errors, events, or system states that are perceived to have compromised patient safety, deviated from an organizational standard, or sometimes simply constrained the delivery of care.

These reports usually contain some or all of the following^5,20:

• time of occurrence

• site of occurrence

• roles of participants and reporters

• patient demographics and clinical attributes

• free-text description of the event or state

• characterization, classification or typing including severity, preventability

• identification of systems that failed

• outcome

Reporting templates, whether paper based or electronically formatted, often contain fields for free text narrative and fixed categorical selection options. A typical electronically formatted incident reporting system⁷ can be seen at http://www.anaesthesie.ch/cirs/.

The final product of an incident report typically is a combination of structured data and unstructured free text, the content of which often is limited to the description of the discrete event.

Although incident reporting can be used to identify a large number of incidents,²¹ studies have found that an even larger number of incidents are unreported.^22,23,24 A number of reasons have been identified for the lack of reporting including fears of blame, high workload, and the belief by the staff that the incident did not warrant a report.⁹ Thus, analyses of incident reports may not yield reliable estimates of errors. However, their analyses still can provide valuable insight on common causes and types of errors.

Root Cause Analyses Reports

A root cause analysis (RCA) report is the product of a comprehensive investigation conducted by a hospital team to identify the causal or contributing factors associated with a specific incident.¹¹ The RCA process usually is multidisciplinary and led by a risk manager, safety officer, or other designated personnel. Although the RCA process has been a safety tool in high-risk industrial domains for some time, it has only recently been adopted for use in the health care domain.²⁵ JCAHO now requires institutions to conduct RCAs of major adverse events (so-called “sentinel events”).²⁶ The final product of an RCA report is a description of the incident, detailed information on factors (human, environmental, equipment, and organizational) contributing to the incident and an action plan for reducing risk of recurrence.¹¹ In addition, for certain types of events, JCAHO requires that institutions investigate a set of contributory factors that are potential causes.²⁷ For example, for an incident involving surgery performed on the wrong anatomic site, the institution must conduct detailed inquiries into the following seven factors: staffing levels, patient identification process, physical assessment process, orientation and training of staff, communication with patient or family members, communication among staff members, and availability of information.

Departmental Case Reviews (Morbidity and Mortality Case Reviews)

To fulfill accreditation requirements or hospital bylaws, clinical departments and divisions perform regular (monthly or quarterly) reviews of cases the clinical courses of which included an unexpected complication or outcomes of which deviated from a standard or expected course (e.g., death, prolonged length of stay, unexpected intensive care admission). The purpose of such reviews is to present the details of the case and the outcome in a peer-review forum, and to encourage open discussion and critique.

Because most departments rely on self-reporting by the attending or admitting physician, the process is subject to bias and may not capture all cases that meet criteria for review. For the identified cases, a manual chart review is performed by either the principal clinician or an independent departmental auditor to acquire relevant data and prepare a report. Typically, case data are summarized in a combination of narrative text and structured tabular format. The narrative text includes some of the preadmission historical data (e.g., comorbidities, previous therapeutic interventions and outcomes, preadmission evolution of the illness or condition). The narrative text also tends to preserve the temporal sequence of events and may include a description of some of the decision-making processes involved. Structured data, if included, often are limited to patient attributes (e.g., medical record number, age, gender, admission diagnosis, discharge or postmortem diagnosis, procedures performed, and a classification of the unexpected outcomes or complications).

In addition to report preparation, cases also are discussed and critiqued in a peer forum. Additional information relating to decision making, constraints, and issues relating to planning and diagnostic or treatment uncertainty can emerge from these discussions. Occasionally, the discussion is recorded and transcribed as a permanent record.

Malpractice Claims

Malpractice claims occur when there were actual or perceived medical errors, and hence can serve as a potentially useful source of patient safety data. In fact, claims data have been used to analyze adverse drug events (ADEs)²⁸ and patient safety risks in anesthesia.^29,30 Despite their strengths, claims data have important limitations. First, because claims represent the perceptions of the different parties involved, such data can contain conflicting details of the actual event. Second, malpractice claims do not accurately reflect the quality or safety of care for a specific institution or practitioner. Comparing multiple sources of data, there appears to be poor correlation between a claim and the actual quality of care (i.e., whether care was negligent or nonnegligent).^31,32 These studies have found that the incidence of negligent care exceeded the incidence of malpractice cases and that malpractice claims frequently were made even in the absence of negligent care. Despite these limitations, claims data are valuable because the thorough investigations pursuant to a claim can reveal specific safety or risk issues that led to the event. Furthermore, claims data emerge from multiple different institutions and can reflect common systemic problems. Finally, malpractice claims often represent events that are associated with especially severe outcomes.

Electronic Patient Data Surveillance and Analysis

Data from electronic medical records and administrative databases can be used to identify the occurrence of adverse events. Computer-based surveillance programs can monitor these databases to identify the occurrence of safety-related events in “real-time.” In addition, retrospective review of these data can be used for identification of events for research and reporting.

Surveillance programs have been implemented for ADE detection in the hospital setting^24,33 and successfully tested for use in the ambulatory setting.³⁴ Similar programs have been used to detect nosocomial infections.^35,36 The programs typically operate by monitoring the electronic patient record for patterns of data indicating a possible event. For ADEs, the signals suggesting an ADE include certain diagnostic codes, sudden medication stop orders, orders for antidotes, and certain abnormal laboratory values when on particular drugs (e.g., low serum potassium with digoxin). The monitoring programs typically have a fairly low positive predictive value for events. In one report, only about 10% of events screened by the monitor were determined to be ADEs.²⁴ Hence, human intervention usually is required for following up each case that is identified by the program.

Computer-based surveillance complements incident reporting systems in identifying safety-related events. One of the limitations of the latter, as mentioned earlier, is that there are selection biases in the cases that are reported and, therefore, not all cases may be reported.²³ Surveillance techniques identify some additional ADEs that otherwise are unreported.^24,33 Jha et al.²⁴ found that the computer monitor identified approximately 45% of ADEs, whereas only 4% of ADEs were voluntarily reported by providers. However, surveillance techniques are more effective for detecting actual events than identifying errors or near misses. Also, such techniques do not yet cover a broad range of events, although more comprehensive detection tools are being developed. This approach is inexpensive, and it should become possible to detect an increasingly large and representative proportion of adverse events as more clinical data become available electronically.

In addition to real-time monitoring, clinical data have been analyzed retrospectively to identify occurrence of adverse events. One study used ICD-9-CM coded diagnoses and procedures and other clinical data to screen the records for the occurrence of events that required mandatory reporting in the state of New York.³⁷ More than one third of the screened cases were found to be reportable events and previously were unreported. A study using ICD-10 codes had similar success with identifying cases that were not reported by the providers.³⁸ These techniques rely on the reliability of the coding process to identify the occurrence of events. Similar screening for adverse events using paper-based charts has been found to be expensive compared with reporting by providers.³⁹

Sources of Data on General Issues Related to Safety and Hazards

Failure Mode Effects and Analysis (FMEA) is a technique for proactive evaluation of vulnerabilities in a system or a process before near misses or adverse events occur.⁴⁰ For each component of the system or process, the potential failure modes are listed. For each failure mode, possible causes are identified, the risk of failure is determined (risk is a function of the probability of occurrence and the severity of the outcome), and actions to control or eliminate the failure are listed. An FMEA report thus contains a wealth of information on patient safety presented in a predictive or anticipatory context. JCAHO requires accredited institutions to perform at least one FMEA every year.¹³

Other kinds of prospective investigations also can provide patient safety related data. For example, the Risk Management Foundation^‡ (RMF) conducts an office practice evaluation regularly at its member institutions. The clinic being surveyed is evaluated for various quality-related parameters such as organization of medical record files, completeness of records, and follow-up system for laboratory results and other studies. These data can be useful to provide context for a causal analysis of an incident or related incidents.

A few other reliability and hazards analysis techniques currently in use in industrial domains are being adapted for use in medicine. Sophisticated tools including event fault trees and fault hazard analysis,⁴¹ human reliability models,⁴² and simulation models⁴³ are being used on an investigational basis to help estimate the likelihood and consequences of such medical adverse events.

Unconventional Sources of Data

Less conventional but valuable sources of patient safety data may include quality assurance field observations or video recordings of clinical activities (e.g., trauma resuscitations or patient simulator scenarios), patient satisfaction surveys or patient complaints, health services research data (e.g., volume outcome data), and third-party insurer audits of performance, diagnostic test utilization, and procedural-diagnostic codes assigned to claims submissions. These sources vary widely in terms of format, content, and structure. While these sources of data often reveal general safety issues, some, such as patient complaints, may also address a specific incident.

Issues Around Scalability and Generalizability of Representation

The sources cited above represent a very heterogeneous set of data from a variety of different institutions. To support aggregation and analysis under such circumstances, it is necessary to have a representational scheme for the data that are both scalable and generalizable. Scalability refers to the ability of the representational scheme to facilitate aggregation and analyses of large volumes of data. Generalizability refers to the ability of the representational scheme to reliably model concepts across organizations, application, and patient safety domains and to be applied to different sources of data.

Volume of Data

The traditional focus of healthcare safety management has been outcome driven and retrospective, starting from an unexpected hospital- or specialty service–based poor outcome and investigating backwards. This limited the scope of what data were collected and analyzed. More recently, with the realization that many serious events do not result in measurable patient injury (i.e., near misses⁴⁴)^§ yet share many of the same features of cases with unexpected outcomes, a wider range of events are being reported and investigated.^45,46 Because the number of so-called “near misses” is much larger than the number of serious adverse events (▶), the potential size of the patient safety data set and the range of concepts that might be used to classify specific events begins to enlarge dramatically. The number of reportable events including near misses has been estimated to exceed 5 million per year in the United States.¹⁶ In comparison, the incident reporting system for the aviation industry in the United States, the Aviation Safety Reporting System, averages less than 35,000 incident reports every year.⁴⁷ Thus, any system created for representing patient safety data must be capable of supporting the analyses of large volumes of data. For example, efficient index-based information retrieval on this large patient safety data set would require a terminology of sufficient detailed granularity so that the user would not be overwhelmed with a large number of irrelevant results.

Figure 1.

The patient safety iceberg. Reported adverse events form the tip of the iceberg. However, a large volume of adverse events and an even larger volume of unreported near-miss incidents lie hidden underneath the surface. Many other adverse events, not preventable given what we know today, also occur.

Aggregating Data Across Organizations

To conduct meaningful analyses on types of events that occur rarely, or perform comparative analyses across classes of institutions, it is necessary to collect data at a regional or national level.⁴⁸ The larger data sets generated through aggregation strengthen statistical analysis and also enable individual institutions to learn from the experience of others.¹⁶ These are some of the reasons why the Institute of Medicine¹ and the Quality Interagency Coordination (QuIC) Task Force⁴⁴ of the U.S. Federal Government recommend development of systems for mandatory and voluntary reporting of adverse events across all 50 states in the United States. A recent report from the Institute of Medicine also calls for the development of standard format (or data models) and coding terminologies for patient safety data reporting for a broad range of applications.⁴⁹ On a local level, and in a smaller scale, efforts are being made to aggregate data across different institutions. The RMF has been leading a project to aggregate and analyze patient safety data from medical institutions affiliated with the Harvard Medical School. Even at this local level, aggregation and analysis are hampered because multiple different proprietary incident reporting systems in use by member institutions have led to variations in coding.

Sharing Data Across Applications and Clinical Domains

As noted above, there are a number of different systems currently available or under development to collect and manually classify adverse event data. Even within a single institution, there may be different reporting systems across different departments with different classification schemes. Custom systems exist for reporting medication errors,²⁰ intensive care unit (ICU) errors,⁸ and transfusion errors.⁶ The custom systems capture data that are at least partially idiosyncratic to the specific clinical practice domain or the safety application (e.g., medication error reporting). Medication error reporting systems, for example, feature tools that enable classification of the event in terms of the specific type of error (e.g., wrong drug, wrong dose), the step in which the error occurred (e.g., prescribing, administering), and the cause of the error (e.g., mislabeled medication container, handwritten prescription). While facilitating data collection and analysis for pharmacy settings, this approach restricts the usefulness of the data for more fundamental analyses of patient safety issues.

To illustrate the importance of this issue, consider a hypothetical scenario in a hospital in which the following errors occur:

1. A patient is given an incorrect chemotherapy medication because the provider administering the medication misidentified the patient.

2. A patient is administered blood of the wrong type. The blood administered was intended for another patient on the same floor.

3. An operation is performed on the wrong anatomic site because the surgeon viewed another patient's radiographs during the surgery.

Analyzed in isolation, these cases might appear as individual lapses. Analyzed collectively, the cases suggest a systematic problem with patient identification, which, as Chassin and Becher⁵⁰ have emphasized, is widespread and can have serious consequences (n.b., “Improve the accuracy of patient identification” was one of the six National Goals for Patient Safety⁵¹ established by JCAHO for the year 2003). An integrated approach to the analysis of patient safety data would improve the ability to identify general or systemic factors associated with a series of otherwise disparate incidents. Such an integrated system would allow global representation of high-level issues, support the representation of detailed information that is specific to different types of errors, and allow assessment using a variety of “lenses.”

Combining Data from Different Sources

In addition to analyzing different incidents or events from a common cause perspective, there also is value in looking at similar events from multiple different perspectives. So-called “sentinel” events, for example, often generate several different reports. Within each report, the event may describe different attributes or features of the event that might collectively lead to a comprehensive understanding of what happened. An incident report generated shortly after the incident by one of the care team members involved might identify the type of the index event itself but might be relatively vague on the sequences of events leading up to or after an event. A root-cause analysis, which retrospectively describes the event, often provides detailed descriptions of causative and contributory factors, specific interventions prompted by the incident, and recommendations for preventing future incidents. Documents associated with a malpractice claim provide details about patient perceptions and expectations surrounding the event as well as long-term consequences of one or more adverse events. The claims' documents also include multiple, different perspectives from the patient, family members, providers, lawyers, jury members, and the insurers.

Each of these sources contains different but potentially interrelated information that might be used in many different ways. As an example, consider a scenario in which a Director of Patient Safety is reviewing incident reports of a type of error that appears to have increased in frequency. To understand the underlying causes, she checks if any root cause analyses were performed and reviews those documents. She then would like to find what actions were taken and preventive measures were implemented to counter the causative factors at her hospital and at other institutions. She also might wish to check national databases to see if this error is common at a national level and whether there are recent recommendations for preventing the error. A unified representation of patient safety data that can integrate different sources of data is needed to support this type of use.

Content of Patient Safety Data and Existing Terminologies

Model of Patient Safety Data

The examples cited above illustrate the variation in detail and comprehensiveness of patient safety data. In spite of this apparent variation, there is a subset of data that is common to all sources and that might serve as a framework for a canonical or reference data model of clinical adverse events. Reports about a specific incident, such as from RCAs and incident reporting systems, tend to organize data around the following key categories or questions:

• the type of the error

• the context of the error

  • where it occurred
  • when it occurred
  • who was involved
  • the patient's state
  • the environmental conditions prior to the occurrence of the error

• associated causative and contributing factors

• the clinical outcome for the patient

• the remedial and preventive actions taken

Similarly, data about general hazards identify factors that might lead to certain types of errors and adverse clinical events and possible preventive actions to mitigate the risk of an untoward incident from occurring.

We believe that a common reference model can be created for representation of patient safety data. As mentioned earlier, the reference model can be refined to create concrete data models to represent data from each of the sources. HL7 has created a methodology called HL7 Development Framework that has among its objectives the refinement of the HL7 Reference Information Model to create a data model for a messaging standard. This refinement framework could be modified for application to patient safety data representation.

In addition to a reference data model, a common terminology is needed to enable us to associate data from different sources and systems with one another. The following sections discuss the issue of terminologies in patient safety data representation.

Terminologies for Patient Safety Data

Existing medical terminologies and indexing schemes such as SNOMED, ICD-9 CM, or CPT^52,53,54 that are used widely to support other clinical applications, do not contain terms or concepts relating to medical errors and their attributes.⁵⁵ Most notably, these terminologies lack detailed representation of error typology (e.g., wrong-sided surgery) and other attributes such as causative and contributing factors (e.g., patient misidentification, failure of communication).⁵⁶ To address these deficiencies, several new, but limited terminologies have been created to support institutional or commercial incident reporting applications (e.g., enable annotation of free text narrative reports).

We are not aware of and could not find any reports in Medline of a broadly applicable, nonproprietary patient safety terminology that could support the range of analytical tasks described earlier. Commercially available general incident reporting tools such as DoctorQuality Inc.'s Risk Prevention and Management System use proprietary but limited terminologies. Specific applications such as for medication error reporting,²⁰ blood transfusion error reporting,⁴⁶ or for specific domains such as primary care^57,58 have very narrowly focused terminologies. The Medical Dictionary for Regulatory Activities (MedDRA) is a terminology created by the “International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use.” MedDRA is intended for recording data during pre- and postmarketing phases of drug development⁵⁹ and does not contain terms for medical errors and their causes outside of its drug regulation perspective. The RMF has developed and uses a broad terminology related to errors. However, this proprietary terminology has a focus on medico-legal claims. ▶ contains a listing of some of these terminologies and representative categories and terms.

Table 1.

Partial Listing of Medical Error Terminologies with Representative Categories and Terms

Purpose	Selected categories	Sample terms
NCC-MERP's Taxonomy of Medication Errors
Medication error reporting	Patient information	Written miscommunication-trailing zero
	Patient outcome	Dose omission
	Product information	Floor stock
	Dosage form	Adult day health care
	Packaging
	Type (of error)
	Causes
	Name confusion
	Contributing factors
DoctorQuality Inc.'s Risk Prevention and Management System's Terminology
Hospital incident reporting	Error type	Fall from bed chemical restraints involved
	Event classification	Critical results not reported
	Adverse Clinical Event	Hemolytic reaction confirmed
	Fall
	Level of impact
	Contributing factor
	Drug name
Medical Event Reporting System for Transfusion Medicine's Terminology (developed by Columbia University)
Blood transfusion	Product check-in	Data entry incomplete
	Order entry	Sample tubes mixed up
Error reporting	Sample testing	Outdated component in stock
	Product storage	Knowledge-based errors
	Latent errors
	Active errors
International Taxonomy for Errors in General Practice
Primary care error classification	Process errors	Records unavailable
	Office administration	Wrong test ordered
	Investigations	Wrong or missed diagnosis
	Knowledge and skill errors
Risk Management Foundation's Taxonomy
Claims coding	Locations	Post mortem procedures
	Services	Sexual misconduct
	Allegations	Lack of any consent
	Medical treatment	Clinician did not receive results because report went to wrong clinician
	Provider behavior
	Hospital policy and procedure
	Risk management issues
	Communication
	Clinical systems

Among the proprietary terminologies, no one single system is broad enough in coverage yet sufficiently detailed to encode patient safety data from all clinical domains.⁶⁰ Often, it may not be possible to map terminologies to each other because of differences in granularity (e.g., the NCC-MERP Taxonomy of Medication Errors²⁰ has a very detailed classification of product labeling issues as a cause of error that is not matched by the codes in DoctorQuality's system), or asymmetries in classification (i.e., assigning terms under different categories in different terminologies).

Additionally, many of the terminologies have been developed using ad hoc approaches, typically by organizing terms in simple type–subtype hierarchies. More complex semantic relationships among terms often are lacking. ▶ illustrates how the lack of formal semantic representation may cause the analysis of data to be unreliable. Ad hoc development also can lead to incomplete, inconsistent, or redundant terminological representations. An excerpt from the NCC-MERP Taxonomy of Medication Errors²⁰ is shown in ▶. As is seen, there are five terms for different types of health care professionals listed. In another section (Section 60—Personnel Involved) of the taxonomy, there are 28 terms for health care professionals listed. In fact, four of the five terms (except “Other”) from ▶ have a different semantic or lexical form in section 60. For these and other reasons, then, existing patient safety terminologies lack the fundamental properties required for reliable and consistent data annotation, aggregation, and intelligent analysis.

Table 2.

Selected Terms Related to Misidentification of Patient, Substance, or Record from a Terminology for Medical Errors

Lab Issues

Specimen mislabeled

Incorrect patient ID used

Operative/invasive procedure

Wrong patient

Radiology

Mislabeled

Wrong patient

Transfusion-related

Incorrect patient received product

The patient misidentification terms are represented under several different categories. Selection of cases, such as those described in the “Sharing Data Across Applications and Clinical Domains” section, from an incident report database relies on the human to know and recall all related terms for patient misidentification.

Table 3.

Excerpt from the NCC-MERP Taxonomy of Medication errors

90.6 Lack of availability of health care professional

90.6.1 Medical

90.6.2 Other Allied Health Care Professional

90.6.3 Pharmacy

90.6.4 Nursing

90.6.5 Other

Five terms for different types of medical professional are listed.

There are several possible reasons for the lack of formal structure in current patient safety terminologies. First, many of the proprietary and application-specific terminologies were developed in response to an urgent demand for incident reporting systems, and their ad hoc properties likely reflect the urgency with which these systems were designed and deployed. Second, until very recently, developers of incident reporting systems did not plan for aggregation of data from various applications, systems, or institutions or the extension or evolution of terminologies over time. Finally, although terminologies that are developed using formal methods are far more useful, the development process requires considerable time, effort, and expertise.

General Models of Errors from Other Industries

In comparison with terminologies and taxonomies for medical errors, which are very domain specific in classifying or describing errors, industrial and transportation safety applications are based on generic models of human performance and organizational–operational aspects of incidents. These classification schemes and terminologies cover such domain-independent concepts as organizational structure, basic human cognition and task execution, process management, control theory, communications, regulatory controls, and policy science.

Donald Norman's categorization of action slips⁶¹ and Stages of Action Model⁶² and Hollnagel's classification,⁶³ for example, were developed initially as a high-level framework for characterizing human–computer interaction but have been applied more generically to classify cognition–action couplings. Rasmussen's Skills-Rules-Knowledge model⁶⁴ and Reason's Generic Error-Modelling System (GEMS)⁶⁵ have been used in a variety of applications. A variety of methods and taxonomies have also been developed for analyzing the causal factors of accident and incident data from the industrial and transportation domains. These include the TRIPOD Basic Risk Factors,⁶⁶ PRISMA,⁶⁷ and UK CIRAS rail reporting system classification schemes;⁶⁸ the classification scheme associated with the Accident Evolution and Barrier Function Method;⁶⁹ and the Taxonomy of Unsafe Operations.⁷⁰ The Eindhoven Classification Model focuses specifically on organizational factors,⁷¹ and the Office of Aviation Medicine's HFACS system focuses on both human and organizational factors.⁷² Each of these models strives to classify or characterize latent (“hidden”) and system-based factors that contributed to an incident.

Several of these models have been applied to medical errors. Reason's GEMS framework has been used as the basis of a protocol for investigating medical accidents.⁷³ Norman's Stages of Action Model has been proposed as the basis of a taxonomy of human errors in medicine.⁷⁴ The Eindhoven classification model has been adopted for causal classification and analysis of blood transfusion errors.⁷⁵ Because these models and taxonomies contribute to our understanding of errors, it would be important to incorporate their features in patient safety terminologies.

Development of a Common Patient Safety Terminology Using Formal Methods

As discussed earlier, a common patient safety terminology is needed to extend the use of data across organizations, applications, clinical domains, and proprietary information systems. However, a terminology that can be applied across this wide range of domains does not exist. Hence, a terminology for patient safety must be developed.

While commonly used clinical terminologies are inadequate for safety reporting, a new terminology effort might be able to build on existing clinical terminologies. First, in addition to terms describing error types and contributing factors, safety reporting also requires terms describing the clinical context such as roles of various actors in the incident, the location of an incident (e.g., intensive care unit, pharmacy, nursing floor), and the specific medication administered or procedure that was performed. Such terms usually are well covered⁵⁶ in the clinical terminologies and would not have to be created de novo. Second, if a compositional approach⁷⁶ is used to create the terminology, as suggested in the next paragraph, several of the atomic terms required to compose compound terms for error types or contributing factors can be found in clinical terminologies. Our analysis⁵⁶ indicates that one or more of the atomic terms exist in the source vocabularies of the UMLS metathesaurus⁷⁷ for more than 90% of safety-related terms.

To maximize scalability and generalizability, formal terminology modeling principles should be used to develop a standard patient safety terminology. Clinical terminologies have been modeled using conceptual graphs⁷⁸ and description logics.⁷⁹ These logic-based approaches can support the development of consistent, valid, and extensible terminologies. In addition to assertions of type–subtype relationships, other semantic relationships among terms can be defined, e.g., part–whole, temporal. On the basis of these definitions, terms can be classified more flexibly, consistently, and automatically into multiple hierarchies.

The terminology should be founded on one or more theoretical models of error and reliability to enable axiomatic analyses of the data. For example, terms could be classified according to concepts in Reason's GEMS framework as Unsafe Acts, Preconditions, and Latent Failures. The terminology should be broadly applicable across clinical domains and specific applications to enable aggregation of data from different sources. Clinical terminologies such as SNOMED-CT⁵² show the feasibility of creating a terminology that spans across a broad range of clinical domains. In a broadly applicable terminology, domain- and application-specific terms can be derived from the more general terms, e.g., miscalculation of a medication dose can be derived from a calculation error term and can thus be associated with miscalculation of radiation dose and miscalculation of cardiac output. One recent effort to develop a terminology for communications errors based on Reason's model of errors and expressed in conceptual graphs exemplifies such a principled terminology modeling approach.⁸⁰

An issue with the current patient safety terminologies and with safety terminologies from other domains is the lack of formal validation of these terminologies prior to their use and of continuing evaluation. A standard patient safety terminology should be evaluated prior to large-scale use to assess its coverage, validity, and consistency. Furthermore, the terminology should also be continuously evaluated for its value in identifying and analyzing patient safety problems.

Conclusion

Lack of standards for representation of patient safety data has significant implications for the success of data-driven interventions, i.e., causal modeling, care improvement policies, benchmarking, and compliance. Without a common representation of the data, it becomes very difficult to analyze data across event types or event sites. Benchmarking, in particular, becomes worthwhile only when standard adverse event definitions are in place and similar surveillance is being conducted across different sites. To date, regulators have not insisted on the use of standard surveillance methodologies or data reporting formats, but that eventually may change.

A critical ingredient of large-scale efforts to collect, share, and analyze patient safety data is a common representation for the data. The representation for patient safety data would consist of a common reference data model and a standard terminology. The representation must be well designed,⁸¹ broad, extensible, and flexible to accommodate the needs of a variety of applications.