Applying the WHO conceptual framework for the International Classification for Patient Safety to a surgical population

Abstract

Objective

Efforts to improve patient safety are challenged by the lack of universally agreed upon terms. The International Classification for Patient Safety (ICPS) was developed by the World Health Organization for this purpose. This study aimed to test the applicability of the ICPS to a surgical population.

Design

A web-based safety debriefing was sent to clinicians involved in surgical care of abdominal organ transplant patients. A multidisciplinary team of patient safety experts, surgeons and researchers used the data to develop a system of classification based on the ICPS. Disagreements were reconciled via consensus, and a codebook was developed for future use by researchers.

Results

A total of 320 debriefing responses were used for the initial review and codebook development. In total, the 320 debriefing responses contained 227 patient safety incidents (range: 0–7 per debriefing) and 156 contributing factors/hazards (0–5 per response). The most common severity classification was ‘reportable circumstance,’ followed by ‘near miss.’ The most common incident types were ‘resources/organizational management,’ followed by ‘medical device/equipment.’ Several aspects of surgical care were encompassed by more than one classification, including operating room scheduling, delays in care, trainee-related incidents, interruptions and handoffs.

Conclusions

This study demonstrates that a framework for patient safety can be applied to facilitate the organization and analysis of surgical safety data. Several unique aspects of surgical care require consideration, and by using a standardized framework for describing concepts, research findings can be compared and disseminated across surgical specialties. The codebook is intended for use as a framework for other specialties and institutions.

Introduction

Since the landmark publication by the Institute of Medicine, To Err is Human, improving patient safety and reducing medical errors has been a priority for healthcare researchers around the globe [1]. Despite the proliferation of patient safety research, progress remains hindered by inconsistent language and varying definitions of the central concepts. Several taxonomies for the classification of patient safety terms and concepts have been proposed, but none have been widely implemented [2–6]. This lack of standardization has limited the ability to aggregate, organize and compare information across disciplines, thus decreasing the dissemination of advancements made by individual fields. The overall result has been delayed improvement healthcare systems [7, 8].

Event reporting, which relies on those involved in safety events to report the occurrence, is a major part of the effort to detect safety vulnerabilities. Although event reporting systems are present in most hospitals, utilization varies by clinical specialty and role in the healthcare team [9]. Team debriefings take a more proactive approach by actively soliciting input from team members at the conclusion of a shift, incident or procedure [10, 11]. Both methods have the advantage of involving frontline personnel in identifying safety hazards for the organization. However, the data collected are in the form of narrative comments, making aggregation, organization and analysis challenging as well as labor intensive.

The International Classification for Patient Safety (ICPS) is a standardized set of concepts and terms organized into a conceptual framework to enable consistent organization of the major events associated with patient safety. The development of the ICPS was first identified as a key initiative by the World Health Organization (WHO) World Alliance for Patient Safety in 2005 [12]. The conceptual framework and accompanying taxonomy provide a method of organizing patient safety data for the purpose of aggregation, analysis and translation into usable information, as well as guidance of patient safety research [13]. The ICPS was created to be broadly applicable in the context of research and institutional quality and safety improvement. It has thus far been applied in a range of specialties around the country, but has not yet been applied to surgical care [14–20]. Subsequent to the development of the ICPS, the WHO began working to advance practical developments based on this Conceptual Framework. One such development is the Minimal Information Model for Reporting Patient Safety Incidents (MIM-PS), which serves as a template for a minimum set of common data categories to facilitate a guide for reporting, while allowing for comparison. The MIM-PS incorporates the ICPS, as well as the International Classification of Disease (ICD), International Classification of Functioning (ICF), Disability and Health. The purpose of this study was to test the application of the ICPS to surgical safety debriefing data.

Methods

Setting and study population

Adult liver and kidney transplant surgeries at a large urban tertiary referral center from April 2010 to April 2011 were included in the study. Abdominal organ transplantation was chosen as a model of a highly complex surgical system and limited physiologic reserve of the patients. Clinicians (physicians, nurses, trainees and surgical technologists) involved in the intraoperative care of kidney and liver transplant donors and recipients were identified via the electronic health record within 24 h of the surgery during which they provided care and were asked to complete a web-based debriefing survey via email. Institutional Review Board approval was obtained prior to the solicitation of participation or collection of data.

Safety event data collection

A web-based safety debriefing tool was developed by a multidisciplinary team of patient safety researchers, clinicians and researchers based on the Institute for Healthcare Improvement debriefing methodology. The web-based safety debriefing tool solicits comments on all patient safety related concerns through several thematic prompts and four open ended questions, allowing for further elaboration when any of the initial thematic areas of care were designated as problematic.

The thematic prompts are: Communication with the Patient and the Family, Inter-Provider Communication, Distractions, Care Coordination, Patient Identification, Information Technology, Access to Necessary Clinical Data, Patient Education/Teaching, Medications, Coordination with Care Facilities/Hotel, Discharge Planning, Discharge Instructions, Appointment Scheduling, Blood/Transfusions, Labs/Studies, Equipment/Physical Environment, Patient Vitals Monitoring, Non-specific Adverse Events, Bed Availability/Staffing, Transitions/Transportation/Handoffs. The four open questions are: Did you encounter other issues or barriers that made your work more difficult? Did you follow up on issues described in this debriefing? How? Did you have suggestions to address the issues described in this debriefing? How would you improve patient safety? [21].

Within 24 h of completion of transplant surgeries, emails were sent to all individuals listed in the operating room nursing personnel report requesting participation via a hyperlink to the web-based safety debriefing. The link contained a consent form that described how information obtained from the web-based safety debriefing would be used to improve the care processes. Participants provide consent electronically and their responses are gathered anonymously and analyzed in aggregate. Non-respondents receive reminder emails at 24 and 48 h after the initial e-mail request.

Classification of collected data

The World Health Organization ICPS

The WHO ICPS was created via a Delphi process in order to facilitate comparison of patient safety research findings between institutions and disciplines [22]. The ICPS is based on a conceptual framework that includes 10 high-level concepts integrated into a framework (Incident Type; Patient Outcomes; Patient Characteristics; Incident Characteristics; Contributing Factors/Hazards; Organizational Outcomes; Detection; Mitigating Factors; Ameliorating Actions; Actions Taken to Reduce Risk (Fig. 1) [23]. For this study we focused on three classes of organization: (i) the 4 levels of safety incident severity (Box 1): Adverse event, No Harm Incident, Near Miss and Reportable Circumstance; (ii) the 13 primary classes of incident type, defined as an event or circumstance that could have resulted, or did result, in preventable harm to the patient (Fig. 2a) and (iii) the 6 primary classes of contributing factors/hazards, defined as the circumstances, actions or influences that are thought to have played a part in the origin or the development of an incident or to have increased the risk of a patient safety incident (Fig. 2b). Application of the classification requires a mutually exclusive classification of the incident type. The technical report and complete taxonomy can be found at http://www.who.int/patientsafety/implementation/taxonomy/publications/en/.

Figure 1.

The conceptual framework for the International Classification of Patient Safety (reprinted with permission from the World Health Organization).

Box 1.

A reportable circumstance is a situation in which there was significant potential for harm, but no incident occurred.

A near miss is an event that could have resulted in unwanted consequences, but did not because, either by chance or through timely intervention, the event did not reach the patient.

A no harm incident is one in which an event reached a patient but no discernible harm resulted.

An adverse event is a harmful incident that results in harm to a patient, resulting from a medical intervention and not due to the underlying condition of the patient.

Figure 2.

(a) Incident type primary classifications and (b) contributing factor primary classifications.

The recently developed MIM-PS is the WHO's latest effort to strengthen effective reporting by identifying the key data features that can provide minimal meaningful learning, while allowing for comparison. The MIM-PS incorporates the ICPS, as well as the ICD, ICF, Disability and Health and serves as a template for a minimum set of common data categories to facilitate a guide for reporting.

Pilot coding

Two separate physician reviewers (A.D. and L.M.) independently coded a preliminary set of web-based safety debriefing responses. The process involved: (i) identification of patient safety incidents within debriefing text, (ii) classification of patient safety incidents severity (reportable circumstance, near miss, no-harm event or adverse event), (iii) identification of any listed contributing factors and (iv) application of primary and secondary codes to each patient safety incident and contributing factor based on the WHO technical report. The additional step of identifying the independent issues for classification was necessary as individual debriefing responses frequently included many issues.

Data collection and determination of inter-rater reliability

Codes were submitted via the electronic coding form to a central database. The data were then aggregated and verified and reviewed manually for completeness and accuracy. The preliminary review by two physicians demonstrated a 93% agreement between coders. Coders involved in pilot coding met the faculty (D.W., D.L.) who were involved in the development of the web-based safety debriefing tool and the codebook and had personal experience in coding qualitative safety incident data to discuss coding discrepancies. For each discrepancy, all possible codes were discussed and standardized responses were chosen via consensus and compiled into a codebook. Each identified incident was re-reviewed by an independent analyst.

Additional coder training

Coder training began with pre-reading of the articles published in the International Journal of Quality in Healthcare describing the WHO ICPS [8, 22–24]. This was followed by a 1 h didactic session led by the faculty (D.W., D.L.) who were involved in the development of the web-based safety debriefing tool and the codebook and had personal experience in coding qualitative safety incident data. Coders were oriented to the structure of the classification and the organization of the technical report. Example safety debriefing reports were reviewed discussed in relation to the ICPS coding structure and the applicability of codes as defined in the codebook. In a subsequent session, the electronic coding tool and codebook were introduced and coders and faculty reviewed several example debriefings and applied appropriate codes as a group. Following the second session, coders were asked to code a set of 10 debriefings using the codebook and provide feedback regarding codebook clarity and overlapping codes. This feedback was used for further refinement of the codebook.

Codebook refinement and validation

Following primary review, a multidisciplinary panel of patient safety experts and transplant clinicians was convened to review the aggregate data and apply the preliminary codebook (v1.0) and determine the applicability for use by researchers (Appendix). Discrepancies in the first two coding sessions were discussed and the coding process was refined iteratively. An electronic coding tool was created using Google Forums. The refined codebook (v2.0) was applied for purposes of validation by a team of 6 coders (A.S., D.L., D.W., A.D., T.C., E.W.) using an additional 20 debriefings. Following the initial 10 debriefing reports, there was a 53% agreement between coders. Each coder received individual feedback regarding their coding discrepancies via email. Suggestions were made for further refinement of the codebook. A further refined version (v3.0) of the codebook was disseminated to the group with a different set of web-based safety debriefing responses. Codes were aggregated, and the final review demonstrated a 76% agreement between coders. This was followed by a final meeting of all six coders to review and resolve discrepancies in coding and finalize the codebook (Table 1).

Table 1.

Debriefing respondents and incident severity

Respondent	n (%)
Nurse	107 (47.1)
Attending surgeon	65 (28.6)
Anesthesiologist	27 (25.2)
CRNA	16 (7)
Physician Assistant	37 (16.3)
Trainee	38 (16.7)
Other	30 (13.2)
Patient safety incident severity	n (%)
Reportable circumstance	148 (65.2)
Near Miss	12 (5.3)
No-harm incident	26 (11.5)
Adverse events	40 (17.6)

Results

A total of 320 web-based safety debriefings were used for the initial review and codebook development (version 1.0–3.0). In total, the 320 debriefing responses contained 227 patient safety incidents (range: 0–7 per debriefing) and 156 contributing factors/hazards (0–5 per response). The most common severity classification was ‘reportable circumstance,’ followed by ‘near miss.’ The most common incident types were ‘resources/organizational management,’ followed by ‘medical device/equipment.’ Results related to incident types are presented in Table 2. Examples of safety debriefing reports by incident type are presented in Table 3.

Table 2.

Patient safety incident types

Incident types (k)	n (%)
Clinical administration	13 (5.7)
Admission	6
Consent	4
Handover	3
Clinical process/procedure	56 (24.6)
Screening/prevention	1
Diagnosis/assessment	6
Procedure/treatment/intervention	41
General care/management	3
Tests/investigations	3
Specimens/results	2
Documentation	19 (8.4)
Charts/medical records/assessments/consultations	14
Forms/certificates	4
Not enough information	1
Medication/IV Fluids	18 (7.9)
Delivery	8
Preparation/dispensing	5
Administration	3
Prescribing	2
Oxygen/gas/vapor	1 (0.4)
Delivery	1
Medical device/equipment	44 (19.4)
Failure/malfunction	22
Lack of availability	7
Poor presentation/packaging	5
Inappropriate for task	4
Unclean/unsterile	3
Dislodgement/misconnection	1
User error	1
Not enough information	1
Behavior (Staff)	5 (2.2)
Noncompliant/uncooperative/obstructive	2
Risky/reckless/dangerous	2
Not enough information	1
Patient accidents	4 (1.8)
Piercing/penetrating	2
Threat to breathing	1
Other	1
Resources/organizational management	64 (28.2)
Human resources/staff availability/adequacy	30
Bed/service availability/adequacy	13
Organization of teams/people	11
Matching of workload management	9
Protocols/policies/procedure/guideline	3
Availability/adequacy	1

Table 3.

Examples of debriefing reports by incident type

Primary classification	Secondary classification	Debriefing report example
Resources/organizational management	Human resource staff availability/adequacy	No available OR transplant nursing staff to start the second liver. Had to use the general call people to set up the case. Need to have another transplant team to cover the case if … it will have a two rooms
	Matching of workload management	Remove distractions. Surgeons want all messages relayed to them. With multiple surgeons and fellows in OR. Answering pagers and phone is a full time job. There is only one scrub nurse with two hands
Clinical process/procedure	Procedure/treatment incomplete/inadequate	Due to patient's previous vascular surgery, it was not possible to place a femoral arterial catheter
Medical device/equipment	Device failure/malfunction	Powercharta was down for 3 h 30 min of the case and unable to get lab values from the computer

^aPowerchart: electronic health record for inpatients.

Operating room scheduling and logistics

Operating room scheduling and logistics, such as the availability of designated operating rooms, equipment processing and scheduling of staff, were reported in 64 (20%) debriefings. The management of the operating rooms was classified as ‘resources/organizational management,’ with applicable secondary codes including-organization of teams/people’ and ‘human resource/staff availability.’ When operating room staff scheduling resulted in the lack of available operating rooms, the secondary code ‘organization of teams/people’ was chosen. The lack of staff for any other reason was coded as ‘human resource/staff availability/adequacy,’ with further details illustrated by using contributing factors. This division allows differentiation between staff shortages and logistic difficulties despite adequate staffing.

Delays

Delays were reported in 41 (12.8%) debriefings. In several instances, the delay was noted as a consequence of another safety incident: ‘there was no available surgeon to talk with the patient in pre-operative holding and the case was delayed 30 min’. However, delays were also frequently reported as safety incidents, with further consequences as a result: ‘the surgeon was late to the operating room, so the timeout was rushed and not everyone's concerns were addressed’. In other instances, delay was both the safety incident and the outcome: ‘There was a delay in processing the medication orders, so the immunosuppression was not given on time.’ While a delay in patient care may be identified as an outcome, we felt that it was important to code delays as individual incidents were applicable for several reasons. First, delays are often ignored in surgical settings as a tolerated part of patient care. By identifying delays as individual safety incidents, we were able to assess the overall burden of delays within the system of surgical care. Secondly, while delays are an outcome, they in turn can lead to hazardous conditions and subsequent outcomes in other aspects of care, including care of patients scheduled for surgeries later in the day. Lastly, in transplantation, delays can translate to an increase in cold ischemic time, which can compromise transplant outcome.

Trainees

Trainees were mentioned in 27 (8.4%) debriefings. References to trainees most commonly commented on interactions between trainees and faculty, which can introduce unique safety vulnerabilities, such as ‘The case took longer because the fellow was not listening to the surgeon and they got into some bleeding’. The hierarchy was reported to complicate communication, with less communication from trainee to faculty, and team members of all positions reported concerns with trainee performance. While it is important to include and analyze these data, the challenge lies in adopting a single code for trainee-related issues. Hence, in the final coding it was felt that trainee behavior and performance lies at the intersection of cognition, performance and behavior. For this reason, the following coding was used: ‘staff contributing factor of Behavior/Performance/Cognition.’

Pagers and cell phones

Pagers and cell phones were mentioned in 23 (7.2%) debriefings. Debriefings submitted discussed the distraction created when operating room staff needed to balance answering pages and phones to facilitate communication from outside of the operating room with their other work tasks such as, ‘Pagers constantly going off and not enough help to answer them and focus on the patient. There is only one nurse with two hands.’ Both ‘resources/organizational management’ with either ‘organization of teams/people’ or 'matching of workload’ as secondary codes applied. The latter was chosen to better reflect the burden of distraction, with a secondary added code of electronic communication.

Biologics

Biologics were mentioned in 9 (2.8%) debriefings. Four of these referred to blood vessels used as vascular conduits during the surgical procedure, four referred to an organ and one referred to tissue transfer. No unique codes exist for graft materials used in surgery, such as PTFE grafts of blood vessels (internal jugular, saphenous vein) for vascular conduits. Medical device/equipment is an applicable code, however, to distinguish between machinery/equipment and biologics in the aggregated results, ‘blood/blood products’ appears more appropriate.

Handoffs and transfer of care

Six debriefings mentioned handoffs or transfers of care. All of these debriefings referred to perioperative transfers of care, either pre-operative to the operating room or the operating room to intensive care unit (ICU) or post-anesthesia care unit (PACU). While there was some overlap with reported delays, delays were generally reported as outcomes in this setting. As incident class of ‘Clinical Administration’ has both handoff and transfer of care codes, transfer of care was limited to the physical transfer of a patient from one location to another, while handoffs were limited to changing of individual providers.

Contributing factors

Preliminary coding identified 191 contributing factors. However, inter-rater reliability was significantly lower for contributing factors than for patient safety incidents (26%). There was a tendency for coders to assume contributing factors based on prior clinical experience. For this reason, the final consensus was to code only contributing factors explicitly written in the debriefing text, such as ‘Due to patient's previous vascular surgery, it was not possible to place a femoral arterial catheter’. In addition, several frequently encountered surgical incidents were assigned designated contributing factors.

Discussion

Prior research in patient safety has demonstrated the value of voluntary event reporting from front line clinical team members [25–27]. However, the qualitative data generated by incident reporting systems and clinician debriefings is difficult to aggregate and labor intensive to analyze, thus limiting its clinical applicability [16, 28]. Although numerous taxonomies have been developed, none is currently widely used. Most hospitals also lack processes or protocols for analyzing or acting upon aggregated report data. This leads to a lack of feedback that discourages further event reporting by health professionals. We applied the ICPS to reports from a web-based safety debriefing. Other fields, such as hospital medicine and radiology, have previously applied the ICPS successfully, but to our knowledge this is the first study to apply it to a surgical specialty [15, 16].

The coding of the 320 web-debriefing from 325 transplant surgeries based on the ICPS allowed the identification of recurrent (>10) system and process vulnerabilities that would otherwise not be captured: 41 clinical process/procedure issues concerning procedures/treatment/intervention, 30 resources/organizational management issues concerning human resources/staff availability/adequacy, 22 medical device/equipment issues concerning failure/malfunction, 14 documentation issues concerning charts/medical records/assessment/consultations, 13 resource/organizational management issues concerning bed/service availability/adequacy and resources/organizational management issues concerning organization of teams/people. These findings highlight recurrent vulnerabilities related to liver and kidney transplantation and, therefore, provide excellent opportunities for targeted interventions. For example at a hospital level, the recurrence of device/equipment failure/malfunction can be targeted through engagement of the device company for troubleshooting and route cause analysis [29].

It is widely accepted that the ICPS is a conceptual framework rather than a system of classification. As the WHO works to advance the practical application of the ICPS through projects such as the MIM-PS, several unique aspects of surgical care should be considered [24, 30].

The structure of the debriefing tool allows for free-text responses regarding issues that threaten patients' safety. While this contributes to the richness of information collected, we found a high incidence of repetition of issues by respondents. This repetition led to frequent double coding of issues by coders. This led us to adopt a dual-reconciliation process for our codes, beginning first with reconciliation of issue identification then progressing to coding of individual issues applying the ICPS code, followed by a second reconciliation.

While a large number of issues reported in debriefings were coded as ‘resources/organizational management’ the challenging remains interpret the secondary codes, (e.g. ‘matching of workload management’, ‘bed/service availability/adequacy’), as they do not reflect which issues are due to the institutional culture or the result of unpredictable events such as prolonged surgical cases and staff shortages. Hence, the strength of the ICPS categorization of the data is to highlight which areas of care experience recurrent issues, with most reliability given to the primary code (e.g. clinical process/procedure vs resource/organizational management). While the secondary codes offer insight into the types of vulnerabilities that occur, they do require further route cause analysis prior to targeted interventions.

Medical devices and equipment codes required manual reconciliation of mentioned device names. In addition, the distinction between which devices augment the care of the patient and which are integral to the surgery itself (grafts, prosthetics) was lost when the data was reviewed in aggregate; however, these can be easily examined further by the appropriate parties. The coding provided excellent categorization of the reported vulnerabilities and a high-level overview.

Delays were shown to be relevant as primary safety incidents, contributing factors to safety incidents and outcomes of incidents. Distinction of these three types of delays, as well as direct vs indirect delays in surgical care, would help target interventions.

Communication with operating room staff requires unique processes due to the sterility requirements. There are two separate thoroughfares of communication that occur simultaneously: the communication between team members within the operating room and the communication into the operating room via cell phones and pagers. Communication between team members in the operating room was most frequently coded as ‘behavior’, while communication into the operating room via phone or pager was most commonly perceived as a distraction or interruption. There are no codes for distraction within the ICPS, so our method was to attempt to report the incident that occurred subsequent to the page or phone call, with the method of communication as a contributing factor. Similarly, ‘Transfer of Care’ and ‘Handoff’ codes failed to reflect the complexity of communication and various settings under which these situations occur. Differentiate in aggregate between perioperative transfers of care (pre-operative holding to the operating room, operating room to the PACU or ICU) and handoffs that occur following perioperative events (shift change).

Trainee-faculty interactions involve significant overlap between behavior, cognition and skill in influencing trainee performance. Debriefing reports commented on the degree of trainee-autonomy afforded by the faculty, as well as interpersonal interaction. Because this is a necessary but sometimes challenging experience to witness, we were hesitant to combine reports focused on trainee-faculty interaction with those reports focused on skill-based error, such as Foley catheter misplacement or inadequate resuscitation. A trainee-related code may be warranted moving forward.

This analysis has the following limitations: we applied the ICPS and developed our codebook based on debriefing reports specific to abdominal organ transplant at a single institution. These patients serve as an excellent model for surgical patients due to the complexity of the procedure and the polymorbidity of the patients; however, the codebook will require further validation in other specialties and at other institutions. In addition, the ICPS is vast and incorporates several other layers including mitigating factors and patient outcomes. We did not apply the whole framework, but rather focused on incident types and contributing factors. Finally, compared with active surveillance methods such as medical record review, in-person debriefings and direct observation, passive surveillance methods such as event reporting and web-based debriefings capture only a fraction of safety events [31, 32]. These passive methods may not reliably characterize the epidemiology of safety problems.

In summary, the ICPS is a framework for organizing patient safety concepts, now integrated into the MIM-PS along with the ICD and the ICF, Disability and Health. Our results demonstrate that templates such as the MIM-PS can be easily and consistently applied for surgeries to facilitate the aggregation, organization and analysis of reported patient safety incidents in a standardized fashion. The aggregate data were hypothesis generating and allowed for the identification of several institutional threats to patient safety not previously identified. Several unique aspects of surgical care require special consideration to ensure research findings can be disseminated across surgical specialties. The codebook developed by our group can be used as a framework for other specialties and institutions.

Funding

This study was supported by the Agency for Healthcare Research and Quality and National Institutes of Health/National Institute of Diabetes and Digestive and Kidney Diseases T32 Training Grants (McElroy 5T32HS000078-15, T32DK077662).

Conflict of interest statement

None declared.