Adverse Events in Pediatric Critical Care Nonsurvivors With a Low Predicted Mortality Risk: A Multicenter Case Control Study*

OBJECTIVES:

Some patients with a low predicted mortality risk in the PICU die. The contribution of adverse events to mortality in this group is unknown. The aim of this study was to estimate the occurrence of adverse events in low-risk nonsurvivors (LN), compared with low-risk survivors (LS) and high-risk PICU survivors and nonsurvivors, and the contribution of adverse events to mortality.

DESIGN:

Case control study. Admissions were selected from the national Dutch PICU registry, containing 53,789 PICU admissions between 2006 and 2017, in seven PICUs. PICU admissions were stratified into four groups, based on mortality risk (low/high) and outcome (death/survival). Random samples were selected from the four groups. Cases were “LN.” Control groups were as follows: “LS,” “high-risk nonsurvivors” (HN), and “high-risk survivors” (HS). Adverse events were identified using the validated trigger tool method.

SETTING:

Patient chart review study.

PATIENTS:

Children admitted to the PICU with either a low predicted mortality risk (< 1%) or high predicted mortality risk (≥ 30%).

INTERVENTIONS:

None.

MEASUREMENTS AND MAIN RESULTS:

In total, 419 patients were included (102 LN, 107 LS, 104 HN, and 106 HS). LN had more complex chronic conditions (93.1%) than LS (72.9%; p < 0.01), HN (49.0%; p < 0.001), and HS (48.1%; p < 0.001). The occurrence of adverse events in LN (76.5%) was higher than in LS (13.1%) and HN (47.1%) (p < 0.001). The most frequent adverse events in LN were hospital-acquired infections and drug/fluid-related adverse events. LN suffered from more severe adverse events compared with LS and HS (p < 0.001). In 30.4% of LN, an adverse event contributed to death. In 8.8%, this adverse event was considered preventable.

CONCLUSIONS:

Significant and preventable adverse events were found in low-risk PICU nonsurvivors. 76.5% of LN had one or more adverse events. In 30.4% of LN, an adverse event contributed to mortality.

RESEARCH IN CONTEXT

• Adverse events occur in 22–76% of all PICU patients.

• The majority of PICU patients has a predicted mortality risk of less than 1%. Subsequently, a small portion of this group dies. Unplanned admissions and complex chronic disorders are associated with mortality in these “low-risk nonsurvivors.”

• The occurrence of adverse events in “low-risk nonsurvivors” is unknown. The contribution of adverse events to mortality in these patients is of great interest but still unknown.

WHAT THIS STUDY MEANS

• In three of four PICU “low-risk nonsurvivors,” one (or more) adverse event(s) occurred. They have more adverse events compared with low-risk survivors and compared with nonsurvivors with a high predicted mortality risk.

• More than 90% of the “low-risk nonsurvivors” has an underlying complex chronic disorders.

• In 30% of the “low-risk nonsurvivors,” an adverse event contributed to mortality.

Despite the introduction of several safety programs, adverse events (AEs) remain a great threat to modern healthcare, leading to patient harm, morbidity, increased healthcare costs, and even death. AEs occur in 22–76% of admissions in the PICU (1–5). Ninety percent of AEs in the PICU do not cause permanent harm (4). PICU mortality in affluent countries has decreased over the last decades to 2–4%, but PICU patients often have underlying complex chronic conditions, receive multiple drugs, need invasive supportive technologies, depend on many clinical decisions being made, and are at risk for iatrogenic harm (6).

Validated mortality prediction models like the “Pediatric Index of Mortality” (PIM) and “Pediatric Risk of Mortality” (PRISM) and their updates are used in the PICU for benchmarking and for research purposes (7–10). A significant portion of PICU patients has a low predicted mortality risk, as measured by these prediction models. Nevertheless, some of the “low-risk” patients die in the PICU. Unplanned admissions and underlying complex chronic conditions are known risk factors associated with mortality in this group (11–13). These factors increase the risk for mortality significantly, but it seems that more factors are involved in the death of the “low-risk” PICU population. Specifically, the contribution of AEs in these “unexpected deaths” is unknown but of great interest. Although one may expect that AEs mainly occur in the most complex, critically ill PICU patients, two small studies showed that low-risk PICU patients who die have a high occurrence of AEs (14, 15). In order to gain more insight into the occurrence and relevance of AEs in low-risk PICU nonsurvivors, we performed a nation-wide study in The Netherlands. More knowledge about the role of AEs might reveal opportunities to increase safety in the PICU.

The primary aim was to study the occurrence of AEs in PICU nonsurvivors with a low predicted mortality risk (low-risk nonsurvivors [LN])‚ compared to low-risk survivors (LS)‚ high-risk nonsurvivors (HN) and high-risk survivors (HS). Secondary aims were to compare the severity, preventability and nature of AEs between LN and LS and high-risk patients and to establish the contribution of AEs to mortality.

MATERIALS AND METHODS

Study Design and Setting

We conducted a case control study, in which admissions were selected from the national PICU registry containing anonymized information of all seven PICUs in The Netherlands (“Pediatric Intensive Care Evaluation” [“PICE-registry”]) (https://pice.nl/)/SKIC). The PIM2 and PRISM-II (further referred as “PRISM”) scores of all PICU admissions were collected, and the models were recalibrated to predict overall mortality in the 11-year cohort without altering the relative weights of the risk factors (12, 14, 16). Mortality in the database was registered as mortality during PICU admission (12).

Study Population

PICU admissions between January 1, 2006, and January 1, 2017, were stratified into four groups based on risk profile and outcome, comparable with previous studies (12, 14, 17). The study group consisted of LN, defined as “admissions with a mortality risk in the simply recalibrated PIM2 and/or recalibrated PRISM of <1% and PICU-nonsurvivor.” The three control groups consisted of “LS” (mortality risk < 1% and survivor), “HN” (mortality risk > 30% and nonsurvivor), and “HS” (mortality risk > 30% and survivor). Nonsurvivors were defined as patients who died during PICU admission. After stratification, a random sample of the four groups was selected by a computer-based randomizer (18).

The methodology was equal compared with a pilot study which was performed in two PICUs. Based on the results of a pilot study, with an anticipated occurrence of patients with greater than one AE of 80% in the LN group and 60% in the HS group, with alpha of 0.0167, beta of 0.2, and power 80%, 420 patients were needed (14). Anticipating 15% exclusions, a total of 4 × 125 (500) admissions were selected. To obtain sufficient patients in the LN group (n = 125), patients were selected from a large time frame. Inclusion criteria were children less than 18 years with PIM2 and PRISM scores. Exclusion criteria were patients who were admitted for palliative reason or who were brain dead at admission, premature patients, patients in whom the medical record was unavailable, or patients who did not fulfill criteria for being high or low risk after the PIM2 and PRISM scores were checked for errors. Details are shown in the additional file, Table S1 (http://links.lww.com/PCC/C245).

Data Collection

Data were collected using a validated two-staged record review method (4, 14).

The first stage of the analysis was performed by a team of three trained medical students and the primary investigator. The primary investigator is a pediatric intensivist with over 20 years of clinical experience. PIM2 and PRISM scores were checked for errors based on physiologic and laboratory data. Patient characteristics were extracted from the registry and from the medical record (Table S2, http://links.lww.com/PCC/C245). All medical records and nursing records were manually screened for potential AEs using a PICU trigger tool method which was adapted from Agarwal et al (4) and used in the exploratory study (Table S3, http://links.lww.com/PCC/C245) (14).

During the second stage, performed by the primary investigator, patient records were reviewed for diagnoses, health status at PICU admission, mode of death (if applicable), and AEs. For diagnosis classification, the diagnostic code list of the Australian New Zealand Pediatric Intensive Care society was used (19).

Health status of the patient at PICU admission was based on the presence of an underlying complex chronic condition (CCC) or non-CCC, according to a modified Feudtner’s list (Table S4, http://links.lww.com/PCC/C245) (11, 12, 14, 20). Because the presence of CCCs does not always differentiate between children with a short life expectancy and children who are able to survive for many more years, a tool to categorize life expectancy before PICU admission was developed. Life expectancy was based on patient history including CCCs and using professional judgment from the primary investigator (13, 21, 22). An expert panel of (pediatric) intensivists (J.A.H., J.v.d.H., J.L.) was available if problems were encountered in judgment of AEs.

Outcome Measures

Definitions and outcome measures are shown in Table 1. Primary outcome was the occurrence of AEs. An AE was defined as unintended injury that results in prolonged hospital stay, temporary or permanent disability, or death, caused by healthcare management rather than by the patient’s underlying disease process (23). Secondary outcomes were severity, preventability, nature and timing of AEs, and contribution of AEs to mortality.

The severity of AEs was rated according to the criteria of the National Coordinating Council for Medication Error Reporting and Prevention (24). Regarding grade I AEs (“contributed to or resulted in the patient’s death”), three subcategories were developed: I-1: “AE partially contributed to death,” I-2: “AE substantially contributed to death,” or I-3: “death completely caused by AE”. All AEs contributing to mortality were discussed within in the expert panel.

A preventable AE was defined as “an AE resulting from mismanagement due to failure to follow accepted practice at an individual or system level” (23). Accepted practice was taken to be “the current level of expected performance for the average practitioner or system that manages the condition in question,” using guidelines and protocols that were valid at that time/period (25). Preventability of AEs was scored using a six-point Likert scale. AEs with a preventability score of 4–6 were considered as preventable (21, 23).

AEs were grouped into nine categories, based on the classification made by Hogan et al (22), for example “clinical monitoring,” “drug or fluid related,” “infection related,” or “technical problems.” A category was added for extracorporeal membrane oxygenation and procedures taking place outside the PICU (“surgical procedure”) (Table 1). AEs that occurred before PICU admission and were related to the PICU admission, were included in the total number of AEs as “AE before PICU admission,” modified from the Canadian AE Study (23). As they occurred before and not during PICU admission, they were not incorporated in the AE rate (number of AEs/PICU day). Data that could not be retrieved were categorized as “missing.”

TABLE 1.

Definitions and Outcome Measures

AE
An unintended injury that results in temporary or permanent disability, death, or prolonged hospital stay and that is caused by healthcare management rather than by the patient’s underlying disease process.
Timing of AE
1	AEs that occurred during the index PICU admission.
2	AEs that occurred shortly before PICU admission and were related to the PICU admission, were scored as “AE before PICU.” “AEs before PICU admission were not incorporated in the AE rate.”
AE rate
Number of AEs occurring during PICU admission divided by PICU length of stay.
Severity of AEs according to National Coordinating Council for Medication Error Reporting and Prevention categories (33)
E	Contributed to or resulted in temporary harm to the patient and required intervention
F	Contributed to or resulted in temporary harm to the patients and required initial or prolonged hospitalization
G	Contributed to or resulted in permanent patient harm
H	Required intervention to sustain life
I	Contributed to or resulted in the patient’s death
	For category I, subcategories were developeda:
		I-1	Partially contributed to death
		I-2	Substantially contributed to death
		I-3	Death was completely caused by AE
Preventability
The degree of preventability of AEs was measured on a six-point Likert scale
1	(Virtually) no evidence for preventability
2	Slight to modest evidence of preventability
3	Preventability not quite likely (less than 50/50, but “close call”)
4	Preventability more than likely (more than 50/50, but “close call”)
5	Strong evidence of preventability
6	(Virtually) certain evidence of preventability
AEs with a preventability score of 4–6 were defined as preventable AEs.
Classification
Based on the classification made by Hogan et al (32)
Clinical monitoring			Failure to act upon results of tests or clinical findings, set up monitoring systems or respond to such systems or increase intensity of care when required.
Diagnosis			Missed, delayed or inappropriate diagnosis as a result of failure to perform an adequate assessment of patient’s overall condition including appropriate tests or lack of focused assessment when required.
Drug or fluid related			Side effects, inappropriate use, failure to give prophylactic care, anaphylaxis, etc.
Technical problems			Related to a device, an operation or procedure whether on ward, in a diagnostic situation or in theatre and including inappropriate or unnecessary procedures (other than technical problems related to extracorporeal life support).
ECMOb			Problems related to ECMO including technical problems, haemorrhage.
Infection related			Healthcare-associated infections including infections from indwelling device.
Resuscitation			Problems in resuscitation including cardiopulmonary resuscitation such as delay in beginning resuscitation, problems related to resuscitation technique, resuscitation medication/fluids, resuscitation equipment.
Surgical procedureb			Problems related to a procedure taking place during PICU admission but outside the PICU, e.g. an operation or heart catheterization (other than standard procedures performed in the ICU like intubations, insertion of central catheters, insertion of pneumothorax, ECMO cannulations etc.).
Other			Any other problem not fitting categories above or a combination of categories above.

AE = adverse event, ECMO = extracorporeal membrane oxygenation.

^a Modification from original National Coordinating Council for Medication Error Reporting and Prevention criteria.

^bModification from original classification by Hogan et al (32).

Data Analysis

Normal distribution of continuous variables was tested using sampling distributions and skewness and kurtosis tests. Skewed distributed data were reported by median and interquartile range (IQR) and were tested by nonparametric tests (Mann-Whitney U). For categorical variables, chi-square test was used (software: IBM statistics 22).

LN patients were compared with LS, HN, and HS patients. Because of multiple testing, a Bonferroni correction was applied, and therefore an alpha of 0.0167 was considered significant.

Reliability Study

To assess the reliability of the review process, a sample of 24 medical records was independently reviewed by a panel of three pediatric intensivists, for the presence and preventability of AE(s). The panel was not part of the core team and was blinded for the study results. A k-value between 0.00 and 0.20 was classified as “slight,” between 0.21 and 0.40 as “fair,” between 0.41 and 0.60 as “moderate,” between 0.61 and 0.80 as “substantial,” and between 0.81 and 1.00 as “almost perfect” (26).

Ethical Approval

The study protocol was approved by the Research Ethics Committee of the Radboud University Medical Center in Nijmegen (File number: 2017-3526). The committee waived the need for informed consent. Data were anonymized and handled according to the principles of good clinical practice. The collection of data started in 2018 and ended in 2021.

RESULTS

Patient Characteristics

The entire cohort contained 53,789 PICU admissions (mortality 3.0%), including 33,961 low risk admissions (mortality 0.5% [n = 180]) and 1,250 high risk admissions (mortality 48.2% [n = 603]) (Fig. 1, flowchart). In total, 419 and 81 unique patients were included and excluded, respectively. Five LN patients and one HN patient were also part of the pilot study (14). LN had more unplanned admissions (71.6%) than LS (35.5%) but less than HN (94.2%) and HS (91.5%) (Table 2). LNs were more often admitted outside office hours and were more often medical (nonsurgical) admissions compared with LS. The prevalence of complex chronic conditions was higher in LN (93.1%) than LS (72.9%), HN (49.0%), and HS (48.1%). A majority of LN (88.2%) had a shorter life expectancy before PICU admission. Many HNs (43/104 [41%]) were admitted after cardiac arrest preceding PICU admission. Mortality risk at admission of LN was slightly but significantly higher than LS and (by definition) lower than HN and HS. LN had a longer length of stay (LoS) (median [IQR], 10 d [5–27 d]) compared with LS (2 d [2–3 d]) and HN (3 d [2–6 d]) (p < 0.001). The mode of death between LN and HN was different. In HN, 39.4% of patients died because they were brain death. In 71.6% of LN, patients died after treatment was limited or withdrawn.

Figure 1.

Flowchart of the study. ^aIn total, 43 of 35,168 admissions had discrepancies between the mortality prediction models: they were low risk in one model and high risk in the second model, therefore fulfilling criteria for both low-risk and high-risk (e.g. “low-risk according to PIM2 and simultaneously high-risk according to PRISM”). Four admissions both LN and HN. Thirty-nine admissions both LS and HS. HN = high-risk nonsurvivors, HS = high-risk survivors, LN = low-risk nonsurvivors, LS = low-risk survivors, PIM2 = Pediatric Index of Mortality 2, PRISM = Pediatric Risk of Mortality.

TABLE 2.

Patient Characteristics

Characteristics, n	Low-Risk Nonsurvivors, N = 102	Low-Risk Survivors, N = 107	High-Risk Nonsurvivors, N = 104	High-Risk Survivors, N = 106
Gender: male	55 (53.9)	61 (57.0)	65 (62.5)	67 (63.2)
Age group
1–28 d	7 (6.9)	(5.6)	15 (14.4)	32 (30.2)b
29–365 d	33 (32.4)	21 (19.6)	29 (27.9)	28 (26.4)
1–4 yr	14 (13.7)	30 (28.0)	25 (24.0)	26 (24.5)
5–17 yr	48 (47.1)	50 (46.7)	33 (32.7)	20 (18.9)
Age, median (IQR) (yr)	3.5 (0.3–13)	4.0 (0.8–10)	2.0 (0.25–8)	0.7 (0.0–2.0)b
Weight, median (IQR) (kg)	15 (5–42)	17 (9–35)	13 (6–29)	8 (3–15)b
Socio economic status: low	18 (17.6)	21 (19.6)	19 (18.3)	20 (18.9)
Unplanned admission	73 (71.6)	38 (35.5)b	98 (94.2)b	97 (91.5)b
Cardiac arrest before PICU admission	0 (0)	0 (0)	43 (41.3)b	17 (16.0)b
Medical admission	80 (78.4)	44 (41.1)b	84 (80.8)	75 (70.8)
Admission outside office hours	50 (49.0)	31 (29.0)a	66 (63.5)	64 (60.4)
Readmission within 48 hr	4 (3.9)	1 (0.9)	0 (0.0)	1 (1.0)
Chronic condition
Complex chronic condition	95 (93.1)	78 (72.9)a	51 (49.0)b	51 (48.1)b
Noncomplex chronic condition	3 (2.9)	15 (14.0)	5 (4.8)	6 (5.7)
No chronic condition	4 (3.9)	14 (13.1)	48 (46.2)	49 (46.2)
Health status before PICU admission
Healthy	4 (3.9)	15 (14.0)b	47 (45.2)b	48 (45.3)b
Chronic condition, normal life expectancy	6 (5.9)	51 (47.7)	12 (11.5)	16 (15.1)
Chronic condition, shorter life expectancy	90 (88.2)	41 (38.3)	42 (40.2)	40 (37.3)
Unknown	2 (1.9)	0 (0.0)	3 (2.9)	2 (1.9)
Recalibrated Pediatric Index of Mortality 2 mortality risk, median (IQR) (%)	1.1 (0.9–4.3)	0.9 (0.4–1.7)b	41 (16–71)b	21 (7–39)b
Recalibrated Pediatric Risk of Mortality risk, median (IQR) (%)	0.8 (0.6–2.4)	0.6 (0.4–0.9)b	45 (29–65)b	37 (12–51)b
Mechanical ventilation	94 (92.1)	52 (48.6)b	102 (98.1)	100 (94.3)
Ventilator days, median (IQR)	7 (3–20)	0 (0–1)b	3 (2–6)b	6 (3–12)
Length of stay, median (IQR) (d)	10 (5–27)	2 (2–3)b	3 (2–6)b	8 (5–19)
Mode of death
Brain death	10 (9.8)		41 (39.4)b
Maximal treatment including CPR	9 (8.8)		10 (9.6)
Maximal treatment without CPR	10 (9.8)		11 (10.6)
Limiting or withdrawal of therapy	73 (71.6)		42 (40.4)

CPR = cardiopulmonary resuscitation, IQR = interquartile range.

^ap < 0.01 compared with low-risk nonsurvivor (LN).

^bp < 0.001 compared with LN.

All numbers are expressed as the number of patients (% column) unless specified otherwise.

Adverse Events

In total, 196 AEs were found in 78 of 102 LN patients (76.5%) (Table 3). The occurrence of AE in LN was higher compared with LS (13.1%) and HN (47.1%) (p < 0.001) and not significantly different from HS (67.0%). The AE rate of LN (median [IQR], 10.00 [0.00–19.05] AEs/100 d) was higher compared with LS (0.00 [0.00–0.00]) in LS and not significantly different from HN (0.00 [0.00–16.15]) and HS (5.90 [0.00–14.29]). Of all AEs in LN, 31.1% was preventable. No significant difference in preventability was found between the groups.

TABLE 3.

Outcome—Adverse Events

Outcome Measure	Low-Risk Nonsurvivors	Low-Risk Survivors	High-Risk Nonsurvivors	High-Risk Survivors
Patients
Patients, n	102	107	104	106
Patients with > 1 AE, n (%)	78 (76.5)	14 (13.1)b	49 (47.1)b	71 (67.0)
Number of AE/ patient, median (IQR)	1 (1–3)	0 (0–0)b	0 (0–1)b	1 (0–2)
AE rate (no/100 d) median (IQR)	10.00 (0.00–19.05)	0.00 (0.00–0.00)b	0.00 (0.00–16.15)	5.90 (0.00–14.29)
Patients with > 1 AE contributing to death	31 (30.4)		27 (26.0)
Patients with > 1 preventable AE contributing to death	9 (8.8)		10 (9.6)
AEs
Total number of AEs, n	196	21b	86b	161
Timing of AEs, n (%)   Before PICU admission	7 (3.6)	6 (28.6)b	20 (23.5)b	19 (11.8)a
During PICU admission	189 (96.4)	15 (71.4)	65 (76.5)	142 (88.2)
AE severity, n (%)
Grade E (temporary harm)	117 (59.7)	14 (66.7)b	40 (46.5)	109 (67.7)b
Grade F (prolonged hospitalization)	4 (2.0)	4 (19.0)	1 (1.2)	7 (4.3)
Grade G (permanent harm)	2 (1.0)	1 (4.8)	1 (1.2)	8 (5.0)
Grade H (intervention to sustain life)	41 (20.9)	2 (9.5)	17 (19.8)	37 (23.0)
Grade I (contributing to death)	32 (16.3)	0 (0.0)	27 (31.4)	0 (0.0)
I-partially	10 (5.1)		7 (8.2)
I-substantially	19 (9.7)		20 (23.5)
I-completely	3 (1.5)		0 (0.0)
AE preventability, n (%)
Not preventable	131 (66.8)	10 (47.6)	43 (50.0)	103 (64.0)
Preventable	61 (31.1)	11 (52.4)	36 (41.9)	56 (34.8)
Unknown	4 (2.0)	0 (0.0)	7 (8.1)	2 (1.2)
AE classification, n (%)
Clinical monitoring	10 (5.1)	3 (14.3)a	16 (19.3)a	13 (8.1)
Diagnosis	4 (2.0)	1 (4.8)	3 (3.6)	5 (3.1)
Drug/fluid related	33 (16.8)	7 (33.3)	13 (15.7)	33 (20.6)
Technical problems	15 (7.7)	2 (9.5)	10 (12.0)	17 (10.6)
Extracorporeal membrane oxygenation	11 (5.6)	0 (0.0)	7 (8.4)	9 (5.6)
Surgical procedure	11 (5.6)	3 (14.3)	7 (8.4)	17 (10.6))
Infection related	65 (33.2)	4 (19.0)	15 (18.1)	39 (24.4)
Resuscitation	2 (1.0)	0 (0.0)	1 (1.2)	2 (1.3)
Other	45 (23.0)	1 (4.8)	11 (13.3)	25 (15.5)

AE = adverse event, IQR = interquartile range.

^ap < 0.01 compared with low-risk nonsurvivor (LN).

^bp < 0.001 compared with LN.

All numbers are expressed as the number of AEs (% column) unless specified otherwise.

LN suffered from more severe AEs compared with LS and HS, including 41 of 196 AEs grade H (20.9%) (intervention needed to sustain life) and 32 of 196 AEs (16.3%) contributing to death (grade I).

Details of the AEs that contributed to death in LN are presented in Table 4. In 31 of 102 LN patients (30.4%), an AE contributed to death, including 8.8% having a preventable AE. In three LN patients, death was completely caused by an AE. In one of these patients, the AE was considered preventable. In 18 LNs, an AE substantially contributed to death (of which five were preventable), and in nine LN, an AE partially contributed to death (three preventable).

TABLE 4.

Adverse Events (n = 32) Contributing to Death in Low Risk Nonsurvivors (N = 31)^a

ID	Severity	Prev	Class	Description of the adverse event	CCC	Description CCC
1	I-3	Y	surg	Severe hypotension during elective cardiac catheterization leading to intestinal necrosis	CCC	cong heart dis
2	I-3	Unk	surg	Massive hemorrhage after tear in atrium after atrial septal defect repair	nCCC	cong heart disd
3	I-3	N	other	Occlusion of pulmonary arteries after cavo pulmonary shunt and repair of pulmonary artery	CCC	cong heart dis
4	I-2	Y	mon	Resuscitation during MRI inpatient on high flow oxygen with respiratory insufficiency	CCC	hem dis
5	I-2	Y	mon	Cardiac arrest inpatient with asthma during PICU admission	nCCC	asthma
6	I-2	Unk	mon	Sudden circulatory collapse with electrocardiogram abnormalities, leading to death	CCC	leukemia
7	I-2	Y	diagn	Missed diagnosis of pulmonary mycosis	CCC	hemat dis
8	I-2	Y	ECMO	Suction of heparin in ECMO system	CCC	cong heart dis
9	I-2	N	ECMO	Asystole after replacement of artificial kidney on ECMO	CCC	cong heart dis
10	I-2	Y	inf	CLABSI inpatient with short bowel	CCC	short bowel
11	I-2	N	inf	Hospital acquired pneumonia after spinal surgery, underlying severe psychomotor retardation	CCC	chrom abn
12	I-2	N	inf	Ventilator acquired pneumonia	CCC	cong heart dis
13	I-2	N	inf	Septic shock acquired during PICU admission	CCC	epilepsy
14	I-2	N	inf	Septic shock acquired during PICU admission	CCC	chrom abn
15	I-2	N	inf	Septic shock acquired during PICU admission	CCC	cong brain dis
16	I-2	N	inf	Sepsis, pulmonary hypertension inpatient with high output stoma and multiple abdominal adhesions	CCC	syndrome or malformation
17	I-2	N	inf	Aspergillus infection	CCC	neoplasm
18	I-2	N	surg	Thrombosis left ventricular assist device, resuscitation followed by multiple organ failure and cerebral infarction	CCC	cardiomyopathy
19	I-2	N	surg	Thrombi in Fontan circuit ultimately leading to death	CCC	cong heart dis
20	I-2	N	other	Resuscitation during intubation inpatient with underlying cong heart dis	CCC	cong heart dis
21	I-2	N	other	Abdominal compartment syndrome inpatient with typhlitis	CCC	leukemia
22	I-1	Y	mon	Delay of intervention in dysfunction of intraventricular drain	CCC	brain tumor
23	I-1	N	drug	Possible allergic reaction, leading to deterioration of fragile respiratory balance	CCC	hem dis
24	I-1	N	drug	Pulmonary veno-occlusive disease after chemotherapy	CCC	neoplasm
25b	I-1	N	drug	Liver insufficiency, possibly iatrogenic (medication) or septic	CCC	chrom abn
25	I-1	N	inf	CLABSI inpatient with infected intravascular thrombi	CCC	chrom abn
26	I-1	N	inf	Resuscitation inpatient with pulmonary mycosis	CCC	leukemia
27	I-1	N	inf	Possible CLABSI on ECMO leading to forced decannulation	CCC	cong lung dis
28	I-1	Y	inf	Systemic fungal infection inpatient with neutropenia, no prophylaxis given	CCC	neoplasm
29	I-1	Y	other	Cerebral herniation partly due to osmotic changes with continuous veno-venous hemofiltration and compression of jugular vein by central venous catheter	CCC	cdh
30	I-1	N	other	Lung bleeding, partially caused by mechanical ventilation with large tidal volumes (20 mL/kg)	CCC	cong lung dis
31	I-1	N	other	Cerebral ischemia due to several episodes of hypotension	CCC	chrom abn

CCC = complex chronic condition, cdh = congenital diaphragmatic hernia, chrom abn = chromosomal abnormality, CLABSI = central catheter–associated blood stream infection, cong brain dis = congenital brain disease, cong heart dis = congenital heart disease, cong lung dis = congenital lung disease, ECMO = extracorporeal membrane oxygenation, hemat dis = hemataologic disease, nCCC = noncomplex chronic condition‚ neoplasm = malignant solid organ neoplasm.

^aOne patient (patient ID 25) had two adverse events partially contributing to death.

^bMany complex congenital heart diseases are CCC, some simple congenital heart diseases are nCCC.

Severity of adverse event: I-3: death was completely caused by adverse event; I-2: substantially contributing to death; I-1: partially contributing to death. Prev: Preventability of adverse event: Y: preventable; N: not preventable; Unk: preventability unknown. Class: Classification of adverse event: mon, clinical monitoring; diagn, diagnosis; drug, drug or fluid related; inf, infection related; surg, surgical procedure; other, other.

Most prevalent AEs in LN were infection-related AEs (33.2%) and drug/fluid-related AEs (16.8%). Details on severity, preventability, and classification of all AEs are shown in Tables S5 and S6 (http://links.lww.com/PCC/C245). Most preventable AEs both in LN and other groups were related to “infections,” “drugs/fluids,” and “clinical monitoring.” The number of AEs during the years remained stable (Supplementary Figs. 1 and 2, http://links.lww.com/PCC/C245).

Interobserver Variability Study

The interobserver agreement of the determination of AEs was almost perfect (κ = 0.83), and agreement on preventability of AEs was moderate (κ = 0.60). Results are shown in Table S7 (http://links.lww.com/PCC/C245).

DISCUSSION

In this multicenter study, a significant number of AEs was found in a PICU subpopulation of LNs. In total, 76.5% of LN suffered from an AE, of which one third was preventable. The occurrence of AEs in the LN group was higher compared with the LS and HN groups and not different from AEs in the HS group. Most AEs were infections or drug/fluid-related AEs. In 30.4% of LN, an AE contributed to death, and in 8.8% of LN, a preventable AE partially contributed to death.

This is a large study determining the contribution of AEs to unexpected deaths among PICU patients. The study population was derived from a large cohort representing all Dutch PICU admissions. The trigger-tool is a validated and commonly used method to detect AEs. Interobserver variability on the presence and preventability of AEs was relatively high compared with other studies (22, 27, 28). However, our study does also have limitations.

First, there is no gold-standard for low risk for mortality (12, 15). A combination of low PIM2 and/or low PRISM mortality risk was used to classify low risk patients. The overall performance of the prediction models in our large PICU cohort was reasonably well. The mortality rate among the cohort of low-risk patients was 0.5%. LN consist of a small subgroup from all low risk patients and have different characteristics compared with LS. Some factors might influence mortality risk prediction in certain subgroups (16). In long-stay patients, such as many LN and HS, mortality prediction models perform less well, since the used variables in the prediction models are measured early after initial admission (16). Dynamic changes occurring after the first 24 hours are by definition not incorporated in the prediction models. Perhaps more importantly, the majority of LN had an underlying CCC, not being reflected in the mortality prediction models. Over the last decades, an increasing number of CCC patients, with a higher mortality rate and longer LoS, is being admitted to the PICU (11, 29). We modified the original list of CCCs based on a study performed in 2012 (11, 20). There has been a 2014 update from the list of CCCs that we did not incorporate in our study (30). However, our list of CCCs reflects many diagnoses incorporated in the updated list. Even though the real mortality risk for LN was higher than presumed, we think that it is worthwhile to develop methods to discover a cohort of “unexpected deaths” and subsequently evaluate quality of care in these patients. Awareness of the possible role of AEs in outcome of children with a low predicted mortality risk but with a CCC is the first step in quality improvement.

Second, there were large differences in patient characteristics between the groups. This may, in part, explain the difference in AE occurrence. As mentioned before, both LN and HS had a long LoS. It is difficult to determine in retrospect whether AEs caused a longer LoS or the longer LoS led to more AEs. Not many AEs resulting in prolonged hospitalization (grade F) were found, but in retrospect it is difficult to estimate if an AE was the cause of a longer LoS. The difference in LoS does not explain the complete difference in occurrence of AEs. If we correct the number of AEs for LoS by using the AE rate, it was higher in LN compared with LS and not significantly different from HN and HS. There were significant differences in the mode of death between LN and HN. The majority of LN died after therapy was restricted or withdrawn. This decision was often made after a long PICU stay. The patients were not admitted with do-not-resuscitate orders at the time of PICU admission. In some cases, the decision was influenced by injuries caused by AEs.

Third, a general weakness of retrospective studies is hindsight bias (31). The primary investigator, who performed both the categorization of life expectancy and determined the presence of AEs, was not blinded to the study group. Knowledge of the outcome of the patient might influence judgment of severity and preventability of AEs. By using clear definitions and a predefined, validated trigger tool, using a panel of intensivists for questions and judgment of preventability and an interobserver reliability study, we tried to avoid the effects of hindsight bias.

Fourth, during the study period, safety programs were developed and implemented. Theoretically it is possible that during the study period, the prevalence of AEs declined. The study was not powered to analyze the occurrence of AEs during different time frames. We did not see a decline on the number of AEs during the years. It is likely that the prevalence of AEs has not changed.

The severity of AEs found in our study contrasts with several studies in the general PICU-population who mainly found low grade AEs (2, 4). In a cross-sectional multicenter study, 62% of all PICU patients had at least one AE, and 10% of the found AEs were classified as severe (contributing to permanent harm or worse) (4). In our study, the percentage of severe AEs was higher among LN (38%), HN (52%), and HS (28%). The higher occurrence of AEs and the more severe AEs in our study can be explained by differences in case-mix. In order to get an effective study sample, we did not randomly select patients from the total cohort but stratified patient categories and selected a relatively high proportion of LN, HN, and HS patients. The proportion of LS patients (with few AEs) was low, and the “intermediate” risk group was not represented at all in our study. Therefore, our results cannot be generalized to the total PICU-population.

Only a few small studies focused on LN. The results of the present study are consistent with our previous study and with another single-center study on LN (14, 15). In the study by Ruegger (15), LN had four times more AEs than LS, although the cut off point for “low-risk” (PIM2 mortality risk < 10%) was different compared with our study. LN seem to be associated with serious AEs, including preventable AEs. Evaluating deaths and especially “unexpected deaths” is an efficient way to obtain valuable information on iatrogenic harm (32‚ 33).

What this study adds is more insight into the occurrence of AEs in low-risk PICU nonsurvivors and their contribution to mortality. PICU deaths are often multifactorial. AEs contribute to death almost a third of LN, but the degree of the contribution to death may vary. Are deaths of LN preventable? In one patient death could be considered avoidable, since a preventable AE was completely responsible for death. In five and three LN, respectively, a preventable AE substantially or partially contributed to death, so death might be possibly preventable. Underlying complex chronic conditions seem to play a role in death of LN. Patients with chronic conditions may be “sicker” than predicted by the standard PICU severity of illness models. But, this cannot explain fully why LN die. Although CCCs are present in more than 90% of LN, they are also present in more than 70% of LS and therefore cannot explain the huge difference in AE occurrence between these groups.

Despite safety programs that have been developed over the last decades, our results show that there is still a large potential for improvement. One third of the AEs was considered preventable, which is comparable with other studies (2, 4, 23). So far, safety programs have succeeded to a certain extent. Although quality improvement programs have been implemented extensively in The Netherlands over the last decades‚ preventable AEs were still encountered (34).

Future research might focus on the interaction between CCCs and AEs. We have seen examples of patients where a CCC makes a patient more prone for AEs, for example patients with immune disorders who are more prone to hospital-acquired infections. It would be interesting to study further the interaction between CCCs and AEs.

Increasing patient safety remains an urgent but complex task. The focus of patient safety shifts more and more from what goes wrong (“Safety-I”) to why things go right (“Safety-II”) (35). A key to a safer PICU might be the development of resilient teams, capable of acting in a complex setting.

CONCLUSIONS

Significant and preventable AEs were found in low-risk PICU nonsurvivors. 76.5% of LN had one or more AEs. In 30.4% of LN an AE contributed to mortality.

ACKNOWLEDGMENTS

This work would not have been possible without collaboration between seven PICUs in The Netherlands, the SKIC (“Stichting Kinder Intensive Care”, Dutch Collaborative PICU Research Network) and Pediatric Intensive Care Evaluation (PICE) working group. We would like to thank Idse Visser, data scientist of the PICE-registry for his contribution on the stratification and randomization process, Cynthia van der Starre for her contribution to the PICU trigger tool and prof. Dr. J. Legemaate and prof. Dr. B. Bos, for their contribution as “confidential committee.”