Prevalence and severity of patient harm in a sample of UK-hospitalised children detected by the Paediatric Trigger Tool

Abstract

The measurement and examination of adverse events (AEs) that occur in children during hospital admissions is essential if we are to prevent, reduce or ameliorate the harm experienced. The UK Paediatric Trigger Tool (UKPTT) is a method of retrospective case note review that measures harm in hospitalised children.

Objectives

To examine the harm resulting from the processes of healthcare in hospitalised children from centres providing data to the National Health Service (NHS) Institute UKPTT data portal, to understand the positive predictive values of triggers and to make recommendations for the further development of the trigger tool.

Setting

25 hospitals across the UK, including secondary, tertiary and quaternary paediatric centres.

Participants

Randomly selected children who were admitted to hospital for longer than 24 h.

Outcome measures

The primary outcome measure was the rate of harm (the percentage of children experiencing one or more AEs during a hospital admission). Secondary measures were the severity of harm and performance of triggers.

Results

Data from 3992 patient admissions were reviewed across the hospitals and submitted to the trigger tool portal from February 2008 to November 2011. At least one AE was reported for 567 (14.2%) patients, with 211 (5.3%) experiencing more than one event. There were 1001 AEs identified. Where harm occurred, it was considered temporary for 923 (92.2%) AEs; however, 43 (4.3%) AEs resulted in the need for life-sustaining interventions, 18 (1.8%) AEs led to permanent harm and for 17 children (1.7% of AEs) the AE was believed to have contributed to death.

Conclusions

There is a significant, measurable level of harm experienced by children admitted to hospitals in the UK. While most of this harm is temporary, some of it is serious. The UKPTT offers organisations the means to measure and examine the AEs occurring in their hospital in order to reduce harm.

Strengths and limitations of this study.

• This study estimates that one in seven children experience harm during admission to hospital in the UK. Most of this harm is temporary, but a significant minority is serious. This should generate discussion about patient safety in paediatrics.

• The study used the UK Paediatric Trigger Tool that can be used by any hospital to learn about harm in order to prevent or reduce it.

• The study does not distinguish between preventable and non-preventable harm; however, we believe that there is an opportunity to learn from any harm event.

Introduction

The provision of care that is safe and reliable is a fundamental goal of modern healthcare. Patient safety is the prevention, reduction and amelioration of medical harm.^{1 2} Medical harm (synonymous with the terms patient harm and adverse event, AE) is defined as unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalisation or that results in death.³ Efforts to improve patient safety have been hampered by a lack of reliable data on the prevalence and nature of harm in all areas of practice. Patients and healthcare professionals need to understand the burden of harm in healthcare in order to develop effective interventions.⁴

Development of Trigger Tool methodology

The Global Trigger Tool (GTT) for measuring harm was developed by the Institute for Healthcare Improvement for use in adult care settings.³ Trigger tools have also been developed for specific populations and settings, including acute hospitals, surgery,⁵ critical care⁶ and primary care.⁷ One study used the GTT to measure harm at a large academic children's hospital in the USA and recommended the development of a paediatric specific tool.⁸ Paediatric-specific trigger tools have been developed for neonatology,⁹ paediatric critical care,^{10 11} medications^{12 13} and a general trigger tool for harm in hospitalised children.¹⁴ A UK version of the acute adult GTT had already been developed, but this was not applicable to a paediatric population. In 2008, the National Health Service (NHS) Institute for Innovation and Improvement undertook to develop a UK Paediatric Trigger Tool (UKPTT) that could be applied to all levels of acute paediatric care.

Trigger tool method for reviewing case notes

The trigger tool method is a retrospective review of 20 sets of healthcare records each month, using a standardised methodology. A random sample of 20 inpatient case notes is selected using a randomisation matrix on a monthly basis. The healthcare record is examined in a structured process for 20 min to search for ‘triggers’. A ‘trigger’ is a predefined event that alerts the reviewer to the possibility of patient harm. Once a trigger is identified, the reviewer uses clinical expertise to examine the records in more detail to understand the circumstances around the event. If harm is suspected, a second reviewer (usually a physician) is consulted to confirm and grade the AE using the National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) grading system (table 4).¹⁵

Table 4.

Severity of adverse events

NCC MERP Grade15	Descriptor	Adverse events	Total adverse events (%)
E	Temporary harm to the patient and required intervention	605	60.4
F	Temporary harm to the patient and required initial or prolonged admission	318	31.8
G	Permanent patient harm	18	1.8
H	Intervention required to sustain life	43	4.3
I	Patient death	17	1.7
Total		1001

National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) Index.²³

An example of a trigger is the administration of the antidote medication naloxone (a trigger) to reverse the effects of opiates. This alerts the reviewer to a possible overdose of opioids. The reviewer examines the relevant parts of the healthcare record to assess whether the use of naloxone was for this reason or not. If this is the reason, then the harm is graded (see online supplementary file).

Development of the UKPTT

In 2008, the NHS Institute sponsored the development of a Paediatric Trigger Tool because at the time there was no tool available for hospitalised children. The tool design was informed by the early (and prepublication) findings of a Canadian Paediatric Trigger Tool (CPTT) study,¹⁴ and the UK Acute Trigger Tool for adults.¹⁶ The development was a coproduction involving the collaboration of patient safety experts from the NHS Institute, international leaders in Paediatric Trigger Tool development, and clinical experts from nine UK hospitals including children's hospitals and district general hospitals. Following discussion and testing, the group agreed on 40 paediatric oriented triggers to be included in the UKPTT. These were based on the triggers used in other tools, UK evidence of AEs and the experience of the reference group in harm and AEs (a subset of the coproduction group). Production of a UK tool was intended to enhance ownership by the clinicians, who would use it in practice and to modify triggers that were not appropriate for the UK setting. We also added a category for ‘other harm’ to capture harm that was not detected by one of the listed triggers.

The UKPTT advocates a working definition of patient harm as ‘anything, which you would not like to happen to yourself or a member of your family as a result of, or contributed to by, medical care’. The decision to aim for a broad definition was to focus on the patient rather than on the medical system—a less defensive approach. This is a broader definition than that given by Griffin and Resar³ or the Canadian tool and aimed to encourage clinicians to explore a holistic concept of harm than that traditionally reported. It allows the inclusion of acts of omission as well as commission. The definition includes missed or delayed diagnosis along with physical and psychological harm.

Through the coproduction, support was developed for UKPTT users such as face-to-face training, online and printed guidance and standardised data collection forms.

Data collection

As part of the UKPTT development, the NHS Institute created a web-based trigger tool portal into which participating hospitals entered anonymised data. The portal calculated harm rates and produced run charts that hospitals could download. Contributing hospitals consented to their data being collated and published to further the understanding of harm in hospitalised children in the UK. Participating hospitals developed local administrative and governance arrangements for PTT reviews following the standard guidance (see online supplementary file).

Aims and methodology

The aims of this study are to:

1. Describe the rate and severity of harm occurring in hospitalised children from UK centres submitting data to the NHS Institute's Trigger Tool portal.

2. Report the frequency and positive predictive value (PPV) of triggers to detect harm.

3. Make recommendations for further application and development of the tool.

Participating hospitals, which voluntarily decided to use the PTT, included secondary, tertiary and quaternary centres. Reviewers were trained in trigger tool methodology either by experts at the NHS Institute or by using online resources with telephone support. Data were collected through the online trigger tool portal that opened in February 2008.

Results

Data from 3992 case note reviews from 25 hospitals submitted to the trigger tool portal between February 2008 and November 2011 were analysed. Nine of the hospitals were children's hospitals; the remainder was classed as district general hospitals. Data from four additional hospitals that used the portal were excluded because they each submitted less than 10 case entries. Harm was recorded as occurring for 567 patients (14.2%) while the majority (85.8%) of patients experienced no evidence of harm. Reviewers identified 1001 AEs, an average of 1.8 events per patient experiencing harm.

There was considerable variation between hospitals in the number of case notes reviewed (12–622), the number of triggers detected (17–1877) and the overall harm rate reported (0—73.3%). Results from each hospital are reported in table 1. Of the 567 children who suffered an AE, the majority (n=356, 63%) experienced a single event. However, 211 (37%) patients suffered more than one AE within the same admission. One patient was reported to have suffered 10 AEs in a single admission. A summary of the number of AEs per case is presented in table 2.

Table 1.

Number of case reviews, positive triggers and adverse events by individual hospital

Hospital	Case notes reviewed	Positive triggers	Average number of triggers per case note review	Adverse events (AEs)	Average number of AEs per case note review	Number of individual patients harmed (%)
A	622	1877	3.02	309	0.50	162 (26)
B	369	579	1.57	66	0.18	31 (8)
C	321	415	1.29	60	0.19	37 (11.5)
D	309	481	1.56	117	0.38	49 (15.9)
E	285	414	1.45	84	0.29	43 (15.1)
F	271	454	1.68	112	0.41	54 (20)
G	260	484	1.86	48	0.18	39 (15)
H	241	418	1.73	6	0.02	4 (1.7)
I	195	432	2.22	14	0.07	14 (7.2)
J	195	52	0.27	3	0.02	3 (1.5)
K	190	446	2.35	45	0.24	40 (21)
L	124	173	1.40	17	0.14	15 (12.1)
M	71	52	0.73	0	0.00	0 (0)
N	70	171	2.44	8	0.11	7 (10)
O	68	141	2.07	8	0.12	7 (10.3)
P	68	107	1.57	1	0.01	1 (1.5)
Q	66	79	1.20	15	0.23	11 (16.7)
R	62	84	1.35	15	0.24	12 (19.3)
S	60	121	2.02	32	0.53	15 (25)
T	59	90	1.53	2	0.03	1 (1.7)
U	23	22	0.96	4	0.17	3 (13)
V	19	14	0.74	1	0.05	1 (5.2)
W	17	26	1.53	4	0.24	4 (23.5)
X	15	50	3.33	27	1.80	11 (73.3)
Y	12	17	1.42	3	0.25	3 (25)
Overall	3992	7199	1.65	1001	0.26	567 (14.2)

Table 2.

Number of AEs per patient

Number of AEs per case	Number of patients (n=3992)	Proportion (%) of patients experiencing one or more AEs (n=567)
0	3425	NA
1	356	62.8
2	111	19.6
3	40	7.0
4	28	4.9
5	16	2.8
6	8	1.4
7	5	0.9
9	2	0.4
10	1	0.2

AEs, adverse events; NA, not applicable.

Individual triggers varied in their ability to lead to detection of harm. The trigger Complications of procedure or treatment yielded the greatest amount of harm (182 AEs). The PPV varied from 80.0% for surgical site infection to 2.62% for missing observations/early warning scores. Also, the PPV was generally low in frequently identified triggers, such as missing observations/early warning scores (PPV 2.6%) and unplanned admission (PPV 4%). The positive triggers, AEs and PPV for each trigger are displayed in table 3.

Table 3.

Trigger descriptors, AE and positive predictive value

Trigger Code	Trigger description	Adverse events	Positive triggers	Severity of harm					Trigger PPV (%)
				E	F	G	H	I
PG8	Complication of procedure or treatment	182	257	99	63	6	11	3	70.8
PG3	Readmission to hospital within 30 days	107	462	36	68	0	1	2	23.2
PG2	Tissue damage or pressure ulcer	81	250	66	12	0	1	2	32.4
PG4	Unplanned admission	68	1668	23	41	0	3	1	4.1
PO1	Other (specify)	60	425	48	10	0	1	1	14.1
PS3	Surgical site infection	48	60	24	22	0	1	1	80.0
PM5	Anti-emetic given	41	507	40	1	0	0	0	8.1
PG10	Hypoxia O2 sat <85%	36	157	31	2	3	0	0	22.9
PG1	EWS or baseline observations missing/incomplete or score/observation requiring response	35	1362	26	8	0	1	0	2.6
PG9	Transfer to higher level of care (inc admission to specialist unit, ICU/HDU)	35	273	15	14	0	5	1	12.8
PS1	Return to theatre	33	75	15	15	2	1		44.0
PM7	Intravenous bolus≥10 mL/kg colloid or crystalloid given	31	386	22	5	1	1	2	8.0
PG11	Cancelled elective procedure/ delayed discharge	24	55	10	12	1	0	1	43.6
PL14	Positive blood culture	23	55	18	4	0	1	0	41.8
PL13	Nosocomial pneumonia	21	28	8	10	0	2	1	75.0
PL5	Na+ <130 or >150	14	71	12	1	0	1	0	19.7
PG5	Cranial imaging	10	141	4	2	3	0	1	7.0
PL8	Hyperglycaemia (>12 mmol/L)	11	65	10	1	0	0	0	16.9
PS2	Change in planned procedure	11	37	6	5	0	0	0	29.7
PL3	Abrupt drop in Hb or Hct (>25%)	10	65	9	1	0	0	0	15.4
PM8	Abrupt medication stop	10	52	8	2	0	0	0	19.2
PL8	Hypoglycaemia (<3 mmol/L)	10	46	9	1	0	0	0	21.7
PL9	Drug level out of range	10	32	8	2	0	0	0	31.3
PL6	K+ <3.0 or >6.0	9	69	8	0	0	1	0	13.0
IP1	Readmission to ICU or HDU	9	16	5	1	0	3	0	56.3
PS4	Removal/injury or repair of organ	9	43	3	5	1	0	0	20.9
PG6	Respiratory/cardiac arrest/crash call	9	41	0	2	0	7	0	22.0
PM5	Chlorpheniramine given	9	82	7	2	0	0	0	11.0
PL2	Transfusion	8	143	6	1	0	1	0	5.6
PL4	Rising urea or creatinine (>2× baseline)	6	54	4	2	0	0	0	11.1
PL15	Thrombocytopenia	6	54	4	1	0	0	1	11.1
PL1	High INR (>5) or APTT>100 s	6	31	6	0	0	0	0	19.4
PM4	Glucagon or glucose≥10% given	6	50	6	0	0	0	0	12.0
PG7	Diagnostic imaging for embolus/thrombus +/− confirmation	4	24	2	1	1	0	0	16.7
PM2	Naloxone given	4	16	3	0	0	1	0	25.0
PL11	Clostridium difficile	4	12	3	1	0	0	0	33.3
PM1	Vitamin K given (except routine neonatal dose)	1	33	1	0	0	0	0	3.0
PM3	Flumazanil given	0	2	0	0	0	0	0	NA
PL10	MRSA bacteraemia	0	0	0	0	0	0	0	NA
PL12	Vancomycin-resistant enterococcus	0	0	0	0	0	0	0	NA
	TOTAL	1001	7199	605	318	18	43	17

AE, adverse events; APTT, activated partial thromboplastin time; Hb, haemoglobin; Hct, haematocrit; HDC, high dependency unit; ICU, intensive care unit; INR, international normalised ratio; MRSA, methicillin-resistant Staphylococcus aureu; NA, not applicable; PPV, positive predictive value.

The majority of AEs (n=923, 92.2%) resulted in temporary harm to the patient (grades E and F—see table 5¹⁵); 43 AEs required life-sustaining interventions and 18 resulted in permanent harm. In 17 cases, the AE was believed to have contributed to the child's death (table 4).

Discussion

The complexity of uncovering harm is reflected in the numerous ways that one has to measure it.¹⁷ Traditional methods such as incident reporting have limitations, especially that of under-reporting, due to the reliance on individual clinicians to recognise and report AEs, as well as a tendency to focus on error rather than on harm. The measurement and examination of harm, rather than that of error, recognises that efforts to improve patient safety benefit from focusing on incidents that result in actual harm, identifying high-risk situations, considering preventability and looking for means of early detection and harm limitation.¹⁸ This deeper understanding of the harm to which patients are exposed is a recent phenomenon and in paediatrics the potential risks to safety are multiple.¹⁹ The call for zero harm and the focus on safety in recent reports^{20 21} reflect the importance of the identification and understanding of harm as an essential part of patient care. Clinicians have not known the actual levels of harm caused and have relied on a reporting system for clinical incidents.

Harm rates vary widely because of multiple factors, such as the definition of harm used, the methodology employed and the population studied. Until recently, most studies sought to establish harm rates for benchmarking purposes over large populations with suggestions that between 3% and 17% of patients experience an AE during a hospital admission.²² Most of these studies used retrospective, unstructured case note reviews which are labour-intensive, costly and impractical for the routine monitoring of harm.²³ Trigger tool methodology is accepted as one of the ways to measure harm in a way that allows local learning,^{1 3 2} Trigger tools have also been reported to provide consistent, reliable and relevant data at low cost,^{2 3} although the cost may vary between different hospitals.

This paper represents the largest study of paediatric harm using trigger tool methodology in the UK. While the primary purpose of a trigger tool is to gain local data for understanding harm in order to improve patient safety locally, we can learn about harm by examining the pooled experiences of the contributing hospitals.

The overall harm rate (the percentage of individual children experiencing one or more AEs during an admission) identified in this study was 14.2%. Previous studies focusing on hospitalised children have identified harm rates between 1% and 25.8% per admission for the general paediatric hospital population,^{12 13 18} and higher rates within the paediatric intensive care population of 26.1% to 62%.^{10 11 24} Two recent studies have examined harm in paediatric hospital populations using trigger tool methodology. The CPTT found a physician who reported a harm rate of 15.1% of admissions during a validation study across six paediatric hospitals in Canada.²⁵ Like the UKPTT, the CPTT has been adapted to make triggers more sensitive and specific to paediatric settings. The second study, at a single paediatric academic medical centre in the USA using the adult GTT, found an overall harm rate of 25.8%.⁸

Variation in harm rates may reflect a number of factors. Different methodologies yield different rates of AE identification. Trigger tools are reported to yield higher rates of AE identification than traditional methods such as self-reporting and unstructured case note review.^{1 24} Definitions of harm vary, as do their interpretation. Professional groups may interpret AEs differently.²⁶ Assessments of inter-rater reliability have reported high levels of agreement between review team members,^{8 26 27} but there is variability between different hospital department teams.²⁸ In addition, some organisations or teams set a lower threshold for what they see as harm and they may change this over time. Finally, different populations are exposed to different levels of harm depending on the complexity of their illness and the intensity and duration of their care.² Most studies report a harm rate per admission, meaning that longer admissions are more exposed to opportunity for harm.

The same reasons that explain the variation between international studies also explain much of the variation between hospitals in this study. Training was provided, but no independent assessment was made of the reviewer's interpretations or competence. The extremes of harm reported or its absence were seen in hospitals uploading low volumes of reviews and may be interpreted as the relative inexperience of the reviewers.²⁹ There is also a wide variety across the level of hospital represented with the corresponding impact on risk due to patient complexity, need for surgery or critical care and length of stay. While we had no means of adjusting for acuity because of the random selection of notes from within hospitals, we believe that the overall group is broadly representative of the population of hospitalised children in the UK.

One could ask whether this level of variation diminishes the findings of the study. On the contrary, we believe it represents a real portrayal of complex issues. It is also a taste of what individual organisations can expect if they start to use the PTT to help understand and reduce the harm in their institution. They will need to consider all of these issues as they interpret their own findings.

The majority (92.3%) of AEs identified in this study represented temporary harm resulting in the child requiring an intervention, admission to hospital or prolongation of their hospital stay. While severe harm (permanent harm or harm that required life-sustaining measures or contributed to death) was rare, it still constituted 7.8% of the harm identified. Similar findings with respect to severity have been reported with 10% of AEs classified as severe in one study of harm in a paediatric intensive care unit.¹⁰ A study of AEs in hospitalised children reported that clinicians do not always recognise harm, even when the consequences to the child are severe.³⁰ In this study, multiple AEs were relatively common, with 37% of those experiencing harm suffering two or more AEs in the same admission, far higher than in previous studies.^{8 25}

Triggers varied in their PPV for AEs. Screening for triggers is the key task of the trigger tool method. Triggers that infrequently identify harm could be removed to increase the efficiency of the tool. Some triggers may be important markers of care quality, such as the missing/incomplete early warning score or baseline observations, despite the inability of the trigger to identify specific patient harm.³¹ This will influence the next iteration of the trigger tool as we refine the triggers and consider taking out some of those that had a low PPV.

A number of studies have examined the possibility of automated trigger detection from electronic medical records, which may make the process easier.^{13 32} We believe that there is value in the manual approach, and that it will be some time before paper-based medical records in hospitals in the UK are converted to electronic medical records. Users of the UKPTT have expressed to us the benefits of having an opportunity to examine the quality of medical and nursing note keeping and observations, which in some centres has resulted in initiatives to improve these elements.

Limitations

There are a number of limitations to this study. The validity of trigger tool methodology is well established and we did not attempt to revalidate it again against another form of medical notes review for harm, as we did not believe this was necessary. The UKPTT differs from other trigger tools only in the constituent triggers. This study has provided PPVs for the 40 triggers included in the UKPTT. This validates the choice of high performing triggers and raises questions about the continued inclusion of low performing ones, which may be used to consider changes to the trigger profile. New triggers may also be suggested and could be tested for future versions of the tool.

The determination of inter-rater reliability may be important within departments but not necessarily between hospitals. The UKPTT is not recommended for benchmarking as the focus is on developing data for improvement rather than data for judgement.³³ The methodology recommends consistency in the reviewing teams so that intrareliability is not an issue.³⁴ We did not attempt to standardise the method of PTT data collection outside of the support provided and the recommendation on randomisation. Individual institutions made their own arrangements in terms of choosing and training reviewers. There were no checks of competence of reviewers or inter-rater reliability or of the accuracy of the data entered via the portal.

Strengths

Parry et al³⁵ note that the approach should be to look at all harm, not only preventable harm. It is our belief that the ability to measure harm and examine case notes using the UKPTT on a regular basis is an effective method of data capture and analysis, which provides hospitals with valuable insights into their quality of care, as every AE provides insight for improvement, whether deemed preventable or not.¹⁰

We did not attempt to find out how many of these AEs were detected through other methods, such as incident reporting (we expect that many were). The purpose of the UKPTT is to extend the ability to detect harm rather than to replace other approaches.

Conclusions and recommendations

In the context of the increasing demands to improve quality and safety for patients, the UKPTT provides a framework for paediatric clinicians to assess the rate of harm for their individual units, and the quality of their record keeping. This study highlights that currently there is a significant, measurable level of harm, which is sometimes severe, experienced by children admitted to hospitals in the UK. There is a range of predictive values for the triggers and some may be more useful than others. These findings will inform future modifications of the PTT including modifying or removing triggers. It will be important to test any new or augmented triggers with paediatric teams to assess their usefulness.

The recognition and examination of AEs through methods such as the UKPTT offers the potential to improve paediatric patient safety by concentrating efforts on strategies that reduce patient harm, rather than errors. The key is to produce information that promotes learning and improvement, with clinicians accepting their role to decrease harm from the perspective of the patient, rather than that of the healthcare provider.

We recommend that the UKPTT be used routinely in hospitals to assess harm and to help develop improvement interventions to reduce it. Although the PTT has been mainly used in children's hospitals, it can be used in district general or community hospitals, with a different spectrum of harm being detected. The UKPTT does not replace other reporting mechanisms, but is a useful addition to the methods already used to understand the harm caused to children in hospital care. Harm needs to be detected and assessed through a number of lenses and this lens allows clinicians to further understand what they do and how harm impacts on children. It provides a way to move from a reactive approach to safety to one that is more proactive and founded on harm free care.