Investigating the cost and efficiency of incident reporting in a specialist paediatric NHS hospital and impact on patient safety

Abstract

Objectives

The aim of this study was to investigate the incident reporting process (IR1s), to calculate the costs of reporting incidents in this context and to gain an indication of how economic the process was and whether it could be improved to yield better outcomes.

Methods

A retrospective analysis of a sample, 10.47% (n=150) selected from 1432 medication incident report summaries, generated at Birmingham Children's Hospital, a specialist tertiary referral paediatric centre, during 2014 and collated through the national Datix incident reporting system software was analysed and the associated staff time required to complete each step of the incident reporting process was costed. The staff costs for various grades of staff were averaged across the staff actually involved, using data calculated by the Personal Social Services Research Unit.

Results

The analysis showed that the incident reporting process involved 262 staff on 2942 occasions (19.16 staff episodes per incident form completed) at a cost of £337.16 per incident form completed.

Conclusions

The study showed that the incident reporting system was a labour intensive process. The numbers of staff involved in the process particularly as a result of the email distribution activity did appear to have room for efficiencies. However, it proved to be relatively inexpensive from a cost perspective. With redesign, arguably the emphasis could be moved away from the recording process to learning in order to gain improved patient safety outcomes.

Introduction

It is argued that a safe culture is an informed culture and this, in turn, depends upon creating an effective reporting culture.¹ The National Health Service (NHS) has become increasingly effective at reporting incidents through the National Reporting and Learning System (NRLS), with over four million incident reports submitted since its inception in 2003, with >600 000 of these having been reported between October 2014 and March 2015 alone.² The goal of every NHS trust is to have a high reporting low harm culture, and one that learns from its errors.

No matter how good a system is the question still ought to be asked whether the system could be improved. Could the current NHS incident reporting system achieve better outcomes and with greater cost-effectiveness?

A safety culture is reflected in an organisation's willingness to develop and learn from errors, incidents and accidents. To be clear, a safety culture is a learning culture.³ Organisations with a healthy safety focus are constantly learning. They learn from their mistakes and those of others. Information regarding prior incidents and accidents is shared openly and not suppressed.⁴

In the absence of frequent bad events, the best way to induce and then sustain a state of intelligent and respectful wariness is to gather the right kinds of data.⁵ A safety culture is one in which data is proactively collected, not just incidents and errors but near misses and concerns. This information is collated and analysed and from the outcomes lessons are learnt and practice changed. In other words, a safety culture is also a reporting culture.

The theme of a reporting culture was first introduced across the NHS with the paper, An Organisation with a Memory, by the Department of Health, in which the four aspects of a safety culture were outlined as being a reporting culture, one in which near misses were systematically reported.⁶ A just culture, not an absence of blame but one in which an atmosphere of trust prevailed the term Fair Accountability has also been coined. A flexible culture being one in which control is allowed to pass to experts at the frontline and a learning culture, a willingness to learn from errors and change practice. As a direct outcome of this report the National Learning and Reporting System (NLRS) and the National Patient Safety Association (NPSA) were created, the role of the NLRS was to collect and collate incident reports from all NHS hospital trusts and the remit of the NPSA was to generate learning from this data and prompt changes in practice.

In an average NHS acute hospital 2.5 million doses of medicines are administered a year. There are 215 000 administration errors and 45 000 prescribing errors per year.⁷ The NHS strategy to reduce errors was to develop the concept of a safety culture, a term first introduced by the International Nuclear Safety Group and given meaning by Reason, who listed the key elements of a safety culture as one that is open, just, reports, learns and adapts.^{1 8} The Department of Health outlined its safety strategy in its paper, an organisation with a memory (2000) requiring NHS organisations to report errors. The NLRS was introduced in 2003. The system enables patient safety incident reports to be submitted to a national database. This data is then analysed to identify hazards, risks and opportunities to improve the safety of patient care.

Since the NRLS was established, over four million incident reports have been submitted through the NLRS system by healthcare staff.⁹ The NHS has become increasingly effective at collecting incident data, for the 6 months October 2014 to March 2015 in specialist and acute trusts there were 642 098 incidents reported of which those categorised as severe amounted to 0.38% (2423).² These figures would indicate that the system is effective but the question is could it be improved further either with regard to safety outcomes and/or from a cost perspective?

The aim of this study was to investigate the incident reporting process (IR1s), to calculate the costs of reporting incidents in this context and to gain an indication of how economic the process was and whether it could be improved to yield better outcomes.

Method

Study design and setting

A retrospective analysis from medication incident report summaries, generated at Birmingham Children's Hospital, a specialist tertiary referral paediatric centre was done. Our hospital has 350 inpatient beds, 39 000 inpatient admissions and 150 000 outpatient appointments each year.

Incident reporting process

The process for completing an IR1 is a composite process initiated by an individual who has identified that an error has taken place. This individual may then approach a colleague for additional input. Once completed the IR1 is submitted electronically and transmitted to the manager of the location where the incident took place for investigation, and in addition to a list of individuals with a related responsibility with that area or incident either managerial or clinically. Once resolved each individual connected no matter how remotely with that incident receives an email summary relaying the outcome of the investigation or a link to where the outcome may be found. In addition, some clinical areas have their own safety team (ST) that reviews the incident in detail, for example, the paediatric intensive care unit (PICU). The PICU ST consists of 12 staff that is made up of doctors, nurses and pharmacists. Finally, all members of the unit or ward where an incident took place are notified of the fact. The incident summaries record the following data, the time and date of the incident, which staff were involved, a brief outline of the incident, an assessment of the impact that the incident may have had on the patient and the staff who was informed of the incident after the form was completed.

Incident reporting process analysis

Data extraction

The complete incident report summaries dataset for 2014 were extracted (n=5147) and the medicine-related errors isolated (n=1432). Of the medicine-related incidents, a subset (n=150), 10.47% of the data was randomly selected, from across the whole period, for detailed analysis in order to calculate a costing algorithm that could be applied to the whole.

Analysis

Staff involved

The analysis recorded the numbers of staff by profession and grade involved in the incident, the actual impact, whether additional therapy was required as a result of the incident and the numbers of staff informed by email of the incident. In addition, the outcomes of the incident report and the error type were also noted using sets of standardised responses for each.

Time dedicated

In addition, a questionnaire was sent by email to a subset of staff (n=97) (33.21%) out of a total staff listed on the incident summaries' circulation list, on the extract subset, as having been informed of that incident. Three questions were asked of those to whom a questionnaire was sent: 1. When you or colleagues complete an incident form entry, how long approximately do you spend? 2. When you are notified of an incident (via email), how much time do you spend processing it? 3. If your ward has a ward ST, how long would the team spend discussing a particular incident, on average? (Only two wards have STs.)

Free text comments were also invited.

Costs

The professional roles of the staff listed on the incident forms were categorised into one of six groupings that matched as closely as possible to the groups described in the reference compiled by the Personal Social Services Research Unit (PSSRU).¹⁰ The categories were ward sister/manager, deputy ward sister/deputy manager, nurse, consultant, pharmacist and pharmacy technician. The PSSRU reference then gave the composite costs per hour for each category of staff including all associate costs (employment, administration, human resources, training and education). In addition staff categories that were not considered to be clinical roles for example, management or administration roles, these were costed using NHS agenda for change pay scales for 2014 and a standard NHS on-cost of 23% was included.

Staffing costs were then calculated per 1 and 10 min units of time (ie, cost per hour obtained from the PSSRU divided by 60 and by 10 to obtain a staff costs per 1 and 10 min units).

The results are relative to (a) the costs for 1 min of the different staff categories, (b) the costs associated with email notification and (c) the costs for each element of the IR1 process are presented in different tables.

Results

Staff involved

Analysis of the subset of incident report summaries (n=150) for the year reviewed, 2014, showed that there 1798 staff episodes in total, comprised of 1020 nursing inputs, 362 doctor inputs, 183 pharmacist inputs, 202 general manager inputs and 31 other staff category inputs. Individual staff being involved on multiple occasions during the incident form completion process. Staff were involved in completing the forms either directly that is completing the incident form themselves or indirectly through being consulted when the incident form was being completed. On the subset of 150 incident report summaries, 262 individual staff were listed.

Of the 97 staff sent a questionnaire 87 responded, an 86.79% response rate, with eight staff having left the trust. As there was more than one incident per ward area contained in this subset some individuals were involved with more than one incident.

Time dedicated

This received a response rate of 40.45% (36/89) and from this it was possible to calculate the average time spent by each respondent on the various tasks that they were involved with. The data was blinded and therefore an individual response was not linked to a particular professional role (figures 1–4).

Figure 1.

The number of responses (Y-axis) and time taken in 5 min blocks to complete an incident form.

Figure 2.

The number of responses (Y-axis) and the time taken in time blocks (minutes) to investigate an incident.

Figure 3.

The number of respondents (Y-axis) and the time spent (again in 5 min time blocks) processing incident summary-related emails.

Figure 4.

Number of responses (Y-axis) and 5 min time blocks for time spent discussing incidents in the paediatric intensive care unit safety team meetings.

Costs

Costs for 1 min of the different staff categories

Costs for 1 min of the different staff categories are presented in tables 1 and 2. The costs for the PICU ST are presented in table 3.

Table 1.

Costs for 1 min of various staff categories

	Per hour	Per 10 min	Per hour with patient contact	Per 10 min	Per minute
Nurse manager	£57.00	£9.50	£139.00	£23.17	£2.32
Deputy manager	£49.00	£8.17	£119.00	£19.83	£1.98
Staff nurse	£41.00	£6.83	£100.00	£16.67	£1.67
Pharmacist	£47.00	£7.83	£94.00	£15.67	£1.57

Table 2.

Calculating a proportionate rate for 1 min of doctors time

Medics	Per hour	Per 10 min	Per hour with patient contact	Per 10 min	Per minute
Consultant (277)	–	–	£139.00	£23.17	£2.32
Registrar (85)	£59.00	£9.83	£71.00	£11.83	£1.18
FY2 (1)	£40.00	£6.67	£48.00	£8.00	£0.80
Composite medical costs					£2.05

Table 3.

The PICU safety team members and unit staffing costs

	Numbers	Unit cost	Total costs
Ward sister/manager	4	£20.67	£82.67
Deputy manager/junior sister	2	£17.33	£34.67
Nurse	2	£14.00	£28.00
Pharmacist	1	£14.00	£14.00
Consultants	3	£17.00	£51.00
10 min block	12		£210.33
1 min block			£21.03

PICU, paediatric intensive care unit.

Costs associated with email notification

From the questionnaire the average time spent dealing with the first notification was found to be 5.71 min. The second email notification was read cursorily. There were 2942 email occurrences to 262 individuals. The PICU ST consists of 12 staff, removing duplication of emails to PICU ST and the individuals directly gave 1773 staff email episodes. Table 4 presents the costs associated with email notification.

Table 4.

Costs associated with email notification

	Numbers	Email list 1 (5.71 min)	Email list 2 (0.5 min)
Consultants	277	£2688.84	£235.45
Doctors	85	£572.71	£50.15
Directorate managers	202	£1810.87	£158.57
Ward managers	137	£1744.46	£152.76
Nurses	883	£8420.02	£737.31
Pharmacists	183	£1640.54	£143.66
Others	31	£47.79	£4.19
PICU ST	51	£6124.15	£536.27
		£23 049.39	£2018.34
	Total per IR1		£167.12

PICU, paediatric intensive care unit; ST, safety team.

Costs for each element of the IR1 process

Table 5 presents the costs for each element of the IR1 process. The total cost for processing an IR1 is £337.16. For the 150 IR1s reviewed, 262 individuals were involved on multiple occasions (2942) giving 19.61 staff episodes per IR1.

Table 5.

Showing costs for each element of the IR1 process

	Investigating manager	Nurses	Doctors	Pharmacist	Others	Email distribution	Ward distribution	PICU safety team (12 people)
Time (minutes)	24.69	9.97	5	9.97	9.97	–	–	10
Numbers	150	148	49	21	9	–	–	53 (35%)
Cost per minute	£1.70	£1.40	£2.05	£1.40	£0.38	–	–	£21.00
Cost per IR1	£41.97	£13.96	£10.25	£13.96	£3.79	–	–	£210.00
Total costs per IR1	£41.97	£13.77	£3.35	£1.95	£0.02	£167.12	£37.58	£71.40

The last three columns are shown in detail in tables 3 and 4.

PICU, paediatric intensive care unit.

Discussion

To our knowledge, our study is the first to investigate the incident reporting process (IR1s), to calculate the costs of reporting incidents in this context and to gain an indication of how economic the process was and whether it could be improved to yield better outcomes.

Literature lacks data

The literature lacks data on the actual process costs of recording IR1s. A literature search reviewed studies that had key words that included, error reporting, patient safety, risk management, organisational memory, intervention and safety management and also terms that explored economics, cost and effectiveness and set in the context of NHS hospitals in the UK. The search yielded 193 papers of which five were deemed to have some relevance and 14 were considered to be remotely relevant. Fung et al looked at the barriers and obstacles that nurses have to making incident reports.¹¹ Ginsburg et al considered the relationship between organisational leadership and learning from patient safety events. Mittmann et al conducted a literature review looking at the costs of achieving 15 selected patient safety targets.^{12 13} Newell et al reviews the impact on safety of applying Toyota production techniques to the medicines administration process at a hospital in Midwest America.¹⁴ Finally, Waring investigated the relationship between incident reporting processes and the degree of participation in the reporting process by medical doctors.¹⁵ It was not the purpose of this study to review the quality of the data captured on IR1 incident reports or even the volume of IR1 reports completed. The aim of this study was not to evaluate the effectiveness of the IR1 recording process but rather to investigate the costs associated with it and in particular incidents that were classified as resulting in no patient harm. Incidents that result in causing harm to patients, follow a more comprehensive investigative pathway.

A wrong system?

The total costs of incident reporting for an incident with low patient impact that is, not requiring additional treatment or further investigation was calculated to be £337.16 per incident form (IR1) completed.

There were on average 19.17 staff episodes generated by the process in order to complete each incident summary report.

What the study did show was that although costs were low, staff episodes required to complete the process was high and calls into question the design of the process and whether or not this level of staff input would not be better allocated to learning from these errors rather than recording them? In other words, the question to be answered next is, could the system be made more efficient? The real reason for recording errors, after all, is to be able to learn from them and improve patient safety.

Limitations

This study has some limitations. This study was done in a single tertiary paediatric centre. However, the method used is reproducible and data could be extrapolated in different centres and countries taking into account the cost per hour for each category of staff and time spend for dealing with email. The time dealing with email was based on the choice of the questionnaire with 5 min blocks that could overestimate or underestimate the time spent to deal with.

Conclusion

The costs associated with reporting, investigating and processing an incident report (IR1) of £337.16 on average did not appear to be unduly high considering the acknowledged positive impact for patient safety. However, the numbers of staff involved (19.61 staff episodes on average) in the process particularly as a result of the email distribution activity did appear to have room for efficiencies. Anecdotally, feedback through the free text element of the survey indicated a certain incident reporting processing fatigue among staff that hinted at disengagement from the process. It could be argued that an element of what many perceive as unproductive processing time could be diverted away from less important tasks in the process to supporting learning from the incidents instead? Indeed the hospital safety committee has begun to consider alternative ways in which the incident recording process might be amended in order to improve learning outcomes.

Key messages.

• What is already known on this subject?

• The National Health Service (NHS) has become increasingly effective at reporting incidents through the National Reporting and Learning System, with over four million incident reports submitted since its inception in 2003.

• The goal of every NHS trust is to have a high reporting low harm culture.

• Could the current NHS incident reporting system achieve better outcomes and with greater cost-effectiveness?

• What this study adds?

• The costs associated with reporting, investigating and processing an incident report (IR1) of £337.16 on average did not appear to be unduly high considering the acknowledged positive impact for patient safety.

• The numbers of staff involved (19.61 staff episodes on average) in the process particularly as a result of the email distribution activity did appear to have room for efficiencies.

• This study indicates that by redesigning the current incident reporting system would allow resources to be diverted from recording incidents to learning from errors with the potential for improved patient safety outcomes.