Identification by families of pediatric adverse events and near misses overlooked by health care providers

Abstract

Background:

Identifying adverse events and near misses is essential to improving safety in the health care system. Patients are capable of reliably identifying and reporting adverse events. The effect of a patient safety reporting system used by families of pediatric inpatients on reporting of adverse events by health care providers has not previously been investigated.

Methods:

Between Nov. 1, 2008, and Nov. 30, 2009, families of children discharged from a single ward of British Columbia’s Children’s Hospital were asked to respond to a questionnaire about adverse events and near misses during the hospital stay. Rates of reporting by health care providers for this period were compared with rates for the previous year. Family reports for specific incidents were matched with reports by health care providers to determine overlap.

Results:

A total of 544 familes responded to the questionnaire. The estimated absolute increase in reports by health care providers per 100 admissions was 0.5% (95% confidence interval −1.8% to 2.7%). A total of 321 events were identified in 201 of the 544 family reports. Of these, 153 (48%) were determined to represent legitimate patient safety concerns. Only 8 (2.5%) of the adverse events reported by families were also reported by health care providers.

Interpretation:

The introduction of a family-based system for reporting adverse events involving pediatric inpatients, administered at the time of discharge, did not change rates of reporting of adverse events and near misses by health care providers. Most reports submitted by families were not duplicated in the reporting system for health care providers, which suggests that families and staff members view safety-related events differently. However, almost half of the family reports represented legitimate patient safety concerns. Families appeared capable of providing valuable information for improving the safety of pediatric inpatients.

It has been estimated that adverse events occur in about 1% of children treated in hospital and that, on average, 60% of these events are preventable.¹ To increase institutional awareness of adverse events, hospitals have implemented systems to encourage health care providers to report adverse events.² The reporting of adverse events can be improved by making electronic systems for reporting readily accessible³ and by ensuring a “just culture,” which includes nonpunitive reporting policies.⁴ However, adverse events reported by health care providers account for only a small fraction of total adverse events as determined by chart review.⁵ Time pressures to treat patients, fear of punishment, lack of belief in the benefit of reporting and differing opinions of what defines a reportable event contribute to low reporting rates.⁶ However, patients and their families are readily available, keen and motivated observers who may not be subject to these reporting barriers. Family members are capable of observing and reporting adverse events in a variety of clinical settings.⁷ It is known that the interpretation of safety events and the threshold for reporting differ among health care disciplines and individual health care providers.⁶ However, it is not clear how families of pediatric patients interpret safety-related events or what their threshold would be for reporting events.

The purpose of this study was to test whether the introduction of an adverse event reporting system for use by families of pediatric patients at the time of discharge from a surgical ward would significantly change the rate of reporting of adverse events by health care providers. We also evaluated the types of events that families reported, the relevance of these events with respect to patient safety, families’ desires for anonymous reporting and families’ assessments of institutional responses to reported events. We anticipated that health care providers’ reporting rates would rise with the introduction of the family reporting system, on the assumption that greater attention would be paid to reporting safety-related events on the ward. We also anticipated that families would provide useful information about safety-related events, at least some of the time. In particular, we thought that facilitating communication from the patient’s family directly to the study institution’s Quality, Safety and Outcome Improvement Department would allow more opportunities to improve safety through changes in practice.

Methods

The study was conducted at British Columbia’s Children’s Hospital, Vancouver, British Columbia. The study ward was a single inpatient ward providing general medical, general surgical, neurologic and neurosurgical care to children from birth through adolescence. In September 2008, we placed posters on the walls of the study ward to inform staff members that the family reporting study was to begin in November 2008. We consulted with nursing and medical staff and addressed confidentiality and liability concerns raised during these discussions. During the study, a research assistant attended weekly ward safety rounds, where the past week’s safety concerns were identified and discussed. We did not inform hospital staff of the study hypothesis.

Reporting by health care providers

For the period November 2007 to November 2009, we collected monthly data on the number of reports submitted by health care providers to the British Columbia Patient Safety & Learning System.

Recruitment of participants

The University of British Columbia/Children’s and Women’s Health Centre of British Columbia Research Ethics Board approved the study on June 30, 2008. Consenting families participated in the study from Nov. 1, 2008, to Nov. 30, 2009. During that period, for each child who was admitted to the study ward for least 24 hours, a research assistant approached the family on the morning of discharge with an invitation to participate in the study by completing a computerized questionnaire. Because of constraints on the availability of research personnel, we recruited families only on weekday mornings. The research assistant approached family members who were parents or legal guardians of patients and who were able to give informed consent. We excluded parents or legal guardians who were unable to read and write in English. The research assistant provided a short tutorial about the computerized questionnaire to families who consented to participate.

Questionnaire items

The questionnaire asked about the occurrence of six types of patient safety problems, as defined in Table 1. We described the face validity, usability and domain relevance of the questionnaire items in a previous publication.⁸

Table 1:

Categories of problems related to patient safety

Category and definition	Question stem
Medication problems: When a medication is not given exactly as it was meant to be given Examples: Medication given in the incorrect amount; patient had an allergic reaction to medication	Do you think a medication problem occurred or was stopped before occurring?
Complication of care: When there is an unwanted result of treatment Examples: Unexpected bleeding occurred; patient was transferred to the intensive care unit because of a complication	Do you think a complication of care occurred or was stopped before occurring?
Equipment problems: When equipment fails or is not used correctly Examples: Equipment not available when needed; an arterial line leaked or became blocked	Do you think an equipment problem occurred or was stopped before occurring?
Miscommunications between staff: When members of the staff give or receive information from other staff about diagnosis, treatment, or care that is inadequate (not enough information), conflicting (information that is different from what someone else gave) or incorrect Examples: A test was repeated because the original result was lost or destroyed; a test was cancelled by mistake	Do you think miscommunication between staff occurred or was stopped before occurring?
Miscommunications between my family and staff: When you or your family gives or receives information from staff about diagnosis, treatment or care that is inadequate (not enough information), conflicting (information that is different from what someone else gave) or incorrect Examples: Medication instructions were not explained to me or my family; staff did not communicate well	Do you think a miscommunication between your family and staff occurred or was stopped before occurring?
Other: When any action not previously described fails or was the incorrect action Examples: A burn occurred; a cut occurred	Do you think any other problems occurred or were stopped before occurring?

If a participant answered “yes” to any of the six questions in Table 1, he or she was prompted to enter a free-text description of the event. Participants reporting an event were asked whether hospital staff were aware of the problem or concern and whether hospital staff had provided an apology. If an apology had been provided, participants were asked to rate the apology as inadequate, adequate or very adequate. These terms were assumed to be self-explanatory, and no definitions were given.

The questionnaire was anonymous, but participants were allowed to identify themselves, their relationship to the patient and their contact information if they wished to participate in future projects related to improving patient safety. Participants who did not supply personal information were identified only by unique random number codes. Reports of important patient safety problems were immediately referred to the Quality, Safety and Outcome Improvement Department at British Columbia’s Children’s Hospital, which handled them according to standard hospital policy.

Evaluation of reports

Two clinical experts not involved with caring for the patient for whom the adverse event or near miss was reported used participants’ categorization of events and associated narratives and the scheme shown in Box 1 to evaluate the degree of harm associated with each report. For these reviews, because of the high degree of variability in applying the harm classification to real-world scenarios, the second reviewer was asked to indicate whether he or she agreed with the first reviewer’s assessment.

Box 1: Classification scheme for degree of harm,^* based on the British Columbia Patient Safety & Learning System, as used at British Columbia’s Children’s Hospital.

Not a patient safety issue: No harm or risk of harm

Near miss: Harm almost occurred but was avoided through chance or timely action

Minor harm: Minor, temporary injury to the patient

Moderate harm: Moderate, temporary injury to the patient

Severe harm: Serious injury, altering hospital stay and/or requiring additional treatment

Insufficient detail to evaluate: Harm that cannot reasonably be assigned to any of the other categories on the basis of information provided

We calculated the agreement between the two reviewers. We determined the proportion of adverse events reported by families that were also reported by health care providers during the study period (Nov. 1, 2008, to Nov. 30, 2009) by attempting to match family and provider reports by date and description. In each case, we categorized the goodness of the match as unlikely, possible, likely, very likely or definite.

Statistical analysis

On the basis of past reporting (by health care providers) for the study ward, we estimated that 544 families had to be enrolled to disprove the null hypothesis that family-based reporting would not lead to an increase in the rates of reporting of safety-related events by health care providers. We analyzed report counts using a quasi-Poisson model to allow for monthly overdispersion. We used generalized estimating equations to fit the model to the reports, to allow for serial correlation over time.

Results

Health care providers submitted a total of 175 patient safety reports in the 12 months before initiation of the family reporting system and 226 reports during the 13-month intervention phase (starting Nov. 1, 2008) (Figure 1). Quasi-Poisson modelling indicated that after initiation of the family reporting intervention, the estimated absolute increase in the rate of reporting by health care providers was 0.5% per 100 admissions (95% confidence interval −1.8% to 2.7%).

Figure 1:

Monthly rate of reports by health care providers per 100 admissions for the period November 2007 to November 2009. Dashed line shows the expected number of reports before and after the initiation, in November 2008, of the family-based reporting system, based on the quasi-Poisson regression model.

A total of 1796 patients whose families were potentially eligible to participate were discharged from the study ward during the intervention phase of the study. Of these, 639 families (36%) were approached to participate in the study. Twenty-three families were excluded because of an inability to communicate in English, and 72 families declined to participate. Therefore, 544 families (85%) agreed to participate and submitted reports. Of the 544 reports submitted, 201 (37%) mentioned one or more adverse events or near misses, and 343 reported no adverse events during the hospital stay. A total of 321 events were identified, of which 153 (48%) were deemed to represent a legitimate patient safety concern (i.e., near miss or some degree of harm) (Table 2). No deaths were reported by the families. Appendix 1 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.110393/-/DC1) provides examples of events for the various categories of harm.

Table 2:

Number of family reports of adverse events, by degree of harm

Degree of harm	Medication problems	Complications of care	Equipment problems	Miscommunication		Other problems	Total no. (%)
				Between staff	Between family and staff
Not a patient safety issue	7	8	11	24	16	25	91 (28)
Near miss	9	2	10	18	10	6	55 (17)
Minor harm	28	15	9	10	6	11	79 (25)
Moderate harm	5	5	2	1	0	0	13 (4)
Severe harm	0	4	0	2	0	0	6 (2)
Insufficient detail to evaluate	14	7	11	21	20	4	77 (24)
Total no. (%)	63 (20)	41 (13)	43 (13)	76 (24)	52 (16)	46 (14)

The agreement between dependent reviewers analyzing the degree of harm of family reports was 93%. Of the 544 respondents, 336 (62%) provided identifying information so that they could be contacted for subsequent institutional efforts to improve patient safety. Mothers (424 [78%]) and fathers (98 [18%]) of patients constituted 96% of respondents.

Of the 321 adverse events reported by families, 313 could not be identified in the health care provider reporting system. We were able to make only 8 (2.5% of the adverse events reported by families) possible, likely or definite matches between provider and family reports. Conversely, 218 of the reports by health care providers were not duplicated by the family reports. Transcripts of the matched reports are given in Appendix 2 ( available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.110393/-/DC1). Seventy-six per cent of families who responded (n = 153) indicated that they believed staff were aware of the reported safety concern. Respondents reported that a total of 139 apologies were offered to families: 33 (24%) for medication problems, 31 (22%) for miscommunications between staff, 22 (16%) for equipment problems, 21 (15%) for complications of care, 18 (13%) for miscommunications between family and staff, and 14 (10%) for other problems. Families rated the apologies as very adequate (29 [21%]), adequate (86 [62%]) or inadequate (24 [17%]).

Interpretation

The results of this study showed that the introduction of a family-based adverse event reporting system administered at the time of discharge from a pediatric inpatient surgical ward was not associated with a change in the rate of reporting of adverse events by health care providers. Furthermore, only 2.5% of reports submitted by families were duplicated in the health care provider reporting system. Almost half of the adverse events reported by families represented valid safety concerns, not merely reports of dissatisfaction. Miscommunication was the most common problem reported by families. In many cases, no apology was offered; however, when an apology was offered, it was usually judged by families to be at least adequate.

Our results indicate that health care providers and family members have different views of adverse events and the importance and effects of reporting. Families reported miscommunication with and between staff most frequently, whereas staff rarely reported such events. Patient safety events frequently entail miscommunication,⁹ to which families may be particularly attuned. The implementation of this family-based patient safety reporting system provided new opportunities to learn about and improve the safety of health care provision without an additional reporting burden for health care providers. Giving families the opportunity to report patient safety events did not remove the barriers to reporting by health care providers (time pressure, culture of blame, fear of reprisal and lack of belief in the value of reporting)⁵ but served to complement such reporting.

We anticipated that there would be few events causing severe harm, with events causing minor harm and near misses accounting for most reported events occurring during the study period. The harm profile of family reports confirmed this expectation. Sixty-two per cent of families identified themselves and volunteered to be contacted for future safety improvement efforts, which indicates that families viewed reporting as a positive action intended to improve care and did not fear potential reprisals as a consequence of their reports. This finding is a strong signal of families’ desire to be engaged and functional components of their health care system and its safety. The very small number of events reported by both families and health care providers reflects the different perspectives of these two groups. The apology results suggest that providers were not aware of many events or of families’ perceptions of the events and indicate that many health care providers could benefit from training about when apologies are valued by families. In the event of a family perceiving harm, an apology from hospital staff would be appropriate and likely welcomed by families. Health care providers’ skills in providing apologies were usually acceptable, however.

Studies on the effect of reporting of safety concerns by patients and families on rates of reporting by health care providers have rarely been performed. In a similar study,¹⁰ researchers used an open-ended question to ask adult patients in a medical inpatient ward to describe any “negative effects” or “complications” (as interpreted by the patient). They found that patients reported poor communication at a rate of 25 reports per 100 admissions, similar to the rate found in this investigation (23 reports per 100 patients). Families in our study frequently reported medication safety problems, which have been predominant in adverse event studies using both patient interviews¹¹ and chart review¹² as data sources. Past work based on reports from adult patients receiving cancer treatment¹¹ yielded a percentage of insignificant events similar to that reported here (23% v. 28%), a higher proportion of events causing minor or moderate harm to the patient (63% v. 29%) and a higher percentage of events causing serious or severe harm to the patient (13% v. 2%). The present study showed that families were able to identify adverse events not captured by the health care provider reporting system, which replicates a previous study.⁷ As with the present study, a previous analysis of the congruence between adverse event reports from health care providers, discharge abstracts and a trigger tool showed that adverse events identified by one method were unlikely to be reported by another.¹³ Some of the discrepancy in reports between the two reporting techniques in the study reported here is likely attributable to the reliance of the health care provider reporting system on a passive surveillance technique, whereas the family reporting system involved actively querying each family at the time of the patient’s discharge from hospital.

Limitations

This study had several inherent limitations. The study was conducted on a single ward of a single tertiary care hospital. Consequently, the relevance of its findings cannot be automatically generalized to other ward types at different locations. Direct in-person solicitation of reports, as was done in this study, will not be practical in many clinical settings. The results were constrained by the preselected categories used to classify events. Free-text reporting might have yielded different or more numerous reports. We did not use incentives and solicitation to improve the rate of family reporting.⁷ It has been shown that patients report more adverse events to lay volunteers than to hospital personnel or research assistants,¹⁴ a factor that might have limited the number of reports submitted in this study. Families who could not be approached for consent and those who declined to participate may have been in a rush to leave the hospital. It is possible that availability of a questionnaire that could be completed from home would have increased the participation rate. The low rate of approaching families to participate in this study was due to our inability to simultaneously collect data from more than one family (because of hardware and personnel limitations), families’ erratic schedules on the day of discharge from hospital, discharges occurring after hours, poor communication between researchers and clinical staff regarding planned discharge dates and a policy of not approaching emotionally distressed families for participation in research studies. Not approaching emotionally distressed families might have led to underreporting of more severe events. There may have been selection bias, whereby patients wanting to vent their frustration with the system were inclined to participate, which would have increased the reporting rate among participating families. We did not analyze the effect of family variables, such as severity of the patient’s disease or frequency of admission to hospital, on reporting.

Conclusions

In this study we found that families observe and report safety problems differently than do health care providers. Families’ observations were rarely documented in the health care provider reporting system. Most families who were approached took the opportunity to report safety problems to the hospital, and many families volunteered to donate their time in future to help prevent recurrence of the adverse event that occurred with their child. Most often, no apology was offered for perceived adverse events, but when apologies were offered, families usually judged them to be adequate or better. The introduction of a family safety reporting system did not alter the rates of safety reports by health care providers. Further research is needed to delineate how best to harness the potential of families to improve the safety of the health care system.

See related commentary by Vincent and Davis on page 15 and at www.cmaj.ca/lookup/doi/10.1503/cmaj.111311