Video intervention to improve incident reporting among medical trainees

Abstract

Introduction/objective

Improving graduate medical trainee involvement with patient safety and incident reporting is an important task in teaching hospitals that has been recognised across the country and led to numerous efforts to address barriers to incident reporting. A variety of studies have started to define the reasons why trainees are not optimally involved and interventions that may be helpful. The present study aims to add to this literature by primarily addressing barriers that can be considered ‘non-technical’ such as fears surrounding potential professional repercussions after submitting a report, perceptions that reporting incidents is not useful, and concerns about anonymity.

Methods

Barriers to incident reporting were previously analysed at our institution. A video was produced to directly target the barriers discovered. A 2-hour educational session was delivered which included the video intervention. The educational session was part of the standard patient safety curriculum at our institution. Paper surveys were used to capture changes in perceived barriers to incident reporting. Baseline and postintervention surveys were analysed for changes using t-tests and a p value of <0.05 to determine significance. Survey development included literature review, patient safety expert discussion and cognitive interviews.

Results

Perceived knowledge about the reporting process significantly improved after the intervention (t=−4.49; p<0.05). Attitudes about reporting also significantly improved with reduction in fear of negative consequences and anonymity. Perceptions of reporting being a futile activity were also diminished after the intervention.

Conclusions

This study demonstrates that targeting non-technical barriers to incident reporting with a video intervention is an effective way to improve perceived knowledge and attitude about incident reporting.

Introduction

Incident reporting is an important skill that all medical professionals should develop. Efforts aimed to improve patient safety have been established across healthcare institutions to facilitate incident reporting, monitoring, and analysis in a systematic approach. A key component of these efforts includes identifying limiting factors to optimal use within organisations.¹ The Accreditation Council for Graduate Medical Education recognises the significance of patient safety in medical education and recommends that training programme include patient safety event reporting in their curricula. This concept has stimulated numerous efforts to improve incident reporting among medical trainees.^2–4 Despite these efforts, incident reporting among medical trainees continues to be an area that needs improvement. Within our organisation, a recent Clinical Learning Environment Review site visit noted that trainee involvement with the incident reporting process is an opportunity for improvement.⁵

Studies aimed at exploring obstacles to incident reporting among medical trainees mainly using questionnaires have uncovered common barriers across institutions, which include lack of knowledge about what to report and how to report, a perception of insufficient time to submit reports, fear of professional retaliation, and a feeling that incident reporting systems are not useful.^{6 7} Previous studies have shown a need to address non-technical barriers to incident reporting such as fear of negative consequences as a result of reporting a ‘mistake’ or the impact on academic progress.⁸ Ultimately, concerns about the personal consequences following incident reporting and fears about being perceived as incompetent are barriers that likely wane the effectiveness of interventions and resulting reporting activities.^{9 10}

Published research suggests that educational interventions to improve incident reporting in general are beneficial. A study of nurses conducted at a medical institution in Japan showed a significant increase in incident reporting for 6 months after the implementation of an educational intervention. However, improvement was not sustained and decreased after 6 months. This suggests the need of long-term efforts to maintain a positive impact.¹¹ A randomised, controlled trial evaluated tailored educational outreach visits and their impact on improving reporting by physicians. The trial showed a possible positive impact changing physicians’ safety behaviour.¹² Similarly, another study that examined the long-term effect of a patient safety course on behaviour changes in incident reporting showed a positive impact on incident reporting attitudes.¹³

Additionally, efforts aimed at improving practical aspects of incident reporting system utilisation have shown benefit. For instance, other institutions have sought to improve the structure and process of reporting by focusing on the ease of reporting through the utilisation of more user-friendly paperless approaches.^{14 15} Baylor University Medical Center in Dallas has shown that with this approach an increase in the number of incident reports can be achieved. The mean increases in the number of reported events at 1, 3, 6, 9 and 12 months after the development of their new paperless system ranged between 31 and 34 events per month.¹⁴ A similar paperless approach was used on a metropolitan tertiary care centre using a web-based system to decrease the burden of the reporting process. Although the study was limited to a single surgical department, implementation of an online tracking system had a potential initiative to improve surgical safety.¹⁵

Among medical trainees, a multitude of interventions aimed at improving incident reporting have shown a variable effect on knowledge, attitudes and behaviours. Studied interventions have shown that regular verbal reminders can increase reporting behaviour,¹⁶ setting an expectation of reporting for advancement provides a brief improvement in reporting behaviour,¹⁷ anonymous, narrative reporting may improve reporting behaviour,¹⁸ and integrating a retirement benefit to an educational campaign improves reporting behaviour.¹⁹

Utilising Donabedian’s quality of healthcare model, the barriers to reporting can be divided into: (1) barriers with the clarity of the structure and how the system influences incident reporting attitudes, (2) barriers associated with the complexity of process and lack of time and understanding of the value or reporting compounded by fears over confidentiality, and legal actions, and (3) barriers of the anticipated outcome of reporting as a results of lack of feedback and fear of blame. In addition, Andersen’s behavioural model which has been effectively applied to clinician responses to quality-based payment incentives, suggests that coordination of health services is based on three concepts, (1) a predisposing motivation in participants’ about their role in the coordinating care impacting their attitude, (2) the availability of supporting resources that allows participants’ to apply the coordination and (3) the need to utilise the coordination. In incident reporting, improving quality of care is the trigger to the behaviour of utilising reporting systems.^{20 21}

While this information is beneficial toward understanding barriers to incident reporting and educational interventions aimed to mitigate these, previous studies have been mostly focused on addressing the lack of knowledge about what and how to report which can be thought of as category 1 barriers using the Donabedian model. Efforts specifically designed to address the ‘non-technical barriers’ such as fear of retaliation, the perception that no change will occur, concern about anonymity or Donabedian model barrier categories 2 and 3 need further study.

The present study focuses on understanding the change in medical residents’ and fellows’ knowledge and attitudes about incident reporting prior to and after an educational intervention targeted at previously identified Donabedian category 2 and 3 barriers.

Methods

Patient and public involvement

We did not involve patients or the public in our work as this was a study of the impact of an educational intervention on graduate medical trainees.

Population

The study evaluated the impact of a patient safety educational session on medical trainees. The population included residents and fellows from medical and surgical specialties and subspecialties. Trainees were assigned to an instructional session as part of their standard patient safety educational curriculum.

Intervention

The intervention delivered was a 2-hour educational session about patient safety and incident reporting. It consisted of a PowerPoint presentation and a patient safety video. The PowerPoint presentation reviewed technical aspects of incident reporting, such as definitions, appropriate classification of adverse events and near misses, the mechanics of submitting a report at our institution, the report review process, and examples of past reports and outcomes. The educational video was 5 min in length and included attending physicians and medical trainees who described past experiences with incident reporting and directly addressed non-technical barriers to reporting including fear of retaliation, a perception that no change occurs after submitting a report, insufficient time to report, and concern about anonymity.

Measures

A preintervention survey was used to measure baseline knowledge and attitudes about incident reporting. The baseline survey was delivered in paper format to trainees attending the educational session. Although participation in the session was a required component of medical training, participation in the survey study was voluntary.

After a 2–4-week interval, trainees who attended the educational session were asked to fill a paper-based, follow-up, voluntary survey to assess changes in knowledge and attitudes.

The survey was developed by identifying instruments existing in the patient safety literature designed to measure barriers to incident reporting.^{8 22–24} A survey instrument specifically tailored to measure non-technical barriers to incident reporting using less than 10 items was not available. A group of experts in patient safety reviewed items that were developed and edited using a modified Delphi approach, initially during email review, and eventually during face-to-face meetings. Cognitive interviews were then conducted with medical trainees who would not be participating in the paper-based survey study.

The survey was brief and included demographic items, such as level of training, age, gender, race/ethnicity, type of training programme, and items assessing knowledge and attitudes toward incident reporting (table 1).

Table 1.

Survey items

	Strongly disagree	Disagree	Neutral	Agree	Strongly agree
1. I know how to create an incident report	1	2	3	4	5
2. I do not have enough time in my day to file an incident report	1	2	3	4	5
3. Filing an incident report will not result in better patient care	1	2	3	4	5
4. I am hesitant to file an incident report because of negative consequences that may affect others involved	1	2	3	4	5
5. I am hesitant to file an incident report because of negative consequences that may affect me	1	2	3	4	5
6. I am concerned because incident reporting is not anonymous	1	2	3	4	5

Analysis

Two-sample t-tests were used to determine significance. Specifically, we conducted one sided independent two-sample tests with a 95% confidence level assuming unequal variance between the two sets of observations on all variables.

Results

A total of 79 trainees attended the educational session. 89% (71) completed the preintervention survey and 80% (57) completed the follow-up survey. Descriptive demographic statistics (table 2) showed the two samples to be comparable in terms of age, gender, ethnicity, nationality, training level and specialty area. This provided grounds to conduct subsequent comparative analyses.

Table 2.

Demographics of survey participants

	Preintervention (%)	Postintervention (%)
Age>30	52.1	49.1
Female	56.3	47.3
US graduates	70.4	74.5
>PGY4	42.3	41.8
Medical specialty or subspecialty	70.4	78.2

After the intervention, participants reported that they understood how to file a report better (t=−4.49; p<0.05); believed that it would lead to better patient care (t=2.35; p<0.01); they were also less concerned about negative consequences that may affect others (t=2.71; p<0.01), themselves (t=2.43; p<0.01) and less concern for anonymity (t=2.01; p<0.01). The only non-significant result was related to time to file a report. Although the trend showed that subjects in the postintervention had improved attitude about the time needed to file a report, the difference was not statistically significant (table 3).

Table 3.

Proportion of respondents who ‘agree’ or ‘strongly agree’ to survey items

	Preintervention (%)	Postintervention (%)	P value
Previously filed a report	10.7
Have the knowledge to file a report	30.0	49.2	<0.01
Lack time to file a report	15.0	8.7	0.07
Reporting does not improve patient care	3.0	1.0	0.01
Fear of consequences to others	24.0	6.8	<0.01
Fear of personal consequences	19.0	5.8	0.01
Concerned about lack of anonymity	24.0	14.5	0.02

Discussion

The results of this study suggest that an educational session, including a video aimed at addressing non-technical barriers to incident reporting, has a positive impact on knowledge and attitudes regarding incident reporting among medical trainees.

It is important to review our results considering the Donabedian quality structure, process, outcomes model and Anderson’s behavioural model indicating the need for predisposing characteristics, enabling resources and needs.^{10 21 25} Previous studies indicated that quality of care suffers when the structures and processes are not in place to promote the outcomes desired.^26–28 We previously found that within our organisation, medical trainees, residents and fellows were not reporting incidents consistently. Our hypothesis was that this was not likely due to a structural issue, as the reporting mechanism was utilised by others within the organisation. However, based on internal query⁷ as well as similar concerns throughout the literature, knowledge and trust of the processes in place are likely culprits reducing the desired outcomes.^{6 29 30} Additionally, the behavioural considerations reinforce this perspective, in that most clinicians possess the predisposing characteristics encompassing the desire to provide quality care to patients, as well as to seek to correct components of that care which are not the patients best interest.³¹ Furthermore, they often understand the need to be involved in the process of identifying and correcting poor process or care patterns.³² However, what often lacks is the enabling resources, either due to lack of knowledge, or perceptions that the resources do not actually provide a good avenue for reporting.6 8 9 30 As such, this intervention provided multiple aspects which addressed structural and procedural educational aspects, but that also focused on the behavioural components that connect residents and fellows to the processes by which change can occur.

Our results indicate that a video-based intervention integrated into a general incident reporting educational conference is an effective way to communicate and educate medical residents and fellows about incident reporting. Specifically, it seems to be an effective way to address non-technical or Donabedian category 2 and 3 barriers among medical trainees. Previous research has indicated the positive impact of video-based interventions in encouraging behavioural change.³³ A randomised control trial focused on video vs in person skill demonstration of sterile surgical techniques concluded that the videos were a superior modality to providing the information.³⁴ The video intervention for this trial was based on face-to-face interviews that included medical trainees sharing their experience with incident reporting. Including trainees in the video is a strategy that has shown to be helpful in previous research.⁴

Future research activities will seek to expand the study to other care sites and use the intervention video via email in isolation. In doing so, the video may provide information on the generalisability of a uniform intervention across sites distinguished by geography, size, as well as a broader medical and residency population.

While this study provides important insight into incident reporting, it is not without limitations. First this study utilises a preintervention and postintervention design that does not include a control group. This potentially leaves unexplained some aspects of either other interventions or educational activities which may help describe the outcomes defined in this study. It is important to note that the intervention was a combination of a lecture using slides as well as the video which limits our ability to attribute the measured change to any portion of the educational session in isolation. In addition, while the breadth of residents and fellows’ backgrounds and locations within the organisation are a strength of the study, the focus on one organisation may reduce the generalisability of the results. Also, important to consider is that our approach only measured the impact of the intervention on attitudes and not behaviours. Although an improvement in attitude and knowledge about incident reporting is important in and of itself as part of the general graduate medical educational curriculum, a corresponding improvement in behaviour would be a desirable outcome. Although there is literature to support the argument that education is not an effective tool to change behaviour,³⁵ one can consider the theory of planned behaviour to predict a change in behavioural intentions and behaviour due to a change in attitude.³⁶Additionally, the impact of the educational session was measured after a 2–4-week interval limiting our ability to understand sustainability of the intervention past this time period. Finally, a general limitation of this study is the assumption that an increase in incident reporting will lead to improved outcomes. Incident reporting is assumed to be a key factor in hospital safety mechanisms across the world and considered important to include as part of a patient safety curriculum within teaching hospitals.⁵ However, there is recent literature highlighting the limitations of health systems to create useful change stemming from incident reports and related root cause analysis.^37–39

However, despite these limitations, the information provided concerning the use of the educational video as well as the focus on alleviating fears associated with incident reporting are valuable for hospitals and educational setting seeking to improve reporting by medical residents and fellows within their organisation.