Skip to main content
NCBI home page
Heliyon logoLink to Heliyon
. 2024 Apr 24;10(9):e30054. doi: 10.1016/j.heliyon.2024.e30054

Development of minimum data set and dashboard for monitoring adverse events in radiology departments

Mahtab Karami a,b,c, Nasrin Hafizi d, Ali-Mohammad Nickfarjam d,e, Soheila Refahi f,⁎
PMCID: PMC11068645  PMID: 38707457

Abstract

Background

To reduce the risk of errors, patient safety monitoring in the medical imaging department is crucial. Interventions are required and these can be provided as a framework for documenting, reporting, evaluating, and recognizing events that pose a threat to patient safety. The aim of this study was to develop minimum data set and dashboard for monitoring adverse events in radiology departments.

Material and methods

This developmental research was conducted in multiple phases, including content determination using the Delphi technique; database designing using SQL Server; user interface (UI) building using PHP; and dashboard evaluation in three aspects: the accuracy of calculating; UI requirements; and usability.

Results

This study identified 26 patient safety (PS) performance metrics and 110 PS-related significant data components organized into 14 major groupings as the system contents. The UI was built with three tabs: pre-procedure, intra-procedure, and post-procedure. The evaluation results proved the technical feasibility of the dashboard. Finally, the dashboard's usability was highly rated (76.3 out of 100).

Conclusion

The dashboard can be used to supplement datasets to obtain a more accurate picture of the PS condition and to draw attention to characteristics that professionals might otherwise overlook or undervalue.

Keywords: Medical imaging, Radiology department, Usability, Development, Evaluation, Patient safety, Dashboard, Adverse event, Minimum data set

Highlights

  • •

    Determining the minimum data set to design a check list and calculate patient safety-related metrics in the dashboard.

  • •

    Producing an electronic checklist to check patient safety-related items pre-, intra-, and post-medical imaging procedures.

  • •

    Developing a dashboard for monitoring adverse events in the medical imaging department.

  • •

    Prioritizing the management of health information over the use of technology.

1. Introduction

Patient safety (PS) is a vital component of care quality in the healthcare system. Even in complex, high-risk contexts, the PS allows healthcare stakeholders to operate without harming patients [1]. Medical errors are one of the leading causes of death worldwide, with millions of victims [2]. A divergence from the expected norm, regardless of any injury, is defined as an error [3].

In a study, Karami et al. explained that “clinical adverse impacts can be categorized as a spectrum ranging from near-miss events to sentinel events. A near-miss event is defined as an event characterized by the detection and correction of an error before harm reaches the patient, and a sentinel event is an unexpected occurrence involving death or serious physical or psychological injury, or the risk thereof” [2].

According to Larson et al. maintaining and enhancing safety levels requires the development of a system and culture that can intelligently integrate personnel with technology and procedures to produce a safer patient care environment [1].

Advances in health information technology in complex environments such as medical imaging departments (MIDs) have the potential to improve service delivery and reduce errors [4].

Errors in medical imaging procedures have a high risk of causing injury [5] and can occur at any point in the imaging chain and across all modalities [6,7]. This topic has been the focus of numerous investigations. In a study, Morbi et al. revealed that vascular interventional radiology techniques carry numerous safety hazards and have a high rate of adverse events. A mistake in interventional radiological procedures, according to Melvin et al. can result in sudden mortality, limb or organ loss, or serious bleeding [8,9].

Incorrect radiation doses to patients, misdiagnosis, disrupted treatments, the patient on the request, insufficient or erroneous clinical facts provided, illegible handwriting, injection of incorrect radiopharmaceutical/contrast medium/drugs, severe reaction to contrast medium, wrong identification of a patient's images, faults in a radiologist's report, considerable overexposure, failure to act on a rogue patient, and failure to meet the requirements, the wrong side or site, the wrong technique, and the wrong patient are just a few examples of these errors [[5], [6], [7], [8], [9], [10], [11], [12], [13]].

To address the mistakes and prevent near miss and sentinel events, it is essential to learn from them. According to the statement of the Joint Commission, all healthcare organizations must have a comprehensive strategy for identifying and addressing safety risks. Patient safety in medical imaging procedures can be improved by using a system to assist staff in monitoring and identifying probable PS-related occurrences, as well as reporting adverse events [14]. To achieve this goal, obtaining critical data should be instilled in the culture of healthcare organizations as part of their daily operations. In addition, effective procedures for carrying them out and regulating them should be created [15].

This study introduces the steps of the development of a dashboard for monitoring adverse events for MIDs before (pre), during (intra), and after (post) an imaging procedure to abate the errors. Since to achieve this system, having a proper minimum data set (MDS) is necessary, we first focus on the determination of the minimum data set, then we start to produce the dashboard. Indeed, we moved from information management to information technology.

2. Material and methods

In this qualitative and developmental research, a multi-method approach including interviews, structured observation, questionnaires, and workplace walkthroughs has been employed, divided into three steps:

Step 1: Analysis of the Status Quo.

After observation and investigation, we found no systematic or electronic method to monitor PS in the MIDs of the study site, and some paper-pencil forms would be used for PS checks. Therefore, we decided to develop an electronic system for tracking PS-related indicators in MIDs.

Step 2: Development of a Minimum Data Set.

The correct definition of metrics is a critical subject in developing a dashboard. We have already determined the key performance indicators (KPIs), associated metrics, and their descriptions for PS in the MID in the two articles mentioned [2,15]. In the present study, we used them (Appendix 1). We needed to define a MDS to calculate most of these KPI metrics. For this purpose, the following stages were defined:

First, a variety of items must be checked to monitor PS before, during, and after an imaging procedure. The dashboard's content is made up of these data pieces, known as the “minimum data set.” In this regard, at first, we collected the existing paper-based forms. Then a literature review [5,[16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43], [44], [45], [46], [47], [48], [49], [50], [51]] was conducted using scientific databases such as Medline (PubMed), Web of Science, Scopus, ProQuest Health, and Google Scholar.

Keywords such as “medical imaging” and/or “radiology department,” along with “patient safety performance indicator,” “patient safety incident,” “patient safety information system,” “near-miss occurrences,” “sentinel events,” and “patient safety monitoring system,” were used for searching.

Later, semi-structured interviews were conducted with five radiologists focused on the task sequence and the information needed to complete the PS checking process (including pre-, intra-, and post-procedure checking). At the end, the extracted MDS were located in the pre-, intra-, and post-procedure parts, and an electronic questionnaire was created.

Second, the Delphi technique was used to confirm these items by a panel of experts (n = 30) comprised of radiologists, medical physicists, and radiology technicians with more than 5 years of professional experience. Chain-referral sampling was used to choose the experts. It is a technique of non-probability sampling.

Experts were emailed the electronic questionnaire. They were asked to determine whether or not an item was required in this process. In addition, an open-ended question was added to the questionnaire to elicit additional data points. Delphi ended after one round because of the lack of additional recommendations in the first round.

For each item, the mean scores of the experts' opinions were determined; a mean score of more than 50 % meant that the item was acceptable, while a mean score of less than 50 % indicated that the item was not acceptable [52]. A list of MDs for use in the dashboard was produced based on the final panel's rating.

Step 3: Development of the Dashboard.

First, the entities and their relationships were graphically represented using ERD (Entity Relationship Diagram) and UML (Unified Modeling Language) diagrams (class, activity, and sequence).

Second, the web-based user interface was designed in the PhpStorm 2016 programming environment using Laravel, which is one of the PHP language frameworks. In addition, Html5, Css3, Jquery, and vue. js were used in the design of dashboard software. The database was also developed using MySQL software, which is fully compatible with the PHP programming language.

About the data feeding of the dashboard, the hospital information systems in the study site could only enter limited information, such as demographic information, medical record number, physician's name, and hospitalization wards, into the dashboard. Therefore, some data had to be entered manually.

Third, three types of evaluations were used to test the dashboard, as follows:

  • -

    The validation of the metrics calculation: These computations were done both manually and by machinery (using the dashboard) to guarantee that the PS metrics were calculated correctly. Finally, the results of the two procedures were compared. The validation was carried out by one of the team's researchers.

  • -

    Verification of the user interface: The user interface (UI) was evaluated by five IT specialists using the ISO 9241-10 standard to guarantee that all UI requirements were fully met. This standard relates to office work with visual display terminals and ergonomic criteria. This checklist included 50 criteria organized into seven primary axes: suitability for the tasks; self-descriptiveness; controllability; compliance with user expectations; error tolerance; suitability for individualization; and suitability for learning [53]. The evaluators were familiar with these topics. They used test data to run the system and compared its features and capabilities to ISO 9241-10. In the end, the problems were found and documented. Evaluation data were analyzed using descriptive statistics in Excel.

  • -

    The usability evaluation: The usability of the dashboard was evaluated using the System Usability Scale (SUS) by ten real users, including radiologists, radiology nurses, and educated radiology technicians who had passed international computer driving license (ICDL) courses. The number of evaluators was ten because most usability issues are detected by five users, so the number of samples should be between five and twelve [54]. In a 2-h workshop, the participants were initially exposed to the dashboard functionalities. To avoid bias, they worked independently with the dashboard and completed the SUS questionnaire. One of the researchers acted as an observer and collected comments, questions, and ambiguities from them.

2.1. Consideration

This study was approved by the Research Ethics Committee of the Kashan University of Medical Sciences Research Council (code of ethics: IR. KAUMS.REC.1395.32). Also, this study was conducted following the guidelines of the Declaration of Helsinki. By the opinion of the above-mentioned Ethics Committee and given the fact that no information about participants is provided in this paper, participants who participated in Delphi and usability studies gave verbal consent to participate in this research.

3. Results

The following are the overall findings based on the work steps:

In step one, 26 PS performance metrics and their corresponding definitions were selected from prior studies (Appendix 1). After the literature review, 143 important data elements were extracted. Following the Delphi phase, 110 of them were chosen and divided into 14 main groups for the three checking processes (pre-, intra- and post-). The selected MDS are listed in Table 1.

Table 1.

The minimum data determined from the Delphi technique.

Row Indicators Required Data agree
 disagree
NO. % NO. %
Data Elements Related to pre Procedurerowhead
1 Patient Scheduled Arrival Time 16 53 14 47
2 Real-Time of Imaging 23 77 7 23
3 Physician Order Physician's Name 19 63 11 37
4 Physician Specialty 21 70 9 30
5 Ward 19 63 11 37
6 Type of Modalities CT Scan 15 50 15 50
7 Nuclear Medicine 15 50 15 50
8 Radiography 18 60 12 40
9 Interventional Imaging 23 77 7 23
10 Cardiovascular Imaging 22 73 8 27
11 Angiography 20 67 10 33
12 CT Angiography 17 57 13 43
13 Heart Scan 16 53 14 47
14 Peripheral Vascular Imaging 18 60 12 40
15 Physician Signature 25 83 5 17
16 Request Time 17 57 13 43
17 Request Date 23 77 7 23
18 Universal Protocol Patient Identification Medical Record Number 20 67 10 33
19 Patient's Name 28 93 2 7
20 Father Name 20 67 10 33
21 Age 25 83 2 17
22 Sex 22 73 8 27
23 Address 15 50 15 50
24 Phone Number 22 73 8 27
25 Procedure Identification Side 24 80 6 20
26 Site 26 87 4 13
27 Medical History Illnesses Current Illnesses 24 80 6 20
28 Past Illnesses 24 80 6 20
29 Surgery 21 70 9 30
30 Drug Drug Use 20 67 10 33
311-
 Medication allergy 29 97 1 3
322-
 Adverse drug reactions 21 70 9 30
33 Imaging With Contrast 24 80 6 20
34 Without Contrast 22 73 8 27
35 Patient Training and Preparation Patient Training How to Imaging 22 73 8 27
36 Imaging Time 18 60 12 40
37 Preparation of Patient 25 83 5 17
38 Lack of Training 18 60 12 40
49 Patient Companion Without 15 50 15 50
40 Type of Companion 26 87 4 13
41 Patient's Condition Inpatient 18 60 12 40
42 Outpatient 17 57 13 43
43 Emergency 26 87 4 13
44 Extra Emergency 19 63 11 37
45 Type of Preparation Medication Oral Administration 20 67 10 33
46 Injection of Contrast Material 24 80 6 20
47 Non-Medication Full Bladder 22 73 8 27
48 Fasting 21 70 9 30
49 Menstruation 16 53 14 47
50 Lack of Metal Objects 23 77 7 23
51 Patient Appropriate Coverage 23 77 7 23
52 Type of Patient Transfer Stretcher 19 63 11 37
53 wheelchair 13 43 17 57
54 Oxygen Equipment 22 73 8 27
55 Anesthesia Equipment 17 57 13 63
56 Without Equipment 6 20 24 80
57 Medication Labeling Patient's Name 21 70 9 30
58 Drug Name 15 51 15 49
59 Drug Dose 26 87 4 93
60 Hand Hygiene Hygienic Procedure Waterless Alcohol 18 60 12 40
61 Gloves 22 73 8 27
62 Lack of Hygienic Procedure 17 55 13 45
63 Radiation Patient's Age 19 63 11 37
64 Pregnant or Non-Pregnant 29 97 1 3
65 History of Radiation 21 70 9 30
66 Duplicate Imaging Procedures 24 80 6 20
Data Elements Related to Intra- Procedurerowhead
1 Vital Signs Control Blood Pressure 23 77 7 23
2 Heart Rate 22 73 8 27
3 Respiration 18 60 12 40
4 Allergy 22 73 8 27
Data Elements Related to Post-Procedurerowhead
1 Image Labeling Patient's Name 29 97 1 3
2 Procedure Side 25 83 5 17
3 Procedure Site 22 73 8 27
4 Time of Procedure 16 53 14 47
5 Date of Procedure 27 90 3 10
6 Number of Images 19 63 11 37
7 Time of Image Delivery 18 60 12 40
8 Type of Image Delay 21 70 9 30
9 Non- Delay 19 63 11 37
10 Image Delivery Paper 16 53 14 47
11 CD 17 57 13 63
12 System 19 63 11 37
13 Image Reporting Patient Name 29 95 1 5
14 Radiologist Name 28 93 2 7
15 Radiologist Signature 27 90 3 10
16 Attending Physician Name 20 67 10 33
17 Date of Interpretation 25 83 5 17
18 Time of Interpretation 15 50 15 50
19 Delivery of Critical Results Reporting 15 51 15 49
20 Time of Delivery of Critical Results Reporting 21 70 9 30
21 Type of Report Submission Electronically 18 60 12 40
22 Manually 18 60 12 40
23 Image Presentation With Report 27 90 3 10
24 Without Report 17 56 13 34
25 Receive of report by Patient 16 53 14 67
26 Physician 16 53 14 47
27 Resident 17 57 13 63
28 Complications Nephropathy 28 89 2 7
29 Fracture 15 50 15 50
30 Dislocation 17 57 13 43
31 Infection 15 50 15 50
32 Pneumothorax 19 63 11 37
33 Intravenous Rupture 21 70 9 30
34 Blood Pressure Changes 20 67 10 73
35 Dizziness 21 70 9 30
36 Nausea or Vomiting 22 73 8 27
37 Patient Fall Without Injury 17 57 13 43
38 With Injury 23 77 7 23
39 Falling Type From Bed 23 77 7 83
40 While Ambulating 15 50 15 50
Open in a new tab

In step two, the dashboard was developed as a prototype software tool for monitoring and managing safety in MIDs, as shown in Fig. 1, Fig. 2, Fig. 3.

Fig. 1.

Fig. 1

Open in a new tab

The screenshot of the main tabs of dashboard.

Fig. 2.

Fig. 2

Open in a new tab

The screenshot of the pre, intra, and post procedure menus.

Fig. 3.

Fig. 3

Open in a new tab

The screenshot of the report formats.

Tabs (pre-, intra- and post-) contain a collection of data sets relating to safety that are linked to KPI indicators. A drop-down list box was proposed for checking the safety items for each patient.

Furthermore, all KPI data could be further filtered using a variety of selection parameters, including modality, dates (day, week, month, or year), examination, and encounter type. The dashboard dynamically re-visualizes the data in real time after the user selects these filters or a combination of filters.

In step three, first, the correctness of KPI metric computations was individually examined and verified. Second, Table 2 gives an overview of UI verification.

Table 2.

Results of the survey about the usability of the RPSMS.

Row Groups Metrics Yes
 No
NO. % NO. %
1 Suitability for the tasks Support the user to perform all daily tasks 5 %100 0 %0
2 Matching data entry with user tasks 5 %100 0 %0
3 Crossing minimal bottlenecks to perform a task 5 %100 0 %0
4 Display all required information on one page 5 %100 0 %0
5 Easy access to needed commands to perform tasks 5 %100 0 %0
1 Self-descriptiveness Equipped with data dictionary and metadata 5 %100 0 %0
2 Comprehensibility the meaning of messages and commands 5 %100 0 %0
3 Provide practical examples to explain tips 5 %100 0 %0
4 The terminology used in software reflects that of environment 5 %100 0 %0
5 Clarity of screen data fields and commands 5 %100 0 %0
1 Controllability Easy and fast return to start menu 5 %100 0 %0
2 Stop running modules at any time
3 Easy to navigate between screens 5 %100 0 %0
4 Go through a set of fixed steps to perform tasks 5 %100 0 %0
5 Enter a letter, character, or code to access menu items quickly 0 %0 5 %100
1 Conformity with user expectations Easily accomplish the tasks due to the well designed and coordinated software 5 %100 0 %0
2 Estimation of the required time to perform the task 5 %100 0 %0
3 Lexical and semantic integrity in different sub- systems 4 %80 1 %20
4 Use the same keys to perform specified tasks 5 %100 0 %0
5 Display issued messages in a specified part of the screen 5 %100 0 %0
1 Error tolerance Request user approval when performing tasks 5 %100 0 %0
2 Provide useful information about getting out of the wrong situation 5 %100 0 %0
3 System alert about potential error situations 5 %100 0 %0
4 Quick identification of the data entry errors 5 %100 0 %0
5 Easy return to the previous (last) action in case of a mistake 5 %100 0 %0
1 individualization Suitability Customize forms, screens, and menus as desired by the user 5 %100 0 %0
2 Adjust the amount the screen tailored to user needs 5 %100 0 %0
3 Conformity of field order with the current process 0 %0 5 %100
4 Set up input/output devices tailored to user need 5 %100 0 %0
5 Adjust the system speed according to the tasks 5 %100 0 %0
1 Learning Suitability Quick and easy learning to work with software 3 %60 2 %40
2 Easy to re-learn after no long-term use of the system 5 %100 0 %0
3 Access the description to use the system when needed (Online or offline Help) 5 %100 0 %0
4 Uses the software at first without asking colleagues 3 %60 2 %40
5 Recall details and tips for proper use of the system 5 %100 0 %0
Open in a new tab

Third, the result of the usability evaluation using SUS showed the users' satisfaction was at the targeted level (76.3 out of 100), according to Table 3.

Table 3.

Results of SUS measurement.

Row Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Scores
R1 5 1 5 4 5 1 5 1 5 4 85
R2 2 2 4 5 5 2 4 1 4 1 70
R3 5 2 1 2 4 1 5 1 4 4 70
R4 5 1 4 1 5 4 5 1 5 5 80
R5 5 1 5 4 5 1 5 1 5 4 85
R6 4 1 5 2 1 4 1 1 4 4 57.5
R7 5 2 4 4 4 2 4 2 4 4 67.5
R8 5 1 5 4 5 1 5 1 5 4 85
R9 4 1 5 4 4 1 5 2 4 2 80
R10 5 1 5 4 5 1 5 1 4 4 82.5
Total of scores 762.5
Average 76.3
Result of evaluation:
Acceptable(76.3) 		Threshold Threshold
100 85 65 0
Recommended range
Not-acceptable 0–64
Acceptable 65–85
Excellent 85–100
Open in a new tab

R= Respond.

Q = Question.

4. Discussion

All healthcare institutions, according to the Joint Commission, must have a comprehensive strategy for identifying and addressing safety risks. PS in medical imaging procedures can be improved by using a system to assist staff in monitoring and identifying probable PS-related occurrences, as well as reporting adverse events [55,56].

In Reason's Swiss Cheese Model of Medical Error, Brook et al. proposed a diagnostic radiological error classification system that considers personnel, communication, cause, and impact. Their system was designed to detect hidden system weaknesses to reduce the likelihood of future errors and the repercussions of such failures. In this regard, we employed all of Karami's safety-related metrics [15], which included all of the components of the cheese model, for the dashboard.

Swensen et al. created a radiological quality value map that traced the patient's path from the referring physician's office to the radiology department and covered the significant phases of ordering, performing, and reporting an examination. This map is a starting point for understanding ways to improve radiologic examination and intervention in terms of safety, reliability, quality, and appropriateness [40].

Accordingly, we decided to design this dashboard to monitor and measure the safety indicators at three stages: pre-procedure (before the patient enters the radiology department), intra-procedure (when the patient is ready for the procedure), and post-procedure (after the procedure and before the patient exits the radiology department).

Multiple studies [32,35,45,51,57,58] have shown that various strategies and technologies have been used to improve the quality and safety of MIDs. For example, Schultz et al. developed a web-based radiology-specific event reporting system. During its initial development, this system detected and addressed significant safety risks and flaws constructively and straightforwardly [32].

Koetser et al. developed a specific radiological patient safety system (RADPASS) checklist for interventional radiology. This checklist has two parts: A (Planning and Preparation) and B (Procedure). The latter part comprises checks just before starting a procedure (B1) and checks concerning post-procedural care immediately after completion of the procedure (B2). The use of the RADPASS checklist reduced deviations from the optimal process by three-quarters and was associated with fewer procedure postponements [57].

In another study, Schultz et al. offered two methods for standardizing work procedures. The first was a checklist for a time-out routine that would comply with the Universal Protocol and reduce interventional procedure errors. The latter was a flow chart used to evaluate pregnant patients' radiological imaging that exhibits lean standardization of a work process to avoid errors [35].

On the other hand, Rafiei et al. mentioned that “a well-designed checklist should effectively address the underlying failure modes for the adverse events that occur in any particular operational environment. In addition, the checklist should be designed to facilitate the reliable execution of the control strategy for those failure modes. Also, they emphasized that patient safety is better served by allowing local teams to build a checklist from a list of items that match the operational requirements of their working environment.” [58].

Gottumukkala et al. employed a video monitoring and feedback system to establish a checklist-based scoring system to rigorously assess compliance, measure performance, and improve the time-out procedure in pediatric interventional radiology. This system improved time-out performance significantly and could address frequent failure scenarios [51].

Also, in research, Corso et al. designed the “Time-Out” safety checklist to improve PS. This checklist had the potential to decrease adverse events in the first year of usage and increase healthcare team involvement and PS-related knowledge [45].

Consequently, we identified e-checklists for controlling and preventing adverse events. This is due to the fact that PS data collection using paper-and-pencil forms was frequently accompanied by rising workloads (especially in the areas of data processing and storage), inaccuracies in data entry, and a high reliance on human resources.

Afterward, we developed the dashboard as e-monitoring software with pre-determined parameters to control safety in different aspects and modalities. The dashboard alerts for error prevention and reports adverse events in table and chart formats.

As shown in some studies, systems' usability, feasibility, and effectiveness are usually explored in a pilot test after some time [[59], [60], [61], [62], [63], [64], [65], [66], [67], [68], [69], [70]]. As can be seen from the findings of UI verification, all participants disagreed with the criterion “no need for a navigator to see the information” in the category “controllability.” Whereas a dashboard should show all of the data on a single screen at a glance, for the filling of the e-checklist, it was required to navigate to the data.

In the category “suitability for individualization,” the capability to “attach comments to indices” was not confirmed. In light of this, it was attempting to add a feature that would allow users to add comments to indices.

In the “suitability for learning” category, participants explained that it was important to learn how to use the system. An instructional file was prepared for users to remove the problems.

The evaluators also discovered some spelling mistakes, which were all fixed. Finally, they found the dashboard to be user-friendly and capable of meeting the primary UI requirements.

In the usability study, some participants explained that the number of items to be checked in the pre-, intra-, and post-procedure tabs is both enormous and time-consuming. Accordingly, the feature of disabling the checking of some items depending on the type of modalities, patients, and other parameters was added to remove this problem.

It's worth noting that the impact of the dashboard on improving PS in the MID was not assessed in this study.

The review of literature allowed us to determine the MDSs to develop a dashboard for monitoring adverse events in MID. After developing the dashboard; its usability testing showed us its functional deficiencies to fix. Since this research was only a development of dashboard, we did not have any measurable outcome of the intervention.

5. Study limitations

Given that the main objective of this dashboard's development is to adhere to safety indicators and enhance the quality of the MID's services, it is recommended that this system include features like systematic training instructions and scientific resources for each intervention in the form of a decision support system (DSS), integration with other hospital information systems, the ability to filter or modify system reporting based on user and department manager access levels and demands, and the capacity to attach various files to the system.

6. Conclusion

This study provides a picture of how to manage health information to achieve health information technology that can effectively make changes in routine work. Understanding workflow, information flow, and provider needs can play a critical role in developing strategies to design an effective e-monitoring system.

This dashboard can display a more accurate picture of PS status and highlight aspects that would otherwise go unnoticed or underestimated by clinicians. For this purpose, healthcare stakeholders must apply a practical implementation strategy for routine data collection.

Acceptance of the PS e-checklist and dashboard by staff can pave the way to extend the dashboard data collection from pre-, intra-, and post-procedures. Maybe the staff considers it an increasing workload. Therefore, they should be educated and justified. This study was focused on information management and the functionality steps needed to develop a dashboard for monitoring adverse events in MIDs and, in the end, provide visibility of useful and vital information for improving safety in this complex environment. We postponed determining the system's impact and only addressed the problems that arose during the testing phase in response to user comments. In order to evaluate its impact in the future, we will conduct a new research plan.

Certainly, by increasing its use, more feedback will be received from users, which can then be utilized to improve its effectiveness. Depending on the feedback, maybe more advanced functionality will be added to the dashboard, or some of its features will be eliminated.

Funding statement

This study was performed as a research plan supported by the Health Information Management Research Center of Kashan University of Medical Sciences (grant number: 9532).

Data availability statement

Data included in the article, supply material, or cited in the article. Also, the datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

CRediT authorship contribution statement

Mahtab Karami: Writing – review & editing, Writing – original draft, Visualization, Validation, Supervision, Software, Resources, Project administration, Methodology, Investigation, Funding acquisition, Formal analysis, Conceptualization. Nasrin Hafizi: Software, Methodology, Formal analysis, Data curation. Ali-Mohammad Nickfarjam: Visualization, Software. Soheila Refahi: Writing – review & editing, Validation, Data curation.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Footnotes

Appendix A

Supplementary data to this article can be found online at https://doi.org/10.1016/j.heliyon.2024.e30054.

Contributor Information

Mahtab Karami, Email: m.karami@ssu.ac.ir.

Nasrin Hafizi, Email: nhafizi89@yahoo.com.

Ali-Mohammad Nickfarjam, Email: amnickfarjam@gmail.com.

Soheila Refahi, Email: soheila52@yahoo.com.

Appendix A. Supplementary data

The following are the Supplementary data to this article.

Multimedia component 1
mmc1.doc (44KB, doc)
Multimedia component 2
mmc2.doc (34.5KB, doc)
Multimedia component 3
mmc3.doc (676KB, doc)

References

  • 1.Larson D.B., Kruskal J.B., Krecke K.N., Donnelly L.F. Key concepts of patient safety in radiology. Radiographics : a review publication of the Radiological Society of North America, Inc. 2015;35(6):1677–1693. doi: 10.1148/rg.2015140277. [DOI] [PubMed] [Google Scholar]
  • 2.Karami M., Hafizi N. Enhancing patient safety using medical imaging informatics. Radiol. Manag. 2017;39(2):27–35. [PubMed] [Google Scholar]
  • 3.Commission J. Accepted: new and revised hospital elements of performance related to CMS application process. Jt Comm Perspect. 2009;29(10):9–16. [PubMed] [Google Scholar]
  • 4.Li J., Paoloni R., Li L., Callen J., Westbrook J.I., Runciman W.B., et al. Does health information technology improve acknowledgement of radiology results for discharged Emergency Department patients? A before and after study. BMC Med. Inf. Decis. Making. 2020;20:1–7. doi: 10.1186/s12911-020-01135-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Johnson C.D., Miranda R., Osborn H.H., Miller J.M., Prescott S.L., Aakre K.T., et al. Designing a safer radiology department. Am. J. Roentgenol. 2012;198(2):398–404. doi: 10.2214/AJR.11.7234. [DOI] [PubMed] [Google Scholar]
  • 6.Magnavita N., Magnavita G., Fileni A., Bergamaschi A. Ethical problems in radiology: medical error and disclosure. La radiologia medica. 2009;114(8):1345. doi: 10.1007/s11547-009-0445-z. [DOI] [PubMed] [Google Scholar]
  • 7.Jones D.N., Thomas M., Mandel C.J., Grimm J., Hannaford N., Schultz T.J., et al. Where failures occur in the imaging care cycle: lessons from the radiology events register. J. Am. Coll. Radiol. 2010;7(8):593–602. doi: 10.1016/j.jacr.2010.03.013. [DOI] [PubMed] [Google Scholar]
  • 8.Morbi A.H., Hamady M.S., Riga C.V., Kashef E., Pearch B.J., Vincent C., et al. Reducing error and improving efficiency during vascular interventional radiology: implementation of a preprocedural team rehearsal. Radiology. 2012;264(2):473–483. doi: 10.1148/radiol.12110530. [DOI] [PubMed] [Google Scholar]
  • 9.Melvin C., Bodley R., Booth A., Meagher T., Record C., Savage P. Managing errors in radiology: a working model. Clin. Radiol. 2004;59(9):841–845. doi: 10.1016/j.crad.2004.01.016. [DOI] [PubMed] [Google Scholar]
  • 10.Gunderman R.B., Phillips M.D., Cohen M.D. Improving clinical histories on radiology requisitions. Acad. Radiol. 2001;8(4):299–303. doi: 10.1016/S1076-6332(03)80498-1. [DOI] [PubMed] [Google Scholar]
  • 11.Hannaford N., Mandel C., Crock C., Buckley K., Magrabi F., Ong M., et al. Learning from incident reports in the Australian medical imaging setting: handover and communication errors. Br. J. Radiol. 2013;86(1022) doi: 10.1259/bjr.20120336. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Danaher L.A., Howells J., Holmes P., Scally P. Is it possible to eliminate patient identification errors in medical imaging? J. Am. Coll. Radiol. 2011;8(8):568–574. doi: 10.1016/j.jacr.2011.02.021. [DOI] [PubMed] [Google Scholar]
  • 13.Seiden S.C., Barach P. Wrong-side/wrong-site, wrong-procedure, and wrong-patient adverse events: are they preventable? Arch. Surg. 2006;141(9):931–939. doi: 10.1001/archsurg.141.9.931. [DOI] [PubMed] [Google Scholar]
  • 14.Sherman H., Castro G., Fletcher M., Hatlie M., Hibbert P., Jakob R., et al. Towards an international classification for patient safety: the conceptual framework. Int. J. Qual. Health Care. 2009;21(1):2–8. doi: 10.1093/intqhc/mzn054. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Karami M. 2009. Development of Key Performance Indicators for Academic Radiology Departments. [Google Scholar]
  • 16.Karami M., Safdari R. From information management to information visualization: development of radiology dashboards. Appl. Clin. Inf. 2016;7(2):308–329. doi: 10.4338/ACI-2015-08-RA-0104. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Sadoughi F., Ahmadi M., Moghaddasi H., Sheikhtaheri A. Patient safety information system: purpose, structure and functions. Medical Sciences Journal Of Mazandaran University. 2011;21(85):174–188. [Google Scholar]
  • 18.Sheikhtaheria A., Sadoughia F., Ahmadia M., Moghaddasib H. A framework of a patient safety information system for Iranian hospitals: lessons learned from Australia, England and the US. International Jornal of Medical Informatics. 2013;8(2):335–344. doi: 10.1016/j.ijmedinf.2012.06.002. [DOI] [PubMed] [Google Scholar]
  • 19.Classen D.C., Munier W., Verzier N., Eldridge N., Hunt D., Metersky M., et al. Measuring patient safety: the medicare patient safety monitoring system (past, present, and future) J. Patient Saf. 2021;17(3):e234–e240. doi: 10.1097/PTS.0000000000000322. [DOI] [PubMed] [Google Scholar]
  • 20.Kim J., Yoon Y., Seo D., Kwon S., Shim J., Kim J. Real-time patient radiation dose monitoring system used in a large university hospital. J. Digit. Imag. 2016;29:627–634. doi: 10.1007/s10278-016-9880-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Boos J., Meineke A., Bethge O.T., Antoch G., Kröpil P. Dose monitoring in radiology departments: status Quo and future perspectives. Röfo. 2016;188(5):443–450. doi: 10.1055/s-0041-109514. [DOI] [PubMed] [Google Scholar]
  • 22.Shimabukuro T.T., Nguyen M., Martin D., DeStefano F. Safety monitoring in the vaccine adverse event reporting system (VAERS) vaccine. 2015;33(36):4398–4405. doi: 10.1016/j.vaccine.2015.07.035. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Al Salman J.M., Hani S., de Marcellis-Warin N., Isa S.F. Effectiveness of an electronic hand hygiene monitoring system on healthcare workers' compliance to guidelines. Journal of Infection and Public Health. 2015;8(2):117–126. doi: 10.1016/j.jiph.2014.07.019. [DOI] [PubMed] [Google Scholar]
  • 24.Samra R., Bottle A., Aylin P. Monitoring patient safety in primary care: an exploratory study using in-depth semistructured interviews. BMJ Open. 2015;5(9) doi: 10.1136/bmjopen-2015-008128. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Rubin D.L. Informatics in radiology: measuring and improving quality in radiology: meeting the challenge with informatics. Radiographics : a review publication of the Radiological Society of North America, Inc. 2011;31(6):1511–1527. doi: 10.1148/rg.316105207. [DOI] [PubMed] [Google Scholar]
  • 26.Magrabia F., Aartsb J., Nohrc C., Bakerd M., Harrisond S., Pelayoe S., et al. A comparative review of patient safety initiatives for national health information technology. Int. J. Med. Inf. 2013;82(5):139–148. doi: 10.1016/j.ijmedinf.2012.11.014. [DOI] [PubMed] [Google Scholar]
  • 27.Rubio E.I., Hogan L. Time-out: it's radiology's turn--incidence of wrong-patient or wrong-study errors. AJR Am. J. Roentgenol. 2015;205(5):941–946. doi: 10.2214/AJR.15.14720. [DOI] [PubMed] [Google Scholar]
  • 28.Brook O.R., O'Connell A.M., Thornton E., Eisenberg R.L., Mendiratta-Lala M., Kruskal J.B. Quality initiatives: anatomy and pathophysiology of errors occurring in clinical radiology practice. Radiographics : a review publication of the Radiological Society of North America, Inc. 2010;30(5):1401–1410. doi: 10.1148/rg.305105013. [DOI] [PubMed] [Google Scholar]
  • 29.Johnson C.D., Krecke K.N., Miranda R., Roberts C.C., Denham C. Quality initiatives: developing a radiology quality and safety program: a primer. Radiographics. 2009;29(4):951–959. doi: 10.1148/rg.294095006. [DOI] [PubMed] [Google Scholar]
  • 30.Thornton E., Brook O.R., Mendiratta-Lala M., Hallett D.T., Kruskal J.B. Application of failure mode and effect analysis in a radiology department. Radiographics : a review publication of the Radiological Society of North America, Inc. 2011;31(1):281–293. doi: 10.1148/rg.311105018. [DOI] [PubMed] [Google Scholar]
  • 31.Thornton R.H., Miransky J., Killen A.R., Solomon S.B., Brody L.A. Analysis and prioritization of near-miss adverse events in a radiology department. Am. J. Roentgenol. 2011;196(5):1120–1124. doi: 10.2214/AJR.10.5373. [DOI] [PubMed] [Google Scholar]
  • 32.Schultz S.R., Watson R.E., Jr., Prescott S.L., Krecke K.N., Aakre K.T., Islam M.N., et al. Patient safety event reporting in a large radiology department. Am. J. Roentgenol. 2011;197(3):684–688. doi: 10.2214/AJR.11.6718. [DOI] [PubMed] [Google Scholar]
  • 33.Donnelly L.F., Dickerson J.M., Goodfriend M.A., Muething S.E. Improving patient safety in radiology. AJR Am. J. Roentgenol. 2010;194(5):1183–1187. doi: 10.2214/AJR.09.3875. [DOI] [PubMed] [Google Scholar]
  • 34.Abujudeh H.H., Kaewlai R., Asfaw B.A., Thrall J.H. Quality initiatives: key performance indicators for measuring and improving radiology department performance. Radiographics : a review publication of the Radiological Society of North America, Inc. 2010;30(3):571–580. doi: 10.1148/rg.303095761. [DOI] [PubMed] [Google Scholar]
  • 35.Kruskal J.B., Reedy A., Pascal L., Rosen M.P., Boiselle P.M. Quality initiatives: lean approach to improving performance and efficiency in a radiology department. Radiographics. 2012;32(2):573–587. doi: 10.1148/rg.322115128. a review publication of the Radiological Society of North America, Inc. [DOI] [PubMed] [Google Scholar]
  • 36.Sari A.B., Sheldon T.A., Cracknell A., Turnbull A. Sensitivity of routine system for reporting patient safety incidents in an NHS hospital: retrospective patient case note review. BMJ Open. 2007;334(7584):79. doi: 10.1136/bmj.39031.507153.AE. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Tuttle D., Holloway R., Baird T., Sheehan B., Skelton W.K. Electronic reporting to improve patient safety. Qual. Saf. Health Care. 2004;13(4):281–286. doi: 10.1136/qshc.2003.009100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Angle J.F., Nemcek A.A., Cohen A.M., Miller D.L., Grassi C.J., D’agostino H.R., et al. Quality improvement guidelines for preventing wrong site, wrong procedure, and wrong person errors: application of the Joint commission “universal Protocol for preventing wrong site, wrong procedure, wrong person surgery” to the practice of interventional radiology. J. Vasc. Intervent. Radiol. 2008;19(3):1151–1154. doi: 10.1016/j.jvir.2008.03.027. [DOI] [PubMed] [Google Scholar]
  • 39.Stevens A. 2007. Establishing quality indicators for medical imaging and the basic quality management toolbox.http://www.e-radimaging.com/site/article.cfm?ID=277 [Available from: [Google Scholar]
  • 40.Swensen S.J., Johnson C.D. Radiologic quality and safety: mapping value into radiology. J. Am. Coll. Radiol. 2005;2(12):992–1000. doi: 10.1016/j.jacr.2005.08.003. [DOI] [PubMed] [Google Scholar]
  • 41.Yam C.S., Rofsky N., Kruskal J., Sitek A. Development of a radiology report monitoring system for case tracking. AJR Am. J. Roentgenol. 2005;187(1):343–346. doi: 10.2214/ajr.184.1.01840343. [DOI] [PubMed] [Google Scholar]
  • 42.McLoughlin V., Millar J., Mattke S., Franca M., Jonsson P.M., Somekh D., et al. Selecting indicators for patient safety at the health system level in OECD countries. Int. J. Qual. Health Care. 2006;18(Suppl 1):14–20. doi: 10.1093/intqhc/mzl030. [DOI] [PubMed] [Google Scholar]
  • 43.Malik S. 2007. Dashboard evaluation: what are the characteristics of a 'best in class' enterprise dashboard?http://www.dashboardinsight.com/articles/digital-dashboards/fundamentals/dashboard-evaluation.aspx [Available from: [Google Scholar]
  • 44.Nielsen J., Molich R. ACM CHI'90 Conf. 1990. Heuristic evaluation of user interfaces; pp. 249–256. Seattle, WA, 1-5 April. [Google Scholar]
  • 45.Corso R., Vacirca F., Patelli C., Leni D. Use of "Time-Out" checklist in interventional radiology procedures as a tool to enhance patient safety. Radiologia Medica. 2014;119(11):828–834. doi: 10.1007/s11547-014-0397-9. [DOI] [PubMed] [Google Scholar]
  • 46.Gemignani Z. 2008. A Dashboard Alerts Checklist. [Google Scholar]
  • 47.Ondategui-Parra S., Bhagwat J.G., Zou K.H., Nathanson E., Gill I.E., Ros P.R. Use of productivity and financial indicators for monitoring performance in academic radiology departments: U.S. nationwide survey. Radiology. 2005;236(1):214–219. doi: 10.1148/radiol.2361040456. [DOI] [PubMed] [Google Scholar]
  • 48.Runciman W.B., Williamson J.A., Deakin A., Benveniste K.A., Bannon K., Hibbert P.D. An integrated framework for safety, quality and risk management: an information and incident management system based on a universal patient safety classification. Qual. Saf. Health Care. 2006;15(Suppl 1):i82–i90. doi: 10.1136/qshc.2005.017467. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 49.wikibooks . 2009. Business Intelligence/Evaluate Dashboard Prototype.http://en.wikibooks.org/wiki/Business_Intelligence/Evaluate_Dashboard_Prototype [Available from: [Google Scholar]
  • 50.Bernie S. 2012. The brilliant dashboards checklist: a systemic approach to assessing your dashboard.https://madetomeasurekpis.com/tag/analysis [cited 2023]; Available from: [Google Scholar]
  • 51.Gottumukkala R., Street M., Fitzpatrick M., Tatineny P., Duncan J.R. Improving team performance during the preprocedure time-out in pediatric interventional radiology. Joint Comm. J. Qual. Patient Saf. 2012;38(9):387–394. doi: 10.1016/s1553-7250(12)38049-5. [DOI] [PubMed] [Google Scholar]
  • 52.Prashant N., Ravi J., Deven J. Delphi methodology in healthcare research: how to decide its appropriateness. World J. Methodol. 2021;11(4):116–129. doi: 10.5662/wjm.v11.i4.116. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 53.Shanbe Zadeh M., Jamshidian M., Yousefian O., Kazemi H. Evaluating the usability of hospital information systems based on ISO 9241-10 standard: a cross-sectional study. Applied Health Information Technology. 2021;2(1):20–29. [Google Scholar]
  • 54.Brooke J. SUS-A quick and dirty usability scale Usability Eval Industry. 1996;189(194):4–7. [Google Scholar]
  • 55.Meeks D.W., Smith M.W., Taylor L., Sittig D.F., Scott J.M., Singh H. An analysis of electronic health record-related patient safety concerns. J. Am. Med. Inf. Assoc. : JAMIA. 2014;21(6):1053–1059. doi: 10.1136/amiajnl-2013-002578. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 56.Schneider E.C., Ridgely M.S., Meeker D., Hunter L.E., Khodyakov D., Rudin R.S. Promoting patient safety through effective health information technology risk management. Rand health quarterly. 2014;4(3):7. [PMC free article] [PubMed] [Google Scholar]
  • 57.Koetser I.C.J., de Vries E.N., van Delden O.M., Smorenburg S.M., Boermeester M.A., van Lienden K.P. A checklist to improve patient safety in interventional radiology. CVIR (Cardiovasc. Interventional Radiol.) 2013;36(2):312–319. doi: 10.1007/s00270-012-0395-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 58.Rafiei P., Walser E.M., Duncan J.R., Rana H., Ross J.R., Kerlan R.K., Jr., et al. Society of interventional radiology IR pre-procedure patient safety checklist by the safety and health committee. J. Vasc. Intervent. Radiol. : J. Vasc. Intervent. Radiol. 2016;27(5):695–699. doi: 10.1016/j.jvir.2016.03.002. [DOI] [PubMed] [Google Scholar]
  • 59.Bellgard M.I., Render L., Radochonski M., Hunter A. Second generation registry framework. Source Code Biol. Med. 2014;9(1):1–6. doi: 10.1186/1751-0473-9-14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Guo J., Iribarren S., Kapsandoy S., Perri S., Staggers N. Usability evaluation of an electronic medication administration record (eMAR) application. Appl. Clin. Inf. 2011;2(2):202. doi: 10.4338/ACI-2011-01-RA-0004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 61.Peute L.W., Jaspers M.W. The significance of a usability evaluation of an emerging laboratory order entry system. Int. J. Med. Inf. 2007;76(2–3):157–168. doi: 10.1016/j.ijmedinf.2006.06.003. [DOI] [PubMed] [Google Scholar]
  • 62.Joe J., Chaudhuri S., Le T., Thompson H., Demiris G. The use of think-aloud and instant data analysis in evaluation research: exemplar and lessons learned. J. Biomed. Inf. 2015;56:284–291. doi: 10.1016/j.jbi.2015.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Tang Z., Johnson T.R., Tindall R.D., Zhang J. Applying heuristic evaluation to improve the usability of a telemedicine system. Telemed. J. e Health. 2006;12(1):24–34. doi: 10.1089/tmj.2006.12.24. [DOI] [PubMed] [Google Scholar]
  • 64.Peute L.W., de Keizer N.F., Jaspers M.W. The value of Retrospective and Concurrent Think Aloud in formative usability testing of a physician data query tool. J. Biomed. Inf. 2015;55:1–10. doi: 10.1016/j.jbi.2015.02.006. [DOI] [PubMed] [Google Scholar]
  • 65.Alhadreti O., Mayhew P., editors. Proceedings of the 30th International BCS Human Computer Interaction Conference 30. 2016. ―Thinking about thinking aloud‖: an investigation of think- aloud methods in usability testing. [Google Scholar]
  • 66.Zakaria N., Wahabi H., Al Qahtani M. Development and usability testing of Riyadh Mother and Baby Multi-center cohort study registry. Journal of infection and public health. 2020;13(10):1473–1480. doi: 10.1016/j.jiph.2020.02.035. [DOI] [PubMed] [Google Scholar]
  • 67.Shahraki A.D., Ghabaee M., Shahmoradi L., Safdari R., Malak J.S. The evaluation of a web-based stroke quality registry system. Journal of registry management. 2018;45(4):173–176. [PubMed] [Google Scholar]
  • 68.Berg M. Implementing information systems in health care organizations: myths and challenges. Int. J. Med. Inf. 2001 Dec;64(2–3):143–156. doi: 10.1016/s1386-5056(01)00200-3. [DOI] [PubMed] [Google Scholar]
  • 69.Solomon D.J., Henry R.C., Hogan J.G., Van Amburg G.H., Taylor J. Evaluation and implementation of public health registries. Publ. Health Rep. 1991 Mar;106(2):142. [PMC free article] [PubMed] [Google Scholar]
  • 70.Mehmood A., Razzak J.A., Kabir S., Mackenzie E.J., Hyder A.A. Development and pilot implementation of a locally developed Trauma Registry: lessons learnt in a low-income country. BMC Emerg. Med. 2013 Mar 21;13:4. doi: 10.1186/1471-227X-13-4. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia component 1
mmc1.doc (44KB, doc)
Multimedia component 2
mmc2.doc (34.5KB, doc)
Multimedia component 3
mmc3.doc (676KB, doc)

Data Availability Statement

Data included in the article, supply material, or cited in the article. Also, the datasets used and analyzed during the current study are available from the corresponding author on reasonable request.

Articles from Heliyon are provided here courtesy of Elsevier

RESOURCES