Impact of IPMOE on nursing tasks in the medical ward: A time-motion study

Abstract

Introduction

The In-patient Medication Order Entry System (IPMOE) was first implemented in the medical ward of Princess Margaret Hospital, Hong Kong. It was a local developed close-loop system including prescription, dispensing and administration modules. Evaluation on its impact on nursing tasks would be important for practice improvement and subsequent system enhancement.

Objective

The study was conducted to quantify the nursing times across medication-associated tasks for paper-based MAR and computer-based IPMOE, including change in the tasks and time patterns before and after IPMOE implementation.

Methods

This was a prospective observation study in medical wards before (Jan 2014–Jun 2014) and after (Mar 2015–Jun 2015) the implementation of IPMOE. We conducted 8-hr observation studies of individual nurses with a customized application to time various pre-categorized nursing tasks. Statistical inferences and interrupted time series analysis was performed to identify the change in the intercept and trends over time after implementation.

Result

The average number of medication-related tasks was significantly reduced from 61.07 to 29.81, a reduction of 31.26 episodes per duty (P < 0.001, 95% CI 22.9–39.63). The time for the medication-related tasks was reduced from 32 min (SD = 21.57) to 26.57 min (SD = 11.35) and the medication administration time increased from 37.93 min (SD = 14.78) to 44.37 min (SD = 19.45), but there was no overall significant difference in the time spent on each duty (P = 0.315) between the two groups. An improving trend in the delayed effect was observed (P = 0.03), which indicated a run-in period for new application was needed in clinical setting.

Conclusion

Our study had shown the time motion observation could be applied to measure the impact of the IPMOE in a busy clinical setting. Through classification of activities, validation, objective measurement and longitudinal evaluation, the method could be applied in various systems as well as different clinical settings in measure efficiency.

1. Introduction

The Computerized Physician Order Entry (CPOE) system developed in 1971 was mainly designed for clinical decision support and helped physicians make prescriptions by following simple steps on a computer. Several commercial CPOE systems with many additional functions have been developed since 1988. The aim of the CPOE system is to enhance patient safety by reducing medication error along the continuum from prescription to administration. There has been much debate over what to include in the COPE system such as the capacity to handle physician orders, alerts of possible errors, and recording responses if an alert is overridden. A recent study defined CPOE as an application system that supports online medication ordering at the point of care [1]. With the increasing number of commercial CPOE systems available on the market, various evaluations have been carried out to identify their respective impacts on clinical efficiency and effectiveness, particularly in regards to errors and patient safety. There have been encouraging findings to support the implementation of CPOE in reducing the hospital length of stay [2,3], reducing medication errors [4], simplifying and optimizing nursing services [5], decreasing the risk of preventable harm [6], enhancing nurse-physician communication [7], and improving patient outcomes [8]. On the other hand, some drawbacks have been identified such as burdens on providers due to the extra time and effort needed [9], and suggestions for better integration into clinical practice have also been made [10]. Because clinicians can have different attitudes toward the CPOE system and its perceived impact largely depends on the clinical context [11], there is an important need to conduct further evaluation research to address the question of whether CPOE can enhance efficiency and to explore its impact on clinical workflows.

In Hong Kong, the in-patient CPOE is called the In-patient Medication Order Entry System (IPMOE), which was developed under the Hong Kong Hospital Authority to support prescription, dispensing and administration. The pilot system was implemented at the Princess Margaret Hospital (PMH) in 2013. The IPMOE system was developed as a module in the Hospital Authority's clinical management system, which includes a closed loop system for doctors to make prescriptions with digital signatures incorporating government e-cert scheme, for dispensing prescriptions with integration to the pharmacy dispensing system, and for administration supported by a wifi connection at the point of care. Medication administration was considered to be one of the major impacted areas, because it required a major change of practice from paper-based administration to computer-based administration, including scheduling, checking prescriptions through barcode scanning, and reminders for outstanding/follow-up tasks. Previous evaluation has been done to measure the impact on medication errors by failure mode and effect analysis (FMEA), which estimated the criticality index to reflect the quantified perception of risk from users. In the captioned evaluation of IPMOE, the overall criticality index was dropped from 1005(pre-stage) to 519 (post-stage), which implied to have a positive impact on likelihood of occurrence, chance of detection and overall severity.

In addition, it was estimated that the IPMOE system would save nursing time in dealing with paper documentation, screening, and managing administration schedules of respective patients. Due to significant changes in the workflow, it was expected that more time would be spent on administration tasks, as more emphasis was placed on preventing medication errors. Therefore, we conducted a study on the shift in the proportion of time nurses spent on the new system and its overall impact on nursing activities, which would provide a yard stick for system enhancement and evidence-based informatics on the CPOE implementation.

This study measured the impact on nursing time in a busy medical ward before and after the IPMOE implementation, and assessed its operation by exploring the shift in the proportion of nursing tasks.

2. Objective

This study was designed to evaluate the impact of IPMOE on the amount of time nurses spent on medication-associated tasks in busy medical wards. Specifically, the study aimed to identify:

• −The amount of time nurses spent across medication-associated tasks between the paper-based medication administration record (MAR) and IPMOE.
• −The shift in the proportion of nursing tasks after IPMOE implementation.
• −Differences in the amount of time nurses spent on medication-related tasks before and after IPMOE implementation.

3. Methodology

3.1. Design

A time and motion study approach was adopted, which involved classifying the medication tasks into simple steps. Performing the sequence of steps was carefully observed and each step was precisely timed.

3.2. Setting

Princess Margaret Hospital is a major acute general hospital in the Kowloon West Cluster, Hong Kong. It is the cluster referral center for various specialties and is equipped with 1442 beds and supported by 3700 staff. It provides services for patients mainly from the Kwai Tsing district with a population over 500,000. IPMOE was first implemented in PMH in 2013. Two busy acute medical wards in medicine and geriatrics departments were selected for the study because these wards had the highest numbers of medication-related activities.

3.3. Data

The pilot study assessed two types of activities, medication-related tasks and medication administration tasks. These activities were broken down as follows:

• Medication-related tasks

  • −
    Post-medical round prescription checks/screening
  • −
    Verification of new/changed prescription by medical staff
  • −
    Verification of new/changed prescription by other nurses
  • −
    Checking MAR upon patient discharge
  • −
    Faxing MAR and tracing by the Pharmacy
  • −
    Patient education
  • −
    Documentation
  • −
    Refill medication
  • −
    Nurse handovers
  • −
    Other
• Medication administration tasks

  • −
    Looking up MAR
  • −
    Medication preparation
  • −
    Medication delivery
  • −
    Documentation in MAR

Participant demographic data including years of experience in existing specialty was used in the subsequent correlation analysis.

3.4. Data collection

We adopted a convenient sampling approach, which included 8 h of observations of individual nurses in three medical wards of PMH carried out by trained observers. The researchers met with the staff nurses and their managers to brief them on the study. The study's aims were explained to nurses before each observation session. A customized application with an electronic data collection form was used to time various pre-categorized nursing tasks including medication-related tasks (10 categories) and medication administration (4 categories) during each observation session (Fig. 1). Data entry included start time, end time, and the selected task. The data was exported to a spread sheet for subsequent analysis.

Fig. 1.

The electronic platform with classified tasks for the time-motion study.

3.5. Analysis

Data was analyzed using SPSS 22 to obtain descriptive statistics including frequencies and percentages of demographic and task variables. Parametric and non-parametric statistical tests including Chi-square Test, independent sample T-test, Mann-Whitney test, Wilcoxon Signed Rank test, and Kruskal Wallis test were applied as appropriate to compare means scores. In addition, an interrupted time series analysis was performed to identify changes in the intercept and trends over time after implementation.

3.6. Ethical considerations

The identity of participants remained anonymous. All associates, colleagues and staff assisting in the study were made aware of their obligations by using ethical questionnaires. All data was encrypted to maintain security. A pseudo code was assigned to the data and no personal identifiers were associated with the dataset. The investigators declare no potential conflicts of interest.

4. Results

4.1. Participant characteristics

Demographic characteristics of the participants are shown in Table 1. There were a total 83 participants (pre: n = 41, post: n = 42). The majority of participants were female (pre: n = 36, 87.80%, post: n = 38, 90.48%). Most participants were educated to at least Bachelor's degree level (pre: n = 38, 92.68%; post: n = 37, 73.81%). Their ages ranged from 26 to 40 years in both the pre and post groups. Most participants had more than 5 years of experience working in their present specialty or working in their present ward since graduating. There were no significant differences between the pre and post groups in their gender (P = 0.62), age (P = 0.44), education (P = 0.84), IPMOE training (P = 0.11), years of experience after graduation (P = 0.92), years of experience in their present specialty (P = 0.93), and years of experience in their present ward (P = 0.93).

Table 1.

Demographic characteristics of participants.

Characteristics	Pre-IPMOE		Post-IPMOE		P-value
	n	%	n	%
Gender
Male	5	12.20	4	9.52	0.62
Female	36	87.80	38	90.48
Age
Under 20 years	0	0.00	0	0.00	0.44
20 - 25 years	9	21.95	8	19.05
26 - 30 years	8	19.51	9	21.43
31 - 35 years	13	31.71	8	19.05
36 - 40 years	7	17.07	14	33.33
41 - 45 years	3	7.32	2	4.76
46 - 50 years	1	2.44	0	0.00
50 years or above	0	0.00	1	2.38
Education
Bachelor	32	78.05	31	73.81	0.84
Master	6	14.63	6	14.29
Other	4	9.76	5	11.90
IPMOE training
Yes	38	92.68	34	80.95	0.11
No	3	7.32	8	19.05
Year of experience after graduation
0 - 5 years	16	39.02	16	38.10	0.92
6 - 15 years	16	39.02	18	42.86
more than 16 years	9	21.95	8	19.05
Year of experience in present specialty
0 - 5 years	20	48.78	19	45.24	0.93
6 - 15 years	15	36.59	17	40.48
more than 16 years	6	14.63	6	14.29
Year of experience in present ward
0 - 5 years	21	51.22	20	47.62	0.93
6 - 15 years	16	39.02	17	40.48
more than 16 years	4	9.76	5	11.90

4.2. Time spent on each task

The distribution of time spent on each associated task as measured from all the observations is shown in Table 2. Overall, there were no significant differences in the time spent on the medication-related tasks, except time spent on “post-medical round prescription check/screening” was significantly increased (9.62 s, P < 0.01). We expect the extra time will be needed to adjust to a new system, especially when working with a new computer interface compared to traditional tasks performed on the paper-based MAR system. The time to perform medication administration tasks was observed to be significantly longer for “look up MAR” (17.13 s, P < 0.01) and “medication preparation” (16.16 s, P < 0.01). The time needed to scan a bar code and visually check against the information of each prescription was longer than visually checking the paper MAR and medication labels. On the other hand, the time for “medication delivery” (−6.34 s, P < 0.01) was observed to be significantly decreased, which could be explained as time saved by scanning the bar code to identify the patient at the bedside.

Table 2.

Distribution of time spent on each associated task.

Tasks	Time (in seconds) spent on the associated task (Pre-PMOE)				Time (in seconds) spent on the associated task (Post-IPMOE)				Change	P-value
	Frequency	Mean	SD	Median	Frequency	Mean	SD	Median
Medication-related tasks
Post-medical round prescription check/screening	677	22.07	47.54	9	245	31.69	30.28	22	9.62	0.00
Verify new/changed prescription with medical staff	171	18.82	18.17	13	38	18.39	17.03	10	−0.43	0.89
Verify new/changed prescription with other nurses	690	13.34	51.65	7	306	16.08	27.06	11	2.74	0.38
Check MAR upon patient discharge	71	36.17	34.38	29	14	60.93	54.02	42	24.76	0.07
Fax MAR and trace with Pharmacy	284	17.46	13.64	14	25	26.20	25.76	18	8.74	0.14
Patient education	129	30.94	40.30	16	55	40.04	48.92	26	9.10	0.19
Documentation	96	28.76	26.28	19	74	29.47	32.83	16	0.71	0.88
Refill medication	27	46.22	36.64	43	14	57.00	106.44	18.5	10.78	0.11
Nurse handover	319	66.94	236.55	10	401	52.08	180.47	12	−14.86	0.35
Others	40	19.80	15.34	16	23	21.83	23.57	13	2.03	0.95
Medication administration tasks
Look up MAR	872	9.17	12.13	5	867	26.30	24.09	19	17.13	0.00
Medication preparation	1149	36.91	41.35	25	1120	53.07	56.90	35	16.16	0.00
Medication delivery	787	28.70	54.57	17	885	22.36	29.07	12	−6.34	0.00
Documentation in MAR	759	6.16	15.97	3	611	6.62	5.37	5	0.46	0.46

4.3. Number of tasks per shift

The distribution of the number of tasks per shift is shown in Table 3. A total of 2504 episodes were recorded from 41 shifts before IPMOE implementation and 1252 episodes were recorded from 42 shifts after IPMOE implementation. We observed significant reductions in the frequency of the tasks of “post-medical round prescription check/screening” (−10.38, P < 0.01), “verify new/changed prescription with medical staff” (−2.26, P < 0.01), “verify new/changed prescription with other nurses” (−8.78, P < 0.01), and “fax MAR and trace with Pharmacy” (−4.85, P < 0.01). These reductions could be explained by the automation of information and the ability to display a summary, which decreased the number of required tasks in daily operations. On the other hand, we observed no significant changes in the medication administration tasks, as the frequency of such tasks depends on the number of patients and prescriptions. There was a shift in the proportion of nursing tasks from general medication-associated tasks to administration tasks, which we postulate was due to the increased focus on patient safety.

Table 3.

Distribution of the number of tasks per shift.

Tasks	Distribution of number of task per shift (Pre-IPMOE)				Distribution of number of tasktask per shift (Post-IPMOE)				Change	P-value
	Total Frequency	Number of Nurses handled the task	Average frequency per shift	Median	Total Frequency	Number of Nurses handled the task	Average frequency per shift	Median
Medication-related tasks
Post-medical round prescription check/screening	677	41	16.51	15	248	40	6.13	5.5	−10.38	0.00
Verify new/changed prescription with medical staff	171	38	4.50	4	39	17	2.24	2	−2.26	0.00
Verify new/changed prescription with other nurses	690	41	16.83	16	330	38	8.05	5.5	−8.78	0.00
Check MAR upon patient discharge	71	24	2.96	2	14	10	1.40	1	−1.56	0.07
Fax MAR and trace with Pharmacy	284	41	6.93	7	25	12	2.08	1	−4.85	0.00
Patient education	129	37	3.49	3	55	20	2.75	2	−0.74	0.38
Documentation	96	30	3.20	2	77	26	2.85	2	−0.35	0.56
Refill medication	27	18	1.50	1	14	7	2.00	1	0.50	0.46
Nurse handover	319	35	9.11	6	427	40	10.03	10	0.92	0.61
Others	40	14	2.86	1	23	12	1.92	2	−0.94	0.93
Medication administration tasks
Look up MAR	872	41	21.27	20	917	41	21.15	21	−0.12	0.94
Medication preparation	1149	41	28.02	25	1163	41	27.32	27	−0.70	0.75
Medication delivery	787	41	19.20	20	923	41	21.59	21	2.39	0.08
Documentation in MAR	759	41	18.51	16	641	40	15.28	15	−3.23	0.10

4.4. Time spent on specific tasks per shift

The distribution of time spent on specific tasks per shift is shown in Table 4. We estimated the net time spent on specific tasks per shift, adjusting for the effects of the changes in time spent and frequency per shift. Overall, the time spent on medication-associated tasks was reduced, except for patient education, but this may depend on patient interaction rather than on system automation. We observed significantly reduced times for “verify new/changed prescription with medical staff” (−43.59 s, P < 0.01), “verify new/changed prescription with other nurses” (−95.01 s, P = 0.03), and “fax MAR and trace with Pharmacy” (−66.40 s, P < 0.01). The time for medication administration tasks were significantly increased for “look up MAR” (361.22 s, P < 0.01) and “medication preparation” (415.44 s, P < 0.01), which could be explained as more time spent scanning the bar code for validation of the medication information against the prescription on the computer. We postulate the increased time spend in this process was due to enhanced patient safety.

Table 4.

Distribution of time spent on associated tasks per shift.

Tasks	Time (in seconds) spend on associated tasks per shift (Pre-PMOE)				Time (in seconds) spend on associated tasks per shift (Post-IPMOE)				Change	P-value
	Total Frequency	Number of Nurses handled the task	Average total time per duty	Median	Total Frequency	Number of Nurses handled the task	Average total time per duty	Medium
Medication-related tasks
Post-medical round prescription check/screening	677	41	364.39	233.00	248	40	194.12	176.00	−170.27	0.86
Verify new/changed prescription with medical staffs	171	38	84.71	63.50	39	17	41.12	22.00	−43.59	0.00
Verify new/changed prescription with other nurses	690	41	224.46	166.00	330	38	129.45	108.50	−95.01	0.03
Check MAR upon patient discharge	71	24	107.00	77.00	14	10	85.30	61.00	−21.70	0.84
Fax MAR and trace with Pharmacy	284	41	120.98	114.00	25	12	54.58	32.50	−66.40	0.00
Patient education	129	37	107.86	65.00	55	20	110.10	69.50	2.24	0.74
Documentation	96	30	92.03	72.00	77	26	83.88	52.00	−8.15	0.46
Refill medication	27	18	69.33	49.50	14	7	114.00	42.00	44.67	0.83
Nurse handover	319	35	610.11	344.00	427	40	522.15	330.50	−87.96	0.55
Others	40	14	56.57	30.50	23	12	41.83	28.50	−14.74	0.50
Medication administration tasks
Look up MAR	872	41	195.00	151.00	917	41	556.22	520.00	361.22	0.00
Medication preparation	1149	41	1034.34	971.00	1163	41	1449.78	1473.00	415.44	0.00
Medication delivery	787	41	550.88	465.00	923	41	482.66	397.00	−68.22	0.43
Documentation in MAR	759	41	114.00	59.00	641	40	101.05	520.00	−12.95	0.69

4.5. Interrupted time series analysis before and after IPMOE

A two-period time series was plotted from January to June 2014 (pre-IPMOE) and from March to June 2015 (post-IPMOE). The time count in seconds was adjusted by the number of patients. Forecast by linear regression (Fig. 2) identified improvement trends, such as the time to complete a set of tasks was gradually reduced (P = 0.008) by 25.27 s per time point of observation. To align the period of observation and to adjust for seasonal effects, only the same period in 2014 and 2015 was included in the comparison in Fig. 3. An interrupted time series was applied from week 9 to week 22 in 2014 and 2015 to gauge the changes in the trends over time. We observed no significant changes between the series (P = 0.248), although there was a progressive reduction of 18.51 s per time point of observation.

Fig. 2.

Interrupt time series before and after IPMOE implementation.

Fig. 3.

Interrupt time series before and after IPMOE implementation with adjustment for seasonal effects.

5. Discussion

5.1. Main findings

Out of the ten medication-related tasks, most of the activities (8 out of 10 tasks) showed a reduction in the average number of tasks per duty. These reductions were observed in the “post-medical round prescription check/screening” (−10.38, P < 0.01), “verify new/changed prescription with medical staff” (−2.26, P < 0.01), “verify new/changed prescription with other nurses” (−8.78, P < 0.01), and “fax MAR and trace with Pharmacy” (−4.85, P < 0.01) tasks. For the total time required for the medication-related tasks per duty, most of the activities (8 out of 10 tasks) had reduced total time to complete per shift, although only three tasks showed significantly reduced times, including “verify new/changed prescription with medical staff” (−43.59 s, P < 0.01), “verify new/changed prescription with other nurses” (−95.01 s, P = 0.03) and “fax MAR and trace with Pharmacy” (−66.40 s, P < 0.01). After adjusting for the number of patients under each nurse, the time for most medication-related activities (9 out of 10 tasks) was reduced, although only two tasks showed significantly reduced times, including “verify new/change prescription with medical staff” (−2.83 s, P = 0.03) and “fax MAR and trace with Pharmacy” (−6.38 s, P < 0.01).

For the medication-related tasks, the number of tasks dropped after introduction of the IPMOE system as expected. The total number of the episodes per shift dropped from 67.88 before IPMOE implementation to 39.45 after IPMOE implementation, representing a 41.93% reduction. The time reductions equated to 460.91 s (25.1%) reduction per shift and 41.95 s (20.4%) reduction per patient.

For the medication administration tasks, the number of episodes per shift between pre-IPMOE and post-IPMO showed no significant reductions, although the times for two tasks were found to be reduced but were not significant, “medication delivery” (−68.22 s, P = 0.431) and “documentation in MAR” (−12.95 s, P = 0.687). In contrast, the times for “look up MAR (361.22 s, P < 0.01) and medication preparation (415.44 s, P < 0.01) per shift were significantly increased. Overall, the total time for medication administration tasks was increased by 695.49 s per shift and 114.91 s per patient.

There was a shift in the proportion of nursing tasks with a reduction in the frequency and time for medication-related tasks. Correspondingly, the time for medication administration tasks was increased, mainly in the time spent on “look up MAR” and “medication preparation” tasks, which led to an overall increase of 234.58 s per shift and 72.96 s per patient.

To understand the increase in nursing times, we examined two periods before (January 2014 to June 2014) and after (March 2015 to June 2015) IPMOE implementation. A longitudinal change was observed after IPMOE implementation with a delayed improvement trend of −18.51 s per time point of observation after adjusting for seasonal effects.

5.2. Favorable implications

This study identified shifts in the proportion of nursing tasks from general medication-related tasks to medication administration tasks. As in previous studies [12], we found IPMOE changed the clinical workflow by removing certain medication-related tasks (e.g., looking up the paper records, transcribing orders, fax communication with the pharmacy department). However, more time was spent on medication administration tasks, which included system-generated schedules for administration, barcode scanning for patient and drug identification, and reminders for outstanding and follow-up tasks via a computerized dash board. In addition, some interruptions to workflow were reduced, such as the number of faxes and phone calls between the ward and pharmacy for checking prescriptions through to dispensing, which was also evidenced in other CPOE studies [13]. We expect patient safety would be enhanced through modifications to the workflow and medication administration, which would reduce medication error, streamline workflow, enhance communication among caregivers, and improve medication documentation.

5.3. Challenging implications

We observed the use of the IPMOE system increased the time for nursing tasks such as “look up the MAR” and “medication preparation”. There were several cases of difficulty in using the system due to interfacing between the hardware and software, including the response and sensitivity of the touch screen. The nurses also found it difficult to interact with the computers particularly when using fingers to interface with the computer. Different from previous studies [14], this was not an issues due to computer literacy, because there were no significant differences in the times between the stratified groups by years of nursing experience in this study. A previous study showed that nurses had a positive attitude toward the use of computers in healthcare [15], meanwhile hardware issues were considered to be an important factor to facilitate the use of such systems [16].

5.4. Implication for further study

The interrupted time series from this study identified an improved time saving trend in the nursing tasks. Longitudinal evaluation should be used to evaluate the trends in time spent, changes in the distribution of tasks, and differences in the proportion of time spent over the study period. A reasonable run-in period should be allowed before starting such data collection, as it is anticipated such results would change over this time. As staff becomes more familiar with the system and hardware, it is anticipated that the time spent will decrease along the continuum of system adaptation. In addition, modifications to the system and hardware may occur during the initial period, which may confound the evaluation data. Overall, we suggest longitudinal measurements with at least three phases of data collection after a reasonable run-in period.

5.5. Limitations

The study adopted a convenient sampling approach for the pre- and post-IPMOE data collection, which might not account for changes in the population during the two study periods. In addition, paired comparison could not be made due to limitations of operation and shift duty characteristics. The study might not be generalized to a surgical stream ward because there may be differences in the tasks and workflows between the two such specialties. Even with these limitations, the results from this study could be used to enhance CPOE implementation and inform further research in this area for complete rollout of IPMOE in Hong Kong.

6. Conclusion

This study showed the amount of time spent by nurses across medication-related tasks per shift between paper-based MAR and computer-based IPMOE were 32 min and 26.57 min, respectively, whereas time spent on medication administration per shift were 37.93 min and 44.37 min, respectively. This equates to an overall additional time of 3.91 min per shift under the IPMOE system. It should be noted that the time differences between the two study periods may overlap with system run in period. The seasonal-adjusted interrupted time series analysis also found an improving trend of a reduction of 18.51 s along each time point of measurement. The shift in the proportion of nursing tasks after IPMOE implementation and the significant reduction in the number of medication-related task will allow more time to focus on medication administration to enhance patient safety. For example, barcode scanning was adopted to check medications against the prescription and patient identity. The study highlights the need to adopt a longitudinal approach for evaluating the impact of CPOE implementation on efficiency, workflow, and the nurse-system interaction, and could identify changes in time patterns and areas for enhancement. Our study had shown the time motion observation could be applied to measure the impact of the IPMOE in a busy clinical setting. Through classification of activities, validation, objective measurement and longitudinal evaluation, the method could be applied in various system as well as different clinical settings in measure efficiency.

Conflicts of interest

There should be no conflicts of interest.

Appendix A. Supplementary data

The following is the supplementary data related to this article: