Enhancing clinical effectiveness of pre-radiotherapy workflow by using multidisciplinary-cooperating e-control and e-alerts

Abstract

Radiotherapy (RT) is useful in managing cancer diseases. In clinical practice, early initiation of RT is crucial for enhancing tumor control. But, delivering precise RT requires a series of pre-RT working processes in a tight staff-cooperation manner. In this regard, using information system to conduct e-control and e-alerts has been suggested to improve practice effectiveness; however, this effect is not well defined in a real-world RT setting.

We designed an information system to perform e-control and e-alerts for the whole process of pre-RT workflow to shorten processing time, to improve overall staff satisfaction, and to enhance working confidence.

A quality-improving study conducted in a large RT center.

Externally validated data were retrospectively analyzed for comparison before (from Sep. 2012 to Dec. 2012, n = 223) and after (from Sep. 2013 to Dec. 2013, n = 240) implementation of pre-RT e-control and e-alerts.

Applying the e-control with delay-working e-alerts in pre-RT workflow was the main intervention.

Nine workstations were identified in pre-RT workflow. The primary outcome measure was the processing time in each pre-RT workstations before and after implementing the e-control and e-alerts. Secondary measures were staff-working confidence and near-missing cases during the process of pre-RT workflow.

After implementing e-control, overall processing time of pre-RT workflow was shortened from 12.2 days to 8.9 days (P < .001). Follow-up data (till Jul. 2016) showed a durable effect of 9.2 days, being still below the predefined threshold of <10 days.

Using a multidisciplinary-cooperating information system is useful to conduct e-control and e-alerts in the whole process of pre-RT workflow. Clinical effectiveness, staff satisfaction, and working confidence are able to be enhanced obviously.

KEY POINTS

• Strength 1: Using a multidisciplinary-cooperating information system to conduct e-control and to increase effectiveness of pre-RT workflow is the main study strength.

• Strength 2: The present study confirmed clinical benefits of applying e-control and e-alerts on pre-RT workflow in a real-world medical setting in a large RT center.

• Strength 3: Additional benefits of implementing pre-RT e-workflow are enhancements of staff satisfaction and working confidence, leading to a decrease of staff-turnover rate.

• Limitation 1: Although the analyzed data are validated both internally and externally, the present study investigates these data in a retrospective manner; thus, potential biases are inevitably existed, such as unobserved variables, which may have affected our results.

• Limitation 2: Our data are obtained from a single RT department; further parallel expanding to gain more data (eg, clinical outcome and user experience) is required.

1. Introduction

1.1. Radiotherapy and the role of pre-RT workflow

Radiotherapy (RT) is effective in managing cancer diseases,^[1] but it requires a series of pre-RT workflow to provide precision and safety of irradiation delivery.^[2–4] For example, well-fixed simulation by using three- or four- dimensional computed tomography (CT) is essential to convey a high reproducibility during the whole course of RT. In addition, carefully target contouring by using a highly matched image-fusion technique plays a cornerstone for improving tumor controls. And, a well-designed RT treatment planning is crucial to reduce RT side effects.

However, a clinical dilemma exists in performing pre-RT workflow. That is, for more precise irradiation delivering, adequate processing time is required for each workstation. But, delayed initiation of RT harbors risks of decreasing tumor control, resultantly increasing potential cancer recurrence likelihood. For example, in head and neck cancer patients, initiating RT more than 6 weeks after surgery impairs clinical outcomes.^[5] As a result, most—if not all—modern RT departments defined clear goals to increase clinical effectiveness of pre-RT workflow, with intentions of shortening processing time but without a cost of impairing care quality.

1.2. One clinical constraint in shortening pre-RT workflow in large RT centers: a lack of systemic mechanism to effectively alert processing deadline

The whole process of pre-RT workflow involves a series of several workstations that require multidisciplinary cooperation.^[2–4] For providing a high quality of RT care, each pre-RT workstation should be processed carefully and optimized thoroughly. However, in addition to re-define effective standard operating procedure (SOP) for each workstation, conventional human-based manipulation still requires to overcome one constraint for shortening pre-RT workflow, that is, a lack of systemic mechanism to effectively alert processing deadlines according to individual patient condition and cancer disease burden.

In this regard, implementing electronic control (e-control) and electronic alert (e-alerts) may be useful to increase clinical effectiveness. However, evidence is largely lacking in a real-world RT setting.

1.3. Problem description: Multidisciplinary procedures are required before initiation of RT—lacking e-control and e-alerts limits working effectiveness

Many factors are involved into the performance of a team work, such as working confidence and satisfaction.^[6–8] This is also true in daily work of radiation oncology. For enhancing clinical effectiveness of pre-RT workflow, multidisciplinary efforts are required, including radiation oncologists, medical physicists/dosimetrists, oncological nurses, radiation technologists, and administrative staffs. Using information system to provide e-control and e-alerts may be useful in improving working confidence and satisfaction (and then enhancing team performance); however, evidence is largely lacking in a real-world RT setting.^[9]

1.4. Rationale for applying an e-control workflow

To resolve the mentioned problem, we constructed a web-based information system to conduct an e-control for enhancing effectiveness of working processes before RT. Two rationales for this conduction were as follows.

First, shortening pre-RT workflow is crucial for increasing tumor control and then decreasing cancer recurrence.^[5]

Second, information-system-based e-control and e-alerts have been reported to increase effectiveness of clinical workflow; however, evidence is limited in the field of radiation oncology,^[10] especially in large busy RT centers.

1.5. Objects and hypothesis

Hence, the present study was designed to test whether applying information-system-based e-control and e-alerts is able to increase clinical effectiveness of pre-RT workflow.

Our hypothesis was that applying e-control and e-alerts can effectively shorten the whole process of pre-RT workflow, resulting in enhancements of staff satisfaction and working confidence, when compared with conventional human-manipulated practice.

1.6. Specific aims

1.6.1. Aim 1

Conducting e-control to shorten pre-RT workflow and enhance clinical effectiveness.

1.6.1.1. Trigged observation/reason/event

Conventional human-manipulated practice showed limitations in shortening pre-RT workflow. Obvious processing delays and staff complaints were noted frequently.

1.6.2. Aim 2

Conducting e-alerts to avoid human miss and then enhance staff satisfaction and working confidence.

1.6.2.1. Triggered observation/reason/event

As mentioned above, our RT staffs were unsatisfied for conventional human-based manipulation in the whole process of pre-RT workflow. Time wasting, poor effectiveness, and easy to miss/delay were complained frequently.

2. Methods

2.1. Ethic statement

The present study obeyed the Helsinki Declaration (written in 1975 and revised in 1983) and reported according to the standards for quality improvement reporting excellence (SQUIRE), mainly according to its new revision (version 2.0).^[11–13] Before data collection and analysis, a formal approval of our institute's review board (IRB) was gained (registered number, B10501024). The IRB waived a requirement of written informed consents because only anonymous secondary data were retrospectively analyzed.

Privacies of patients and staffs involved were adequately protected. Only validated data that retrospectively retrieved from our institute's quality-improving and accreditation database were used for analysis. All data were reported, and no additional data are available.

2.2. Setting and context (participants and dataset used)

For comparing effectiveness of our pre-RT workflow, we retrospectively analyzed data from our quality-improving projects that were conducted from Sep. 2012 to Dec. 2012 for pre-improved referencing. Note that these projects were doubly validated by our cancer center (internal checks per 3 months) and the National Quality and Safety Committee (external reviews in yearly quality-control activities). In the condition-check period, a total of 223 anonymous patients were treated in our RT center. In addition to processing time of each workstation, self-scored staff satisfaction and working confidence were also analyzed retrospectively.

For condition checking, we identified 3 time-waiting intervals for pre-RT workflow, as follows: waiting for simulation (2.8 days), waiting for planning (4.8 days), and waiting for the first-fraction RT treatment (4.6 days). As a result, an overall average processing time of 12.2 days was summated in the whole pre-RT workflow.

3. Definition

The time period of “overall pre-RT workflow” was defined from the day of first RT visit to the day of the first-fraction RT (Fig. 1). Five disciplinary staffs were involved, as follows.

Figure 1.

Simple cartoon depicting pre-RT workflow and corresponding RT staff. [A], waiting time interval for CT simulation (oncology nurse and radiation oncologist); [B1], waiting time interval for RT targeting (radiation oncologist); [B2], waiting time interval for treatment planning (medical physicist); and [C], waiting time interval for first-fraction RT (radiation technologist). CT = computed tomography; RT = radiotherapy.

First, the time period of “waiting for simulation” was defined from the day of first RT visit to the day of RT simulation. Two disciplines of RT staffs were involved: oncology nurse and radiation technologist.

Second, the time period of “waiting for RT planning” was defined from the day of RT simulation to the day of completed RT treatment planning. Radiation oncologist (for contouring targets) and medical physicist/dosimetrist (for designing and optimizing plan) manipulated this working process subsequently and then simultaneously.

Third, the time period of “waiting for first-fraction RT treatment” was defined from the day of completed RT treatment planning to the day of initiating first-fraction RT. Administrative staff (mainly for orienting medical environment) and radiation technologist (for arranging treatment schedule) were responsible for this process.

3.1. Condition analysis (before implementation of pre-RT e-control)

Before implementing e-control, we retrospectively analyzed data for searching potential time-wasting factors.

First, from September 2012 to December 2012, we retrospectively calculated processing time in each pre-RT workstation (involved patients, n = 223).

Second, after our analysis, 3 main working processes that demonstrated massive time waiting were identified: waiting for treatment planning (mean waiting time, 4.8 days), waiting for first-fraction RT (4.6 days), and waiting for RT simulation (2.8 days). As a summary, mean working time of 12.2 days was found in the whole process of pre-RT workflow before implementation of e-control.

Third, as shown in Fig. 2, we used the Pareto 80/20 rule to identify the majority of the pre-RT waiting time,^[14] being summated with 77% of “waiting for treatment planning” and “waiting for the first-fraction RT.”

Figure 2.

Pareto chart for analyzing pre-RT workflow. Three time-burdened working processes were identified, as follows: time interval [B1] + [B2], waiting time for RT treatment planning (i.e., time elapsed from CT simulation to complete RT treatment planning); time interval [C], waiting time for first-fraction RT (i.e., time elapsed from completed treatment planning to first-fraction RT); and, time interval [A], waiting time for RT simulation (i.e., time elapsed from first RT visit to CT simulation).Note 1: The corresponding time intervals (i.e., [A], [B1], [B2], and [C]) are depicted in Fig. 1. 1. Note 2: The waiting time intervals of CT simulation (i.e., [C]) and RT planning (i.e., [B1] + [B2]) are identified as the major time-burdened working processes, according to the Pareto chart and the Pareto rule. CT = computed tomography; RT = radiotherapy.

Fourth, as shown in Fig. 3, for “waiting for treatment planning,” two sub-workstations were identified, that is, physician targeting treatment area (mean working time, 2.1 days) and medical physicist designing/optimizing treatment plan (mean working time, 2.7 days).

Figure 3.

Dismantled analysis for “pre-RT workflow.” Nine sub-workstations were identified, as follows: A1. RT ordering: radiation oncologist confirmed indication and ordered RT; A2. Patient educating: oncology nurse educated the patient for during-RT self-nursing care; A3. CT simulation: radiation technologist performed CT simulation; B1. Target contouring: radiation oncologist contoured target area; B2. Treatment planning: medical dosimetrist designed and then medical physicist optimized RT treatment planning; C1. RT plan approving: radiation oncologist approved the RT treatment plan; C2. TNHI bill setting: administrative staff set TNHI billing accordingly; C3. Treatment appointing: radiation technologist appointed the timing of the first-fraction RT for individual patients; and, C4. First-fraction RT: first-fraction RT was delivered. Note 1: At the timing of RT plan approving, the attending radiation oncologist checks the optimized treatment planning thoroughly, in terms of both adequate tumor dose coverage and limited normal-tissue dose distribution. If the plan optimization is inadequate, the RT treatment plan will be returned to medical physicist for re-designing and re-optimizing (i.e., the processing step will backward from C1 to B2). Note 2: This Fig. 3 is sub-divided from prior Fig. 1. For example, sub-workstations of A1, A2, and A3 are included within the processing time interval of [A] in the Fig. 1. And, workstations of C1 to C4 are embraced in the time interval of [C] in the Fig. 1. CT = computed tomography; RT = radiotherapy; TNHI = Taiwan National Health Insurance.

Fifth, also shown in the Fig. 3, for “waiting for first-fraction RT,” three sub-workstations could be defined, that is, physician approved plan, administrative process, and first-fraction RT appointment. Of these, the main time-waiting burdened workstation was the “waiting for first treatment appointment (n = 89).” Several etiologies for delay appointment were found (Table 1). The main etiology for a delayed scheduling was a too full treatment schedule to allocate new patients (84.3%, 75/89).

Table 1.

Etiology of delayed initiation of first-fraction RT.

Sixth, analyzing the causes of full schedule of RT treatment machines showed 3 etiologies: (1) patient cannot match residual timing in the near-full treatment schedule; (2) daily portal images required double confirmation of physician and radiation technologist, prolonging each treatment duration; and, (3) excessive under-treatment patients: our average daily treating patients of 43 per treatment machine was excessively large than an average of 32 patients per treatment machine of other institutes.

Seventh, we defined the goal for this improvement was to shorten overall pre-RT workflow to be less than or equal to 10 days. The goal-setting reasons were as follows: (1) cancer patients should start RT as soon as possible to increase survival and to decrease tumor recurrence^[5]; and, (2) considering essential processing time for workstations: waiting for CT simulation (1 day), target contouring (3 days), treatment planning (3 days), and first-fraction RT scheduling (3 days).

3.2. Interventions

For minimizing unnecessary time wasting and then enhancing clinical effectiveness, we constructed an integrated information platform to provide e-control and e-alerts for the whole pre-RT workflow. Several steps were as follows.

Firstly, as shown in Figs. 1 and 3, we conducted extensive processing analysis to define details of pre-RT workflow, including essential working records and forms.

Secondly, we held regular multidisciplinary meeting between designers (i.e., information engineers) and users (e.g., radiation oncologist, oncological nurse, and medical physicist/dosimetrist) for establishing the e-control information system.

Thirdly, in these weekly combined meeting, frequent communications focused on constructing user-centered e-structure and e-checkpoints.

Fourthly, before each new function (including e-control and e-checkpoint) was applied on-line, a 14-day-at-least trial period was performed for identifying potential bugs.

Fifthly, modifications were allowed for polishing the information system, if indicated and consensus was achieved.

Sixthly and finally, we applied the finally polished information system to conduct e-control for the whole pre-RT workflow (Fig. 4). Systemic auto-alerts for preventing human miss and processing delay were also defined according to the consented e-checkpoints.

Figure 4.

Information-system web-based workstation: conducting e-control and e-alerts for the whole process of pre-RT workflow. Note 1: The information-system web-based workstation performs data e-capture from both health information system (HIS) and RT-specific information systems, including treatment-machine-operating ones. Thus, information (mainly actually done timing of each working process) is able to be presented as a timeline list, electronically. More notably, e-alerts are able to be conducted in a systemically automatic way. Note 2: The dot lines represent feedback cross-workstation communication between the information-system web-based virtual workstation and the 9 pre-RT real ones. Auto-alerts and systemic monitoring are allowed for enhancing clinical effectiveness and protecting patient safety (i.e., avoiding human-manipulated miss). CT = computed tomography; RT = radiotherapy; TNHI = Taiwan National Health Insurance.

3.3. Study of interventions

Using information system has been reported to be effective in conducting processing e-control in clinical practice^[15]; however, its role is rarely uncovered in the field of radiation oncology. Herein, we designed and conducted a multidisciplinary integrated information system to perform e-control for our pre-RT workflow. The benefit effects on processing efficacy were predictable and reasonable.

3.4. Measurements: 3 aim-specific end points

According to our specific aims, we defined 3 study end-points: for the Aim 1, the level of processing-time shortening, and overall staff satisfaction for clinical effectiveness; and for the Aim 2, overall staff working confidence for conducting pre-RT workflow. Again, we used visual-analog-scale-based (VAS) questionnaire to survey subjective staff satisfaction and their working confidence, as previous recommendation.^[16–18]

Briefly, VAS-based questionnaire classified staff-reported satisfaction and working confidence by using a 10-cm scale, ranging from 0 (on the bottom) to 10 (on the top of the scale). On reporting, staff pointed their satisfactory/confident level on the scale. A pointed level that was more closed to the bottom “0” represented a lower satisfaction/confidence. On the other hand, the pointed level that was more approached to the top “10” demonstrated a higher satisfaction/confidence. The staff-pointed level was measured in length to gain a score. These scores (i.e., staff-reported outcomes) were treated as a continuous variable in further statistical analysis.

3.5. Statistical analysis

Data were analyzed by using SPSS (Statistical Product and Service Solutions; version 12, IBM SPSS Inc., Chicago, USA).^[19] Student t test was applied to determine significance of difference between continuous-variable outcomes. As reported previously,^[19] 2 independent biostatisticians analyzed and validated the data respectively. P value of less than .05 was defined for representing statistical significance.

4. Results

4.1. Developing events and RT staffs involved in pre-RT e-workflow

Developing events of our pre-RT e-workflow were as follows. First, in late 2012, we started to generate the preliminary framework of e-control for our pre-RT workflow. Next, in late 2013, we completed e-tracing function for each patient in our pre-RT workflow. Finally, in middle 2014, we completed waiting-for-each-workstation e-patient list for helping clinical management, such as waiting for targeting, 3D CT simulation, treatment planning, plan checking, and billing. In addition, e-alerts for helping RT staffs to prevent processing delays were built at this timing.

Note that before the use of e-control, we used conventional human-practiced paper-based manipulation for processing pre-RT workflow. After successful introduction of e-control, good feedbacks were noted between users (i.e., our RT staffs) and system designers (i.e., information technicians: Miss Liu and Mr. Chung).

For constructing and implementing the e-control of pre-RT workflow, 22 RT staffs were involved, as follows: radiation oncologists (n = 4), medical physicists (n = 3), medical dosimetrists (n = 2), radiation technologists (n = 8), oncological nurses (n = 3), and administrative staffs (n = 2). However, after validation, only 21 questionnaires were considered as effective ones for further data analysis (effective rate, 95.5% [21/22]).

4.2. Improved aim-specific study end-points, in terms of processing effectiveness and staff satisfaction

Two measurements were conducted for data validation: first, 3 weeks later after online use of the e-control (Table 2); and second, 3 months thereafter (data not shown; results were similar with that shown in Table 2).

Table 2.

Shortened processing time of pre-RT workflow after e-control.

As shown in Table 2, when compared with conventional human-based manipulation, most aim-specific end-points showed improved results after implementation of e-control, as follows.

First, waiting time for physician targeting was shortened from 2.7 days to 1.4 days, with a time-saving rate of 48% (P < .001). The proportion of completed targeting within 3 days was improved, with an increase rate of 24%.

Second, time for RT treatment planning was decreased from 2.1 days to 1.2 days, with a time-saving rate of 46% (P < .001). Note that after we implemented e-control, the complexities of the treatment planning were increased, that is, the ratio of volumetric-modulated arc therapy (VMAT; an extremely high-precise RT technique) compared with intensity-modulated radiotherapy (IMRT; a high-precise RT technique), supporting an improvement of our treatment quality.

Third, after we certified image-validating ability of radiation technologist, on-board image-validating time was reduced from 288.4 to 168.7 seconds, with a time-saving rate of 41% (P < .001). Note that this time saving of on-board image validation effectively decreased an average on-board time per patient in the treatment room, leading to that more patients were able to initiate their first-fraction RT more quickly (as shown the fourth points).

Fourth, time from completed treatment planning to first-fraction RT was reduced from 4.6 days to 3.5 days, with a time-reducing rate of 24% (P = .001).

Fifth, as a result, the overall “pre-RT workflow” was reduced from 12.2 days to 8.9 days, with a time-saving rate of 27% (P < .001). In the maintained period, the results were durably kept (i.e., <10 days) for more than 2 years (till 2016).

More notably, in addition to shortened process time, 2 staff-aspect benefits were noted after pre-RT e-control: first, overall staff satisfaction for working effectiveness was increased from 5.3 (standard deviation [SD], 1.7) to 8.9 (SD, 0.9); and second, overall working confidence was improved from 5.9 (SD, 2.1) to 8.8 (SD, 0.8).

Note that pre-e-control data were retrieved from Sep. 2012 to Dec. 2012 (n = 223), and post-e-control data were retrieved from Sep. 2013 to Dec. 2013 (n = 240).

4.3. Three value-added benefits of implementing e-control: improved clinical documentation, increased processing safety, and then decreased a staff-turnover rate

As mentioned above, applying e-control obviously enhanced effectiveness of pre-RT workflow, leading to non-negligible time saving. The benefit of time saving allowed our RT staffs to record their clinical documentation more efficiently (with a higher quality). This finding was compatible with a prior report.^[20] More notably, an enhanced quality of cross-disciplinary handover was also observed, as previously declared.^[21] Remarkably, a main value-added benefit was identified after implementing e-control; that is, our RT-staff-turnover rate was decreased from >9% to <1% (P < .001).

4.4. An unexpected benefit of applying e-control and e-alerts: internalized an invisible positive department culture

In addition to increase staff satisfaction and working confidence, an invisible culture was internalized into our daily RT practice gradually, that is, a working habit of “requesting for more e-control and e-alerts to further increase clinical effectiveness, user friendliness, and working-system safety.” However, this benefit cannot be measured effectively. Further studies may be required to demarcate its real effects.

5. Discussion

5.1. Summary: key findings, including relevance to rationale and specific aims

The present study demonstrated several aim-specific achievements, as follows.

First, when compared with human-based manipulation, implementing e-control and e-alerts statistically shortened overall processing time of pre-RT workflow (from 12.2 days to 8.9 days; P < .001), with a 2-year durable maintaining effect till 2016 (still below the threshold goal of <10 days). This effect mainly based on improved clinical effectiveness of 3 workstations:

• (1)Time waiting for physician targeting, from 2.7 days to 1.4 days, with a time-saving rate of 48% (P < .001, the Aim 1);
• (2)RT planning time, from 2.1 days to 1.2 days, with a time-saving rate of 46% (P < .001, the Aim 1); and,
• (3)Time waiting for initiating first-fraction RT, from 4.6 days to 3.5 days, with a time-saving rate of 24% (P = 0.001, the Aim 1).

Second, overall staff satisfaction for clinical effectiveness (from 5.3 ± 1.7 to 8.9 ± 0.9; P < .0001) and their working confidence (from 5.9 ± 2.1 to 8.8 ± 0.8; P < .0001) were increased after implementing e-control and e-alerts (the Aim 2).

Third, we observed several value-added benefits, in terms of improved quality of documentation, cross-disciplinary handover, and safety of working processing—leading to a decreased staff-turnover rate (from >9% to <1%; P < .001)

Our observed results were compatible with previous reports,^[22–24] suggesting that establishing a web-based information system to conduct e-control and e-alerts should be critically considered for enhancing clinical effectiveness in daily RT practice.

5.2. Interpretation

Using a well-designed information system to apply e-control and e-alerts is useful in optimizing clinical workflow. Several benefits may be observed, such as improved clinical documentation^[20] and cross-disciplinary handover.^[21] However, these effect and their effective sizes are rarely reported in a real-world RT setting.^[10]

Herein, we constructed a web-based information system to conduct e-control and e-alerts for the whole process of pre-RT workflow in a large RT center. We observed and confirmed several benefits, including enhanced clinical effectiveness, improved staff satisfaction, and working confidence. In addition, qualities of working documentation and clinical handover were also improved. Remarkably, the staff turnover rate was largely decreased. These observations were similar with other's reports.^[10,20,21]

5.3. Study strength

Clinical practice of modern RT is largely depended on daily application of information systems,^[25,26] such as health information system (HIS) and radiotherapy treatment planning systems ]RTPS; e.g., Varian Eclipse^TM (CA, USA),^[27] Oncentra (Nucletron, CA, USA),^[28] and Elekta Monaco (Sunnyvale, CA, USA)^[29]]. However, the 2 types of information systems are varied and mutually exclusive in data sharing.

In daily routine, therefore, RT members require checking data between the two types of information systems frequently. This kind of clinical practice decreases working effectiveness and harbors potential errors. Several information systems have been reported for improving this practice.^[30–32] For example, Patel and Kirby^[9] reported their self-established information system to assist quality control of radiation physics. And, Pan et al^[33] published their data e-captured information system to facilitate clinical documentation. Herein, our reported web-based system not only improved quality control of radiation physics and clinical effectiveness of e-documentation, but also provided e-control and e-alerts in the whole pre-RT workflow. Three study strengths were as follows.

First, using a web-based information system to conduct e-control and e-alerts for the whole process of pre-RT workflow is the main strength of the present study. The web-based design made it possible that our staffs can log on from every RT workstations.

Second, the e-control and e-alerts were codesigned and cooptimized by our information technician and our multidisciplinary RT staffs for achieving better usability and performance, as previously recommended.^[34,35]

Third, the present study confirmed a clinical role of implementing e-control and e-alerts in enhancing effectiveness of pre-RT workflow in a real-world RT setting, suggesting further parallel expand.

5.4. Study limitations

Inevitably, as mentioned previously, the present quality-improving study has limitations.

First, results are reported in a single RT center. More data should be collected and analyzed after further parallel expand.

Second, retrospective analysis by using secondary quality-improving data harbors intrinsic bias, though the data were validated internally and externally. For example, unobserved variables may exist. Thus, interpreting our results should be carefully; further prospective studies may be required.

6. Conclusion

Replacing conventional human-manipulated practices with information-system-based e-control and e-alerts should be critically considered in a large RT center for enhancing clinical effectiveness of multidisciplinary pre-RT workflow. Staff satisfaction and working confidence may be simultaneously improved, leading to a low staff-turnover rate.

Acknowledgment

The authors would like to thank all of our RT staffs. Without their help and feedback, our e-control and e-alerts function cannot be reached its current version and the best effects.