Risk assessment and quality management in AIO based on CT‐linac for nasopharyngeal carcinoma: An improved FMEA and FTA approach

Abstract

Background

All‐in‐one radiotherapy workflow (AIO) is a novel one‐stop solution that integrates the multiple conventional radiotherapy steps from simulation, contouring, planning, image guidance, beam delivery, and in vivo dosimetry into a single device (integrated computed tomography linac, the uRT‐linac 506c), making the treatment process more efficient and convenient while reducing errors for cancer patients' initial radiotherapy. Despite its numerous advantages, the implementation of AIO faces challenges such as interdisciplinary coordination, software and hardware complexity, and reliance on artificial intelligence. To ensure its safety and effectiveness, it is necessary to conduct a risk assessment and identify appropriate quality management measures.

Purpose

To perform risk assessment on the AIO for nasopharyngeal carcinoma using failure mode and effects analysis (FMEA) and fault tree analysis (FTA), and to validate the effectiveness of the quality management measures.

Methods

A flowchart was established for the AIO of nasopharyngeal carcinoma. FMEA analysis was conducted based on the flowchart, and quantitative assessments of each failure mode (FM) were performed to obtain $O$ (occurrence), $S$ (severity), and $D$ (Detectability). Weighted $O_{wi}$, $S_{wi}$, and $D_{wi}$ were obtained using the similarity aggregation method (SAM), and the final risk priority number (RPN) was calculated by multiplying these values. The FMs were then evaluated into two groups based on whether quality management (QM) measures were implemented, and sorted by the RPN. Finally, FTA analysis was conducted on the highest‐risk FMs identified through ranking.

Results

A flowchart of AIO for nasopharyngeal carcinoma was established, consisting of 5 main steps and 28 sub‐steps. After FMEA analysis, 86 FMs were identified. In the group without implementing QM measures (QM‐free group), the RPN of FMs ranged from 13.5 to 186.2, with a threshold of 94.6 for the top 20% RPN scores, resulting in 17 high‐risk FMs. Additionally, 21 FMs had $S_{wi}\ge 8$, with a cumulative total of 25 high‐risk FMs after removing duplicates. In the group that implemented QM measures (QM group), the RPN of FMs ranged from 3.0 to 46.7, showing an overall decrease compared to the QM‐free group. There was a statistically significant difference in RPN between the QM‐free (55.80 ± 38.40) and QM (16.17 ± 10.99) groups (p < 0.001), validating the effectiveness of the QM measures. Finally, FTA analysis was performed on the highest‐risk step identified in the QM‐free group with the highest RPN.

Conclusion

The improved FMEA and FTA analysis methods are practical and operational, allowing for a comprehensive analysis of potential failures and risks in the AIO for nasopharyngeal carcinoma. They can effectively assist in establishing and evaluating QM standards for AIO of nasopharyngeal carcinoma. Moreover, the analytical methods and QM measures of this study can be effectively applied to AIO for tumors in other sites.

1. INTRODUCTION

Radiotherapy (RT) is a crucial modality for cancer treatment. The routine RT is multi‐step, team‐based, and time‐consuming, with significant reliance on human expertise. In high‐volume centers, delays from patient simulation to the first treatment session can span one to two weeks, introducing uncertainties that can affect treatment outcomes. Factors such as tumor progression, changes in body size, and alterations in internal anatomical structures can result in the planning CT no longer accurately reflecting the patient's current anatomical situation at the time of RT implementation. These discrepancies ultimately cause significant deviations in the dose distribution received by the patient's tumor and organs at risk (OARs) from the planned dose. ¹ , ² , ³ , ⁴ Given these limitations, it is imperative to seek changes in routine RT workflows to enhance the efficiency, accuracy, and efficacy of treatments.

In 2023, Yu et al. proposed a simplified artificial intelligence (AI)‐driven online RT workflow called the “All‐in‐one” to optimize routine RT workflows, standardize and expedite RT procedures, and make treatments more accurate and efficient. ⁵ All‐in‐one radiotherapy workflow (AIO) integrates simulation, contouring, planning, and the first treatment session on an integrated computed tomography linac (CT‐linac), with physicians, physicists, and therapists completing these steps consecutively in the same treatment room within 30 min (Details in section 2.1). Currently, the CT‐linac enabling the AIO is the uRT‐linac 506c, introduced in 2019 by United Imaging Healthcare (UIH) Co., Ltd. (Shanghai, China). ⁵ , ⁶ The uRT‐linac 506c combines a 16‐slice helical CT scanner with a C‐arm linac, enabling the acquisition of high‐quality KV fan beam CT (FBCT) images that serve both diagnostic and simulation purposes for precise target delineation and treatment planning. ⁵ , ⁶ , ⁷ , ⁸ By consolidating routine standalone RT steps into an online, one‐stop workflow with the assistance of AI, and avoiding the need for a change of equipment or procedure rooms, AIO reduces the initial treatment wait time from days, or even weeks, to within 30 min. This enhances the efficiency of clinical RT workflows and eliminates errors associated with switching between different devices. It minimizes system uncertainties between simulation and treatment, and mitigates the impact of anatomical changes (e.g., tumor progression and weight variations after simulation) on treatment accuracy, thereby enhancing the patient's treatment experience and maximizing treatment benefits. There are not many published studies on AIO and its associated devices yet, these are mainly technical notes on this emerging workflow, ⁵ advantages in positioning accuracy, ⁹ commissioning of related equipment, ⁶ evaluation of daily CT for in vivo dosimetry, ¹⁰ dosimetry effect evaluation of the couch, ¹¹ and treatment efficiency advantages brought by the rotatable 540° gantry. ⁶ , ¹² However, this workflow has been implemented in at least 12 RT centers in China for the treatment of tumors such as nasopharynx, rectum, lung, breast, brain, cervix, with over 400 successful cases, gaining recognition from these RT teams.

Although AIO offers numerous clinical advantages and benefits for patients, for such a new workflow introduced in 2023, as members of the RT team, we should not only focus on its advantages but also ensure the safety and effectiveness of its implementation. The implementation of AIO is complex and challenging, primarily in two aspects. First, there are the inherent risks posed by the sophisticated hardware and software systems. AIO relies on a unique CT‐linac equipped with both traditional MV‐CBCT and diagnostic‐grade KV‐FBCT. This requires precise calibration and synchronization between the CT and Linac components. The increased data interaction demands higher‐speed data channels and more robust network infrastructure, leading to a hardware complexity that surpasses standalone Linac and CT devices. Meanwhile, maintenance and quality control procedures for this integrated system are inherently more complex and time‐consuming, increasing the risk of system failures. On the software side, the CT‐linac operates exclusively with UIH's proprietary integrated TPS+OIS platform (uRT‐TPOIS) and treatment delivery system (TDS). The implementation of AIO incorporates multiple AI modules for tasks such as automated contouring, automated planning, and automated quality assurance (QA), as well as in vivo dosimetry for online QA. Furthermore, the AI modules used by each RT center typically need to be retrained according to the specific requirements and historical data of the center, as different centers have varying practices and experiences. AIO involves both a greater quantity and higher complexity of software and hardware components compared to traditional RT, necessitating special attention to the stability, reliability, and accuracy of both hardware and software. The AI modules, in particular, pose potential risks due to their lack of transparency.

Another challenge in implementing AIO is meeting the high demands placed on the treatment team to balance both efficiency and accuracy. During the implementation of AIO, patients remain fixed on the couch after positioning while waiting for the start of the first treatment. Given the patient's tolerance, the implementation efficiency of AIO must be as high as possible to avoid any delays or procedural failures due to procedural errors. However, AIO differs significantly from routine RT workflows, involving more steps, complex procedures, multiple software and hardware transitions, and real‐time communication and coordination among various team roles. The pressure to perform efficiently can lead to operational mistakes and oversights. Therefore, the entire treatment team must undergo thorough training to ensure they can correctly, skillfully, and swiftly complete all operations according to standard procedures.

In 2016, the American Association of Physicists in Medicine (AAPM) released the report of Task Group 100, advocating for the establishment of corresponding quality management (QM) standards for clinical workflows. ¹³ This report recommends using a combination of failure mode and effects analysis (FMEA) and fault tree analysis (FTA) to assess the risks in RT processes and establish and improve QM standards. Several reports internationally have confirmed the effectiveness of FMEA and FTA in risk analysis for RT, including plan design, plan‐specific QA, brachytherapy, etc. ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ However, traditional FMEA methods are often influenced by subjective factors of experts, resulting in significant differences in scores for the same failure mode (FM). Conservative experts may assign higher scores, while optimistic experts may assign lower scores. Furthermore, differences in experience and knowledge among experts also affect the reliability of scores. ¹⁸ , ¹⁹ , ²⁰ To make the FMEA analysis results more objective and reliable, the traditional FMEA method can be improved using the similarity aggregation method (SAM). Hsu et al. proposed SAM to address the issue of expert opinion inconsistency in multi‐criteria decision‐making scenarios involving group decisions. ²¹ , ²² SAM establishes a procedure for aggregating expert opinions, first measuring each expert's degree of consensus with other experts based on the relative consistency (RC) of their opinions. Then, it aggregates the importance of each expert to obtain the final opinion weight of each expert, thus resolving the impact of subjective opinion inconsistency on decision‐making results.

This study aims to use SAM to determine appropriate scoring weights for each expert by comprehensively considering the consistency of opinions among experts, as well as the experts' backgrounds such as education, work experience, and professional titles. The weighted expert scores will then be used to conduct the FMEA analysis. Based on the improved FMEA method, an expert team will collaborate to conduct a risk analysis of the AIO for nasopharyngeal carcinoma, identify high‐risk segments in the workflow, and propose QM standards and recommendations to enhance the safety and effectiveness of AIO for nasopharyngeal carcinoma, thereby maximizing treatment benefits for patients.

2. MATERIALS AND METHODS

Since April 2022, the RT department of Sun Yat‐sen University cancer center has conducted over 200 successful cases of AIO for nasopharyngeal carcinoma, accumulating extensive clinical experience and fostering an experienced team dedicated to AIO implementation. This study is conducted against this backdrop, the complete improved FMEA analysis process is shown in Figure 1.

FIGURE 1.

Implementation process of improved FMEA based on SAM in this study.

2.1. Implementation of AIO

Each implementation of AIO requires a team consisting of at least one physicians, one physicist, and two therapists to work simultaneously (Figure 2). Specifically, during the offline phase, it is crucial to complete all preparatory work of the AIO for nasopharyngeal carcinoma to save time during the online phase. This includes patient immobilization and MR scanning, with the physicians pre‐contouring the primary gross tumor volume (GTVp) on the MR images and importing it into the uRT‐TPOIS. Additionally, the medical physicist should preset various AIO protocols according to the RT prescription, enabling the TPS to correctly perform automated contouring and treatment planning in subsequent steps. The online phase begins with patient positioning, where the radiation therapist uses the CT component of the CT‐linac to acquire CT images, which are automatically transmitted to the uRT‐TPOIS via a preset IP address. The physicians then register the GTVp from the MR to the CT images and manually corrects it before performing automated contouring of other targets and OARs. After the physicians completes all necessary corrections to the automatically contoured regions of interest (ROIs), the physicist proceeds with automated planning and collaborates with the physicians to evaluate and modify the plan. The automated contouring and planning software are integrated into the corresponding functional modules of uRT‐TPOIS. Once the plan is approved, it is transferred to the TDS for the radiation therapist to administer the initial treatment. The treatment execution process includes pre‐treatment image guidance and in vivo dosimetry, which is based on the electronic portal imaging device (EPID) and integrated QA software within the TDS. Each step in the AIO is sequentially connected, requiring all team members to maintain real‐time responsiveness to ensure timely communication and resolution of any potential issues, thereby ensuring the smooth completion of the entire process.

FIGURE 2.

The implementation steps of AIO, as well as the corresponding facilities and treatment team roles for each step.

2.2. Flowchart

A multidisciplinary research team was established, consisting of eight experts including physicians, physicists, dosimetrists, therapists, and R&D engineers from the manufacturer of the uRT‐linac 506c. All members have clinical experience in implementing/developing AIO for nasopharyngeal carcinoma and are familiar with various steps of this workflow. Defining the stage from the physician's initial delineation of the GTVp based on the patient's MR images to the completion of the patient's first RT session as the AIO stage. A member familiar with the entire AIO for nasopharyngeal carcinoma drafted the flowchart, which was then discussed and revised by all team members to obtain the final flowchart used in this study. The principle of flowchart design is to decompose and refine each step to visually display the execution process of the entire workflow without overlooking important details, while avoiding the impact of excessive size and redundancy on analysis.

2.3. Improved FMEA based on SAM

The supplementary material, Figure S1, provides specific examples of each step in the implementation process of the improved FMEA based on SAM.

2.3.1. Identify the failure modes (FMs) for each sub‐step

According to the flowchart of AIO for nasopharyngeal carcinoma, team members collectively discussed and brainstormed from both clinical and software/hardware development and maintenance perspectives. These discussions were informed by clinical practice and took into account real‐world scenarios, and all potential FMs that could occur in each sub‐step were identified through this process. The $i$‐th FM is denoted as $F{M}_i$. The causes, frequency, and clinical impact of each FM were clarified, and each expert provided FMEA scores for each FM.

2.3.2. Initial FMEA scoring

Each FM was quantitatively evaluated according to predetermined unified scoring criteria (Table 1), resulting in three initial scores for each FM: $O$ (occurrence), $S$ (severity), and $D$ (detectability). The scoring range for $O$, $S$, and $D$ is 1 to 10, with a higher $O$ indicating a higher probability of occurrence, a higher $S$ indicating a more severe impact of the FM, and a higher $D$ indicating a lower probability of detection during the workflow execution process. The three initial scores assigned by the $k$‐th expert for $F{M}_i$ are denoted as $O_i^k$, $S_i^k$, and $D_i^k,$ respectively.

TABLE 1.

Scoring criteria of the $O$, $S$, and $D$ as suggested in TableII of the report of Task Group 100 of the AAPM.

	Occurrence ($O$)		Severity ($S$)		Detectability ($D$)
Rank	Qualitative	Frequency (%)	Qualitative	Categorization	Estimated probability of failure going undetected (%)
1	Basically impossible	0.01	No impact	–	0.01
2		0.02	Waste of time	Waste of time	0.2
3	Relatively few	0.05			0.5
4		0.1	Small dosage Error	Waste of time, or suboptimal plan/treatment	1
5		<0.20	Severe waste of time	Suboptimal plan/treatment, or severe waste of time	2
6	Occasionally happens	<0.50	Lower toxicity response or insufficient target dosage	Wrong dose, dose distribution, location, volume	5
7		<1.00	Potential severe toxicity response or insufficient target dosage		10
8	Repeatedly occurs	<2.00	Cause extremely severe toxicity response or insufficient target dosage	Serious error in dose, dose distribution, location, volume	15
9		<5.00			20
10	Inevitably occurs	>5.00	Catastrophic Accident		>20

2.3.3. Weighted FMEA scoring based on SAM

To enhance the accuracy and objectivity of expert scoring results, the final weight of each expert's score for each FM is obtained by linearly combining the expert's personal background weight and the relative consistency weight of their scores based on SAM. The personal background weight of the $k$‐th expert is denoted as $W_{bk}^k$. The relative consistency weight and the final weight of the $k$‐th expert's score for $F{M}_i$ are denoted as $R{C}_{xi}^k$ and $W_{xi}^k$, respectively, where $x$ represents $O$, $S$, or $D$. The final weighted scores used for calculating the risk priority number (RPN) for $F{M}_i$ are denoted as $O_{wi}$, $S_{wi}$, and $D_{wi}$ respectively.

(1) Calculation of expert's personal background weight $W_{bk}^k$

The total personal background score of the $k$‐th expert, $Scor{e}_{bk}^k$, is obtained by summing the individual scores of each item according to the evaluation criteria in Table 2. The personal background weight $W_{bk}^k$ is calculated based on the proportion of their total score:

$$W_{bk}^k=\frac{Scor{e}_{bk}^k}{{\sum }_{k=1}^{8}Scor{e}_{bk}^k}$$ (1)

(2) Calculation of relative consistency weight for each expert's scores of each FM

TABLE 2.

Weighting criteria and score of experts.

Title	Working experience (years)	Educational level	Age(years)	Score
Senior professional title	>30	PhD	>50	4
Sub‐advanced title	21∼30	Master	41∼50	3
Intermediate title	10∼20	Bachelor	31∼40	2
primary title	6∼9	Junior college	25∼30	1
Other	<6	Other	<25	0

Calculate the average distance $A{D}_{xi}^k$ between the initial scores $O_i^k$, $S_i^k$, $D_i^k$ given by the $k$‐th expert and those given by other experts for $F{M}_i$:

$$A{D}_{xi}^k=\frac{1}{7}\sum _{\begin{array}{c}v\ne k\\ v=1\end{array}}^8\left|x_i^k-x_i^v\right|.$$ (2)

Then, use the normalization method to calculate the relative consistency weight $R{C}_{xi}^k$ for the $k$‐th expert's scores:

$$R{C}_{xi}^k=\frac{A{D}_{xi}^k}{{\sum }_{k=1}^{8}A{D}_{xi}^k},$$ (3)

where $|x_i^k-x_i^v|$ represents the distance between the initial scores of $F{M}_i$ given by the $k$‐th and $v$‐th experts, with $x$ representing $O$, $S$, or $D$.

(3) Calculation of final weight $W$ for each expert's scores of each FM

The final weight $W_{xi}^k$ for the $k$‐th expert's scores of $F{M}_i$ is given by:

$$W_{xi}^k=\alpha W_{bk}^k+\beta R{C}_{xi}^k.$$ (4)

Similarly, $x$ represents $O$, $S$, or $D$. Here, $\alpha$ and $\beta$ are relaxation factors reflecting the relative importance of the two weights, satisfying $0\le \alpha \le 1$, $0\le \beta \le 1$, and $\alpha +\beta =1$. In this study, $\alpha$ and $\beta$ are both set to 0.5. Decision‐makers can adjust the size of the relaxation factors to determine the importance of expert background and scoring consistency in the final expert weight.

(4) Calculation of the final weighted scores

The final weighted scores $O_{wi}$, $S_{wi}$, and $D_{wi}$ of $F{M}_i$ used for calculating the RPN are calculated as follows:

$$O_{wi}=\sum _{n=1}^8{W}_{Oi}^n{O}_i^n,$$ (5)

$$S_{wi}=\sum _{n=1}^8{W}_{Si}^n{S}_i^n,$$ (6)

$$D_{wi}=\sum _{n=1}^8{W}_{Di}^n{D}_i^n,$$ (7)

2.3.4. Calculation of the weighted RPN

The final $RP{N}_i$ for $F{M}_i$ is obtained by multiplying the values of $O_{wi}$, $S_{wi}$, and $D_{wi}$,

$$RP{N}_i=O_{wi}\times S_{wi}\times D_{wi}.$$ (8)

RPN ranges from 1 to 1000, where a higher value indicates a higher level of risk for the $F{M}_i$. All FMs are then sorted in descending order based on their RPN. FMs with RPN within the top 20% range or with $S_{wi}\ge 8$ are defined as high‐risk FMs.

2.4. Assessments are conducted separately for the QM‐free and QM groups

To ensure the safe and effective implementation of RT, QM measures were developed for the AIO for nasopharyngeal carcinoma. The QM measures currently in use include establishing checklists, enhancing clinical training for personnel, strengthening communication and education for patients, switching from single‐person to double‐check verification for certain tasks, conducting regular specialized QA for software and hardware systems, and adding automatic verification functions (Table S1 of the Supplementary material). The research team conducted assessments of all FMs using SAM. While the QM measures listed in Table S1 were all in use during the treatments and assessment, evaluators first estimated the O, S, and D scores assuming QM measures were not implemented (QM‐free group), and then conducted a second assessment with the actual QM measures in place (QM group), allowing for a comparative analysis.

2.5. FTA

Conduct FTA analysis on the step associated with the highest‐risk FM to identify all the causes leading to failure or errors in those steps. Present the findings in a fault tree diagram, with potential failures on the left side and all possible causes on the right. The causes are connected to the potential failures on the left through logic gates, visually illustrating the pathways by which various factors contribute to the failure of the process.

2.6. Statistical analysis

Statistical analysis was conducted using Statistical Package for the Social Sciences software (IBM SPSS, version 27.0). Since the measurement data did not conform to normal distribution, the non‐parametric Wilcoxon signed rank test was employed to compare the RPNs between the QM‐free and QM groups, with p < 0.05 indicating statistical significance.

3. RESULTS

3.1. Flowchart of AIO for nasopharyngeal carcinoma

The most important aspect of a well crafted flowchart is the definition of research detail standards. ¹³ , ²³ Focusing too much on details can result in a flowchart that is overly large and redundant, diminishing the importance of key steps. On the other hand, being too broad may cause some steps to be hidden and potentially lose important details. After multiple discussions and revisions by team members, the flowchart for AIO for nasopharyngeal carcinoma was finalized (Figure 3). The entire process of AIO for nasopharyngeal carcinoma consists of 5 main steps and 28 sub‐steps.

FIGURE 3.

Flowchart of AIO for nasopharyngeal carcinoma.

3.2. Analysis results of improved FMEA

Based on the flowchart of AIO for nasopharyngeal carcinoma, the research team conducted FMEA analysis, resulting in a total of 86 FMs (Table S1). The RPN of FMs in the QM‐free and QM groups ranged from 13.5 to 186.2 and from 3.0 to 46.7, respectively. FMs within the top 20% range of RPN or with $S_{wi}\ge 8$ were considered as high‐risk FMs. In the QM‐free group, the threshold RPN for the top 20% was 94.6, identifying 17 FMs, including ignored spurious points, delineating with incorrect slices, and incorrect modification of the objective function. The spurious points are defined as small delineations of an ROI typically with a volume less than 0.1 cm³. Additionally, 21 FMs had $S_{wi}\ge 8$, including excessive changes in patient size and the use of accounts without permissions. After removing duplicates, a total of 25 high‐risk FMs were identified, which should be given special attention when developing QM standards. Among all high‐risk FMs, human error was the primary factor, followed by inadequate training, software/hardware errors, and communication deficiencies, indicating that the main factors affecting the safety and effectiveness of AIO were operational norms in various steps and the team's familiarity with the workflow. Table 3 lists all high‐risk FMs and their QM measures, including the potential FMs, the sub‐steps where each high‐risk FM occurs, the QM measures, and their weighted $O_{wi}$, $S_{wi}$, $D_{wi}$, and $RP{N}_i$.

TABLE 3.

All high‐risk FMs and their QM measures.

Branching step	Potential failure modes	$O_{wi}$	$S_{wi}$	$D_{wi}$	$RP{N}_i$	QM measures
Top 20% FMs in $RP{N}_i$ ranking
Modify all other ROIs	Ignored spurious points a	3.5	8.1	6.5	186.2	Automated verification by software
Modify GTVp	Delineating with incorrect slices a	4.2	8.4	4.5	156.4	Dual verification by doctors & physicists
Modify all other ROIs	Incorrect substitution of ROIs a	3.2	8.4	4.9	132.0	Dual verification by doctors & physicists
IP configuration	Misconfigured IP a	5.9	8.7	2.5	127.5	Building checklists for offline preparateion
Modify all other ROIs	Delineating with incorrect slices a	3.2	8.2	4.7	121.9	Dual verification by doctor and physicist
Modify GTVp	Ignored spurious points a	4.2	8.3	3.4	117.8	Automated verification by software
Automatically delineate other ROIs on CT	Delineating with incorrect slices	3.5	7.6	4.5	116.2	Dual verification by doctors and physicists
Modify the objective function	Incorrect modification ofthe objective function	5.0	4.3	5.3	112.9	Building an autoplan Checklist
Modify the objective function	Omitting to modify the objective function	5.2	4.0	5.4	112.1	Building an autoplan Checklist
Automatically delineate other ROIs on CT	Without added couch a	3.5	8.0	3.9	108.5	Automated verification by software
Automatically delineate other ROIs on CT	Ignored spurious points	4.4	5.6	4.4	108.4	Automated verification by software
Modify all other ROIs	Omitting to modify some ROIs a	3.2	8.4	4.0	106.8	Dual verification by doctor and physicist
Setup for patient	Excessive changes in patient size a	5.1	8.2	2.4	101.8	Monitor weight changes and intervening promptly
Approval of the plan	Plan approval failed a	5.0	8.3	2.4	100.7	Check machine communication
Implementation of radiotherapy	Poor patient tolerance a	4.7	8.2	2.6	99.3	Pretraining for patients
Modify GTVp	Omitting to modify ROIs for some slices a	3.5	8.1	3.4	97.0	Add final ROIs modification verification
Modify all other ROIs	Misdeleted ROIs a	3.2	8.0	3.7	94.6	Automated verification by software

FMs with $S_{wi}$ ≥ 8
MR scanning	Metal parts carried by the patient were not removed	2.1	8.5	1.7	29.6	Prechecking for metal carriers
MR scanning	Patient allergic to contrast media	2.2	8.4	1.4	25.6	Inquire about the patient's allergy history
Automatically delineate other ROIs on CT	Incorrectly matching of couch positions	2.1	8.4	3.9	65.9	Automated verification by software
Delineating GTVp based on MR images	The lesion range was not fully covered during scaning	5.6	8.2	1.8	83.3	Verification of scanned request forms
Image‐guided patient positioning	CT malfunction	4.1	8.2	2.5	81.0	Periodic reboot of the machine
Automatically delineate other ROIs on CT	Incorrect selection of CT to electron density curve	1.5	8.1	4.5	52.9	Building an ROIs outlining checklist
Creating an automated plan	Used an account withoutpermissions	3.2	8.1	2.4	62.6	Building an autoplan checklist
Transmitting plan	Communication failure between TPS and TDS	4.4	8.1	1.4	49.5	Regular quality assurance

^a $S_{wi}>8$ for the respective FM.

3.3. Assessment results for the QM‐free and QM groups

The research team independently conducted two assessments of all FMs, with the first assessment without QM measures and the second assessment with QM measures implemented, comparing the results of QM‐free and QM groups to validate the effectiveness of the QM measures. Statistical analysis of the two sets of evaluation data was performed using the nonparametric Wilcoxon signed‐rank test. The results showed that the RPN for the QM‐free group was 55.80 ± 38.40, and for the QM group was 16.17 ± 10.99. After implementing QM measures, the RPN for each FM decreased overall compared to those before implementing QM measures. The difference in RPN between the two evaluation groups was statistically significant (p < 0.001), confirming the effectiveness of QM measures and demonstrating that the proposed QM measures can effectively reduce the risk of AIO for nasopharyngeal carcinoma.

Figure 4 shows box plots and line plots of the weighted RPN for all FMs after two assessments by the two groups. The box plot displays the mean, minimum, maximum, upper quartile, median, and lower quartile of the RPN for both evaluation groups. The line plot indicates that after implementing QM measures, the RPN for all FMs decreased. However, there were still some FMs (highlighted in red boxes) with less significant decreases in RPN after implementing QM measures, including TPS communication errors with TDS, discovery of significant setup errors in image guidance, and failure to create in vivo plans.

FIGURE 4.

Box plots (left) and line plots (right) of RPN for each FM in QM‐free and QM groups.

3.4. Results of FTA

The FTA visually illustrates the causes of failures or errors in the step associated with the highest‐risk FM, facilitating the analysis of various risk factors within this step and aiding in the selection of appropriate QM measures for effective risk management. When QM measures were not implemented, the FM “ignored spurious points” had the highest RPN of 186.2 and ranked first. The FTA results for the ROIs outlining step are shown in Figure 5, where the fault tree uses logical gates to connect the relationship between various steps and FMs, the FMs highlighted in red boxes indicate high‐risk FMs, with the red numbers following them representing the RPN of these high‐risk FMs. As shown in Figure 5, several risk factors, including errors in patient positioning, can lead to ROI outlining failure/error. These risk factors can primarily be attributed to categories such as human error, inadequate training, and software/hardware errors.

FIGURE 5.

The FTA for the delineation of ROIs.

4. DISCUSSION

AIO is an emerging workflow based on CT‐linac with promising application prospects, enhancing patients' RT benefits by reducing treatment errors and improving the patient experience. As more CT‐linac equipment are put into use, the implementation of AIO is expected to become increasingly widespread in the future. The results of this study are significant for the clinical application of AIO. By systematically identifying and assessing potential risks, we can take preventive measures in advance, reduce treatment errors, and further enhance both the safety and benefits of the treatment for patients.

This study acknowledges the complexity and challenges of implementing AIO. Based on the report of Task Group 100 of the AAPM, it is the first to propose an improved FMEA method for risk analysis of AIO using nasopharyngeal carcinoma as an example. Compared to traditional FMEA, the FMEA improved by SAM considers the personal background and consistency of experts' scores, making risk assessment more objective and reliable. Research by Guo et al. indicates that SAM has significant advantages in addressing discrepancies in expert opinions. ²² Especially when there is a considerable difference in expert scores, SAM can mitigate this impact by altering the contribution of expert opinions, diminishing the influence of less credible experts on the final scores, and skewing the final scores towards the opinions of high‐weight, highly credible experts. Our study further validates its effectiveness in AIO risk management. The improved FMEA method can accurately identify and assess high‐risk FMs in AIO, enabling the development of effective QM measures, and thus enhancing the safety and effectiveness of AIO.

Through FMEA analysis, it was determined that the FM with the highest RPN without QM measures was the “ignored spurious points,” with an RPN of 186.2. The S and D scores were 8.1 and 6.5, respectively. The high severity and detectability are the reasons for this FM having the highest RPN. On one hand, if the high‐dose spurious points of the GTVp fall on critical serial organs, such as the spinal cord, and are not detected and corrected, it can result in severe RT side effects, such as patient paralysis. On the other hand, spurious points are generally small, and nasopharyngeal carcinoma patients have dozens of ROIs, making such a small spurious point easily overlooked visually. We employed an automated pre‐planning quality check of all ROIs as a QM measure to address this FM, overcoming the drawbacks of time‐consuming and ineffective visual inspection. This significantly reduced its RPN from 186.2 to 35.3, substantially decreasing the risk associated with this FM. This demonstrates the effectiveness and necessity of the QM measure. Certainly, there are also FMs such as communication errors between TPS and TDS, identification of significant setup errors during image guidance, and failure in creating in vivo plans, where the reduction in RPN after implementing QM measures is not significant (Section 3.3). Currently, there are no effective methods for preemptive prevention of these FMs, or the associated manpower costs are high. Fortunately, the RPN values for these types of errors are generally low. We hope that targeted development of better QM methods will be pursued in future research.

Among the 25 high‐risk FMs, 14 are associated with the main step of “Delineating the ROIs.” This concentration of issues is understandable given that errors in other steps typically result in delays within the AIO. In contrast, the accurate delineation of ROIs is foundational to treatment planning and directly influences the evaluation of dose distribution quality. Inaccuracies in ROI delineation could lead to severe adverse effects, inadequate target dosing, or potentially catastrophic outcomes. Additionally, to expedite the AIO and enhance efficiency, ROIs delineation heavily relies on AI. However, AI inevitably exhibits shortcomings including lack of transparency and dependence on historical training data. Since patients are highly individualized, significant disparities between a patient's anatomical data and the training data of the AI model can severely impact its automatic delineation effectiveness. In such cases, doctors may need to spend a considerable amount of time correcting each ROI manually. In the worst‐case scenario, this could lead to serious consequences. Therefore, doctors should avoid blindly or completely relying on AI models.

Among all high‐risk FMs, human error was the primary factor, followed by inadequate training, software/hardware errors, and communication deficiencies (Section 3.2). Based on these findings, we recommend that the implementation of AIO should particularly focus on identifying and addressing these factors. Establishing stringent QM measures is advised to ensure the accuracy of each sub‐step and to minimize system errors. First, for high‐RPN FMs resulting from human error, such as ignored spurious points, a multi‐tiered verification mechanism including automated detection and manual review should be employed. Second, regular training and assessments for the treatment team should be conducted to optimize workflow, thereby reducing patient waiting time on the couch and improving treatment efficiency and outcomes. Training should encompass simulation exercises, case studies, QA standards, and methods for handling common errors and emergencies, enhancing team collaboration and problem‐solving skills. Furthermore, ensuring regular QA and maintenance of CT‐linac equipment and associated software can effectively reduce mechanical errors and system uncertainties, thereby improving treatment accuracy. Lastly, establishing a comprehensive patient management and follow‐up system will enable timely monitoring of patient treatment responses and potential side effects. This feedback can be used to continuously refine the AIO process and treatment plans, enhancing overall patient benefits and healthcare experiences. By implementing these measures, the safety and efficacy of AIO can be significantly improved, maximizing the potential of this emerging workflow in radiation therapy. These practices can also serve as a reference for other institutions to optimize their AIO implementation processes.

In AIO, efficiency and safety are two crucial themes, but they often contradict each other. Therefore, this study briefly discusses some suggestions that can help balance both. First, thorough preparation before treatment is crucial, ensuring that tasks that can be completed prior to treatment are finished in advance. These tasks may include preheating the CT, configuring the IP, and setting up AIO protocols, etc. Second, anticipating potential issues and taking preventive measures beforehand, rather than waiting for problems to arise, is essential. Keeping records of past failures and analyzing them can aid in this process. Third, establishing a dedicated AIO team is crucial for fostering cohesion and reducing communication time. Providing thorough training for team members is crucial; it should not only focus on operational skills but also delve into the underlying principles. Understanding both the “what” and the “why” is essential. This ensures that when problems arise, the team can swiftly identify the point of occurrence, pinpoint the cause, and address the issue effectively. Fourth, the overall operational process has evolved into a set routine. During procedures, it is preferable to proceed cautiously rather than hastily. Taking a few extra seconds to confirm the accuracy of each step is crucial, as rushing may lead to errors. Fifth, developing additional automated scripts reduces the risk of human error and facilitates automated QM. Currently, many tasks are automated, such as selecting CT to electron density curves and matching the treatment couch position. These scripts not only enhance efficiency but also prevent numerous human errors, such as incorrect expansion of PTV margins or forgetting to add the treatment couch. The sixth point for discussion is whether the thermoplastic mask can be removed after the CT scan and before treatment begins. Prolonged use of the mask can cause discomfort, and concerns about patient tolerance may increase pressure on the treatment team, potentially affecting safety. We suggest removing the mask after the FBCT scan and reapplying it before beam delivery to enhance patient comfort, while ensuring treatment accuracy through image‐guided radiation therapy (IGRT) prior to treatment. Although wearing the mask from simulation to beam delivery helps reduce positional errors during the first fractionated treatment, its benefit is limited since patients undergoing AIO for nasopharyngeal carcinoma typically receive 33 fractions of RT. The improvement in patient comfort from removing the mask temporarily during AIO outweighs the marginal benefit of continuous mask use.

It should be noted that in this study, subjective experience accumulated from past AIO conducted in our institution was used for scoring during FMEA analysis. Therefore, the research findings are more applicable to our center. While other institutions may reference the results of this study to develop corresponding QM measures and optimize their own AIO, it is not advisable to replicate them entirely. Furthermore, this study is still influenced by subjective factors to some extent, such as the scoring criteria for experts' personal backgrounds and the relaxation factors $\alpha$ and $\beta$, which are determined manually. Therefore, even though the SAM method was used for weighted processing, the influence of subjective factors on FMEA cannot be completely eliminated. Moreover, the overall workflow of AIO for other cancers is similar to that for nasopharyngeal carcinoma, but there may be slight differences in details. For instance, lung cancer patients require additional 4DCT imaging and do not use MR imaging as nasopharyngeal carcinoma patients do. Therefore, the number of high‐risk FMs and the RPN ranking in the AIO workflow for lung cancer may change, and the FMEA analysis results of this study cannot be directly applied to other cancers. However, the results of this study still hold significant reference value, and the research methodology can also be fully extended to other cancers. In the future, establishing a shared system for reporting failure events across our center and potentially multiple centers will facilitate continuous optimization of the AIO. Additionally, the development of more automated tools to implement automatic QM measures will significantly contribute to enhancing the safety and effectiveness of AIO.

5. CONCLUSION

Based on the report of Task Group 100 of the AAPM, this study conducted a risk assessment of the AIO for nasopharyngeal carcinoma using improved FMEA and FTA analysis. The analysis comprehensively examined potential failures and risks within the workflow, revealing that the improved FMEA and FTA analysis are highly practical and operationally feasible. Moreover, the QM measures proposed by our center effectively mitigate the risks associated with AIO for nasopharyngeal carcinoma. Additionally, the findings of this study provide valuable insights for enhancing the safety of AIO for other tumor sites. The analysis methods and QM specifications can be further promoted for broader application.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflicts of interest.

Supporting information

Supplementary Material

Wang G, Ding S, Yang X, et al. Risk assessment and quality management in AIO based on CT‐linac for nasopharyngeal carcinoma: An improved FMEA and FTA approach. Med Phys. 2025;52:2425–2437. 10.1002/mp.17620