Abstract
Purpose
To evaluate the implementation of a magnetic resonance (MR)-only workflow (ie, implementing MR simulation as the primary planning modality) using failure mode and effects analysis (FMEA) in comparison with a conventional multimodality (MR simulation in conjunction with computed tomography simulation) workflow for pelvis external beam planning.
Methods and Materials
To perform the FMEA, a multidisciplinary 9-member team was assembled and developed process maps, identified potential failure modes (FMs), and assigned numerical values to the severity (S), frequency of occurrence (O), and detectability (D) of those FMs. Risk priority numbers (RPNs) were calculated via the product of S, O, and D as a metric for evaluating relative patient risk. An alternative 3-digit composite number (SOD) was computed to emphasize high-severity FMs. Fault tree analysis identified the causality chain leading to the highest-severity FM.
Results
Seven processes were identified, 3 of which were shared between workflows. Image fusion and target delineation subprocesses using the conventional workflow added 9 and 10 FMs, respectively, with 6 RPNs >100. By contrast, synthetic computed tomography generation introduced 3 major subprocesses and propagated 46 unique FMs, 15 with RPNs >100. For the conventional workflow, the largest RPN scores were introduced by image fusion (RPN range, 120-192). For the MR-only workflow, the highest RPN scores were from inaccuracies in target delineation resulting from misinterpretation of MR images (RPN = 240) and insufficient management of patient- and system-level distortions (RPN = 210 and 168, respectively). Underestimation (RPN = 140) or overestimation (RPN = 192) of bone volume produced higher RPN scores. The highest SODs for both workflows were related to changes in target location because of internal anatomy changes (conventional = 961, MR-only = 822).
Conclusions
FMEA identified areas for mitigating risk in MR-only pelvis RTP, and SODs identified high-severity process modes. Efforts to develop a quality management program to mitigate high FMs are underway.
Introduction
Magnetic resonance imaging (MRI) provides excellent soft tissue contrast that is substantially superior to that provided by computed tomography (CT), which has led to the incorporation of MRI into the radiation therapy treatment planning (RTP) process. Traditionally this has taken the form of using MRI as an adjunct to the CT simulation image (CT-SIM) for delineation of target or organ-at-risk volumes.1 In this conventional workflow, MR images are first rigidly registered to the CT-SIM, target volumes are contoured on the MR images, and the contours are then mapped to the CT-SIM for use during treatment plan optimization and evaluation. Aside from increasing the clinical workload, the main limitation of this method is the systematic uncertainty, reported to be up to 4 mm in the pelvis,2, 3 added by the registration process. Importantly, this systematic uncertainty persists throughout treatment, cannot be mitigated with image guided radiation therapy, and could therefore lead to geometric misses and compromised tumor control.
Thus, MR-only treatment planning has emerged as an attractive alternative to the traditional combined MR/CT workflow. However, before MR-only treatment planning in the pelvis can undergo widespread implementation, several important practical issues must be addressed, including the mitigation of geometric distortions and the development of a method for generating the electron density information necessary for accurate dose calculation. At this time, at least 2 methods have met European regulatory approval (been CE (Conformité Européene) marked) for use in the pelvis (Spectronic MriPlanner4 and Philips MR-CAT) with one also receiving Food and Drug Administration approval5 (Philips MR-CAT).
As these major obstacles are addressed and MR-only treatment planning moves closer to clinical implementation, it is imperative that the MR-only workflow be comprehensively evaluated to identify potential high-risk areas that may require additional safeguards and quality assurance procedures to be put in place. Additionally, a thorough evaluation of the workflow is necessary to educate all involved members of the radiation therapy team. This work aims to address this unmet need by examining the MR-only workflow using failure mode and effects analysis (FMEA). FMEA is widely used in industrial settings for systematically assessing risk by identifying possible failure modes (FMs) that can occur throughout the entire workflow and prioritizing actions to reduce risk based on 3 major aspects: (1) the severity of the effects from FMs, (2) their frequency of occurrence, and (3) the detectability of their occurrence.6 FMEA is often applied to either validate the design of a process or to monitor and improve existing procedures as part of a continual improvement process.7 Recently, several groups have applied this methodology within radiation oncology for a variety of applications to develop more robust quality management programs.8, 9, 10 In this study, we applied design FMEA to an MR-only simulation workflow that is proposed for use in our clinic and then compared it with the conventional multimodality workflow (MR simulation [MR-SIM] used as an adjunct to CT-SIM) for pelvis external beam RTP. This comparison was performed to highlight sources of error that may arise that are unique to the MR-only treatment planning environment and will require careful attention to mitigate risk in a future implementation as well as to identify areas of commonality where we can carry over existing processes. As we aim to bring this emerging technology into widespread clinical practice, a need exists to understand potential problems and to mitigate any foreseeable errors that may arise during the implementation of this methodology.
Methods and Materials
FMEA
The first step in FMEA involves identifying key processes within the overall workflow. For each process, all possible methods that might lead to the failure of any task within that process (ie, FMs) are identified. Each FM is then given 3 scores based (1) on the most severe effect resulting from the failed task (S for severity), (2) the frequency with which the FM would occur (O for occurrence), and (3) the likelihood that the failure would be detected given the currently used quality control/quality assurance framework (D for detectability). These assigned S, O, and D scores are then used to calculate a risk metric that is used to assign a priority level for addressing the FMs.
A 9-member multidisciplinary team consisting of 3 medical physicists, 2 radiation oncologists, 2 radiation therapists, and 2 MR technologists performed FMEA for our institutional MR-only workflow and our current multimodality pelvis MR-SIM process. The medical physics group began by developing the high-level process maps for each workflow that are shown in Figure 1, wherein main processes were identified for each workflow. For each main process, a list of necessary tasks was developed and listed as subprocesses under one of these main categories. Team members were assigned to the most appropriate process for their discipline (eg, therapists for the immobilization/positioning process, MR technologists for the MRI process). A list was compiled to include FMs for each subprocess as well as any new subprocess previously overlooked. Hence, all process maps were reviewed by members of the team, and additional subprocesses were identified. Each FM was discussed with the other team members. For implementation, a list of failure pathways leading to each FM was then generated and assigned a score with respect to S, O, and D by the medical physicist group.
Figure 1.
High-level process map showing the main processes for both the magnetic resonance (MR)–only and conventional multimodality workflows as well as select subprocesses unique to each workflow. Green indicates those processes unique to the multimodality workflow, red indicates those processes unique to the MR-only workflow, and blue indicates those processes that are identical between the 2. Abbreviations: CT = computed tomography; DICOM = Digital Imaging and Communication in Medicine; DRR = digitally reconstructed radiograph; IV = intravenous; MRI = magnetic resonance imaging; RT = radiation therapy; TPS = transaction processing system.
For this study, to reduce subjectivity in scoring, we used the ranking system defined by the American Association of Physicists in Medicine Task Group 100.11 The system uses a 10-point scale for each category, with 1 defined as the least severe, least common, and most easily detected for the S, O, and D categories, respectively. Typically the risk metric used for determining the relative risk for each FM is the risk priority number (RPN), which is the product of the S, O, and D values that were scored for each FM.12 Although the RPN number has become a standard metric for evaluating the relative risk of FMs, there exist some limitations with using this simple product.13 For example, only 120 unique RPN scores exist for 1000 different combinations of S, O, and D values, and these unique RPN values are not evenly distributed on a continuum from 1 (RPN = S*O*D = 1*1*1) to 1000 (RPN = S*O*D = 10*10*10), with the result that 90% of possible S, O, and D combinations result in RPN scores less than 405. Because no priority is given to any individual category, it is possible to have an RPN score for an FM with very severe effects (eg, S, O, D = 9, 3, 4) that is the same as that for an FM that has low severity but moderately higher occurrence and detectability scores (eg, S, O, D = 3, 6, 6). Thus we have employed an alternative metric that preferentially weights the severity of the FM: the SOD score.
SOD was first introduced as an alternative to RPN in the Automotive Industry Action Group’s FMEA manual.14 The SOD is a simple composite of the 3 scores, in which the severity of the failure is the first digit, followed in order by rate of occurrence and detectability (eg, scores of 8, 3, and 4 for S, O, and D, respectively, give an SOD value of 834). The purpose of using both RPN and SOD values is that they provide complementary information. RPN values provided a surrogate for relative patient risk, and SOD values were used for prioritizing the high-severity FMs within risk categories for the MR-only and combined MR/CT workflows.
Fault tree analysis
Fault tree analysis (FTA) is part of the process of identifying elements in a chain of causation that could lead to the various FMs. FTA is used to identify the possible negative outcomes that could occur given an initiating event and to represent the failure pathway via a block diagram. A detailed FTA was performed for the FM that was found to yield the highest severity scores.
Radiation treatment planning workflows
Figure 1 illustrates high-level process mapping, highlighting the 3 overall shared processes (in blue) that are identical between workflows. For the conventional multimodality workflow used in our clinic, the patient undergoes a CT-SIM using a Brilliance Big Bore (Philips Healthcare, Cleveland, OH) CT scanner. This is followed by an MR-SIM imaging session in our dedicated 1.0T Panorama High Field Open (Philips Medical Systems, Best, Netherlands) MR simulator using a flat tabletop and the same immobilization devices as used in CT-SIM. Both CT and MR images are then imported into the Eclipse treatment planning system (Varian Medical Systems, Palo Alto, CA) where the MR images are rigidly registered to the CT, a process that introduces several unique subprocesses and FMs. The physician contours the target on the MR images, contours are mapped to the CT-SIM, and a treatment plan is generated. In the conventional multimodality workflow, the CT data set is used to determine the electron density values required for dose calculation and to generate digitally reconstructed radiographs (DRRs), which are used for patient localization at the time of treatment for image guided radiation therapy. For our pelvis population, we typically treat using intensity modulated arc therapy or 9 static-field intensity modulated radiation therapy treatments.
The MR-only workflow begins with the patient undergoing an MR-SIM on the previously mentioned Philips 1.0T Panorama. This is followed by generation of a synthetic CT (synCT) from the acquired MR images. SynCTs are used for dose calculation and DRR generation in an MR-only setting and are typically generated via atlas-based15, 16 or intensity-based17, 18, 19 approaches. The approach used in this study was developed at our institution and involves a voxel-based weighted summation method that has been described in detail in a previous publication.19 Briefly, our synCT workflow requires manual segmentation of pelvic bones followed by automated segmentation of air, soft tissue, fat, and fluid using k-means clustering and morphologic operations. Each synCT voxel value is then calculated as a summation of intensity values from the acquired MR images that have been weighted by sequence- and class-specific factors. The synCTs and MR images are then imported into the Eclipse treatment planning system, where the MR images are used for target and organ-at-risk definition, and the synCTs are used for dose calculation and DRR generation.
Results
The process map developed by the team is shown in Figure 1. Seven main processes were identified for the multimodality workflow and 6 for the MR-only workflow. The main processes captured the key steps in our simulation and treatment planning workflows. Included subprocesses were broad enough to include several FMs and provided a good overview of the focus of each main category. Processes 1 to 5 for the multimodality workflow and 1 to 4 for the MR-only workflow mainly focused on departmental logistics, MRI procedure and simulation, and data transfer domains. These were largely the same between the 2 workflows, even with respect to subprocesses and FMs. Processes 3 and 4 (MRI and CT simulation) for the multimodality workflow were combined into a single MR-SIM process (process 3) in the MR-only workflow. However, the subprocesses and FMs in the MR-only process were the same as in the 2 MRI and CT simulation processes in the multimodality workflow. The 3 main processes that the 2 workflows shared were ordering of simulation procedures, immobilization and positioning of the patient at time of simulation, and transfer of data to the treatment planning system. For these processes, the risk-mitigation strategies used for our current multimodality workflow (eg, checklists, staff training procedures, physical barriers preventing unqualified personnel from entering certain areas) can be used without modification for our proposed MR-only workflow. In all, these consisted of 24 subprocesses and 125 FMs.
Although Figure 1 shows significant overlap between the MR-only and combined MR/CT treatment planning workflows, several key areas were identified as unique to MR-only planning. For example, geometric distortions, which can be categorized as either machine specific or patient specific, may have a significant impact on the electron density map in an MR-only planning environment. The magnitude of machine-specific distortions can be characterized in phantoms and then accounted for in postprocessing,20, 21 with the overall behavior increasing in magnitude as the distance away from magnetic isocenter increases,20 which is relevant for the large fields of view required in pelvis radiation therapy. Patient-specific distortions, on the other hand, depend on field strength and can be reduced to within acceptable tolerances for many body sites using appropriate MR sequences, although these can be nonnegligible near tissue/air interfaces.22 The extra processes required to generate a synCT also introduce additional FMs in an MR-only setting.
Process 6 was entirely different between workflows. For the MR-only workflow, process 6 comprised synCT generation, whereas for the multimodality workflow, process 6 focused on MR-to-CT fusion. The addition and use of a synCT introduced 3 additional subprocesses and propagated 46 FMs (15 with FMs >100) unique to the MR-only workflow. Table 1 best summarizes the RPN and SOD results that were unique to the synCT generation process, 8 of which had RPNs >100. The use of synCT also introduces additional unique FMs based on the interpretation of images, impact of segmentation and density assignments on dose calculation, and bone segmentation affecting DRR accuracy. In the conventional workflow (MR/CT combined), the image fusion and target delineation subprocesses added 19 total FMs (6 being >100). One of the most significant FMs was inaccurate localization of the tumor volume, which could have several causes. FTA for this particular FM was performed to identify all events that fall along the causal chain that ultimately lead to the inaccurate localization. Four failure pathways were identified with a total of 14 branches, where each pathway consisted of one or more technical failures along with a failure in supervision, often caused by inadequate training of the person overseeing the process. The FTA diagram is displayed in Figure 2. Treatment planning was the seventh process for both workflows, but the FMs were unique for each method.
Table 1.
Unique synCT generation RPN and SOD scores before and after suggested modifications
| Failure mode | Failure pathway | Before mitigation |
Suggested Change | After mitigation |
||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| S | O | D | RPN | SOD | S | O | D | RPN | SOD | |||
| 1. Bone segmentation | ||||||||||||
| Incorrect bone classification | Bone/air indistinguishable in MR images | 4 | 9 | 1 | 36 | 491 | Consult with radiologists; apply consensus guidelines; attend education programs through national meetings (eg, RSNA) | 4 | 9 | 1 | 36 | 491 |
| Nonbone material classified as bone | 5 | 8 | 1 | 40 | 581 | 5 | 4 | 1 | 20 | 541 | ||
| Bone volume underestimated | 4 | 7 | 4 | 112 | 474 | 4 | 4 | 3 | 48 | 443 | ||
| Bone volume overestimated | 4 | 8 | 4 | 128 | 484 | 4 | 5 | 3 | 60 | 453 | ||
| Uncertainty from interobserver differences in manual bone segmentation | 4 | 5 | 3 | 60 | 453 | 4 | 4 | 3 | 48 | 443 | ||
| 2. Tissue classification/density assignments | ||||||||||||
| SynCT not representative of average anatomy | Long scan time leads to changes in internal anatomy (bladder/rectal filling) | 4 | 7 | 1 | 28 | 471 | Minimize number of acquired sequences; minimize acquisition time for each sequence | 3 | 7 | 1 | 21 | 371 |
| Varied physiologic states for different data sets needed for synCT | 4 | 7 | 1 | 28 | 471 | 3 | 7 | 1 | 21 | 371 | ||
| Changed target location because of state | 6 | 5 | 2 | 60 | 652 | 4 | 5 | 2 | 40 | 452 | ||
| Patient-specific distortion corrections for air/tissue may be inaccurate | 4 | 4 | 5 | 80 | 445 | 3 | 4 | 5 | 60 | 345 | ||
| Tissue misclassification/inaccurate HU assignment | Inaccurate autosegmentation | 4 | 6 | 3 | 72 | 463 | Standardize sequences; increase the number of patients to ensure a representative group of patients in the training set | 3 | 4 | 3 | 36 | 343 |
| Patient not well represented by population-based values | 4 | 3 | 9 | 108 | 439 | 4 | 1 | 9 | 36 | 419 | ||
| Population-based values derived from a nonrepresentative set of patients | 4 | 1 | 9 | 36 | 419 | 2 | 1 | 9 | 18 | 219 | ||
| Not enough patients used to determine population-based values | 4 | 1 | 9 | 36 | 419 | 2 | 1 | 9 | 18 | 219 | ||
| Inaccurate segmentation | Image nonuniformity affecting automated intensity-based segmentation approaches | 4 | 5 | 5 | 100 | 455 | Check constancy of vendor-implemented correction software; Implement independent postprocessing assessment and correction tools and QA procedures | 4 | 3 | 4 | 48 | 434 |
| Inadequate distortion correction | 6 | 6 | 3 | 108 | 663 | 2 | 4 | 3 | 24 | 243 | ||
| 3. Overall synCT process | ||||||||||||
| External contour incorrect | System-level geometric distortion not taken into account | 5 | 9 | 3 | 135 | 593 | Implement robust QA/QC including verification tests performed on phantoms; training of radiation oncology staff with respect to proper coil use | 5 | 5 | 3 | 75 | 553 |
| Image artifacts preventing accurate external delineation | 3 | 8 | 1 | 24 | 381 | 3 | 6 | 1 | 18 | 361 | ||
| External anatomy incomplete | 3 | 8 | 1 | 24 | 381 | 3 | 6 | 1 | 18 | 361 | ||
| Anatomy deformed by coils | 3 | 7 | 2 | 42 | 372 | 3 | 4 | 2 | 24 | 342 | ||
| Inaccurate synCT | Missing images required for generating synCT | 4 | 2 | 1 | 8 | 421 | Standardize sequences | 4 | 1 | 1 | 4 | 411 |
| Organ location inaccurate | System-level geometric distortion not taken into account | 6 | 7 | 4 | 168 | 674 | Standardize sequences; optimize sequence parameters to minimize acquisition time; implement vendor-independent postprocessing software | 2 | 2 | 5 | 20 | 225 |
| Patient-induced distortions near interfaces present | 6 | 7 | 5 | 210 | 675 | 5 | 6 | 5 | 150 | 565 | ||
| Patient anatomy is not standard for patient—unable to reproduce anatomy | 7 | 3 | 3 | 63 | 733 | 7 | 1 | 3 | 21 | 713 | ||
| Long scan time leads to changes in internal anatomy (bladder/rectal filling) | 6 | 5 | 2 | 60 | 652 | 4 | 5 | 2 | 40 | 452 | ||
| Varied physiologic states for different data sets needed for synCT | 6 | 3 | 2 | 36 | 632 | 5 | 2 | 2 | 20 | 522 | ||
| Changed target location because of state | 8 | 2 | 2 | 32 | 822 | 6 | 2 | 2 | 24 | 622 | ||
Abbreviations: HU = Hounsfield unit; MR = magnetic resonance; QA = quality assurance; QC = quality control; RPN = risk priority numbers; RSNA = Radiological Society of North America; SOD = severity-occurrence-detectability; synCT = synthetic computed tomography.
Figure 2.
Fault tree analysis (FTA) diagram for potential failure modes for inaccurate tumor localization. Abbreviation: MRI = magnetic resonance imaging.
Figure 3 shows the distribution of calculated RPN scores for the FMs unique to either the MR-only or multimodality workflows. The MR-only workflow yielded a larger number of FMs (47 compared with 19 for the conventional) that were generally weighted toward lower RPN values (mean RPN scores of 75 ± 42). However, the highest RPN scores between both workflows were obtained for FMs specific to the MR-only workflow, as shown in Table 1. Table 2 provides the 5 failure pathways with the highest RPN and SOD scores for the MR-only workflow. As seen, the FMs with the highest RPNs for MR-only treatment planning were inaccurate target/organ delineation arising from inadequate training of MRI interpretation (RPN = 240) and patient- and system-level distortion corrections not being properly managed (RPN = 210 and 168, respectively). For the conventional multimodality workflow, the highest RPN values were calculated for poor image fusion quality and misinterpretation of multimodality information leading to inaccurate delineation (RPN range, 120-192).
Figure 3.
Histogram of risk priority number (RPN) scores for failure modes unique to either magnetic resonance (MR)–only or the multimodality (CT/MR) workflows. Abbreviation: FMEA = failure mode and effects analysis.
Table 2.
Five highest scoring failure modes in terms of RPN and SOD
| Failure mode | Failure pathway | S | O | D | RPN | SOD |
|---|---|---|---|---|---|---|
| Wrong data set used for contouring | Inadequate training/MR images | 6 | 5 | 8 | 240 | 658 |
| Organ location inaccurate | Patient-induced distortions near interfaces present | 6 | 7 | 5 | 210 | 675 |
| Inaccurate dose calculation | Bone volume overestimated | 6 | 8 | 4 | 192 | 684 |
| Organ location inaccurate | System-level geometric distortion not taken into account | 6 | 7 | 4 | 168 | 674 |
| Treatment volume not properly identified | Physician interpretation of images | 5 | 4 | 8 | 160 | 548 |
| Organ location inaccurate | Changed target location because of state | 8 | 2 | 2 | 32 | 822 |
| Inaccurate dose calculation | Bone volume underestimated | 7 | 5 | 4 | 140 | 754 |
| Organ location inaccurate | Patient anatomy is not standard for patient—unable to reproduce anatomy | 7 | 3 | 3 | 63 | 733 |
| Wrong treatment volume delineated | Wrong image set used | 7 | 2 | 6 | 84 | 726 |
| Inaccurate dose calculation | Bone volume overestimated | 6 | 8 | 4 | 192 | 684 |
Abbreviations: RPN = risk priority number; SOD = severity-occurrence-detectability.
The distribution of SOD values shown in Figure 4 and Table 1 demonstrates a different pattern in which the bulk of FMs unique to the MR-only process are of medium severity (77% of unique SODs falling between 400 and 700). Conversely, FMs unique to the multimodality workflow were shifted toward higher SOD scores (32% >700). These higher SODs emphasize the generally more severe effects of systematic uncertainties in target localization that arise in this workflow. The highest calculated SOD values fell under the category of inaccurate target localization caused by varying bladder and rectal filling status for both conventional (961, RPN = 54) and MR-only (822, RPN = 32) workflows.
Figure 4.
Histogram of severity-occurrence-detectability (SOD) scores for failure modes unique to either magnetic resonance (MR)–only or the multimodality computed tomography (CT)/MR workflows. Abbreviation: FMEA = failure mode and effects analysis.
Discussion
In this study, FMEA was performed to systematically evaluate our proposed MR-only workflow and to compare it with the existing multimodality standard of care workflow in our clinic. This comparison indicated that a significant number of existing processes and procedures that are in place for the multimodality workflow can be used without modification for MR-only RTP because so many of the main processes are identical between the 2 workflows. We also identified key areas of opportunity for reducing uncertainties in the treatment planning simulation process in the MR-only workflow.
The most significant FMs introduced by the MR-only methodology in terms of RPN and SOD focused mainly on the generation of synCTs and their use in RTP, meaning that this step provides the highest measure of risk and that this is in part a result of these FMs having the most severe consequences within this workflow. These FMs generally fell into 2 categories, with the highest scores arising from (1) inexperience in handling and interpreting MR data and (2) systematic errors arising from inadequate postprocessing of the images, such as residual geometric distortion and intensity nonuniformity corrections. To mitigate the effects of the first category of FMs as users gain experience in an MR environment, it is important for radiation oncology staff to become more familiarized with MRI sequences used for select disease sites. This can be accomplished through consultations with radiologists, the application of consensus guidelines,23 or education programs provided at national meetings such as that of the Radiological Society of North America.24 Furthermore, standardization of imaging sequences and the contrasts presented can be emphasized. Opportunities to mitigate this risk and improve the detectability include the addition of contour review rounds25 or developing checklist items on the physics chart check to ensure the proper MRI data set was used for contouring.
To address postprocessing corrections, a robust quality control/quality assurance program must be implemented that thoroughly characterizes the magnet to determine the need for additional gradient nonlinearity distortion and nonuniformity corrections in addition to vendor-provided corrections. Automated checks of the DICOM (Digital Imaging and Communication in Medicine) header can verify that the 3-dimensional vendor-supplied distortion correction was applied. We have previously found that system-level distortions are magnet dependent26; uncorrected distortions can be up to 7.4 mm only 23 cm away from the magnetic isocenter for our 1.0T MR-SIM, necessitating an additional postprocessing step. Distortion corrections may be necessary for both the MR/CT and MR-only treatment planning workflows in large body sites such as the pelvis, and each body site will have its own unique challenges and separate workflows. Our daily MR-SIM QA program verifies constancy and identifies any deviations from baseline.
The generation of synCTs was the major source of unique FMs for MR-only RTP. Table 1 provides the failure pathways identified and the corresponding RPN and SOD scores for this process. They can be categorized into those that arise from errors in the acquired MR data and those that are caused by the image processing required to generate the synCT. The first category can be mitigated through methods similar to those listed earlier: by development of a standard set of MR sequences along with training of the MR staff to identify problems in the MR images, such as apparent intensity nonuniformity, as they are being acquired. The second can be accomplished through consistent quality assurance of the synCT generation. Quality control tests must be incorporated that can adequately identify gross errors (>5%) because more subtle errors in Hounsfield Unit (HU) assignment will have less than 1% impact on dose calculation for photon planning.27 As can be seen from Table 1, the re-evaluated FMEA scores after the proposed changes were added to the prospective workflow could also be split into 2 groups with increased training and QA/QC procedures leading to decreases mainly in the occurrence of the errors, and standardization of procedures and implementation of postprocessing solutions mainly reducing the severity of the residual error. The main improvements were seen in the patient-independent process of correcting for the geometric distortion and intensity homogeneity by establishing a correction method independent from the vendor-supplied software. These measures shifted a high-risk (RPN = 168, SOD = 674) pathway to a low-risk pathway (RPN = 20, SOD = 225).
One limitation of this study is that it includes the experience of only a single institution. Although efforts were made to include reviewers with different perspectives and a broad range of experience, knowledge was still limited mainly to experience with the equipment available in our department using our own processes and procedures. Although one would expect there to be significant overlap in the major processes and subprocesses identified by separate institutions, each institution would most likely identify several unique FMs and subprocesses that arise from different workflows and MR system designs. MR safety and contraindications were shared by both workflows, have been evaluated in detail previously, and are not unique to MR-SIM.28
Experience gained from our analysis may be used to evaluate our current QA processes to develop more robust chart checklists in an MR-only workflow. For example, additional checklist items relating to the QA of generated synCT images (eg, if bone segmentation were appropriate and if air pockets were properly characterized) would serve to add increased scrutiny to this unique aspect of the MR-only workflow. The FMEA performed here for the pelvis can also serve as a template for performing further analyses on different body sites such as the cranium. Most of the FMEA will be similar to MR-only of the pelvis. Although the coregistration uncertainty tends to be reduced,29 synCT generation tends to be more complex,30 and system-level distortions are likely to be lower with the brain positioned near the magnet isocenter.21
Conclusions
Implementation of the FMEA has led to identification of key areas of focus for risk mitigation in an MR-only pelvis RTP. Use of the SOD metric allowed us to prioritize high-severity FMs. As was expected, obtaining CT data as part of the conventional multimodality workflow led to increased detectability of FMs, partially because of redundant information and preexisting clinical safeguards. Strategies for mitigating the highest-risk FMs have been established, and future updates will include incorporation of image guidance and delivery. Robust design and process FMEA is an essential component of continual process improvement for safe integration of emerging technologies into the clinic.
Acknowledgments
The authors would like to thank their Radiation Therapy Technologists (Denise Socia and Kim Sandhu) and magnetic resonance techs (Michelle Mayhew, Stephanie Schachermeyer) for their contributions, which were essential to the work.
Footnotes
Sources of support: This work was supported by an Internal Mentored Grant and the National Institutes of Health (R01CA204189).
Conflicts of interest: Henry Ford Health System holds research agreements with Philips Healthcare and Varian Medical Systems.
References
- 1.Debois M., Oyen R., Maes F. The contribution of magnetic resonance imaging to the three-dimensional treatment planning of localized prostate cancer. Int J Radiat Oncol Biol Phys. 1999;45:857–865. doi: 10.1016/s0360-3016(99)00288-6. [DOI] [PubMed] [Google Scholar]
- 2.Nyholm T., Nyberg M., Karlsson M.G. Systematisation of spatial uncertainties for comparison between a MR and a CT-based radiotherapy workflow for prostate treatments. Radiat Oncol. 2009;4:54. doi: 10.1186/1748-717X-4-54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Dubois D.F., Bice W.S., Prestige B.R. CT and MRI derived source localization error in a custom prostate phantom using automated image coregistration. Med Phys. 2001;28:2280–2284. doi: 10.1118/1.1406525. [DOI] [PubMed] [Google Scholar]
- 4.Siversson C., Nordstrom F., Nilsson T. Technical note: MRI only prostate radiotherapy planning using the statistical decomposition algorithm. Med Phys. 2015;42:6090–6097. doi: 10.1118/1.4931417. [DOI] [PubMed] [Google Scholar]
- 5.Tyagi N., Fontenla S., Zhang J. Dosimetric workflow evaluation of first commercial synthetic CT software for clinical use in pelvis. Phys Med Biol. 2017;62:2961–2975. doi: 10.1088/1361-6560/aa5452. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Teng S.H., Ho S.Y. Failure mode and effects analysis: An integrated approach for product design and process control. Int J Qual Reliab Manage. 1996;13:8–26. [Google Scholar]
- 7.Teoh P.C., Case K. Failure modes and effects analysis through knowledge modelling. J Mater Process Technol. 2004;153:253–260. [Google Scholar]
- 8.Ford E.C., Gaudette R., Myers L. Evaluation of safety in a radiation oncology setting using failure mode and effects analysis. Int J Radiat Oncol Biol Phys. 2009;74:852–858. doi: 10.1016/j.ijrobp.2008.10.038. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Perks J.R., Stanic S., Stern R.L. Failure mode and effect analysis for delivery of lung stereotactic body radiation therapy. Int J Radiat Oncol Biol Phys. 2012;83:1324–1329. doi: 10.1016/j.ijrobp.2011.09.019. [DOI] [PubMed] [Google Scholar]
- 10.Younge K.C., Lee C., Moran J.M. Failure mode and effects analysis in a dual-product microsphere brachytherapy environment. Pract Radiat Oncol. 2016;6:e299–e306. doi: 10.1016/j.prro.2016.03.003. [DOI] [PubMed] [Google Scholar]
- 11.Huq M.S., Fraass B.A., Dunscombe P.B. The report of Task Group 100 of the AAPM: Application of risk analysis methods to radiation therapy quality management. Med Phys. 2016;43:4209. doi: 10.1118/1.4947547. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Carbone T.A., Tippett D.D. Project risk management using the project risk FMEA. Eng Manage J. 2008;16:28–35. [Google Scholar]
- 13.Wheeler D.J. Quality Digest; 2007. Problems With Risk Priority Numbers: Avoiding More Numerical Jabberwocky.https://www.qualitydigest.com/inside/quality-insider-article/problems-risk-priority-numbers.html Available at: Accessed December 6, 2016. [Google Scholar]
- 14.Potential Failure Mode and Effects Analysis: FMEA Reference Manual. 4th ed. AIAG; Southfield, MI: 2008. [Google Scholar]
- 15.Dowling J.A., Lambert J., Parker J. An atlas-based electron density mapping method for magnetic resonance imaging (MRI)—alone treatment planning and adaptive MRI-based prostate radiation therapy. Int J Radiat Oncol Biol Phys. 2012;83:e5–e11. doi: 10.1016/j.ijrobp.2011.11.056. [DOI] [PubMed] [Google Scholar]
- 16.Chen S., Quan H., Qin A. MR image-based synthetic CT for IMRT prostate treatment planning and CBCT image-guided localization. J Appl Clin Med Phys. 2016;17:6065. doi: 10.1120/jacmp.v17i3.6065. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Johansson A., Karlsson M., Nyholm T. CT substitute derived from MRI sequences with ultrashort echo time. Med Phys. 2011;38:2708–2714. doi: 10.1118/1.3578928. [DOI] [PubMed] [Google Scholar]
- 18.Hsu S.H., Cao Y., Huang K. Investigation of a method for generating synthetic CT models from MRI scans of the head and neck for radiation therapy. Phys Med Biol. 2013;58:8419–8435. doi: 10.1088/0031-9155/58/23/8419. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Kim J., Glide-Hurst C., Doemer A. Implementation of a method for generating synthetic CT images from magnetic resonance imaging data sets for prostate cancer radiation therapy. Int J Radiat Oncol Biol Phys. 2015;91:39–47. doi: 10.1016/j.ijrobp.2014.09.015. [DOI] [PubMed] [Google Scholar]
- 20.Liney G.P., Moerland M.A. Magnetic resonance imaging acquisition techniques for radiotherapy planning. Semin Radiat Oncol. 2014;24:160–168. doi: 10.1016/j.semradonc.2014.02.014. [DOI] [PubMed] [Google Scholar]
- 21.Price R.G., Kadbi M., Kim J. Technical Note: Characterization and correction of gradient nonlinearity induced distortion on a 1.0 T open bore MR-SIM. Med Phys. 2015;42:5955–5960. doi: 10.1118/1.4930245. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Stanescu T., Wachowicz K., Jaffray D.A. Characterization of tissue magnetic susceptibility-induced distortions for MRIgRT. Med Phys. 2012;39:7185–7193. doi: 10.1118/1.4764481. [DOI] [PubMed] [Google Scholar]
- 23.Schimmoller L., Quentin M., Arsov C. MR-sequences for prostate cancer diagnostics: Validation based on the PI-RADS scoring system and targeted MR-guided in-bore biopsy. Eur Radiol. 2014;24:2582–2589. doi: 10.1007/s00330-014-3276-9. [DOI] [PubMed] [Google Scholar]
- 24.Barentsz J, Futterer J, Mus R, et al. Prostate MRI (hands-on). Presented at: Radiological Society of North America 2016 Scientific Assembly and Annual Meeting. November 27–December 2, 2016; Chicago IL. Available at: archive.rsna.org/2016/16002005.html Accessed January 20, 2017.
- 25.Bajaj A., Solanki A.A., Small W. Prospective case review in radiation oncology prior to treatment delivery. Oncol Times. 2016;38:14–15. [Google Scholar]
- 26.Price R.G., Knight R.A., Hwang K.P. Optimization of a novel large field of view distortion phantom for MR-only treatment planning. J Appl Clin Med Phys. 2017;18:51–61. doi: 10.1002/acm2.12090. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Zurl B., Tiefling R., Winkler P. Hounsfield units variations: Impact on CT-density based conversion tables and their effects on dose distribution. Strahlenther Onkol. 2014;190:88–93. doi: 10.1007/s00066-013-0464-5. [DOI] [PubMed] [Google Scholar]
- 28.Thornton E., Brook O.R., Mendiratta-Lala M. Application of failure mode and effect analysis in a radiology department. Radiographics. 2011;31:281–293. doi: 10.1148/rg.311105018. [DOI] [PubMed] [Google Scholar]
- 29.Price R.G., Kim J.P., Zheng W. Image guided radiation therapy using synthetic computed tomography images in brain cancer. Int J Radiat Oncol Biol Phys. 2016;95:1281–1289. doi: 10.1016/j.ijrobp.2016.03.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Zheng W., Kim J.P., Kadbi M. Magnetic resonance-based automatic air segmentation for generation of synthetic computed tomography scans in the head region. Int J Radiat Oncol Biol Phys. 2015;93:497–506. doi: 10.1016/j.ijrobp.2015.07.001. [DOI] [PubMed] [Google Scholar]