Assessment of Learning Curve for Radiofrequency Ablation in Twin Reversed Arterial Perfusion Sequence: A Simulation Model Study

ABSTRACT

Objective

This study characterized the procedural learning curve of novice practitioners in mastering radiofrequency ablation (RFA) in a simulated twin reversed arterial perfusion sequence (TRAPS) model.

Method

Twelve novices practiced RFA in a TRAPS model, which was evaluated for validity. A learning curve CUSUM analysis was performed to define the number of procedures required to achieve competency. The learning plateau of needle insertion time and the number of procedures required to surpass 90% of the learning plateau were calculated.

Results

The overall model rating of 4.26 ± 0.58 serves as validating the high learning performance. A success rate of 92.8% was achieved across 767 procedures. The average number of procedures required to achieve technical competency was 29 relative to years of experience in minimally invasive prenatal testing. After reaching this competency level, the success rate improved from 88.2% to 96.7% (P < 0.001). The needle insertion time learning curve indicated that 90% of the learning plateau was reached after 21 procedures, with the learning plateau occurring at 27.5s.

Conclusion

Performance in RFA within the simulated TRAPS model improved over time. Achieving competency enhanced technical success and shortened the needle insertion process. This simulation training provides practical skills for inexperienced surgeons.

Trial Registration

TCTR20221005001

Summary.

• What's already known about this topic?

  • ◦Radiofrequency ablation (RFA) is a treatment option for twin reversed arterial perfusion sequence (TRAPS). Simulation training is essential for acquiring the necessary skills for this invasive technique.

• What does this study add?

  • ◦Novice surgeons obtain surgical skills from this simulation training by completing 29 procedures. The attainment of a competency level for technical success depends on both the training received and prior experience in performing invasive prenatal procedures.

1. Introduction

Twin reversed arterial perfusion sequence (TRAPS) is a complication of monochorionic twin pregnancy in which one fetus lacks typical cardiac structures (referred to as the “acardiac fetus”), whereas the other must supply both circulatory systems (referred to as the “pump fetus”) [1]. Reversed oxygen‐deprived blood flow is distributed to the acardiac fetus, which impairs morphogenesis. Without intervention, the pump fetus has a perinatal death rate of 55% throughout pregnancy [2]. TRAPS is associated with a higher risk of congestive heart failure in the pump fetus, polyhydramnios, and preterm delivery when the acardiac‐to‐pump twin weight ratio exceeds 70% [3]. There is a debate on whether and under what conditions TRAPS is treated [4, 5, 6, 7, 8].

Radiofrequency ablation (RFA) can be used to ablate intrafetal vessels of the acardiac fetus during the second trimester of pregnancy, with an overall survival rate of 80% in the pump fetus [9, 10]. However, incomplete intrafetal ablation leads to potential risks of intrauterine fetal death and preterm delivery.

Accurate placement of the RFA needle is critical for successful vessel ablation. Simulation‐based practice is valuable for acquiring surgical skills and ensuring patient safety, particularly for this unique procedure [11, 12]. The Accreditation Council for Graduate Medical Education emphasizes the assessment of an operator's competency [13]. To minimize complications from inexperienced operators and reduce unnecessary training for experts, a learning curve analysis is recommended to identify when an operator has reached the required level of competency. Therefore, this study was designed to determine the learning curve for novices practicing RFA using a validated TRAPS model.

2. Methods

2.1. Study Design and Population

This prospective study was conducted at the Department of Obstetrics and Gynecology, Faculty of Medicine Siriraj Hospital, Mahidol University, between November 2022 and September 2023. This study was conducted according to the ethical principles of the Declaration of Helsinki. Furthermore, the study protocol was approved by the Institutional Review Board of Siriraj Hospital (COA No. Si696/2022) and was registered in the Thai Clinical Trials Registry (TCTR20221005001). Funding for this study was provided by a grant from the Faculty of Medicine Siriraj Hospital. Note that the grant provider played no role in the study design, data collection, statistical analysis and interpretation, manuscript preparation, or decision to publish.

2.2. Study Participants

In this study, 12 maternal–fetal medicine specialists and trainees were invited to participate with informed consent. Although all participants were familiar with invasive prenatal procedures, none had prior experience with RFA. However, each had observed this procedure on TRAPS at least once. The participants were expected to attend simulation training sessions once a week, with each session lasting no more than 3 h, until achieving competency. Once competent, each participant performed 35 operations, allowing for an acceptable error rate of 10%, a confidence level of 95%, and a predicted success rate of 90%.

2.3. TRAPS Model

A box model was developed using a stainless steel container, measuring 35 × 18 × 15 cm, covered with a rubber sheet. This design facilitates the passage of RF energy through grounding pads attached beneath the container. Inside the box model, a model simulating an acardiac fetus was immersed in warm saline solution at 40°C. To simulate the amniotic sac of the pump fetus, a rubber condom was inflated with 500 mL of tap water and placed inside the box model. The body of the acardiac fetus was simulated using a 300‐g piece of porcine tissue [14] wrapped in a latex condom with both ends open. A fresh human umbilical cord (10 cm in length and 1 cm in diameter) from term gestation was inserted into the core of the acardiac fetus after being flushed with warm saline through the umbilical vein to ensure vessel patency. An orogastric tube (8 French) was inserted into the umbilical vein to a depth of 2 cm and connected to a Hotline fluid warmer. During the practice session, warm saline solution was infused into the umbilical vein (100 mL/min) to generate a realistic ultrasound image of the acardiac fetus (Figure 1A) [15]. This model was validated by three fetal surgeons experienced in RFA for TRAPS. Using a 5‐point Likert scale (ranging from strongly disagree (1) to strongly agree (5)), the average ratings for the realism of materials, value and relevance of this model as a skill training and appraisal tool, and accessibility of the procedure were 4.2, 5.0, and 4.0, respectively [16, 17, 18]. This model has been patented to Mahidol University, Bangkok, Thailand (patent number 2303000415).

FIGURE 1.

Radiofrequency ablation (RFA) training in a twin reversed arterial perfusion sequence (TRAPS) simulator: (A) TRAPS model; (B) axial view of a simulated acardiac fetus demonstrated by ultrasonography; (C) longitudinal view of a simulated acardiac fetus demonstrated by ultrasonography; (D) cut surface of a simulated acardiac fetus after RFA demonstrating tissue coagulation.

2.4. RFA

A RITA StarBurst probe (17G) equipped with three retractable tines and thermocouples for temperature feedback was connected to the RF generator. This RF needle, measuring 12 cm in length, created a spherical ablation with a diameter of 2 cm. The RITA 1500× RF generator (AngioDynamics Inc., Manchester, GA, USA) generated monopolar 460‐kHz RF energy. The RF power was fixed at 150 W, and the temperature was 105°C. Two grounding pads were attached beneath the box model.

Under ultrasound guidance (Voluson, General Electric, Medical System Kretztechnik GmbH & Co. OHG, Zipf, Austria, with a 2–6‐MHz abdominal transducer) (Figure 1B and 1C), the RF needle was introduced into the acardiac fetus and directed toward the internal vessel. When approaching the target site, the tines were deployed to a diameter of 2 cm. RF energy was then generated, gradually increasing the temperature of the umbilical vessel to 105°C and sustained for 10 min, followed by a 30‐s cooling period. After the procedure, the acardiac fetus was removed from the box model. Warm saline was infused through the umbilical vein of the acardiac fetus (1 mL/sec). Completion of ablation was determined when no fluid was observed coming out of the other end of the umbilical vein (Videos S1 and S2). The success of the operation (binary) was determined by the completion of ablation. In contrast, insufficient ablation or inadvertent puncture of another sac was considered failure. The acardiac fetus was sectioned at the site of needle insertion (Figure 1D). The size of the ablation was measured using a vernier caliper, which can precisely measure up to 0.01 mm.

2.5. TRAPS Model Evaluation

After completing the practice sessions, each participant was instructed to rate the model based on its physical attributes and material realism, value and relevance of the training model, and procedural accessibility. Each scale item was scored using a 5‐point Likert scale (Table 1). Cronbach's alpha was used to assess the reliability of the survey by measuring the consistency of responses across all survey items for all participants [19]. To assess interrater agreement, the intraclass correlation coefficient (ICC) was calculated.

TABLE 1.

Survey items for evaluating the twin reversed arterial perfusion sequence model.

Physical attributes and realism of materials
No.	Assessment	Rating scores
		1 Not at all realistic	2 Lacks many key features to be realistic	3 Don't know	4 Adequate realism, but could be improved	5 Highly realistic, no changes needed
1	Acardiac fetus visualized on ultrasound images
2	Intrafetal vessels visualized on ultrasound images
3	Texture of the acardiac fetus
4	Tissue coagulation demonstrated by ultrasound
5	Does this simulator represent the expected experience during RFA of the acardiac fetus?

Value and relevance of this training model
No.	Assessment	Rating scores
		1 No value/relevance	2 Little value/relevance	3 Don't know	4 Some value/relevance	5 A great deal of value/relevance
6	Value of this simulator as a skill training tool
7	Value of this simulator as a skill appraisal tool
8	Relevance of this simulator to clinical practice

Accessibility of the procedure
No.	Assessment	Rating scores
		1 Too difficult to perform	2 Very difficult to perform	3 Difficult to perform	4 Somewhat easy to perform	5 Very easy to perform
9	Proper placement of the needle
10	Deployment of the tines

[Correction added on 22 April 2025, after first online publication: The table 1 is updated in this version.]

2.6. Data Collection

The following variables were collected for analysis: baseline characteristics of the participants; years of experience in invasive prenatal procedures, defined as the number of years the participant has performed any invasive prenatal procedures; operation success or failure; needle insertion time (seconds), defined as the time from RFA needle insertion into the box model until the full deployment of the tines; total procedural time (seconds); and the horizontal and vertical diameters of the ablated tissue in millimeters.

2.7. Statistical Analysis

The learning curve cumulative summation (LC‐CUSUM) approach was used to determine the point at which a skill achieved a predefined level of proficiency. The score started at zero and changed by −0.087 for each successful procedure and by +0.560 for each failure, forming the LC‐CUSUM graph. Proficiency was attained when the score exceeded the lower decision limit (h_lc). An h_lc of −1.25 was selected for the LC‐CUSUM using simulations involving 10,000 samples to maintain type I and type II error rates of 1.35% and 3.13%, respectively, within 100 procedures. Upon achieving a sufficient performance level in the LC‐CUSUM test, the CUSUM model was used to ensure that the performance level was maintained. Each failure increased the CUSUM score by +0.916, whereas each success decreased the score by −0.182. The CUSUM score exhibited a barrier that kept the score at zero as successful procedures accumulated. If the score exceeded the upper decision limit (h_c) of 2, the performance was deemed inadequate, with a type I error rate of 1.3% and a type II error rate of 5%. Supporting Information S1: Appendix S1 provides an explanation of the LC‐CUSUM and CUSUM methodologies.

The learning phase was defined as the period preceding proficiency, whereas the subsequent period was referred to as the control phase. Success rates during the learning and control phases were compared using the chi‐squared test. Regression analysis was performed to estimate the association between the number of procedures required to achieve proficiency and years of experience in invasive prenatal procedures.

The secondary objective was to define the learning curve of RFA in the TRAPS model regarding needle insertion time. Needle insertion time was logarithmically transformed to normalize the skewed distribution. For needle insertion time, the inverse regression model (y = a + b/x, where x represents the consecutive number of procedures and y represents the log₁₀ needle insertion time) was used to identify the learning plateau. The learning plateau was determined as the intercept of the regression equation (10^a), and the number of procedures required to surpass 90% of the learning plateau was then calculated. The procedures were divided into seven consecutive groups of 10. The last group was selected as the reference group, assuming that it had the lowest mean log10 needle insertion time. Moreover, the mean log10 needle insertion time for each group was compared with that of the reference group using the Tukey method for multiple comparisons.

The participants were classified into two groups based on years of experience with invasive prenatal procedures (≥ 5 years, n = 6 vs. < 5 years, n = 6). A mixed analysis of variance was performed to assess the association between years of experience with invasive prenatal procedures and needle insertion time for each consecutive procedure group. A two‐sided p‐value of 0.05 was considered statistically significant. All analyses were performed using the Statistical Package for the Social Sciences (version 29; IBM Corp., Armonk, NY, USA).

3. Results

The study population comprised 10 female physicians and two male physicians, with a median age of 33 years (interquartile range (IQR): 31.25–36.25 years) and median years of experience in invasive prenatal testing of 4.5 years (IQR: 2.25–5.75 years). Following RFA, the average horizontal and vertical diameters of the ablated tissue were 20.91 ± 2.77 and 22.84 ± 2.88 mm, respectively. The total number of procedures performed by each participant ranged from 57 to 72.

3.1. TRAPS Model Evaluation

The observed average values for physical attributes and material realism, value and relevance of this training model, and procedural accessibility are displayed in Figure 2. The average ratings for material realism, value and relevance of this model as a skill training and appraisal tool, and procedural accessibility were 4.03, 4.92, and 3.88, respectively. The scores did not significantly differ from those of experienced surgeons (p = 0.635, 0.484, and 0.754, respectively). Ratings for the value as a skill training tool, value as a skill appraisal tool, and relevance of this simulator to clinical practice achieved the highest scores (5, 4.83, and 4.83, respectively). All item rating scores demonstrated a high degree of correlation (Cronbach's alpha of 0.872) and good interrater agreement (ICC = 0.851; 95% confidence interval (CI): 0.666–0.956).

FIGURE 2.

Observed average rating scores across 10 survey items.

3.2. Learning Curve and Competency of the Participants for Procedural Success

Twelve individuals performed 767 procedures, yielding an overall success rate of 92.8% (712 of 767 procedures; 95% CI: 91%–95%). Figure 3 presents the LC‐CUSUM and CUSUM plots for each participant. On average, 28.92 ± 4.68 procedures were required to achieve proficiency. A comparison of the two phases before and after proficiency revealed that the total success rate was significantly higher during the control phase than during the learning phase (96.7% vs. 88.2%; p < 0.001). The operative point of proficiency was negatively correlated with the participants' years of experience in invasive prenatal testing, best‐fitted to the inverse regression curve: the operative point of proficiency = 23.499 + (18.977/years of experience), R ²  = 0.544 (Figure 4).

FIGURE 3.

Learning curve cumulative summation (LC‐CUSUM) and cumulative summation (CUSUM) graphs for radiofrequency ablation training of all participants. The X‐axis represents the consecutive number of procedures, and the Y‐axis represents LC‐CUSUM and CUSUM scores. Decision limits for LC‐CUSUM and CUSUM analyses are illustrated by the lower dashed line at −1.25 and the upper dashed line at 2.0, respectively.

FIGURE 4.

Scatter plot diagram and regression analysis correlating the operative point of proficiency in technical success with the participant's years of experience in invasive prenatal testing (operative point of proficiency = 23.499 + (18.977/years of experience), R ² = 0.544).

3.3. Learning Curve and Competency of the Participants Regarding Needle Insertion Time

The needle insertion time learning curve was represented and fitted to the model as follows: log₁₀ needle insertion time (s) = 1.440 + (0.468/consecutive number of procedures), p < 0.001 (Figure 5). Thus, a needle insertion time of 27.54 s is the point at which the learning plateau is reached. After 21 procedures, 90% of the learning plateau for needle insertion time was achieved. In multiple comparisons of consecutive grouping (Table 2), the average log₁₀ needle insertion time for groups 1 and 2 (procedures 1–20) was significantly higher than that for Group 7 (procedures 61–70) (% difference between groups 1 and 7 = 76.69, p < 0.001, and % difference between groups 2 and 7 = 41.54, p < 0.001). No significant differences in any other pairwise comparisons of the consecutive group were observed. The needle insertion time at each consecutive number of procedure group was not significantly associated with years of experience with invasive prenatal procedures (p = 0.182).

FIGURE 5.

Learning curve of needle insertion time along consecutive number of procedures.

TABLE 2.

Percentage increase in needle insertion time using consecutive grouping.

The number of procedures in consecutive groups	Increased needle insertion time (%)	95% adjusted confidence interval	Adjusted p‐value
1–10	76.69	37.31–127.35	< 0.001
11–20	41.54	10.00–82.14	< 0.001
21–30	9.2	−15.14 to 40.51	0.946
31–40	5.84	−17.74 to 36.21	0.994
41–50	12.96	−12.22 to 45.34	0.785
51–60	5.87	−17.83 to 36.40	0.994
> 60	Reference	Reference	Reference

4. Discussion

This study introduced a validity‐supported TRAPS model for RFA training. Clinically, a clinician needs to perform approximately 29 of these RFA procedures to reach a defined level of competency. Prior experience with minimally invasive prenatal procedures can reduce this number. Needle insertion time, a key component of the procedure, plateaus at around 27.5 s after about 21 attempts. This information is valuable for designing training programs, suggesting that around 29 procedures should be sufficient for most trainees to achieve competency. The data also highlights the rapid improvement in needle insertion time with initial practice, suggesting that focused training early on is crucial.

Surgical trainees and programs face increasing pressure to efficiently demonstrate practical skills competency, alongside a focus on patient safety and outcomes. Fetal intervention is particularly amenable to using simulation technology. Previous studies support the effectiveness of surgical simulation in transferring skills to the operating room [20, 21, 22]. A number of recent research projects have examined models and simulations intended for fetal surgical interventions [23, 24, 25, 26]. While some advocate for animal labs for advanced training, their high costs and ethical concerns make them impractical [27, 28]. Our model's design makes it more affordable. The novel simulator was assessed for construct validity to support and ensure inferences related to performance evaluation. According to the Standards for Educational and Psychological Testing (Standards), jointly developed by the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education, validity evidence concerning test content, internal structure, response processes, and relationship to other variables was addressed. The highest evaluation ratings were attributed to the value of the stimulator as a skill training and appraisal tool and its applicability to clinical practice. The rating consistency between experienced and novice surgeons strengthens evidence related to response processes. Furthermore, good interitem and interobserver reliability provides evidence relevant to the internal structure. This simulator effectively assesses and distinguishes skill levels, as evidenced by the inverse relationship between the operating point of proficiency in procedural success and the participants' years of experience in invasive prenatal testing. Therefore, this simulation model can be used to train for this type of procedure.

A learning curve analysis is a common skill qualification approach. In this study, the LC‐CUSUM test was used to determine the operative point of proficiency in relation to procedural success in this simulator [29]. On average, the participants reached skill competence after approximately 29 RFA procedures. To ensure the maintenance of acceptable performance, the CUSUM test was performed after achievement of the competency level. The type I and type II error rates were 1.3% and 5%, respectively, indicating test efficiency. It was observed that 20 procedures were required to identify poor performance, with a chance of false alarms occurring every 77 procedures. During the control phase, each participant maintained their skill levels for the simulation, achieving a greater success rate than during the learning phase. This finding highlights the significantly improved performance following the operative point of proficiency. Moreover, the operative point of proficiency is influenced by experience in invasive prenatal procedures, emphasizing the essential nature of this fundamental skill.

Procedural time is another critical factor in fetal intervention. Prolonged procedural time can increase the complication rate. Fetal movement, which is typically active, can also impact the procedure's outcome. Minor increases in procedural time may predispose to technical failure. Regardless of the operator's skill level, mastery of at least 20 procedures is required to become proficient in needle insertion time using this simulator. Therefore, additional RFA training using this simulator is necessary to achieve technical success beyond simply aiming to reduce the operative time. These findings also indicate that using this simulator to achieve the target competency level in RFA practice can lead to shorter operative times and enhanced technical success.

4.1. Strengths and Limitations

This study is a preliminary investigation aimed at characterizing the RFA learning curve using the proposed TRAPS simulator. Importantly, this innovative simulator was validated using various sources of validity evidence to ensure conclusions regarding the evaluation of surgical performance using this simulator [16, 17, 18]. The affordability, practicality, and ease of creation of this simulator further highlight its potential benefits. Because the primary objective of this study was to define the learning curve of each participant on the simulator, a sufficient number of procedures were performed by each participant to ensure the accuracy of the findings. Note that experts in RFA were not included in this study because the simulator was designed for individuals lacking expertise in RFA to practice, and the results provide the learning curve for these non‐experts. Moreover, the competency assessed in this study was related to proficiency in the simulation model rather than clinical competency.

In contrast, studies have demonstrated that surgical simulation facilitates the transfer of skills from the laboratory to the operating room [20, 21, 22]. Because RFA in TRAPS is an uncommon procedure, trainees should achieve a predetermined level of performance in simulation‐based training before transitioning to patient‐based settings. Although trainees may benefit from repetitive task practice before clinical operations, such practice may not always be feasible in clinical settings. The umbilical vessels in this simulation, potentially larger than second‐trimester vessels, could make targeting easier but ablation more challenging. This study is limited by the inability to control infusion pressure during the leak check and the absence of data regarding physiological pressure within the vessels of the acardiac fetus. Further implementation studies in real‐world scenarios will provide additional data regarding pregnancy outcomes and continuous skill monitoring, aiming at reducing complications and improving fetal health.

5. Conclusion

The proposed TRAPS model is realistic and applicable to RFA training. Achieving proficiency in this simulator enhances technical efficiency. The operator's experience with invasive procedures contributes to technical success.

Ethics Statement

The study protocol was approved by the Institutional Review Board of Siriraj Hospital (COA No. Si696/2022).

Consent

Written informed consent was obtained from the participants of this study. The participants agreed to participate in this study freely and voluntarily and can withdraw from the study at any time without penalty or loss of benefits.

Conflicts of Interest

The authors declare no conflicts of interest.

Supporting information

Supporting Information S1

Video S1

Video S2

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.