Skip to main content
Scientific Reports logoLink to Scientific Reports
. 2021 Mar 17;11:6109. doi: 10.1038/s41598-021-85607-6

Validation of a high-fidelity training model for fetoscopic spina bifida surgery

Luc Joyeux 1,2,3,4,✉,#, Allan Javaux 5,#, Mary P Eastwood 1,2,6, Felix R De Bie 1,2,7, Gert Van den Bergh 2, Rebecca S Degliuomini 1, Simen Vergote 1,2,3, Talita Micheletti 1,8, Geertje Callewaert 1,2,3, Sebastien Ourselin 9, Paolo De Coppi 1,2,3,4, Frank Van Calenbergh 10, Emmanuel Vander Poorten 5, Jan Deprest 1,2,3,11,✉
PMCID: PMC7969952  PMID: 33731777

Abstract

Open fetal surgery for spina bifida (SB) is safe and effective yet invasive. The growing interest in fetoscopic SB repair (fSB-repair) prompts the need for appropriate training. We aimed to develop and validate a high-fidelity training model for fSB-repair. fSB-repair was simulated in the abdominal cavity and on the stomach of adult rabbits. Laparoscopic fetal surgeons served either as novices (n = 2) or experts (n = 3) based on their experience. Technical performance was evaluated using competency Cumulative Sum (CUSUM) analysis and the group splitting method. Main outcome measure for CUSUM competency was a composite binary outcome for surgical success, i.e. watertight repair, operation time ≤ 180 min and Objective-Structured-Assessment-of-Technical-Skills (OSATS) score ≥ 18/25. Construct validity was first confirmed since competency levels of novices and experts during their six first cases using both methods were significantly different. Criterion validity was also established as 33 consecutive procedures were needed for novices to reach competency using learning curve CUSUM, which is a number comparable to that of clinical fSB-repair. Finally, we surveyed expert fetal surgeons worldwide to assess face and content validity. Respondents (26/49; 53%) confirmed it with ≥ 71% of scores for overall realism ≥ 4/7 and usefulness ≥ 3/5. We propose to use our high-fidelity model to determine and shorten the learning curve of laparoscopic fetal surgeons and retain operative skills.

Subject terms: Experimental models of disease, Paediatric research, Preclinical research, Translational research, Spinal cord diseases

Introduction

Open fetal repair for spina bifida aperta (SBA) effectively reduces postnatal morbidity1,2. Fetoscopic SBA repair (fSBA-repair) may minimize maternal risks and preterm delivery with similar neonatal neuroprotective effects3–6. However, this approach is challenging with a long learning curve (LC) of at least 56 cases7. A valid training model would reduce the LC and avoid or limit training on clinical subjects8–11. Such simulator would also accelerate the safe and ethically acceptable transition from the open approach to fetoscopy or its implementation in a center without experience based on the available case volumes. It should be paired with off- and on-site clinical training and guidance from established fetal centers10. Subsequently, it would enable skill retention and dissemination of the procedure12.

The IDEAL recommendations for surgical innovation state that preclinical studies, including simulators and valid animal models, are essential prior to first-in-human trials12,13. Simulators may be low-fidelity (e.g. computer simulators and inanimate bench-top trainers) or high-fidelity (e.g. in vivo animal models and human cadavers) training models. For fSBA-repair, two low-fidelity inanimate models have been proposed yet without validation14,15. However, they may be useful in initial training and help reducing the numbers of animals needed in further training. Two in vivo SBA models have previously been proposed for feasibility studies, i.e. fetal rhesus monkey and fetal lamb. While only the latter was used for fetoscopic surgery16, neither were validated17 or specifically designed for training. Among smaller animal models, mice and rats cannot be used for surgical training purposes, due to their size.

High-fidelity rather than low-fidelity surgical models enhance training realism and thus minimize potential harms from the LC in animals as well as in humans following translation to clinical practice12,13. Such a model for fSBA-repair requires (1) complex surgical steps like port insertion, dissection and suturing (2) simulated in a realistic environment, i.e. with proper depth perception, live motion and pulsatile blood flow in arteries and veins (3) using a living and breathing animal. Moreover, the use of large animal models should be restricted in accordance with NCR3-guidelines18,19. Therefore we report on the development and validation of a high-fidelity training model for fSBA-repair in a living and breathing rabbit and its use to determine the number of cases needed for a laparoscopic fetal surgeon to achieve competency.

Methods

Ethical statement

This experiment was approved by the Animal Ethics Committee of the Group Biomedical Sciences of the KU Leuven (P093-2016). It followed the NC3Rs and the ARRIVE guidelines for animal research18,19.

Study design

The validation of this animal model followed the consensus guidelines for validation of surgical simulators20,21. It was assessed in two phases: construct-criterion and face-content validity22. For the former, the study was designed to train three laparoscopic fetal surgeons from a single fetal center, i.e. surgeons experienced in open fetal SBA repair as well as multi-port laparoscopy, yet who had never performed fSBA-repair (LJ, PDC, JD).

We categorized our surgeons into novices and experts in our model for simulated fSBA-repair based on surgical experience. Since we aimed to validate a training model and not the port-access approach, we hypothesized that one of them being a single-port and multiple-port laparoscopic neonatal surgeon was an expert (PDC). The multi-port approach is being clinical used for fetoscopic SBA repair5,23. By contrast, the single-port approach currently applied in fetoscopy for twin-to-twin transfusion syndrome (TTTS) or congenital diaphragmatic hernia is the ultimate minimally-invasive technique to achieve24,25. The other surgeons (LJ, JD) having overall less experience in multiple-port laparoscopy and no experience in single-port laparoscopy were considered as novices. They were thus trained in the model until competency in single-port fSBA-repair was reached (Supplementary Methods 1 and 2.1, Supplementary Fig. S1, Video 1). When these two surgeons had completed that training, they were then referred to as experts. In the end, these three experts performed multi-port fSBA-repair to confirm their competency (Supplementary Methods 2.2, Supplementary Fig. 1, Video 1). Overall novices and experts performed the same procedure consisting of 10 surgical steps yet used a different port-access approach (single- or multi-port).

Description of the model

This live model was developed to mimic the operative steps and conditions present for a clinical multilayered fSBA-repair.

Clinical procedure to mimic

The gestational age at fSBA-repair in humans typically is around 24 weeks of gestation1,5,26. At that time the fetal weight is 662 ± 77 g27 and the abdominal circumference 187 ± 10 mm27. The region of interest is lumbar in 95% of cases1. The current literature on open and fSBA-repair describes several steps in the procedure, which we summarized into 10 consecutive steps to be simulated (Fig. 1)1,5,26,28,29.

Figure 1.

Figure 1

Comparison of clinical and simulated fetoscopic spina bifida repair in our rabbit training model. (A) Schematic drawing and external view of our model setup showing similarities in working space and presentation of the surgical target. (B) Comparison of the 10 essential steps performed during a clinical (left column) and simulated (right column) fetoscopic repair. Reproduced with permission of and

copyright by the Texas Children’s Hospital, Houston, TX, USA and UZ Leuven. Artist drawing by Allan Javaux; copyright by UZ Leuven, Leuven, Belgium.

Animal model

Rabbits have previously been used for training in pediatric30,31 and fetoscopic32 surgery. We used New-Zealand male adult rabbits (weight, 3-4 kg). They were given water and food at libitum under the standard light–dark cycle until the procedure. They were put under general anesthesia without endotracheal intubation (Supplementary Methods 3). In rabbits, the adequately insufflated abdominal cavity mimics the working space or amniotic cavity, which approximately measures 15 × 10x5cm with a pneumoperitoneum of around 3L with CO2 at 5 mmHg (Fig. 1A)32. Monogastric herbivores, rabbits have a large single-chamber stomach with a circumference of 165 ± 13 mm mimicking the abdominal circumference of a 22–24 weeks human fetus (Fig. 1A)33. Overall 10 clinical steps for fSBA-repair are recapitulated by a laparoscopic gastric Nissen fundoplication34 and the suturing of a patch to the gastric wall (Fig. 1B, Video 1). These procedures require the ability to gently manipulate fragile tissue, perform extensive dissection, hemostasis and suture (Supplementary Methods 3).

Technical performance

Clinical outcomes were total operation time (steps 1 to 9)8,9, fetal repair time (steps 4 to 9), CO2 insufflation volume, Objective Structured Assessment of Technical Skill (OSATS) score and watertightness of the patch repair (step 10, Video 1). The latter was tested post-mortem by fluorescein injection under the patch after completion of the repair (Supplementary Methods 4, Video 1). Operative performance and difficulty were assessed applying an adapted OSATS rating scale on videos of the procedures (Supplementary Methods 4)35,36.

We also used a composite binary outcome for surgical success based on clinically relevant outcomes to measure the LC and competency level of fSBA-repair7. Similar to clinical fetal SBA surgery7, a successful surgical repair of simulated SBA was defined by a watertight repair, an operation time ≤ 180 min in accordance with the FDA Drug Safety Communication about potential risks of general anesthesia in pregnant women37 and an OSATS score ≥ 18/25 (> 70%)35,36.

Validation study

Construct validity

To assess construct validity and therefore discriminate performance levels of our simulator, we determined and compared competency level of novices and experts during their first six cases applying two methods. First, the Competency Cumulative Sum (C-CUSUM) test38 used the aforementioned binary outcome for surgical success and was set with a control limit of hC = 3 (Supplementary Methods 5.1). Subsequently, we applied the group-splitting method9 by comparing performance using the five aforementioned clinical outcomes.

Criterion validity

Criterion validity compares performance of our innovative model to the ground truth which is the clinical procedure in our case since, to the best of our knowledge, no animal training model has been validated yet7. Herein we compared the learning curve (LC) of novices in our model to the LC of novices performing percutaneous fSBA-repair7. It was determined by the predictive validity method39 that uses the Learning Curve CUSUM (LC-CUSUM) test with similar parameters as the C-CUSUM yet with a control limit set at hLC = 0.85 (Supplementary Methods 5.2)7.

Face and content validity

To assess realism (face validity) and usefulness (content validity) of our model, an anonymous online survey (Supplementary Online Survey SurveyMonkey #6VNJMS9) was designed and sent to fetal surgeons (n = 49) worldwide who are currently involved in clinical fetal surgery programs for SBA using an open fetal and/or fetoscopic approach (Supplementary Methods 5.3, Supplementary Table S1)7. 22% (11/49) of the surgeons were performing fSBA-repair in their fetal center at that time. All were invited to try our model out in our research center and under our supervision to get a realistic experience. For obvious geographical reasons, some experts only answered our anonymous online questionnaire (non-users) while others also tried it (users). According to current practice, all survey responses from expert users or non-users were included in the analysis40. We also performed a subanalysis of data from users currently performing fetoscopic repair in humans.

Expert fetal surgeons were categorized according to their specialty (obstetricians and gynecologists, pediatric neurosurgeons and pediatric surgeons) and demographic data were captured by seven questions. We additionally asked six questions on face validity and five on content validity, using a 7-point and 5-point Likert scale respectively. We set validity thresholds at 4/7 (undecided) and 3/5 (neutral) for each scale (Supplementary Methods 5.3)39.

Statistical analysis

We used GraphPad Prism version 7.00 for MacOs X (GraphPad, La Jolla, CA, USA) to analyze the data. Binomial and categorical variables were expressed as percentages with their frequency distribution. Continuous variables were tested for normal distribution using the D'Agostino-Pearson (omnibus K2) or Shapiro–Wilk normality tests. Continuous variables normally distributed were presented as mean and standard deviation (SD) and the others were expressed as median and range or interquartile range (IQR).

For face and content validity, categorical and continuous variables based on the response by the three clinical subspecialties involved in the survey were compared with one-way analysis of variance (ANOVA) or Kruskal–Wallis test as appropriate. For construct validity, the Fisher exact test was used to compare binomial variables. Continuous variables were compared with unpaired two-tailed t test or Mann Whitney test as appropriate. A p value < 0.05 was considered significant. For the construct and criterion validity, we performed C- and LC-CUSUM analysis using an algorithm that we developed in MATLAB software (Mathworks, Natick, MA, USA) based on and verified with the model of Biau et al.7.

Results

Surgical procedures

The data below were collected from 52 completed single-port simulated fSBA-repairs by two novices (n = 34 and n = 18), and from 18 multi-port simulated fSBA-repairs by three experts (n = 6, n = 6 and n = 6).

Construct validity

Figure 2A displays the evolution of the C-CUSUM score of each novice and expert during the first six cases. As C-CUSUM scores of the three experts remained below the competency control limit hC = 3, they were considered competent. In contrast, scores of both novices reached a higher score of 3.07 after the 6th case and were therefore not competent (graphs of Fig. 2A). The two groups were also significantly different for all surgical outcomes measured when applying the group-splitting method (table of Fig. 2A).

Figure 2.

Figure 2

Construct and criterion validity of our training model. (A) Construct validity: competency assessment of novices and experts using competency cumulative sum (C-CUSUM) test (graphs), and comparison of clinical outcomes of the first six cases of novices and experts using the group splitting method (table). (B) Criterion validity: learning curve assessments of novices and experts using learning curve CUSUM (LC-CUSUM) test (graph).

Criterion validity

LC-CUSUM analysis of all the cases performed demonstrated that novice 1 reached competency when his score was 0.89, exceeding the control limit of hLC = 0.85, after 33 out of 34 cases (Fig. 2B, left graph). To reduce the number of animals used, novice 2 performed only 18 procedures and did not reach that threshold. When considering the best-case scenario, it was predicted that this novice reached competency at 31 cases (Fig. 2B, right graph). These numbers involving laparoscopic fetal surgeons are lower than what has been reported as the minimum number required for competency in clinical fSBA-repair performed by non-laparoscopic fetal surgeons (n ≥ 56)7.

Demographics of survey respondents

The response rate from the fetal surgeons for SBA to the online survey was 53% (26/49). 38% (10/26) of the respondents tried the model out to get a realistic experience and 27% (7/26) were performing fSBA-repair in their fetal center (Supplementary Table S2). The demographics of the different subspecialists involved were comparable, except that pediatric neurosurgeons had less experience in laparoscopic surgery (Table 1). None of the respondents was aware of computer simulators or high-fidelity models for fSBA-repair.

Table 1.

Demographics of the three subspecialties involved in fetal surgery.

Specialty Obstetrician and gynecologist Pediatric neurosurgeon Pediatric surgeon p-value
Number 11 6 9
Number of years of experience as a
Specialist 16 ± 13 22 ± 9 17 ± 8 0.566
Laparoscopic surgeon 7 (10) 0 (1) 13 (10) 0.003
Handedness
Right-Handed 11 5 7
Left-Handed 0 1 2
Number of fetal SB open repairs in humans
As first surgeon 1–30 (0;1–30) 31–60 (1–30;91–120) 0 (0;31–60) 0.230
As second surgeon 1–30 (0;31–60) 1–30 (0;1–30) 1–30 (1–30;61–90) 0.363
Number of fetal SB fetoscopic repairs in humans
As first surgeon 0 (0;0) 0 (0;1–30) 0 (0;1–30) 0.977
As second surgeon 0 (0;0) 0 (0;0) 0 (0;1–30) 0.779
Number of training sessions on an open fetal surgery simulator
Virtual reality 0 ± 0 0 ± 0 0 ± 0 NA
Box trainer 0 (0;1–30) 0 (0;0) 0 (0;0) 0.108
Animal model 0 (0;1–30) 0 (0;0) 0 (0;31–60) 0.643
Human cadaver 0 (0;0) 0 (0;0) 0 (0;0) 0.722
Number of training sessions on a fetoscopic surgery simulator
Virtual reality 0 ± 0 0 ± 0 0 ± 0 NA
Box trainer 0 (0;1–30) 1–30 (0;1–30) 0 (0;1–30) 0.449
Animal model 0 (0;1–30) 0 (0;1–30) 0 (0;1–30) 0.648
Human cadaver 0 ± 0 0 ± 0 0 ± 0 NA

Number of surgeries per specialty are displayed in median and range based on 5 block of 30 cases as per the online survey (0, 1–30, 31–60, 61–90, 91–120, ≥ 121). For example 31–60 (1–30;91–120) signifies that a mean of 1–30 cases was performed and the number of cases varied between 1–30 and 91–120.

Face validity

The three subspecialties of respondents considered our live model realistic since all survey questions reached scores ≥ 4/7 in ≥ 60% of cases and recommended it for its realism (≥ 84% of scores ≥ 4/7; Table 2). Good to exceptional scores (63 to 100% of scores ≥ 4/7) were reached for questions that required a living and breathing animal. These encompassed the surgical target, surgical steps such as dissection, resection and suturing, and depth perception (Table 2). Despite some discrepancies, there were no significant differences in ratings between the subspecialists. In addition, the seven fetoscopic experts who used our model confirmed its realism as 95% of survey questions scored ≥ 4/7. They also recommended it for its realism (100% of scores ≥ 4/7; Supplementary Table S3).

Table 2.

Face and content validity of the model.

Subspecialty Obstetrician and gynecologist Pediatric neurosurgeon Pediatric surgeon P value
Number 11 6 9
Face validity Percentages of scores ≥ 4/7 on the Likert scale
Overall realism
Recommendation for realism 88% 94% 84% 0.874
Surgical scene
Animal positioning 80% 83% 75% 0.916
Position of the video monitor 100% 100% 100% 0.216
Position of 1st surgeon 100% 100% 88% 0.557
Position of 2nd surgeon 100% 100% 75% 0.933
Surgical cavity
Humidified environment (Fluid-gas interface) 90% 100% 88% 0.804
Workspace 90% 100% 88% 0.034*
Vision 100% 100% 88% 0.419
Surgical target
Mimic of the fetal lumbar region 70% 84% 73% 0.961
Instrumentation set
Endoscope 90% 100% 100% 0.440
Grasping forceps 90% 100% 100% 0.253
Scissors 100% 100% 100% 0.648
Coagulating and dissecting hook 100% 100% 100% 0.646
Dissector 100% 100% 100% 0.349
Needle holders 100% 100% 100% 0.167
Surgical steps
Exposition 80% 83% 75% 0.750
Port insertion 60% 83% 88% 0.201
Insufflation 100% 100% 100% 0.753
Fetal positioning 70% 83% 63% 0.834
Dissection 70% 83% 63% 0.964
Resection 70% 83% 63% 0.889
Mobilization 80% 100% 63% 0.984
Patch 90% 83% 63% 0.521
Skin 90% 83% 75% 0.159
Quality assessment 100% 83% 88% 0.507
Depth perception
Mimic of clinical conditions (live motions) 80% 100% 100% 0.110
Content validity Percentages of scores ≥ 3/5 on the Likert scale
Overall usefulness
Recommendation for training 80% 83% 71% 0.332
Overall difficulty
As difficult as in humans 80% 100% 71% 0.815
Similar stress 50% 50% 86% 0.104
Instrument handling
Improvement of instrument handling skills 100% 83% 100% 0.690
Suturing
Improvement of suturing skills 100% 100% 100% 0.832
Self-confidence
Overall confidence 100% 100% 100% 0.180
Surgical Tasks
Exposition 80% 83% 71% 0.501
Port insertion 90% 83% 71% 0.568
Insufflation 100% 83% 100% 0.718
Fetal positioning 60% 83% 71% 0.712
Dissection 80% 83% 71% 0.595
Resection 80% 83% 71% 0.642
Mobilization 80% 83% 86% 0.829
Patch 90% 83% 71% 0.939
Skin 90% 83% 100% 0.385
Quality assessment 100% 83% 100% 0.406

There were no significant differences between each of the 3 specialties of fetal surgeons.

Content validity

There were no significant differences in ratings between the subspecialists. Respondents considered the model useful and would recommend it for training (≥ 71% of scores ≥ 3/5) and improving complex fetoscopic skills, such as instrument handling and suturing (≥ 83% and 100% of score ≥ 3/5 respectively; Table 2). Average (≥ 50%) scores varied among the three subspecialties and were obtained when we asked whether the model exposes to stress similar to that in clinical conditions and is useful to train for fetal positioning (≥ 50% and ≥ 60% of scores ≥ 3/5 respectfully). In contrast high scores (≥ 83% of scores ≥ 3/5) concerned the usefulness of the model for instrument handling, suturing, self-confidence, insufflation, tissue mobilization, skin closure and quality assessment of the patch repair. Finally, the seven fetoscopic experts who used our model confirmed its usefulness as 98% of survey questions scored ≥ 3/5. They also recommended it for its training (100% of scores ≥ 3/5; Supplementary Table S3).

Discussion

Main findings

We developed and validated a high-fidelity training model for fetoscopic SBA repair in live rabbits. We first demonstrated that competency of laparoscopic fetal surgeons was reached at 33 consecutives cases. That number is lower than what has been reported for non-laparoscopic fetal surgeons performing clinical multi-port fSBA-repair. Surveyed fetal experts also proved face and content validity.

Clinical interpretation

A recent systematic review with available individual patient data on multi-port fSBA-repair demonstrated that the LC to reach competency was at least 56 cases7. This is of the same order of magnitude of other complex multi-port laparoscopic surgeries such as colectomy41,42 or sacrocolpopexy43 performed by surgeons without previous experience in these techniques. In our high-fidelity model the LC was 33 cases for novices, more precisely surgeons experienced in multi-port laparoscopic surgery but non-experienced with single-port surgery. That number is in keeping with other complex single-port procedures such as colectomy performed by surgeons experienced in multi-port laparoscopic surgery44. It is also similar to other advanced, yet less complex multi-port endoscopic procedures than those above, such as cholecystectomy45, pyloromyotomy46 or the most common fetoscopic operation (laser coagulation for TTTS)24. They are described as less complex as they do not require suturing and extensive dissection (or in case of TTTS none at all) skills. We therefore surmise that the number of 33 reached in our simulator is the LC of surgeons experienced with complex multi-port laparoscopy before translation to clinical practice. It may represent an underestimation for surgeons non-experienced with complex multi-port laparoscopy. Clinically, the challenges of fetal surgery are greater than what can be simulated, such as the complex pathologic anatomy of the lesion, the frailty of human fetal tissue, the interference of fetal monitoring, the presence and vicinity of the placenta, the large number of people and specialties around the operation table, or simply the stress of operating on two patients. These points were suggested by 6/26 (23%) of the survey participants (3/6 pediatric neurosurgeons, 2/9 pediatric surgeons and 1/11 obstetrician and gynecologist).

Strength and limitations

We acknowledge a number of limitations to our study. First, some surveyed fetal surgeons raised concerns regarding the realism of specific aspects of our model. Indeed, the simulation procedure does not mimic the precise dissection and gentle manipulation of the dura, musculo-fascial flaps and fetal skin. However, manipulation, dissection and suturing of the stomach are quite comparable to these clinical steps, as the rabbit stomach can be easily damaged and perforated. This way, those steps unmask potential clinically relevant complications. Secondly, three laparoscopic fetal surgeons were involved in our fetal surgery training to reduce the number of animal required hence following the ethical standards of the NC3Rs-guidelines18,19. Our competency analysis allowed us to confirm our hypothesis about the competency level of novices and experts. Since experts 1 and 2 were previously novices 1 and 2, these surgeons became experts in three-port surgery in our rabbit training model—yet not for clinical fSBA-repair—following their training as novices in single-port surgery. Finally, we only tested our model for either a single- and three-port23. A two-port approach26 currently practiced by some centers can easily be adapted.

Our study also has considerable strengths. Firstly, in the development of a simulator, we followed the consensus guidelines for animal research and validation of animal models18,39 and surgical simulators13,14,22. Secondly, we applied robust methods for assessing subjective and objective validity. Thirdly, we measured the LC and competency level of both experts and novices with standardized methodology38. Finally, our observations seem clinically relevant as we come to numbers that are comparable to what has been demonstrated for complex clinical laparoscopic procedures performed by trained laparoscopic surgeons.

Conclusion

We developed and validated a high-fidelity model for fetoscopic layered SBA repair. It was used to determine the learning curve of laparoscopic fetal surgeons, which was in the range of other complex endoscopic procedures. We propose the use of this model to determine and shorten the learning curve of laparoscopic fetal surgeons, and aid retention of operative skills.

Supplementary Information

Supplementary Video 1. (93.3MB, mp4)

Acknowledgments

We are very grateful to Rosita Kinnart from the Center for Surgical Technologies for her precious help in preparing the surgeries. We also thank the following colleagues for their technical support: Dr. Lennart Van der Veeken, Dr. David Basurto, Dr. Johanna Parra, Dr. Enrico Corno and Dr. Yada Kunpalin.

Author contributions

All authors of this multicenter study contributed substantially to the submitted manuscript. L.J. and A.J. designed the entire study with the help of M.P.E., P.D.C., E.V.P. and J.D. L.J., J.D. and P.D.C. performed the surgeries and animal preparations with help of M.P.E., T.M., R.S.D., G.G. The MATLAB algorithm was developed by A.J. A.J. and L.J. performed the cumulative sum analysis using the custom-made algorithm made by A.J. and validated. F.D.B., R.S.D. and G.V.D.B. independently analyzed and rated the videos of the surgeries and scored them using the OSATS rating scale. L.J. and S.V. created the online questionnaire for the face and content validities. L.J. and A.J. performed the statistical analysis of the data and wrote the manuscript. L.J., A.J. and J.D. drafted and edited the manuscript. All co-authors made substantial contributions to the study design, acquisition and interpretation of data, critically revised and approved the final submitted manuscript.

Funding

LJ is supported by an Innovative Engineering for Health award by the Wellcome Trust (WT101957) and the Engineering and Physical Sciences Research Council (EPSRC) (NS/A000027/1). AJ is supported by a grant from The Research Foundation – Flanders (FWO) [SB/1S63117N]. MPD and TM were funded with support of the Erasmus + Programme of the European Union (Framework Agreement number: 2013–0040). FDB is supported by a Fulbright Research grant. PDC is supported by the National Institute for Health Research (NIHR-RP-2014-04-046) and the BRC NIHR GOSH. JD was a Clinical Researcher of the FWO (FWO-1.8.012.07) and is currently supported by the Great Ormond Street Hospital Charity fund.

Competing interests

The authors declare no conflict of interests. All the instrumentation from the company KARL STORZ were part of a research loan (agreement 1705230948). Above mentioned support of KARL STORZ is in no way affiliated with any other service or procurement decisions on the part of the contractual parties.

Footnotes

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

These authors contributed equally: Luc Joyeux and Allan Javaux.

Contributor Information

Luc Joyeux, Email: luc.joyeux@kuleuven.be.

Jan Deprest, Email: jan.deprest@uzleuven.be.

Supplementary Information

The online version contains supplementary material available at 10.1038/s41598-021-85607-6.

References

  • 1.Adzick NS, et al. A randomized trial of prenatal versus postnatal repair of myelomeningocele. N. Engl. J. Med. 2011;364:993–1004. doi: 10.1056/NEJMoa1014379. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Farmer DL, et al. The Management of Myelomeningocele Study: Full cohort 30-month pediatric outcomes. Am. J. Obstet. Gynecol. 2018;218(256):e251–256. doi: 10.1016/j.ajog.2017.12.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Joyeux L, et al. Fetoscopic versus open repair for spina bifida aperta: A systematic review of outcomes. Fetal Diagn. Ther. 2016;39:161–171. doi: 10.1159/000443498. [DOI] [PubMed] [Google Scholar]
  • 4.Kabagambe SK, Jensen GW, Chen YJ, Vanover MA, Farmer DL. Fetal surgery for myelomeningocele: A systematic review and meta-analysis of outcomes in fetoscopic versus open repair. Fetal Diagn. Ther. 2018;43:161–174. doi: 10.1159/000479505. [DOI] [PubMed] [Google Scholar]
  • 5.Lapa Pedreira DA, et al. Percutaneous fetoscopic closure of large open spina bifida using a bilaminar skin substitute. Ultrasound Obstet. Gynecol. 2018;52:458–466. doi: 10.1002/uog.19001. [DOI] [PubMed] [Google Scholar]
  • 6.Belfort MA, et al. Comparison of two fetoscopic open neural tube defect repair techniques: Single- vs three-layer closure. Ultrasound Obstet. Gynecol. 2020;56:532–540. doi: 10.1002/uog.21915. [DOI] [PubMed] [Google Scholar]
  • 7.Joyeux L, et al. Learning curves of open and endoscopic fetal spina bifida closure: Systematic review and meta-analysis. Ultrasound Obstet. Gynecol. 2020;55:730–739. doi: 10.1002/uog.20389. [DOI] [PubMed] [Google Scholar]
  • 8.Harrysson IJ, et al. Systematic review of learning curves for minimally invasive abdominal surgery: A review of the methodology of data collection, depiction of outcomes, and statistical analysis. Ann. Surg. 2014;260:37–45. doi: 10.1097/SLA.0000000000000596. [DOI] [PubMed] [Google Scholar]
  • 9.Khan N, Abboudi H, Khan MS, Dasgupta P, Ahmed K. Measuring the surgical 'learning curve': Methods, variables and competency. Bju Int. 2014;113:504–508. doi: 10.1111/bju.12197. [DOI] [PubMed] [Google Scholar]
  • 10.Cohen AR, et al. Position statement on fetal myelomeningocele repair. Am J Obstet Gynecol. 2014;210:107–111. doi: 10.1016/j.ajog.2013.09.016. [DOI] [PubMed] [Google Scholar]
  • 11.Szasz P, Louridas M, de Montbrun S, Harris KA, Grantcharov TP. Consensus-based training and assessment model for general surgery. Br. J. Surg. 2016;103:763–771. doi: 10.1002/bjs.10103. [DOI] [PubMed] [Google Scholar]
  • 12.McCulloch P, et al. No surgical innovation without evaluation: the IDEAL recommendations. Lancet. 2009;374:1105–1112. doi: 10.1016/S0140-6736(09)61116-8. [DOI] [PubMed] [Google Scholar]
  • 13.Hirst A, et al. No Surgical innovation without evaluation: Evolution and further development of the IDEAL framework and recommendations. Ann. Surg. 2019;269:211–220. doi: 10.1097/SLA.0000000000002794. [DOI] [PubMed] [Google Scholar]
  • 14.Belfort MA, Whitehead WE, Bednov A, Shamshirsaz AA. Low-fidelity simulator for the standardized training of fetoscopic meningomyelocele repair. Obstet. Gynecol. 2018;131:125–129. doi: 10.1097/AOG.0000000000002406. [DOI] [PubMed] [Google Scholar]
  • 15.Miller JL, et al. Ultrasound based three-dimensional medical model printing for multispecialty team surgical rehearsal prior to fetoscopic myelomeningocele repair. Ultrasound Obstet. Gynecol. 2017 doi: 10.1002/uog.18891. [DOI] [PubMed] [Google Scholar]
  • 16.Joyeux L, et al. Safety and efficacy of fetal surgery techniques to close a spina bifida defect in the fetal lamb model: A systematic review. Prenat. Diagn. 2018;38:231–242. doi: 10.1002/pd.5222. [DOI] [PubMed] [Google Scholar]
  • 17.Joyeux L, et al. Validation of the Fetal Lamb model of spina bifida. Sci. Rep. 2019;9:9327. doi: 10.1038/s41598-019-45819-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG. Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research. PLoS Biol. 2010;8:e1000412. doi: 10.1371/journal.pbio.1000412. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.NC3Rs. Conducting a Pilot Study. https://www.nc3rs.org.uk/conducting-pilot-study#data (2016).
  • 20.Carter FJ, et al. Consensus guidelines for validation of virtual reality surgical simulators. Surg. Endosc. 2005;19:1523–1532. doi: 10.1007/s00464-005-0384-2. [DOI] [PubMed] [Google Scholar]
  • 21.Schout BM, Hendrikx AJ, Scheele F, Bemelmans BL, Scherpbier AJ. Validation and implementation of surgical simulators: A critical review of present, past, and future. Surg. Endosc. 2010;24:536–546. doi: 10.1007/s00464-009-0634-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Dawson B, Trapp RG. Basic and Clinical Biostatistics. London: McGraw-Hill Companies; 2004. pp. 287–289. [Google Scholar]
  • 23.Gine C, et al. Fetoscopic two-layer closure of open neural tube defects. Ultrasound Obstet. Gynecol. 2018;52:452–457. doi: 10.1002/uog.19104. [DOI] [PubMed] [Google Scholar]
  • 24.Peeters SH, et al. Learning curve for fetoscopic laser surgery using cumulative sum analysis. Acta Obstet. Gynecol. Scand. 2014;93:705–711. doi: 10.1111/aogs.12402. [DOI] [PubMed] [Google Scholar]
  • 25.Jani JC, et al. Severe diaphragmatic hernia treated by fetal endoscopic tracheal occlusion. Ultrasound Obstet. Gynecol. 2009;34:304–310. doi: 10.1002/uog.6450. [DOI] [PubMed] [Google Scholar]
  • 26.Belfort MA, et al. Fetoscopic open neural tube defect repair: Development and refinement of a two-port, carbon dioxide insufflation technique. Obstet. Gynecol. 2017;129:734–743. doi: 10.1097/AOG.0000000000001941. [DOI] [PubMed] [Google Scholar]
  • 27.Salomon LJ, Bernard JP, Ville Y. Estimation of fetal weight: reference range at 20–36 weeks' gestation and comparison with actual birth-weight reference range. Ultrasound Obstet. Gynecol. 2007;29:550–555. doi: 10.1002/uog.4019. [DOI] [PubMed] [Google Scholar]
  • 28.Ovaere C, et al. Prenatal diagnosis and patient preferences in patients with neural tube defects around the advent of fetal surgery in Belgium and Holland. Fetal Diagn. Ther. 2015;37:226–234. doi: 10.1159/000365214. [DOI] [PubMed] [Google Scholar]
  • 29.Heuer GG, Adzick NS, Sutton LN. Fetal myelomeningocele closure: Technical considerations. Fetal Diagn. Ther. 2015;37:166–171. doi: 10.1159/000363182. [DOI] [PubMed] [Google Scholar]
  • 30.Luks FI, Peers KH, Deprest JA, Lerut TE. Gasless laparoscopy in infants: The rabbit model. J. Pediatr. Surg. 1995;30:1206–1208. doi: 10.1016/0022-3468(95)90023-3. [DOI] [PubMed] [Google Scholar]
  • 31.Kirlum HJ, Heinrich M, Tillo N, Till H. Advanced paediatric laparoscopic surgery: Repetitive training in a rabbit model provides superior skills for live operations. Eur. J. Pediatr. Surg. 2005;15:149–152. doi: 10.1055/s-2005-837600. [DOI] [PubMed] [Google Scholar]
  • 32.Quintero RA, et al. Hydrolaparoscopy in the rabbit: A fine model for the development of operative fetoscopy. Am. J. Obstet. Gynecol. 1994;171:1139–1142. doi: 10.1016/0002-9378(94)90052-3. [DOI] [PubMed] [Google Scholar]
  • 33.Chitty LS, Altman DG, Henderson A, Campbell S. Charts of fetal size: 3. Abdominal measurements. Br. J. Obstet. Gynaecol. 1994;101:125–131. doi: 10.1111/j.1471-0528.1994.tb13077.x. [DOI] [PubMed] [Google Scholar]
  • 34.Jamieson GG, Watson DI, Britten-Jones R, Mitchell PC, Anvari M. Laparoscopic Nissen fundoplication. Ann. Surg. 1994;220:137–145. doi: 10.1097/00000658-199408000-00004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Martin JA, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br. J. Surg. 1997;84:273–278. doi: 10.1046/j.1365-2168.1997.02502.x. [DOI] [PubMed] [Google Scholar]
  • 36.Swift SE, Carter JF. Institution and validation of an observed structured assessment of technical skills (OSATS) for obstetrics and gynecology residents and faculty. Am. J. Obstet. Gynecol. 2006;195:617–621. doi: 10.1016/j.ajog.2006.05.032. [DOI] [PubMed] [Google Scholar]
  • 37.FDA, U. S. FDA Review Results in New Warnings About Using General Anesthetics and Sedation Drugs in Young Children and Pregnant Women. https://www.fda.gov/Drugs/DrugSafety/ucm532356.htm (2017).
  • 38.Biau DJ, Porcher R. A method for monitoring a process from an out of control to an in control state: Application to the learning curve. Stat. Med. 2010;29:1900–1909. doi: 10.1002/sim.3947. [DOI] [PubMed] [Google Scholar]
  • 39.Boateng GO, Neilands TB, Frongillo EA, Melgar-Quinonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: A primer. Front. Public Health. 2018;6:149. doi: 10.3389/fpubh.2018.00149. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 40.Connell J, et al. The importance of content and face validity in instrument development: Lessons learnt from service users when developing the Recovering Quality of Life measure (ReQoL) Qual. Life Res. 2018;27:1893–1902. doi: 10.1007/s11136-018-1847-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Jamali FR, et al. Evaluating the degree of difficulty of laparoscopic colorectal surgery. Arch. Surg. 2008;143:762–767. doi: 10.1001/archsurg.143.8.762. [DOI] [PubMed] [Google Scholar]
  • 42.Tekkis PP, Senagore AJ, Delaney CP, Fazio VW. Evaluation of the learning curve in laparoscopic colorectal surgery: Comparison of right-sided and left-sided resections. Ann. Surg. 2005;242:83–91. doi: 10.1097/01.sla.0000167857.14690.68. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Claerhout F, et al. Analysis of the learning process for laparoscopic sacrocolpopexy: Identification of challenging steps. Int. Urogynecol. J. 2014;25:1185–1191. doi: 10.1007/s00192-014-2412-z. [DOI] [PubMed] [Google Scholar]
  • 44.Kim CW, et al. Learning curve for single-port laparoscopic colon cancer resection: A multicenter observational study. Surg. Endosc. 2017;31:1828–1835. doi: 10.1007/s00464-016-5180-7. [DOI] [PubMed] [Google Scholar]
  • 45.Moore MJ, Bennett CL. The learning curve for laparoscopic cholecystectomy. The Southern Surgeons Club. Am. J. Surg. 1995;170:55–59. doi: 10.1016/s0002-9610(99)80252-9. [DOI] [PubMed] [Google Scholar]
  • 46.Oomen MW, Hoekstra LT, Bakx R, Heij HA. Learning curves for pediatric laparoscopy: How many operations are enough? The Amsterdam experience with laparoscopic pyloromyotomy. Surg. Endosc. 2010;24:1829–1833. doi: 10.1007/s00464-010-0880-x. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Video 1. (93.3MB, mp4)

Articles from Scientific Reports are provided here courtesy of Nature Publishing Group

RESOURCES