Patient-specific mental rehearsal with three-dimensional models before low anterior resection: randomized clinical trial

Abstract

Background

It was hypothesized that preparing for a surgical procedure, taking into account individual patient characteristics, may facilitate the procedure and improve surgical quality. The aim of this study was to compare different case-specific, preoperative mental rehearsal methods before minimally invasive rectal cancer surgery.

Methods

In this RCT, patients were allocated in a 1 : 1 : 1 : 1 ratio to four groups: systematic mental rehearsal (SMR) using MRI scans; SMR and three-dimensional (3D) virtual models; SMR and synthetic 3D printed models; and routine practice (control group). Surgeons operating on all but the control group underwent mental rehearsal with the visual aids, including axial MRI scans of the pelvis, interactive 3D virtual models reconstructed from axial MRIs, and synthetic models, manufactured by 3D printing. Operations were video-recorded and assessed by two experts blinded to allocation using two validated scores, the Competency Assessment Tool (CAT) and Objective Clinical Human Reliability Analysis (OCHRA). The primary outcome of the study was surgical performance, measured by the CAT.

Results

Forty-nine patients were randomized and allocated to the four groups. There were 12 participants in each of the control, MRI and SMR, and virtual and SMR groups, whereas the SMR using physical models and simulation group included 13. No difference was observed between groups in median CAT scores (control 30.50, MRI 34.25, virtual 31.75, physical 34.00; P = 0.748, partial η² <0.001, where _pη² is indicative of effect size) or OCHRA scores (anterior, posterior, right and left lateral planes, transection P>0.200, _pη² =0.052–0.088). Time spent not performing dissection was significantly shorter for the SMR with MRI group than for the control (57.5 versus 42 respectively; P < 0.001, _pη² =0.212).

Conclusion

Mental rehearsal did not affect CAT and OCHRA scores of consultant surgeons. Reference number: ISRCTN 75603704 (https://www.isrctn.com).

Introduction

Technical difficulty in rectal cancer surgery affects both specimen quality and complication rates^1,2 and is directly associated with anatomical and pathological characteristics^1–4. The level of difficulty is associated with tumour location, pelvic geometry, and the patient’s BMI^1,2. Individualized preoperative planning may result in advanced quality of surgery and better outcomes after rectal cancer surgery.

Patient-specific surgical rehearsals are now facilitated particularly by the advancement of computer-assisted manufacturing technology—surgical simulators⁵. In addition, mental practice may improve subsequent surgical performance⁶. Evidence regarding the impact of mental practice on the surgical skills of experts is non-existent, as comparative studies have thus far recruited only medical students^7–12 and surgical trainees^13–15.

The aim of this RCT was to compare patient-specific mental rehearsal with patient-specific simulation and with the current routine preparation before minimally invasive low anterior resection.

Methods

This RCT was conducted in two centres in the UK, St James University Hospital in Leeds and Pinderfields Hospital, Mid Yorkshire Hospitals NHS Trust, in Wakefield (a large district general hospital), between July 2015 and July 2016. The objectives of this project were: to create an evidence-based methodology for preoperative, patient-specific rehearsals; to standardize/streamline the process with the introduction of mental practice; to assess the feasibility of applying the new methodology in a stressful, time-restrained clinical environment; and to compare the different methods of rehearsal with one another and with current practice.

The study received NHS Research Ethics Committee approval from the Leeds East Committee (reference number 15/YH/0134), the Leeds Teaching Hospital Research and Innovation Department (reference number GA15/070), the Mid Yorkshire Hospitals NHS Trust Research and Development Department (reference number JH/CSC/N:R&D(15/992)), and the Health Research Authority (Integrated Research Application System identification number 165586). All patients provided informed written consent for the study. The trial was registered with the ISRCTN registry database (reference number ISRCTN 75603704). The full protocol is available on request.

Participants

Patients diagnosed with resectable rectal cancer (distance from the anal verge 4–16 cm) and scheduled for minimally invasive (laparoscopic/robotic, laparoscopic/transanal) rectal cancer surgery were eligible. Patients planned for a primary Hartmann resection (no anastomosis planned), abdominoperineal resection or primary open surgery were excluded from the study. Patients unable to represent their own interests and consent themselves for treatment were not included. The operations had to be performed or supervised by a consultant colorectal surgeon who had performed at least 50 laparoscopic anterior resections, thereby ensuring competence in performing this procedure¹⁶. After hospital discharge, no further follow-up was undertaken.

Randomization

Randomization was performed using co-variable adoptive randomization (minimization). The co-variables adapted for were: BMI (underweight, normal, overweight, obese), tumour distance from the anal verge (4–8, 8.1–12 or 12.1–16 cm), cT category, and sex. Patients were allocated to four groups (group 1: systematic mental rehearsal (SMR) using MRI scans; group 2: SMR and three-dimensional (3D) virtual models; group 3: SMR and synthetic 3D printed models; and group 4: routine practice) in a 1 : 1 : 1 : 1 ratio. Patients were approached for consent by the consultant surgeon in charge of their care and the research fellow. An automated software algorithm was used for the randomization process, with necessary data input by the surgical research fellow. Randomization was performed several days before the operation to allow sufficient time for the creation of models; this was done within the premises of the University of Leeds, which are in close proximity to St James’s University Hospital. A detailed explanation of the consent and randomization process can be found in the study’s research protocol, which is available on request.

Design and production of three-dimensional virtual and physical models for allocated patients

3D virtual models were reconstructed from routine preoperative MRI scans, using our in house software VolumeViewer, University of Leeds, Leeds, UK. The model was validated in a small group of expert surgeons using a Likert scale (1-10, 1 being not satisfied at all and 10 very satisfied) for content validity, scoring 8.5 of 10.

After reconstruction of the pelvic bone to a 3D virtual model, a half-oval structure was added to it using SOLIDWORKS^® (Dassault Systèmes, Paris, France), 3D computer-aided design software. The final 3D virtual model was 3D printed, using an Objet1000 Multi-material 3D Printer (Stratasys, Edina, MN, USA) (Fig. 1). A similar process was followed to produce physical models of the rectum and mesorectum. These were then used as a negative cast to create the mesorectal fascial plane, where surgical dissection takes place during total mesorectal excision (TME). Semiliquid silicon was used to cover the solid mesorectum in such a way that, when dried, it provided an accurate representation of the plane. The resulting entity was then placed in the reusable pelvis. The gap between the ‘fascia’ and the ‘pelvic wall’ was filled in by pliable material, leaving a small gap, tightly packed with polyester fibres, representing the so-called ‘angel hairs’ found in the embryological plane separating the mesorectum from surrounding structures. After the pelvic model had been prepared, it was placed into a box trainer for the surgeons to practise the pelvic dissection part of the procedure.

Fig. 1.

Virtual and physical models reconstructed,

a Virtual models: 3D reconstructed models - left to right - pelvis without the mesorectum, pelvic bone with rectum, mesorectum and ureters, pelvic bone and tumour b physical models - left: external view, right: internal view.

Systematic mental rehearsal methods and comparators

Three different SMR strategies were applied for all but the control group. For group 1 SMR was performed combined with MRI scans, for group 2 with interactive 3D virtual models, and for group 3 with synthetic models. It should be noted that for group 3, in addition to SMR, physical practice in the form of a simulated dissection of the TME plane was performed.

All surgeons participating in the study mentioned that they routinely reviewed the MRI scans at multidisciplinary team meetings, in the clinic, and either the day before or the day of the procedure. Although the intervention applied in the MRI group may appear similar to the procedure followed by the control group, surgeons operating on the MRI group were given the opportunity to undergo an evidence-based, structured, streamlined approach based on an international consensus (on how to perform minimally invasive TME)¹⁶ and made possible by the introduction of SMR. The streamlined process was designed by an expert consultant surgeon in collaboration with two psychologists, and is reflected in Table 1 (standard scans).

Table 1.

Framework for systematic mental rehearsal sessions

Step	SMR theme	Virtual model	Physical model	Standard scans
1	Introduction	The outline of the SMR sessions is explained by the researcher
2	Viewing of visual aids	Participants are given opportunity to view the visual aids for 5–10 min
3	Agreement on technical steps	Participants are given a summary of the steps of the procedure. They have the opportunity to change steps according to their individual preference on how they perform the procedure
4	Detailed TME	For each step of the procedure, the surgeon is asked to go mentally through the step and explain how they will do this and what possible difficulties could be encountered
4.1	Posterior plane	Dynamic views of the posterior plane are given	Posterior plane dissection is simulated in pelvic trainer	Posterior plane is visualized in consecutive MRI slides
4.2	Anterior plane	Dynamic views of the anterior plane are given	Anterior plane dissection is simulated in pelvic trainer	Anterior plane is visualized in consecutive MRI slides
4.3	Side walls	Dynamic views of the side walls are given	Side-wall dissection is simulated in pelvic trainer	Side-wall planes are visualized in consecutive MRI slides
4.4	Low pelvic dissection	Dynamic views of low pelvic anatomy are given	Low pelvic dissection is simulated in pelvic trainer	Low pelvic planes are visualized in consecutive MRI slides
5	Strategy changes recorded	Based on the above, participants are asked whether any strategic or technical changes have been made
6	Repetition	Any of the above steps can be repeated, if required by the participant
7	Agreed plan	An operative plan is agreed and recorded

SMR, systematic mental rehearsal; TME, total mesorectal excision.

The SMR sessions were co-ordinated by the surgical research fellow and co-designed with cognitive psychologists. The sessions lasted 20–30 min and were performed within 48 h before the operation. The SMR protocol for the three intervention groups is outlined in Table 1.

Outcome measures

The primary outcome of the study was surgical performance. To assess this, two separate, validated scoring systems were used: the Competency Assessment Tool (CAT)¹⁷ and Objective Clinical Human Reliability Analysis (OCHRA)¹⁸. The CAT score consists of four categories, each representing the steps of the pelvic dissection: posterior mesorectal dissection, lateral mesorectal dissection, anterior mesorectal dissection, and resection and anastomosis. For each category a score from 1 to 4 (1, unsatisfactory performance; 4, excellent performance) was provided for evaluation according to performance in the following subcategories: retraction and exposure; dissection/execution of task; number of errors; and quality of end product¹⁷. OCHRA allows for identification and tagging of previously defined errors and near misses, using video-tagging software¹⁷. This leads to a detailed description of performance. It has been shown previously¹⁸ that a combination of CAT and OCHRA scoring is highly specific and sensitive for reliable identification of surgical competence.

For the OCHRA evaluation process, the pelvic dissection was divided in the following steps: anterior plane dissection, posterior plane dissection, lateral planes dissection, low mesorectal dissection, and transection. In addition to these steps, time spent without any dissection was recorded (under the code name ‘nothing’).

OCHRA aims to identify errors during surgery. These are defined as errors related to either instrument use or tissue handling, and could also be consequential or bear no consequences¹⁸. All errors and time durations for each step of the operation were recorded using BORIS(freeware designed by the University of Pisa, Italy (www.boris.unito.it)).

Modifications to CAT and OCHRA

For the purposes of the present study, two modifications were made to allow for comparisons between groups. As the cases include partial and total mesorectal excisions, the duration of partial excisions was expected to be shorter than that for TME; therefore a smaller number of errors could be attributed to shorter duration rather than improved surgical performance. To ensure fair comparison between partial and total mesorectal excisions, instead of comparing the crude number of errors, the rate of errors per unit of time was used as the measured outcome.

Real-time pelvic dissection was recorded and assessed by two independent blinded experts who reviewed the videos. Total time was calculated from the duration of pelvic dissection in the unedited video of the operation. No identifiable characteristics were included in the recordings to ensure accuracy of the blinding process. In addition to the error rate (number of errors per second) for each procedural step, the amount of time spent without engaging in dissection was compared between groups. This was expressed as the rate of time spent not performing dissection divided by the overall duration of the TME (‘nothing’ time/total time for pelvic dissection), and was used to reflect the inefficiency of the surgical technique¹⁹.

Secondary outcomes included length of stay, complications (Clavien–Dindo classification)²⁰ and specimen quality. The latter was defined according to Quirke et al.²¹: good (dissection in the mesorectal plane and mesorectum is intact with smooth surface and with defects not exceeding 5 mm in depth); moderate (dissection in the intramesorectal plane; the mesorectal surface has irregularities but the muscularis propria is not visible); and poor (dissection in the muscularis propria plane with little bulk of mesorectum and defects to muscularis propria).²¹

Power calculation

Sample size calculations were performed based on the assessment of experts and apprentices’ performances, as published previously²². According to this study,²² the effect size of previous surgical experience (for instance, experts versus non-experts) is 1.245 (Cohen’s d). Assuming a relatively more modest anticipated effect size (Cohen’s d = 1) and the four conditions (for the 4 groups to which the patients were randomized) between subjects, α of 0.05 and power of 0.80, the calculated sample size was 48 individuals. G*Power(Heinrich Heine University, Düsseldorf, Germany (https://www.psychologie.hhu.de)) was used for the above calculations²³.

Statistical analysis

Statistical analysis was performed on GraphPad Prism^® 7.0c (GraphPad, La Jolla, CA, USA), and SPSS^® version 17.0 (SPSS, Chicago, IL, USA). The non-parametric Kruskal–Wallis test was used for discrete metric values. For continuous metric variables, ANOVA was used. Quality of specimen was assessed using the χ² test. P<0.050 was considered statistically significant.

Results

A total of 49 patients were included; their characteristics and clinical outcomes are shown in Fig. 2  and Table 2. There were no protocol violations and no patient dropped out after randomization. Initially, 51 patients were eligible; one patient was excluded because he could not consent or represent his own interests, and another was excluded as there was a disagreement between colonoscopy and radiological imaging regarding the distance of the tumour from the anal verge, which was borderline for inclusion (a sigmoid rather than rectal tumour). Control, MRI and SMR, and 3D virtual and SMR groups included 12 participants each, and the physical and SMR simulation group included 13 participants (Fig. 3). One procedure in the control group was converted to an abdominoperineal resection, after opinion had been sought from a second consultant during surgery.

Fig. 2.

Patient characteristics

a Sex, b distance from anal verge, c T category and d BMI. a  P =0.669, b  P=0.999, c  P=0.667, d  P=0.888 (Kruskal–Wallis test).

Table 2.

Patient characteristics and clinical outcomes

	Control group (n=12)	MRI + SMR (n=12)	Virtual + SMR (n=12)	Physical + SMR (n=13)
Age (years)*	71.6 (63–87)	71.6 (61–84)	61.7 (37–83)	67.5 (43–84)
ASA grade
I	4	4	3	5
II	7	8	9	7
III	1	0	0	1
LOS (days)†	7 (3–41)	7 (3–58)	7.5 (5–36)	14 (6–36)
No. of lymph nodes†	13.5 (8–31)	19 (12–34)	12 (8–20)	8 (5–28)
Type of surgery
Laparoscopic	7	8	8	9
Robotic	4	3	3	3
Laparoscopic/transanal TME	1	1	1	1
Converted to open	3	0	2	1

Values are *mean (range) and ^†median (range). SMR, systematic mental rehearsal; LOS, length of stay; TME, total mesorectal excision.

Fig. 3.

Flow diagram of recruitment and randomization of patients

SMR, systematic mental rehearsal; 3D, three-dimensional.

CAT score and OCHRA evaluation (primary outcome)

Median (i.q.r.) overall CAT scores and CAT scores for individual steps were no different between the groups (control: 30.50 (24.63–38.63); MRI and SMR: 34.25 (30.50–40.50); virtual and SMR: 31.75 (30.13–36.38), physical and SMR: 34.00 (28.50–35.00); P =0.748, partial η² <0.001, where _pη² is indicative of effect size) (Figs 4 and 5).

Fig. 4.

Total Competency Assessment Tool scores in the four groups

Values are median (i.q.r.). CAT, Competency Assessment Tool.

Fig. 5.

Competency Assessment Tool scores for each step of the procedure

a Posterior plane, b Anterior plane, c lateral planes. Individual patient values are shown as well as median (i.q.r.) values. CAT, Competency Assessment Tool. Effect size (_pη²): a 0.046, b 0.146, c 0.026.

Similar to the CAT results, OCHRA scores between groups showed no statistical difference (Table 3 and Table S1).

Table 3.

Objective Clinical Human Reliability Analysis score results (mean and 1-3 IQR)

	Anterior plane	Posterior plane	Right lateral plane	Left lateral plane	Low mesorectal dissection	Transection
Control	0.012 (0–0.02)	0.015 (0.01-0.023	0.012 (0.003–0.039)	0.013 (0.006–0.023)	0.012 (0.005–0.022)	0.004 (0.001–0.026)
MRI + SMR	0.003 (0–0.019)	0.024 (0.013–0.035)	0.024 (0.016–0.038)	0.018 (0.01–0.032)	0.018 (0.01–0.036)	0.009 (0.005–0.018)
Virtual + SMR	0.009 (0.003–0.012)	0.016 (0.009–0.029)	0.011 (0.005–0.017)	0.012 (0–0.017)	0.005 (0–0.016)	0.009 (0–0.018)
Physical + SMR	0.015 (0.007–0.023)	0.019 (0.012–0.036)	0.018 (0.011–0.024)	0.012 (0.005–0.022)	0.017 (0.009–0.024)	0.007 (0.001–0.01)
P*	0.359	0.725	0.204	0.330	0.121	0.900
Partial η2	0.056	0.033	0.067	0.088	0.085	0.052

Values are mean (i.q.r.). SMR, systematic mental rehearsal. *ANOVA.

Secondary outcomes

The mean hospital stay was 7.0, 7.0, 7.5 and 14.0 respectively for the control, MRI, virtual and physical group respectively. With regard to quality of specimen, no differences were observed between the four groups (P = 0.565) (Fig. S2). Complications classified according to the Clavien–Dindo classification²⁰ are shown in Table 4.

Table 4.

Clavien–Dindo complication grades

	Control	MRI + SMR	Virtual + SMR	Physical + SMR
Grade I	2	1	2	2
Grade II	2	2	1	1
Grade III	1	2	0	0
Grade IV	0	0	0	1
Total	5	5	3	4

SMR, systematic mental rehearsal.

The median (i.q.r.) rate of time spent not performing dissection (time spent not performing dissection/total time) was significantly lower for cases rehearsed with SMR and MRI compared with cases that had routine preparation (0.42 (33.50–0.48) versus 0.58 (0.50–0.72) respectively; P < 0.010 with Bonferroni correction, _pη²=0.212). There was no difference in the median (i.q.r.) rate of time spent without performing dissection between the control and virtual group (0.58 (0.50–0.72) versus 0.51 (0.38–0.62) respectively; P =0.13) and control and physical groups (0.58 (0.50–0.72) versus 0.56 (0.39–0.66); P =0.41). Rate of time spent performing no dissection divided by total time is demonstrated in Fig. 6. It should be noted that cases converted to open shortly after initiating the pelvic dissection were included, which accounts for the case with ‘nothing’ time approaching 100 per cent. The mean (i.q.r.) total time to complete the procedure wasted as follows (control: 88.35 (42.70–140.65) min; MRI: 55.29 (33.46–63.87) min; virtual: 59.83 (25.70–84.89) min; physical: 88.31 (46.40–101.44) min).

Fig. 6.

Percentage of time spent doing no dissection for each of the four groups

Individual patient values are shown as well as median (i.q.r.) values. *P<0.01 (ANOVA).

Discussion

This RCT compared patient-specific simulation with mental rehearsal in a real clinical environment for a highly complex minimally invasive procedure. SMR did not lead to improved CAT and OCHRA scores. Mental practice with MRI as a visual aid may increase the efficiency of similar procedures, reducing idle time. This appears to have a greater impact on more technically challenging parts of a procedure, such as dissection of the low plane in minimally invasive anterior resections. This intervention is straightforward and can be applied easily in a teaching or general district hospital environment.

Previous studies conducted by the authors’ group had somewhat different results. The combination of mental rehearsal and 3D anatomical models was found to be effective in enhancing surgical performance in a simulated environment^24,25. Previous studies recruited trainees and medical students, whereas for the present study experts were recruited. One can hypothesize that the impact of preoperative preparation on performance appears limited when experts are involved^26,27. Advanced skills make them potentially more adaptable to unexpected events during surgery. Moreover consultant (expert) surgeons usually follow their patient’s journey from diagnosis to treatment, familiarizing themselves with the unique characteristics of the patient, pathology and imaging. Combined with clinical experience, many expert surgeons have the ability to plan for surgery without the need of specific preparation sessions.

In the literature the effect of SMR is not consistent⁶. Five^10,12–15 of nine RCTs described favourable results. In these studies^10,12–15, a variety of skills, ranging from basic skills to full surgical procedures, were taught. The assessment tools used in these studies were equally diverse and included objective (such as checklists, time to complete task, number of instrumental tip movements) and non-objective (expert assessment) measures. Interestingly, the studies that did not show any significant impact of mental practice on the acquisition of skills used similar tasks and evaluation methods to those that did demonstrate a difference (for example, circle-cutting in a box trainer, and checklists, scoring systems for assessment)^7–9,11.

The authors of the systematic review⁶ attributed the difference in results to the duration of the mental practice sessions and the number of times these were repeated. However, the difference in duration between the studies that did and did not show a statistically significant difference was as short as 5 min⁶, raising the question of whether there is a separate factor contributing to boosting the effect of mental practice.

The findings of the present RCT were essentially negative. Explanations may be that the study was underpowered because of the assumption of a large effect size, or differences in the primary outcome (CAT and OCHRA scores) are absent in expert surgeons. The small sample size was an important limitation of the study. Comparison of four groups demands a sufficient sample size to allow for multiple comparisons.

Reduction of idle time may be an indication of increasing surgical efficiency and therefore relevant, but this was not the primary outcome in this study. As the aim of SMR in the present study was to refine expert performance, metrics that could detect more subtle differences would have been more appropriate. The rationale of SMR is to prepare and anticipate technically challenging aspects of a procedure that could lead to reduced stress and easier adaptability during the operation. In a future study, in addition to idle time and surgical performance, physiological measures and stress levels of the surgeon could be assessed. The surgeon’s physiological parameters while operating can provide an estimation of how well they are coping during a complex procedure^28–31. Conceivably, this outcome measure would have been more appropriate to demonstrate how mental practice influenced the surgeon’s ability to cope effectively with a complex operation.

Funding

Leeds Cares

Pelican Cancer Foundation

 

Presented to the 13th Scientific and Annual Meeting of the European Society of Coloproctology, Nice, France, September 2018