Does mental imagery prior to cystoscopy make a difference? A randomized controlled trial

Abstract

OBJECTIVE

We sought to determine whether mental imagery improves surgical performance of residents novice to cystoscopy.

STUDY DESIGN

We performed a multicenter randomized controlled trial. Residents who had performed ≤ 3 cystoscopies were randomized to preoperative mental imagery sessions or reading a book chapter describing cystoscopy. The primary outcome was comparison of groups’ surgical performance scores. Secondary outcomes were measurements of operative times and resident ratings of helpfulness of their preparation. Scores were compared using 2-factor analysis of variance.

RESULTS

In all, 68 residents were randomized; 33 to imagery and 35 to control groups. Groups did not differ in age, cystoscopic experience, residency level, or sex. The imagery group’s surgical assessment scores were 15.9% higher than controls (P = .03). Operative times did not differ between groups. Imagery residents rated imagery preparation as more helpful than controls (P < .0001).

CONCLUSION

Residents considered mental imagery to be a more useful preoperative preparation. The mental imagery group’s surgical performance was superior to controls.

Evolving demands on surgical training necessitate development of educational techniques that rapidly enhance surgical performance. Traditionally, surgical education has been based on a model of graded responsibility under a mentor. Trainees have been apprentices who learned from repetition. This apprenticeship model is under siege because of decreased resident work hours, ethical concerns regarding “practicing” skills on patients, and limitation in numbers of experienced mentors.¹ Additionally, current resident work hour limitations have raised concerns that surgical volume per resident will decrease.² Concerns are not limited to physicians in training. Equally important, proliferation of new procedures will make even experienced surgeons novice sometime during their careers. Accordingly, it will be necessary to develop innovative methods that improve skill attainment not only for residents but for all practitioners.

Mental imagery has successfully improved skills and enhanced performance in athletics.^3,4 Strictly defined, mental imagery is “the symbolic rehearsal of a physical activity in the absence of any gross muscular movements.”⁴ It has been hypothesized that mental imagery produces cognitive blueprints for movement patterns and mental rehearsal of these blueprints allows movements to become automatic.⁴ Surgery and athletics share similar characteristics. Both require advanced motor skills and the intellect to adapt to new situations.

More recent advances in cognitive psychology support mental imagery’s potential use in surgical education.^5,6 For example, dual coding theory distinguishes between verbal and nonverbal cognition. Researchers believe that coupling of verbal and nonverbal encoding improves learning.^5–7 Accordingly, mental imagery practice, a combination of verbal and nonverbal cognition, could be a superior educational technique for surgical training compared to textbook reading, primarily a verbal activity. Regardless of mechanism, mental imagery practice is particularly useful in tasks with large cognitive components.³ Thus, mental imagery may be well suited to improving surgical performance.^1,8 Despite potential advantages of mental imagery practice prior to surgery, study of its use in surgical training is still in its formative stages.^1,8–10 Mental imagery has been found to be an effective tool to teach medical students cricothyrotomy in a simulated situation as well as animal suturing in an operating department suite.^9,10 Recently, consistent with the dual coding theory of cognition, mental imagery was also found to be a more effective tool than textbook study among medical students.⁷

We aimed to evaluate whether mental imagery improved residents’ surgical performance in actual clinical practice. Our hypothesis was that mental imagery improved surgical performance in the operating room. Our specific goal was to determine whether formal mental imagery practice before cystoscopy improved surgical performance of gynecology residents novice to cystoscopy. For our primary outcome, we compared performance ratings between residents who practiced mental imagery prior to cystoscopy to those who were randomized to reading a textbook. Based on the dual coding theory, mental imagery practice would be superior to textbook study. For secondary outcomes, we compared time for cystoscopy performance and compared resident ratings of helpfulness of their preoperative preparations.

Materials and Methods

We conducted a multicenter randomized controlled trial. Subjects were recruited from 6 academic centers. Gynecology residents who had observed at least 1 cystoscopy and performed ≤ 3 cystoscopies were eligible to participate regardless of house officer level. We required residents to have observed at least 1 cystoscopy before performing cystoscopy because this was standard at most of the institutions. The ≤ 3 cystoscopy cut-off point was decided on by consensus of the investigators. Based on their experience as educators, they agreed that this was a conservative threshold to determine novice status. The primary investigators at each site obtained local institutional review board approval. All subjects gave written informed consent. Evaluations occurred during resident rotations on gynecology services at their respective institutions.

We compared performance ratings between residents who practiced mental imagery prior to cystoscopy to those who did not. Residents were assigned to groups after they gave informed consent. Study participants were assigned to either mental imagery (imagery group) or instructed to read the same cystoscopy chapter in a specific textbook (control group).¹¹ A random numbers table generated assignments using block randomization groups of 4. A research nurse otherwise uninvolved with the study, ensured the allocation sequence was concealed and placed group assignments in sealed, opaque envelopes.

Residents’ age, sex, and level of training were recorded. Although inclusion criteria included performance of ≤ 3 cystoscopies, all residents were asked to record the number of cystoscopies they had performed prior to entering the study. Each resident was also asked to record whether they had completed their assigned preoperative preparation. Intention-to-treat analysis was performed initially; results were analyzed on all randomized residents regardless of whether or not they complied with their preoperative assignment.

Mental imagery sessions were administered by gynecology faculty (imagery educators) at each site. To standardize imagery sessions, a digital video disc (DVD)-recorded template of mental imagery sessions performed at the coordinating site was sent to all mental imagery educators. The DVD served as a template on which mental imagery educators at the different institutions were to fashion their imagery sessions. The DVD recorded a resident from the coordinating site using mental imagery prior to performing cystoscopy. The resident described assembly and insertion of the cystoscope, identification of bladder landmarks, systematic survey of bladder quadrants, identification of potential abnormalities, and urethral inspection. The DVD was not shown to the study subjects since its purpose was to standardize imagery sessions for mental imagery educators.

Mental imagery educators, all gynecology faculty members, individually met with subjects randomized to mental imagery preparation within 24–48 hours of each of the scheduled cystoscopies. Residents rehearsed performance of the procedure with the educators during 1-on-1 sessions. Residents were instructed to envision performing the procedure and described cystoscopy performance to the educator. Educators required residents to describe all the components of the cystoscopy; assembly and insertion of the cystoscope, identification of bladder landmark, and 4-quadrant survey of the bladder. The study protocol specified that imagery sessions last < 20 minutes. Imagery educators were aware of the randomization assignments.

The control group was instructed to individually read the same chapter in a standard text describing cystoscopy within 24–48 hours of each of the 2 planned cystoscopies.¹¹ The chapters included 8 illustrations or photographs of cystoscopes/cystoscopies and 10 pages of text. Investigators estimated reading this chapter would take a similar amount of time as imagery sessions. Postoperatively, residents were asked whether or not they had performed their assigned preoperative assignments.

Attending physicians or urogynecology fellows who had done >50 cystoscopies evaluated residents’ performance. Cystoscopy evaluators were blinded to randomization and were different individuals than the imagery educators. Prior to evaluating resident cystoscopies, evaluators viewed a DVD of 3 mock cystoscopies portraying novice, intermediate, and expert cystoscopies. These mock cystoscopies were performed on a pelvic model by one of the investigators, videotaped, and sent to each site. Evaluators graded each cystoscopy. Intraclass correlation coefficients calculated on the mock cystoscopy scores equaled 0.78, indicative of high interrater reliability.

Residents were evaluated performing 2 separate cystoscopies on patients in the operating department. All cystoscopies were performed with images displayed on a monitor so evaluators, present during the entire procedure, could view all aspects of the procedure. Evaluators used a modification of a validated assessment form, The Global Scale of Operative Performance (GSOP), to rate residents’ surgical performance (Figure 1).^12–14 The fifth item on the original scale (use of assistants) was deleted. Assistants are not usually present or needed for cystoscopy. The evaluation form included 6 individual items (rated 1–5) and a cumulative score (range, 6–30), which was the sum of the items (Figure 1). Higher scores represent superior performance.

FIGURE 1.

Cystoscopy evaluation form

In addition to the GSOP items, Evaluators also recorded 3 other performance measures: time required for cystoscopy, competence to perform the procedure independently (yes or no), and preparedness for the procedure (1 = not at all, 2 = somewhat, 3 = adequately, 4 = very well, 5 = extremely well) (Figure 1). Time for cystoscopy included time from instrument assembly until removal of the cystoscope at the end of the procedure. Lastly, residents were asked to rate helpfulness of their preoperative preparation assignment using a visual analog scale (range, 0–10; 0 = not at all helpful, 10= extremely helpful).

Statistics

Power analysis was performed prior to study initiation. We decided a priori that a 15% difference in performance between groups would be clinically significant. Targeting 80% power and α = 0.05, assuming a 2–3 point difference in total scores, 68–72 subjects were required to find a 15% difference in total GSOP scores between groups.

Statistical evaluation of data included 2-sided t tests for continuous variables, Fisher exact test for categorical variables, and Wilcoxon rank test for ordinal variables. Two-factor analysis of variance (ANOVA) was used to control for confounding categorical variables using SAS type III sums of squares (SAS/STAT 9.1 computer software; SAS Institute Inc, Cary, NC). The resulting weighted averages of scores obtained using ANOVA were reported as least square means (LSM) and their SEs. Differences in LSM scores between groups were used to report percentage differences between group scores ([LSM imagery group score – LSM control group score]/LSM control group score × 100%). Significance was set at P < .05.

Results were first analyzed using the intent-to-treat principal. A subanalysis was then performed after excluding residents who had performed >3 cystoscopies. A subanalysis was also attempted to evaluate the effect of compliance with preoperative preparation.

Results

A total of 68 residents from 6 institutions were enrolled in this study from July 2006 through August 2007. In all, 35 subjects were randomized to the control group and 33 to the mental imagery group (Figure 2). Intention-to-treat analysis included evaluation of all 68 residents randomized irrespective of whether or not they met inclusion criteria. There were no differences between imagery and control subjects’ age, level of residency training, number of previous cystoscopies performed, time between first and second cystoscopies, or sex (Table 1). Four residents, 2 in each group, did not meet study inclusion criteria because they had performed > 3 cystoscopies but were kept in the intent-to-treat analysis. Subanalysis of only the 64 qualified residents, excluding the overly qualified residents, was also performed: 33 in the control group and 31 in the imagery group. Again, there were no differences between imagery and control subjects’ age, residency level, number of previous cystoscopies, time between first and second cystoscopies, and sex (Table 1).

FIGURE 2.

Imagery randomized controlled trial flow diagram

TABLE 1.

Resident characteristics (results for all and only qualified residents)

Characteristic	Control group All (N = 35) Qualified (n = 33)	Imagery group All (N = 33) Qualified (n = 31)	P
Age, y (±SD)
All (N = 68)	29.3 (4.1)	28.7 (2.5)	.49a
Qualified (n = 64)	29.1 (4.1)	28.7 (2.5)	.49a
Year in residency (±SD)
All (N = 68)	1.7 (0.8)	1.6 (0.8)	.79a
Qualified (n = 64)	1.6 (0.7)	1.5 (0.7)	.74a
Previous No. of cystoscopies performed, mean ± SD [rank score]
All (N = 68)	0.77 ± 1.33 [33.6]b	0.76 ± 1.9 [35.4]b	.65b
Qualified (n = 64)	0.55 ± 0.97 [33.5]b	0.35 ± 0.71 [31.3]b	.60b
Female
All (N = 68)	27 (77%)	28 (85%)	.54c
Qualified (n = 64)	26 (79%	27 (87%)	.51c
Time between first and second cystoscopy, mean days ± SD [rank score]
All (N = 68)	16.7 ± 22.7 [36.3]b	19.5 ± 40.5 [32.5]b	.41b
Qualified (n = 64)	17.7 ± 23.0 [34.5]	20.8 ± 41.5 [30.4]	.37b
Did assigned preoperative preparation for first cystoscopy
All (N = 68)	20/29d (68%)	26/27d (96%)	.01c
Qualified (n = 64)	19/28d (68%)	25/26d (96%)	.01c
Did assigned preoperative preparation for second cystoscopy
All (N = 68)	21/30d (70%)	26/28d (92%)	.04c
Qualified (n = 64)	20/28d (70%)	25/27d (92%)	.08c

^aSatterthwaite 2-sample t test;

^bMann-Whitney-Wilcoxon rank sum test;

^cFisher exact test;

^dmissing information from some subjects.

Both overall GSOP and component scores varied between institutions and these institutional differences confounded evaluation of performance results. ANOVA was used to control for the confounding effect of institutions. The resulting data showed the imagery group’s performance measures for the first cystoscopy were superior to controls in multiple categories. Results were similar for both the intent-to-treat analysis (includes residents overly qualified for the study) and the subanalysis of only the qualified residents (Table 2). For the sake of simplicity, we specifically focused on the results of the 64 qualified residents.

TABLE 2.

Global Scale of Operative Performance for first cystoscopy

All N = 68 Qualified n = 64	Control (±SE) n = 35 n = 33	Imagery (±SE) n = 33 n = 31	Pa
Total score–LSM
All	14.71 (.77)	17.05 (.81)	.02
Qualified	14.4 (.78)	16.70 (.81)	.03
Knowledge of procedure–LSM (subscale of total score)
All	2.50 (.15)	2.98 (.16)	.01
Qualified	2.40 (.15)	2.85 (.15)	.02
Knowledge of instruments–LSM (subscale of total score)
All	2.50 (.15)	3.00 (.16)	.01
Qualified	2.38 (.15)	2.86 (.16)	.02
Flow of procedure–LSM (subscale of total score)
All	2.25 (.16)	2.98 (.17)	.0006
Qualified	2.16 (.16)	2.87 (.17)	.002
Respect for tissue–LSM (subscale of total score)
All	2.98 (.16)	3.03 (.17)	.82
Qualified	3.03 (.16)	3.12 (.17)	.67
Time and motion–LSM (subscale of total score)
All	2.26 (.16)	2.53 (.16)	.17
Qualified	2.28 (.16)	2.54 (.16)	.22
Instrument handling–LSM (subscale of total score)
All	2.21 (.16)	2.55 (.16)	.09
Qualified	2.18 (.15)	2.47 (.16)	.16

LSM, least square means.

All subjects (intention-to-treat results) and qualified subjects only (excludes overly qualified residents).

^aAnalysis of variance.

The imagery group’s LSM score, group means adjusted for institution, was 2.3 points superior to controls (Table 2). Thus, the imagery groups GSOP total score was 15.9% higher than controls. The mental imagery group’s ratings were superior to controls in 3 of 6 individual questions contributing to the overall performance rating: knowledge of instrument, flow of operation, and knowledge of procedure (Table 2).

The imagery group also scored higher than controls in 1 of the 3 additional performance measures: preparedness for the procedure (Table 3). There was no difference between imagery and control groups’ competence to perform the procedure independently and time to complete the procedure (Table 3). The imagery residents rated the helpfulness of their preparation for the first cystoscopy much higher than the control group rated their preparation (Table 3).

TABLE 3.

Additional results for first cystoscopy: qualified subjects only (N = 64)

Residents who met study criteria N = 64	Control (±SE) n = 33	Imagery (±SE) n = 31	P
Preparedness–LSM preparedness score (±SE)	2.31 (.16)	2.81 (.17)	.02a
Time for procedure–LSM, min (±SE)	9.65 (.65)	10.08 (.67)	.61a
Can independently perform procedure	3 (9%)	6 (19%)	.15b
Preoperative helpfulness–LSM, VAS score (±SE)	4.04 (.51)	7.46 (.44)	< .0001a

LSM, least square means; VAS, visual analog scale.

^aAnalysis of variance;

^bFisher exact test.

The imagery group’s overall performance rating for the second cystoscopy did not differ from controls’ (19.04 ±.89 vs 17.18 ± .85; P = .10). For the second cystoscopy, cystoscopy evaluators again rated the imagery group as more prepared for the procedure (3.13 ± .16 vs 2.69 ±.15; P =.03) and the imagery residents again rated their preoperative preparation’s usefulness higher than controls (7.40 ± .49 vs 3.77 ± .59; P < .0001) (Table 4).

TABLE 4.

Global Scale of Operative Performance for second cystoscopy: qualified subjects only (N = 64)

Qualified N = 64	Control (±SE) n = 33	Imagery (±SE) n = 31	P a
Total score–LSM	17.18 (.85)	19.04 (.89)	.10
Knowledge of procedure–LSM (subscale of total score)	2.63 (.17)	3.17 (.18)	.02
Knowledge of instruments–LSM (subscale of total score)	2.97 (.16)	3.44 (.17)	.03
Flow of procedure–LSM (subscale of total score)	2.77 (.18)	3.21 (.18)	.06
Respect for tissue–LSM (subscale of total score)	3.27 (.13)	3.50 (.13)	.17
Time and motion–LSM (subscale of total score)	2.74 (.14)	3.11 (.14)	.04
Instrument handling–LSM (subscale of total score)	2.57 (.14)	3.12 (.14)	.003

LSM, least square means.

^aAnalysis of variance.

Imagery and control groups differed in compliance in performing their preoperative preparation. More imagery subjects complied with their assignment than controls (Table 1). This finding also persisted for the second cystoscopy (Table 1). Of the 64 qualified residents, 19 of 28 (68%) controls vs 25 of 26 (92%) of the imagery group complied with the preoperative preparation (P = .01). Thus, group assignment and compliance with performance of preoperative assignments were highly associated. Ten residents did not answer the question, “Did you perform your assigned preoperative preparation?” There was no difference between number of imagery and control subjects who did not answer this question (5/31 = 16% and 5/33 = 15%, respectively; P = 1.0). Results were similar in the intent-to-treat group (Table 1).

We attempted to perform a subanalysis of score results for the 54 residents who gave information regarding compliance with preoperative preparation. However, due to loss of 10 residents’ scores and the close association between group and compliance, neither group assignment nor compliance were significant predictors when entered into ANOVA simultaneously (P = .38 and P = .07, respectively). Thus the effects of the 2 confounders, group and compliance, could not be evaluated separately.

Comment

The surgical education environment is changing. Medical educators, including the Accreditation Council for Graduate Medical Education, have shifted focus of surgical education from lectures and lists of learning objectives to surgical competency.¹⁵ This paradigm shift is exemplified by the work of Reznick,¹³ who developed and validated the GSOP to score technical prowess during a structured examination, the Objective and Structured Assessment of Technical Skills (OSATS).¹⁴ OSATS is most commonly administered to residents using bench models. Several studies have shown OS-ATS has improved resident surgical performance.^16,17 Although OSATS is effective, it is labor-intensive. Our goal was to develop a low-technology method that could complement bench models and could apply to a wide range of training programs.

Mental imagery is a relatively simple teaching technique that may complement OSATS and other more complex preoperative training. Mental imagery improved performance of medical students’ suturing of animals and performance of cricothyrotomy on mannequins.^7,9,10 Our goal was to determine whether mental imagery enhanced resident surgical performance on patients in the operating room. We found mental imagery improved cystoscopic performance of residents novice to the procedure. The 2.3-point difference in total scores represents a 15.9% improvement in scores comparing imagery group to control. Residents who practiced mental imagery preoperatively performed superiorly to residents who did not.

Preoperative mental imagery was clearly helpful for the first cystoscopy evaluated. This advantage did not persist for the second cystoscopy, most likely because the actual execution of the procedure overwhelmed the influence of mental imagery on performance of this relatively simple procedure. This is consistent with findings in sports literature that show mental practice is less effective than physical practice in enhancing performance over time.³ For novices, mental imagery seems to be more effective in learning cognitive tasks compared to physical tasks.³ Because cystoscopy is a relatively simple procedure, it may have a smaller cognitive component compared to lengthier operations. It is possible that for novice surgeons, mental imagery would show greater effect if performed prior to more complicated procedures with larger cognitive requirements.

There are several limitations to our study. First, because the GSOP had been validated previously, we anticipated little variation between institutions. To avoid variation in scoring, cystoscopy evaluators underwent standardized training with a DVD illustrating novice, intermediate, and expert cystoscopies that demonstrated high interrater reliability.^12–14 Nonetheless, like others, we found significant interrater differences in GSOP evaluations, which suggests resident grading is influenced by factors other than actual procedure performance.¹⁶ We controlled for variation between institutions using ANOVA. Future work might better standardize rater evaluations by developing a larger teaching set of graded cystoscopy videos to be viewed prior to initiating the study.

Last, the mental imagery group complied with their preoperative preparation much more consistently than controls. Fortunately, our questionnaire was sufficiently extensive to detect this potential confounder. We did not foresee, however, that significant numbers of residents would also fail to respond as to whether they had complied with their assigned training. By decreasing our sample size this omission prevented us from detecting group differences in performance after controlling for the linkage between group and compliance with preparation.

We chose textbook preparation for the control group because it is a traditional method of preparing for surgery. It also is an example of pure verbal cognition and encoding that contrasts with mental imagery’s use of both verbal and nonverbal encoding. The poor compliance with textbook study was unforeseen but is relevant to actual resident preparation in a training institution. Because of the time burden of resident duties, residents may not successfully incorporate textbook reading into preparation for surgery. Mental imagery is a method of preparation that is time-efficient, individualized, and, because of faculty involvement, more likely to be incorporated into a busy schedule. Although we did not study this here, mental imagery could also be used as a method of self-preparation.

The acceptance of mental imagery by trainees argues for its use in preoperative preparation. Our study shows that trainees in 6 accredited resident training programs repeatedly favored mental imagery over reading and were more likely to prepare for the procedure using this method. In the day-today practice of academic medicine, it may not matter whether it is the teaching technique itself or the likelihood that residents will adopt that technique that ultimately results in improved surgical performance.

There are several important strengths of this study. Its multi institutional design suggests the findings might be generalized to other educational centers. The simplicity of the imagery sessions makes it replicable elsewhere; the preoperative sessions lasted < 20 minutes and the assessment sheet is a single page. The procedure is cost-efficient and requires no additional equipment. Lastly, this study is the first trial evaluating mental imagery as a preoperative practice in a clinical setting involving physicians.

In summary, our study demonstrates the practical use of mental imaging in a real-life surgical setting. In the educational arena, the potential effect of resident work hour’s limitations on surgical caseloads obligates us to seek cost- and time-efficient means of improving residents’ surgical skills. Now and increasingly in the future, these pressures will not be limited to residency training. With the constant introduction of new procedures, all surgeons are challenged to remain current. As perpetual physicians in training, formal mental imagery practice offers the potential of enhancing surgical performance for all.