Impact of Video Game Cross-Training on Learning Bronchoscopy. A Pilot Randomized Controlled Trial

Abstract

Background: Video game playing requires many of the same skill sets as medical procedures such as bronchoscopy. These include visual–spatial awareness, rapid decision making, and psychomotor skills. The role of video game cross-training on learning bronchoscopy is unknown.

Objective: We studied the association of baseline video gaming experience with, and the impact of short-term video game playing on, visual–spatial awareness and acquisition of basic bronchoscopic skills among medical trainees.

Methods: Bronchoscopy-naive medical trainees underwent formal didactic and hands-on instruction on basic bronchoscopy, along with a baseline assessment measuring bronchoscopic and visual–spatial skills. Half of the subjects were subsequently randomized to playing a videogame (Rocket League) for 8 weeks. All participants returned at 4 weeks for a refresher course and at 8 weeks for a final assessment.

Results: Thirty subjects completed the study, 16 of them in the intervention arm who all met the minimum video game playing time requirement. At baseline, video game players had significantly lower airway collision rates (6.82 collisions/min vs. 11.64 collisions/min; P = 0.02) and higher scores on the Purdue Visual Spatial Test: Visualization of Rotations test (27.5 vs. 23.54; P = 0.04). At completion, the intervention group had no significant differences in airway collisions, bronchoscopy time, or Bronchoscopy Skills and Tasks Assessment Tool scores. There was moderate correlation between airway collision rate and mean Purdue Visual Spatial Test: Visualization of Rotations score (Spearman’s rho, −0.59; P < 0.001).

Conclusion: At baseline, learners with former video game–playing experience have higher visual–spatial awareness and fewer airway collisions. The impact of video game playing as an aid to simulation-based bronchoscopic education is uncertain.

Bronchoscopy is a procedure that requires dexterity, rapid decision making, and spatial awareness. More advanced bronchoscopic techniques, such as robotic and navigational bronchoscopy and therapeutic bronchoscopy, may require development of even higher levels of psychomotor and rapid information-processing skills. In athletics, cross-training in an unrelated sport or exercise is used to improve performance (1). This concept has crossed over into healthcare, such that surgery and aviation now use similar cross-training tools in the form of video games and/or simulators (2).

The use of video games has a negative social stigma associated with presumed negative effects on health and education, in addition to being implicated in aggressive behaviors (3). On the other hand, video games have been shown to hone attributes that could be beneficial to training physicians for medical procedures. These benefits include improved hand–eye coordination (4, 5), reaction time (6), visuospatial orientation (7), information processing (8), and performance on neuropsychological tests (9). Improved psychomotor outcomes have already been demonstrated in laparoscopic surgery and gastrointestinal endoscopy through the use of simulators (10–17).

Although these promising findings have been demonstrated in surgical residents, there have been no prospective studies examining the effects of video games on procedural skills among nonsurgical trainees to date. The purpose of this study was to examine the association of baseline video gaming experience with, and the impact of short-term video game playing on, visual–spatial awareness and acquisition of basic bronchoscopic skills among medical trainees. Although there are already ample data to support simulation in bronchoscopy training, this study seeks to explore an accessible supplement (video games) to simulation and bronchoscopy training as an additional strategy to improve bronchoscopic skills in early adopters.

Methods

This randomized controlled single-blinded trial was reviewed and approved by the Johns Hopkins Institutional Review Board (IRB00142377). Subjects were junior medical trainees (medical students or internal medicine residents) without any surgical training or experience and with little or no exposure to bronchoscopy (having witnessed anywhere from zero to two bronchoscopy procedures). Individuals were recruited by e-mailing institutional resident and medical student listserves. All subjects provided written informed consent.

All subjects underwent a half-day standardized course on introduction to bronchoscopy, modeled after the Essential Bronchoscopist (18) curriculum and the introduction to bronchoscopy curriculum designed for first-year pulmonary fellows at our institution. This curriculum consisted of didactic teaching on basic airway anatomy coupled with hands-on training in basic bronchoscopic skills (airway inspection, brushing, forceps biopsy, and bronchoalveolar lavage [BAL]) using both low-fidelity plastic models and a high-fidelity bronchoscopy simulator (CAE EndoVR). On the same day as the introduction to bronchoscopy course, all subjects were asked to complete the revised Purdue Spatial Visualization Tests: Visualization of Rotations (PSVT:R) (19) assessment and undertake a bronchoscopic skills assessment on the high-fidelity simulator, which included the Bronchoscopic Skills and Tasks Assessment Tool (BSTAT) that was graded by an independent board-certified pulmonologist.

The BSTAT is a validated assessment tool for competency in bronchoscopy, commonly used for bronchoscopic assessment of pulmonary fellows and pulmonologists, which has been validated on virtual bronchoscopy (20). It comprises a checklist of tasks used to assess competency. It includes a full airway examination where the bronchoscopist first identifies and enters all pulmonary segments one by one, then identifies and enters certain specific segments as instructed by the assessor, and last performs basic sampling maneuvers, including a BAL, brushing, and forceps biopsy. During these tasks, the assessor is monitoring the bronchoscopist’s body positioning, equipment handling, hand positioning, and airway wall trauma. On completion of the bronchoscopic portion, there is a standardized quiz of pictures of airway pathology for the subject to complete. The scores of all of these assessments pool into a final score out of 100. Usage of the BSTAT helps ensure a standardized series of tasks for each subject to complete. The BSTAT comes coupled with the Bronchoscopic Self-Assessment Tool (BSAT), which allows subjects to rate themselves on multiple aspects of bronchoscopy, with a higher score implying greater self-assessed competence and a maximum score of 50.

The PSVT:R is a visual–spatial assessment tool where the subject is given 30 standardized questions. Each question consists of a three-dimensional object drawn on a two-dimensional surface and then a rotated version of the same. The subject is then presented with a second object and asked to choose from five choices on how to rotate the object in the same manner as the initial object.

After baseline instruction and baseline testing, subjects were randomized in a 1:1 ratio to either intervention or control using the sealed envelope method. The intervention group was asked to play Rocket League (Psyonix) on the Nintendo Switch (Nintendo) for 2.5 hours every week over the next 8 weeks. Our choice of video game and video gaming system was based on the need to require high visual–spatial awareness, hand–eye coordination, and information processing. Furthermore, the controller was selected to mimic the control systems of conventional bronchoscopy. Using the parental control features available on Nintendo Switch, each subject’s playing time was remotely monitored by the investigators to ensure compliance with the designed intervention. All subjects, including control subjects, returned 4 weeks later for an anatomy review and hands-on practice using the bronchoscopy simulator (not graded). Subjects returned another 4 weeks thereafter to complete the final assessment, which involved the revised PSVT:R as well as the bronchoscopy simulator and BSTAT assessment.

Each subject completed the same module and “test case” on the bronchoscopy simulator at both time points to avoid measurement bias. As before, performance was graded using the BSTAT checklist by an independent board-certified pulmonologist who was blinded to experimental allocation. Automatically calculated performance metrics, including airway wall collisions, total bronchoscopy time, and the percentage of airways successfully entered, were also recorded. All measures were summarized using means and percentages as appropriate. Independent sample t tests were used to compare populations at baseline and after intervention. Paired t tests were used to compare subject-wise pre- and postintervention scores within each group. Linear and logistic regression analyses, including analysis of covariance, were conducted to control for baseline differences between groups. Airway collision rate (collisions per minute) was determined using the bronchoscopy simulator metrics by dividing the total number of airway wall collisions by the total bronchoscopy time for each subject. An active video game player (VGP) was defined as a subject who reported ongoing video gaming of >1 h/wk at the time of study enrollment. The study was 80% powered to detect a difference of 10 on the BSTAT score, with α at 0.05. All statistical tests were two-sided, keeping α at 0.05, and performed using Stata software version 14.2.

Results

We enrolled 32 subjects, including 26 medical students and 6 internal medicine residents. Eight were baseline VGPs. Sixteen were randomized to the intervention group. Thirty subjects completed the study. One subject failed to follow up for final assessment and could not be contacted, and another subject dropped out during initial assessment because of competing commitments; these two subjects (both assigned to the control arm) were excluded from analyses. Baseline characteristics of the included subjects are shown in Table 1. There were more VGPs than non-VGPs (43.8% vs. 7.1%; P = 0.02). All subjects assigned to the intervention arm completed the minimum prescribed gaming time as remotely monitored by investigators using the Nintendo Switch parental control features.

Table 1.

Baseline characteristics of all study participants classified by experimental arm

Variable	Intervention (n = 16)	Control (n = 14)	P Value
Age, yr, mean	25.1	26.4	0.35
Year in training, mean*	2.81	2.57	0.76
Female	25	28.6	0.83
Right-handed	93.8	92.9	0.92
Plays an instrument	62.5	57.1	0.77
Athlete	75.0	57.1	0.32
Juggles	0	21.4	0.05
Artist	12.5	14.2	0.89
Can write blindly†	68.8	57.1	0.52
Active video game player	43.8	7.1	0.02

Data presented as percentage unless otherwise noted.

^*Denotes year 1 as a first-year medical student; year 5 would denote a first-year resident.

^†Refers to the self-reported ability to take written notes without looking at the paper (recorded as yes/no, reported here as percentage of subjects who said yes).

Baseline Comparison between Active VGPs and Non-VGPs before Randomization

We analyzed the baseline statistics of the VGPs and non-VGPs, which are illustrated in Table 2. On the bronchoscopy simulator, the total amount of time on the simulator was less for VGPs versus non-VGPs (516.37 s vs. 577.39 s; P = 0.28). VGPs also had fewer airway collisions (mean collisions per bronchoscopy, 57 vs. 111.8; P = 0.01). The difference in mean collision rate was also significant (6.8 collisions/min vs. 11.6 collisions/min; P = 0.02) (Figure 1). These differences remained significant when controlling for sex. The percentage of airways entered was similar between groups (49.5% VGP vs. 42.9% non-VGP; P = 0.17). BSTAT scores were similar between both groups (56.4 VGP vs. 55.8 non-VGP; P = 0.87), as were BSAT scores (25.0 vs. 26.3; P = 0.44). VGPs had significantly higher revised PSVT:R scores than the non-VGP group (27.2 vs. 23.5; P = 0.04), which was not significant when controlling for sex (P = 0.08).

Table 2.

Baseline performance of VGPs and non-VGPs before randomization

Outcome	VGP (n = 8)	Non-VGP (n = 23)	P Value
Collision rate, collisions/min	6.82	11.64	0.02
No. of collisions	57	111.78	0.01
Bronchoscopy time, s	516.37	577.39	0.28
Percent airways entered	49.50	42.96	0.17
Revised PSVT:R score	27.25	23.54	0.04
BSTAT score	56.37	55.78	0.87
BSAT Score	25.0	26.35	0.44

Definition of abbreviations: BSAT = Bronchoscopic Self-Assessment Tool; BSTAT = Bronchoscopic Skills and Tasks Assessment Tool; PSVT:R = Purdue Spatial Visualization Tests: Visualization of Rotations; VGP = video game player.

Data presented as means.

Figure 1.

Airway collision rates at baseline, classified by baseline gaming status. Lower airway collision rates were noted among participants who were active video game players (VGPs) at the time of enrollment.

Outcomes following Randomization

Within the intervention group, there was a nonsignificant trend toward reduced airway collisions at 8 weeks (pre and post difference in means, −25.44 collisions; P = 0.18). Among control subjects, the mean difference was −6.57 collisions (P = 0.53). Mean total airway collisions were similar in both groups at 8 weeks (P = 0.39). Reductions in bronchoscopy time were similar as well (mean change in bronchoscopy time for intervention and control groups being −105.44 s and −112.93 s, respectively; P = 0.90). Outcome metrics comparing the two groups are detailed in Table 3. There was no interaction between assignment to intervention and video gaming time.

Table 3.

Outcomes among intervention and control groups

Outcome	Intervention (n = 16)	Control (n = 14)	P Value
Collision rate, initial	10.60	10.69	0.97
Collision rate, final	10.52	11.96	0.45
Collision rate, change	−0.08	1.27	0.51
Total collisions, initial	105.38	94.43	0.61
Total collisions, final	79.94	87.86	0.61
Total collisions, change	−25.44	−6.57	0.39
Bronchoscopy time, initial, s	568.38	566.50	0.97
Bronchoscopy time, final, s	462.94	453.57	0.85
Bronchoscopy time, change, s	−105.44	−112.93	0.90
Percentage of airways entered, initial	46.0	42.71	0.53
Percentage of airways entered, final	38.31	50.35	0.03
Percentage of airways entered, change	−7.69	7.64	0.01
PSVT:R score, initial	24.13	24.62	0.79
PSVT:R score, final	25.44	24.08	0.57
PSVT:R score, change	1.31	−0.54	0.15
BSTAT score, initial	57.0	54.14	0.66
BSTAT score, final	67.5	56	0.11
BSTAT score, change	10.5	1.86	0.19
BSAT score, initial	25.94	25.64	0.89
BSAT score, final	31.69	27.93	0.29
BSAT score, change	5.75	2.29	0.24
BSAT change, cognitive, %	1.2	−2
BSAT change, pure motor, %	8.2	5.6
BSAT change, mixed cognitive/motor, %	0.3	2.4

Definition of abbreviations: BSAT = Bronchoscopic Self-Assessment Tool; BSTAT = Bronchoscopic Skills and Tasks Assessment Tool; PSVT:R = Purdue Spatial Visualization Tests: Visualization of Rotations.

Data presented as means unless otherwise noted.

The baseline airway collision rate on the bronchoscopy simulator moderately correlated with the revised PSVT:R scores (Spearman’s rho, −0.59; P < 0.001) (Figure 2), but not with BSTAT scores (Spearman’s rho, −0.10; P = 0.58) (Figure 3), BSAT scores (Spearman’s rho, 0.12; P = 0.51), or combined BSTAT and BSAT scores (Spearman’s rho, 0.21; P = 0.26).

Figure 2.

Association between airway collision rates and revised Purdue Visual Spatial Test: Visualization of Rotations (PSVT:R) scores. Higher PSVT:R scores were associated with lower collision rates.

Figure 3.

Association between airway collision rates and Bronchoscopic Skills and Tasks Assessment Tool (BSTAT) scores. Higher BSTAT scores were not associated with lower collision rates.

Bronchoscopy simulator metrics (including airway collisions) or BSTAT scores were not associated with juggling ability, ability to write on paper without looking, or being an athlete or musician.

Discussion

Our study was the first to our knowledge to investigate the impact of video gaming on bronchoscopy learning by nonsurgical trainees (some of whom would go on to become pulmonologists and bronchoscopists). We demonstrated that subjects who regularly played video games at baseline performed better, with fewer airway collisions and superior visual–spatial abilities. Similar findings have also been described in other fields, including surgery and aviation (17, 21), but to our knowledge this is the first attempt at studying medical trainees outside of surgical training programs. Without accounting for various types of games, consoles, and controllers, video gaming in general was associated with better baseline performance in our study, unlike other activities such as playing a musical instrument or engaging in athletics.

On the other hand, our short-term study intervention (cross-training on videogames, 2.5 h/wk for 8 wk) did not measurably add to simulation-based education (which was delivered twice: at baseline and again at 4 weeks) in improving performance of basic bronchoscopy and related tasks. In a study investigating the relationship between performance of robotic surgery and video gaming experience, only those with heavy video gaming experience showed superior performance (20). It is unclear if a higher dose of our intervention would have similarly yielded tangible improvements in the studied outcomes or if video gaming would aid the learning of more advanced bronchoscopic tasks (such as navigational and robotic bronchoscopy or endobronchial ultrasound). Although it is possible that additional video game playing (beyond 8 wk and/or 2.5 h/wk) may improve performance further, it may not be realistic to commit such time for an already demanding medical trainee’s schedule.

Although previous authors have reported an association between video gaming experience and surgical performance (14, 16, 17), there have been very few studies examining the mechanisms explaining these findings. It is unclear if baseline VGPs have developed better visual–spatial awareness as demonstrated by the higher PSVT:R or if intrinsically higher spatial awareness attracted them to play video games in the first place. At the end of our intervention, there was an improvement in PSVT:R scores among non-VGPs as well, suggesting that playing video games may improve visual–spatial awareness. However, this did not translate into better bronchoscopy skills by our metrics (BSTAT), suggesting that performance of a simple bronchoscopy may not be significantly affected by visual–spatial awareness. However, the BSTAT assessment, which is predominantly purely cognitive (56%) rather than psychomotor (34%) or mixed skills (10%), may not have been the optimal choice in bronchoscopy metrics. For this reason, we chose to use additional metrics of psychomotor skills, including collision rate and PSVT:R. In previous studies, improvements in characteristics such as hand–eye coordination (4, 5), visuospatial orientation (7), and information processing (8) were all associated with playing video games. In laparoscopic surgery studies, using videogames did improve specific skill sets with video game playing, such as using instruments to transfer beads, whereas others did not (suturing) (21).

In our study, both the intervention and control groups showed significant improvements in learning bronchoscopy with simulation. One question not addressed by our study is whether video gaming could be used in place of simulation-based education to achieve the same educational outcomes. The benefit of using video games over simulators would be accessibility and cost; however, there are scant data available for use of video games for this purpose and none with bronchoscopy to date. A future study may consider an arm with video games and didactics only or perhaps a video game of bronchoscopy. On the basis of existing literature, video games seem to have a stronger effect if the involved images and tasks are similar to the medical procedure being taught. Also, the choice of gaming controller seems to play a role, as several laparoscopic studies using different controllers seem to illustrate a benefit in certain skills but not others (21). We took these factors into account when making our choice of the video game and gaming controller.

We used validated tools to assess basic bronchoscopy competence (BSTAT) and visual–spatial orientation (PSVT:R) in our study. When we correlated these tools with airway collision rates, the PSVT:R was moderately correlated, whereas the BSTAT was not (Figures 2 and 3). The BSTAT is focused on overall competency and tests predominantly knowledge over psychomotor skills, which may explain this poor correlation. It is also worth noting that there was a nonsignificant trend toward improvements in visual–spatial examination and bronchoscopy simulator metrics in the intervention group but not in BSTAT scores. Considering this, an important takeaway for future research would be that the PSVT:R may potentially be an alternative assessment tool for predicting bronchoscopy-relevant psychomotor skills among novice bronchoscopists. This is not surprising, because novice bronchoscopy learners often focus on being able to keep the bronchoscope centered and to properly maneuver the scope through the airways. The BSTAT focuses on comprehensive competency, with the majority of points counting toward cognitive over motor concepts.

The ability of video gaming to positively impact health outcomes has been shown in surgery before (10, 14, 16, 17, 22). In one study, playing video games led to 37% fewer errors and 27% faster completion on a laparoscopic surgery simulator. Furthermore, a prospective study of surgical residents found that cross-training via a first-person shooter video game, as opposed to playing virtual chess or no video gaming, led to improved performance on both laparoscopic and gastrointestinal endoscopy simulators (17). On the basis of our study, it appears that video gaming may have limited utility in the training of novice bronchoscopists—at least in instances in which simulation-based education is simultaneously provided. Notably, unlike our study, none of the aforementioned studies used blinded outcome assessors. Because there have been several studies showing no significant differences in high-fidelity and low-fidelity simulators, an interesting comparative study would be between low-fidelity simulators and video games, because they may share certain advantages, including availability and possibly cost (23, 24). Furthermore, as video games continue to become more sophisticated, with increasing use of virtual reality and tactile feedback, these technological advances may potentially contribute to procedural cross-training.

This study has several limitations. We studied a diverse pool of medical trainees rather than specifically those internal medicine residents who planned to enter the field of pulmonology and would therefore become future bronchoscopists. Therefore, the applicability of our findings to novice bronchoscopy learners may be limited. Furthermore, the small sample size limited our ability to achieve an optimal level of parity between groups despite randomization (Table 1), prompting the covariate analysis. Associated with a small sample size, our cohort may have a selection bias, as future surgeons or proceduralists may have been more inclined to join this study. This would also likely be reflected in the real world, as proceduralists may gravitate to this subspecialty. In addition, the repeated assessments (8 wk later) on the same simulator may create a potential performance bias, as real-world cases are rarely exactly the same. Also, we could not control for individual level of motivation in performing well on the simulator. However, previous investigators have found that visual–spatial ability may impact performance on a simulator more substantially than motivation (25).

Conclusions

Medical trainees with ongoing video gaming experience have better baseline bronchoscopy skills, demonstrated by fewer airway collisions and better visual–spatial skills. Future efforts should be aimed at studying larger and more specific subject cohorts over a longer period of time to permit further assessment of the effects of video gaming along with their durability.