Abstract
Background
Currently there is no consensus on ideal teaching method to train novice trainees in EBUS. Simulation-based procedure training allows direct observation of trainees in a controlled environment without compromising patient safety.
Objective
We wanted to develop a comprehensive assessment of endobronchial ultrasound (EBUS) performance of pulmonary fellows and assess the impact of a multimodal simulation-based curriculum for EBUS-guided transbronchial needle aspiration.
Methods
Pretest assessment of 11 novice pulmonary fellows was performed using a three-part assessment tool, measuring EBUS-related knowledge, self-confidence, and procedural skills. Knowledge was assessed by 20 multiple-choice questions. Self-confidence was measured using the previously validated EBUS–Subjective Assessment Tool. Procedural skills assessment was performed on Simbionix BRONCH Express simulator and was modeled on a previously validated EBUS–Skills and Task Assessment Tool (EBUS-STAT), to create a modified EBUS-STAT based on internal faculty input via the Delphi method. After baseline testing, fellows participated in a structured multimodal curriculum, which included simulator training, small-group didactics, and interactive problem-based learning sessions, followed by individual debriefing sessions. Posttest assessment using the same three-part assessment tool was performed after 3 months, and the results were compared to study the impact of the new curriculum.
Results
The mean knowledge score improved significantly from baseline to posttest (52.7% vs. 67.7%; P = 0.002). The mean EBUS–Subjective Assessment Tool confidence scores (maximum score, 50) improved significantly from baseline to posttest (26 ± 7.6 vs. 35.2 ± 6.3 points; P < 0.001). The mean modified EBUS-STAT (maximum score, 105) improved significantly from baseline to posttest (44.8 ± 10.6 [42.7%] vs. 65.3 ± 11.4 [62.2%]; P < 0.001). There was a positive correlation (r = 0.81) between the experience of the test participants and the modified EBUS-STAT scores.
Conclusion
This study suggests a multimodal simulation-based curriculum can significantly improve EBUS-guided transbronchial needle aspiration–related knowledge, self-confidence, and procedural skills among novice pulmonary fellows. A validation study is needed to determine if skills attained via a simulator can be replicated in a clinical setting.
Keywords: endobronchial ultrasound, simulation training, pulmonary fellows
Endobronchial ultrasound–guided transbronchial needle aspiration (EBUS-TBNA) is recommended as the first-line procedure for the diagnosis and mediastinal staging of lung cancer by multiple medical societies. However, debate still exists on methods to effectively train and measure EBUS-TBNA performance in trainees (1–4, 6). Compared with the short and steep learning curve for conventional TBNA (7–9), the learning curve for EBUS-TBNA is longer and more complex, resulting in the need for extensive training and experience (4–6, 10). Currently, EBUS-TBNA proficiency is either judged by procedure volume or determined by direct observation, both of which vary widely between institutions worldwide (11). Volume-based certification may be arbitrary, because individuals learn at different speeds, and it is problematic for smaller institutions that serve an insufficient number of patients to meet the threshold for each trainee (12). In observation-based assessment, the lack of a structured protocol introduces potential bias of supervisors because it relies on the individual supervisor’s level of expertise and experience. A 2015 CHEST consensus statement suggested changing to a system that assesses skill acquisition and knowledge by incorporating tools like simulation (13–15). Despite this, a national survey of U.S. pulmonary fellowship program directors revealed only 30% of programs used a structured assessment strategy to evaluate EBUS competency (16).
Currently, there is no consensus on the ideal teaching method for EBUS training in novice learners. Most pulmonary fellowships currently use the traditional apprenticeship model for EBUS-TBNA training, supplemented with some secondary teaching methods (i.e., literature review, videos, and didactics). However, this training model has significant disadvantages in detecting and addressing gaps in trainees’ knowledge and skill, due to overreliance on trainees to recognize their deficiencies and the supervisor’s ability to identify those gaps and teach accordingly. During EBUS-TBNA, the patient’s safety takes precedence, resulting in the supervisor often taking over and interrupting the trainee’s experience or assessment. Trainee involvement during EBUS can also increase procedural duration, anesthesia dose, and complication rates (17, 18). Simulation-based training can reduce the learning curve that novice operators need to conduct an independent, successful EBUS-TBNA (19). EBUS simulators can also accurately discriminate between operators at different skill levels and experience and is one of several methods recommended to assess trainees and help achieve proficiency, complementing traditional apprenticeship training models (20, 21).
Based on this, we wanted to 1) develop a comprehensive assessment of EBUS performance of pulmonary fellows in training at our institution; 2) implement a new multimodal simulation-based EBUS training curriculum; and 3) test the effectiveness of this training curriculum.
Methods
Curriculum Development
For this project, we formed an internal EBUS Expert Committee (EEC) with four teaching faculty physicians in the Division of Pulmonary and Critical Care at our tertiary level hospital and training program. Members of the EEC were considered at the expert level, because each independently performed more than 200 EBUS-TBNA procedures (20). One senior pulmonary fellow designated in the Clinician-Educator track developed this study with the EEC and two simulation-center medical directors. The Clinician-Educator track fellow had received training and experience in other simulation-based assessments and debriefing before this study. Knowledge gaps among our trainees were identified through a needs-assessment survey of the teaching faculty. Based on this survey, we identified deficiencies such as anatomical identification of segmental bronchi and lymph node stations and best practices for procedural technique. We then targeted these deficiencies via a newly created EBUS curriculum to improve EBUS-TBNA–related knowledge, self-confidence, and procedural skills. This study was declared exempt by the institutional review board.
Learning Outcomes and Measures
Three learning outcomes were measured to evaluate the efficacy of the EBUS curriculum: knowledge, self-confidence, and procedural skills. Multiple-choice questions (MCQs) were used for EBUS-related knowledge assessment. Twenty-five MCQs were selected from an online pool of previously validated MCQs by expert bronchoscopists (The Essential EBUS Bronchoscopist, access at http://www.bronchoscopy.org) to address topics per the faculty’s needs-assessment survey (lymph node station anatomy, lung cancer staging, and EBUS procedure technique). To refine the MCQ tool, it was tested on a focus group of the three EEC faculty mentors and three graduating pulmonary fellows (post graduate year [PGY] VI), who were trained in the traditional apprenticeship method. Each of the three graduating pulmonary fellows performed at least 35 EBUS-TBNA procedures at the time of testing. The questions were rated by difficulty based on the focus group’s responses. The difficulty index or ρ of a test item is the proportion of a group of test-takers who responded incorrectly. For example, ρ = 90% is a very difficult question, whereas ρ = 10% is very easy. Questions that had either ρ ⩾ 75% among experts or ρ ⩽ 25% among the graduating PGY-VI class were eliminated. The resulting 20 MCQs were used for the knowledge assessment of the learners in this study.
We included subjective self-assessments as a measurable outcome because the EBUS–Subjective Assessment Tool (EBUS-SAT) (see online supplement) has been previously validated as a tool to measure the change in EBUS skills (22). It also allows trainees to provide feedback on the curriculum. In EBUS-SAT, trainees rate their ability to perform 10 different EBUS-related tasks using a 5-point Likert confidence scale.
Our objective assessment was modeled on the EBUS Skills and Tasks Assessment Tool (EBUS-STAT), a validated 10-item assessment tool developed in 2012 (23). Permission was obtained from one of the tool’s creators (Dr. Henri Colt) before modifying it for our study. Two validated objective assessment tools exist to evaluate EBUS-TBNA: the EBUS Assessment Tool and the EBUS-STAT (22, 23). We chose the latter as a template because it contained more relevant details of TBNA assessment for a novice trainee, was designed as a potential screening tool to assess fundamental skills, and was flexible to be modified per local requirements (24). Certain items in the EBUS-STAT fail to discriminate between novices and experts (24). Therefore, we developed a modified version (mEBUS-STAT) that replaced some items to include tasks measurable by the BRONCH Express simulator (Figure 1) and elements reflective of our institutional practice, while adherent to evidence-based practices of EBUS-TBNA (25). Using the Delphi survey method, the EEC reached a consensus after three survey cycles on the final modifications of the objective assessment tool. The differences between the original EBUS-STAT and the mEBUS-STAT we used are summarized in Table 1.
Figure 1.
BRONCH Express simulator dual-screen display. A = endobronchial ultrasound bronchoscopy view; B = white light bronchoscopy view.
Table 1.
Differences between the original Endobronchial Ultrasound Skills and Tasks Assessment Tool (EBUS-STAT) and the modified EBUS-STAT
| Original EBUS-STAT 10-Point Assessment Tool | Modified EBUS-STAT |
|---|---|
| 1. Able to maneuver the scope through the upper airway into the trachea without trauma or difficulty (5 points for single item tested) | Not included*† |
| 2. Able to maneuver scope using white light bronchoscopy within a tracheobronchial tree without trauma (4 points, no partial points) | Not included† |
| 3. Ultrasound image obtained without artifacts (5 points, no partial points) | Not included‡ |
| 4. Identify major mediastinal vascular structures (4 points per item) | Retained |
| 5. Identify lymph node station (Select 3 targets, 5 points each) | Modified (trainee must identify all 10 targets, 2 points each)§ |
| 6. Able to demonstrate EBUS processor functions (2 points for each item) | Not included‖ |
| 7. Performance of EBUS-TBNA (1 point each, target 15 points) | Modified (target 20 points, 2 points each)¶ |
| 8. Image analysis: CT scans (1 point each, target 10 points) | Retained |
| 9. Image analysis: EBUS views (1 point each, target 10 points) | Retained |
| 10. Decision-making tasks (2 points each, target 10 points) | Retained |
Definition of abbreviations: CT = computed tomography; EBUS = endobronchial ultrasound; EBUS-STAT = Endobronchial Ultrasound Skills and Tasks Assessment Tool; EBUS-TBNA = EBUS-guided transbronchial needle aspiration; EEC = EBUS Expert Committee.
Not reflective of local institutional practice (most of our EBUS-TBNA procedures are performed after endotracheal intubation).
Noted to be not sensitive in previous studies.
Difficult to measure in a simulator; easy to get clear images with minimal skill required for scope manipulation.
EEC agreed that to complete trainee assessment on one module, trainees would be expected to identify all 10 lymph node stations (station 2R, 2L, 4R, 4L, 7, 10R, 10L, 11Rs, 11Ri, and 11L).
Difficult to measure in a simulator; unable to modify gain and depth in EBUS simulator software.
EEC agreed on 20-step TBNA process.
The mEBUS-STAT has two parts: knowledge (image recognition and decision-making tasks) and technical skills assessment (lymph node identification, mediastinal vascular structure anatomy, and TBNA technique). The simulation case scenario for TBNA technique assessment was the same for all participants and met the objectives of “simple, obvious” mediastinal adenopathy (see online supplement). The TBNA assessment was limited to one case, because it accomplished our goals within a 1-hour duration. To maintain consistency of the assessment method, the Clinician-Educator fellow guided the learners through the session with a standard script (see online supplement) but otherwise did not interfere. Any verbal or manual intervention was tracked. Feedback was only given during targeted debriefing sessions to validate the trainee’s accomplishments while highlighting opportunities for improvement. Trainees also completed a questionnaire to assess demographic information such as year of training and EBUS-TBNA experience. The trainee assessment was conducted in a controlled environment in the simulation lab. All simulator case modules were developed by 3D Systems and preloaded into the bronchoscopy simulator.
Before implementation, the EEC approved the grading for the final three-part assessment (MCQs, EBUS-SAT, and mEBUS-STAT). Before this project none of the trainees had been assessed formally and were all trained under the apprenticeship model. Approximately 6 months was necessary to perform the needs assessment, develop the assessment tools, and finalize the educational interventions for this curriculum.
Participants and Procedures
Eleven pulmonary fellows (six PGY-V and five PGY-VI) were the target of this study. After their baseline assessment, each participant successfully attended a didactic lecture, problem-based learning (PBL) session, and individual debriefing session over 4 weeks (Table 2). The one-on-one debriefing practice session targeted the deficiencies in their individual EBUS-TBNA technical skills on the simulator.
Table 2.
Endobronchial ultrasound–guided transbronchial needle aspiration assessment and multimodal simulation-based curriculum
| Step | EBUS Course | Allotted Time (min) |
|---|---|---|
| 1 | Knowledge assessment (20 MCQs) with trainee self-assessment (EBUS-SAT) | 15 |
| 2 | 1:1 Objective skills assessment on the simulator (mEBUS-STAT) | 45 |
| 3 | Didactic lecture by faculty | 60 |
| 4 | PBL cases in a flipped classroom model (3 cases) | 60 |
| 5 | 1:1 EBUS simulation debriefing | 60 |
Definition of abbreviations: EBUS = endobronchial ultrasound; MCQ = multiple choice questions; mEBUS-STAT = modified Endobronchial Ultrasound Skills and Tasks Assessment Tool; PBL = problem-based learning; SAT = Subjective Assessment Tool.
Didactics addressed evidence-based practices related to EBUS, including indications, lung cancer staging, lymph node station anatomy, TBNA performance, and slide preparation (26). A flipped classroom model was implemented for the PBL session, because it is better for learner retention than a passive learning model (27–30). PBL sessions facilitate the learners to remain engaged during the session and take ownership of their self-learning, and faculty act as facilitators to challenge the trainees’ thinking without dictating it (31, 32). The cases for the PBL session were chosen after consulting with the EEC and based on the baseline MCQ testing of the learners and included the following topics: lung cancer staging, the role of mediastinoscopy, assessing TBNA results, and positron emission tomography–negative adenopathy (see online supplement). During the PBL session, fellows perform a literature review and discuss their answers in a brief PowerPoint presentation as a small group, with faculty moderating the sessions.
After 3 months, all 11 fellows were retested using the three-part EBUS assessment. All fellows were allowed access to the BRONCH Express simulator to practice, in addition to performing supervised EBUS-TBNA procedures on actual patients in the interval between baseline and posttest.
Equipment
The simulator used in our study was the BRONCH Express (3D Systems) that consisted of a proxy bronchoscope, a proxy EBUS biopsy needle tool, an interface that tracks equipment movements, and a monitor displaying the computer-generated endoscopic and ultrasound images (Figure 1).
Statistics
Results are reported using mean ± SD. Paired two-tailed t test analyses were conducted with the trainees’ posttest scores compared with their baseline scores. For statistical analysis, SPSS was used (version 20; IBM Corp). The significance level was defined as P < 0.05.
Results
We conducted a single cohort pretest-posttest study in July 2019 and included PGY-V (n = 6) and PGY-VI (n = 5) pulmonary fellows from a single-center tertiary care training program. At baseline pretest assessment, basic demographic data showed that 9 of the 11 participating fellows had assisted in fewer than five EBUS-TBNA procedures under the traditional apprenticeship model. The remaining two fellows were involved in fewer than 10 EBUS-TBNA procedures.
Knowledge Assessment
The mean MCQ score improved significantly from a baseline of 52.7–67.7% posttest (10.5 ± 1.4 vs. 13.5 ± 1.6; P = 0.002). To assess the effectiveness of the educational intervention for the group as a whole, the total class-averaged gain (g) was calculated (g = 31.7%). A minimum significant increase of 30% is considered for rating the intervention as effective (33).
Subjective Self-Assessment
The mean EBUS-SAT confidence scores improved from 26 ± 7.6 to 35.2 ± 6.3 points posttest (maximum score, 50; P < 0.001).
Objective Skills Assessment
The mean mEBUS-STAT score improved from 42.7% to 62.2% posttest (44.8 ± 10.6 vs. 65.3 ± 11.4; maximum score, 105; P < 0.001). The learners improved significantly in the bronchoscopic technical skills portion from 38.4% to 62.9% (30.7 ± 9.6 vs. 50.4 ± 10.9; maximum score, 80; P < 0.001). Construct validity for the mEBUS-STAT tool assessment was supported by a positive Spearman correlation between the experience of the operators taking the test (number of EBUS procedures performed or assisted) and the mEBUS-STAT scores when tested with expert members of the EEC and trainees (r = 0.81). The EEC members had a mean mEBUS-STAT of 76.5% (81.3 ± 3.2). Using the contrasting group method, a pass score of 75 was established (34). At the baseline objective assessment, 0% of learners passed, but at the posttest, 27% passed.
Discussion
Pulmonary fellowship programs commonly determine bronchoscopic competency either by an arbitrary number of supervised procedures or subjective evaluations, despite evidence that it should be evaluated in a more formalized, objective fashion (35). We modeled our EBUS curriculum based on the three-step approach to EBUS-TBNA training proposed by the European Respiratory Society: learning the necessary anatomy, simulation-based training, and supervised performance (36, 37). Simulation training alone has been shown to improve technical skills in EBUS-TBNA (24, 38). The multimodal simulation-based curriculum we used rapidly improved EBUS-TBNA technical skills and knowledge among novice pulmonary fellows. It is notable the posttest mEBUS-STAT scores of the novice learners (n = 11) after the 3-month curriculum were similar to the graduating class (PGY-VI, n = 3) who underwent 3 years of traditional apprenticeship training (62.2% vs. 64.8%; P = 0.69), despite the novice learners’ involvement in much fewer EBUS-TBNA procedures during posttest (<15 among novice trainees vs. >30 procedures among graduates). This suggests that the addition of a standardized simulation-based training may shorten the learning curve for EBUS-TBNA compared with the traditional apprenticeship training model (22, 39). Despite improvement in technical skills, only 27% of learners achieved a passing score on the mEBUS-STAT posttest. It is unclear at this time if the low passing score is from the lack of sufficient self-regulated practice and/or training on the simulator (trainees spent an average of 30 min in self-regulated training on the EBUS simulator), low procedural volume (<15 by posttest), or suboptimal teaching methods in our training curriculum.
Strengths and Limitations
Because multiple teaching approaches were used, it is difficult to discern which modality proved to be the most effective: small group didactics, flipped classrooms and PBL, or the hands-on EBUS simulation training. Currently, there is a lack of consensus on the best method for EBUS training (20–22, 40, 41). The flexibility to individualize teaching to learners’ specific skill or knowledge gaps through one-on-one interaction during the debriefing session and hands-on simulation training were strengths of our curriculum.
Several limitations may influence the interpretation of the findings in this study. First, this is an observational study with a small sample size at a single institution—a common problem among medical education research. Other studies using bronchoscopy simulators had between 6 and 16 participants (22, 42, 43). Expanding the curriculum to learners from other programs is needed to validate these tools externally. We considered a crossover cohort study design with the PGY-V class in the educational intervention group and PGY-VI as the control. However, during baseline testing, both classes reported novice level procedural experience based on procedure logs (<10 procedures) and similar scores on their baseline subjective and objective assessments. We therefore decided to provide the multimodal training program to all 11 participating fellows. A second limitation is using a single unblinded proctor for assessment because of scheduling conflicts involved in having a second proctor. The Clinician-Educator fellow was the proctor for the assessment and debriefing sessions because of the limited availability of the EEC faculty to proctor these sessions, given their clinical responsibilities. Anonymous grading was not feasible, because items on the mEBUS-STAT require direct observation. To minimize proctor bias, a scripted protocol was followed during every assessment session, with documentation of any verbal cues (see online supplement). Third, we considered that acquiescence bias or learners’ unrestricted procedural experience on real patients during the 3-month intervention period before posttest could lead to confounding results. This is unlikely, because the EBUS procedure logged by all 11 learners only increased by an additional average of five cases or fewer by the time of the posttest. The improvement in posttest scores is likely out of proportion to the experience gained by five or fewer additional EBUS cases. Fourth, using the same questions and case scenarios for baseline testing and posttest can introduce test–retest bias. We minimized this with the 3-month washout period, collecting all completed knowledge tests and not providing the MCQ answers to learners after the pretest. Fifth, there can be issues with realism and transfer with all types of simulation training. For example, lymph nodes were easier to identify on the simulator than real patients because of a more defined separation between nodes and vessels. Because additional tools such as a stylet or syringe are not available with the BRONCH Express, we supplemented the debriefing session with a real EBUS needle tool to bridge the realism gap. We were unable to assess learners during real patient cases because of scheduling conflicts and the limited availability of the proctor. Still, all faculty commented on a notable improvement in the bronchoscopic skills of the learners who went through this curriculum compared with the prior graduated classes who went through the traditional apprenticeship model. Finally, although it would be desirable for every novice operator to receive simulator training before patient exposure (44), EBUS simulators are a fragile and expensive resource, which may limit their widespread use. The Simbionix BRONCH Express costs $25,000 to purchase. In addition to equipment costs, conducting simulation training is time intensive and requires dedicated trained personnel and/or teachers. Although an assessment based solely on simulator-generated metrics does not require additional resources from busy educators, a simulator cannot replace a comprehensive curriculum to achieve all learning objectives. Simulation-based education can be an excellent tool to train novice fellows to acquire technical skills, complementing traditional apprenticeship training to improve procedural skills rapidly, but further studies are needed to test the validity of this curriculum.
Conclusions
A multimodal simulation-based EBUS curriculum can improve novice learners’ EBUS-related knowledge, self-confidence, and technical skills. A validation study is needed to determine if skills attained via a simulator can be replicated in a clinical setting.
Supplementary Material
Acknowledgments
Acknowledgment
The authors thank the pulmonary fellowship graduating class of 2019, who helped validate the course material. The leadership of our Department of Education and the support of our Division of Pulmonary and Critical Care Medicine in continuing financial and personnel support for this educational intervention and BRONCH Express simulator has facilitated us to invest time and effort into this project since its conception.
Footnotes
This article has a data supplement, which is accessible from this issue’s table of contents at www.atsjournals.org.
Author disclosures are available with the text of this article at www.atsjournals.org.
References
- 1. Rivera MP, Mehta AC, Wahidi MM. Establishing the diagnosis of lung cancer: diagnosis and management of lung cancer, 3rd ed. American College of Chest Physicians evidence-based clinical practice guidelines. Chest . 2013;143:e142S–e165S. doi: 10.1378/chest.12-2353. [DOI] [PubMed] [Google Scholar]
- 2. Vilmann P, Clementsen PF, Colella S, Siemsen M, De Leyn P, Dumonceau JM, et al. Combined endobronchial and esophageal endosonography for the diagnosis and staging of lung cancer: European Society of Gastrointestinal Endoscopy (ESGE) Guideline, in cooperation with the European Respiratory Society (ERS) and the European Society of Thoracic Surgeons (ESTS) Endoscopy . 2015;47:545–559. doi: 10.1055/s-0034-1392040. [DOI] [PubMed] [Google Scholar]
- 3. Sharples LD, Jackson C, Wheaton E, Griffith G, Annema JT, Dooms C, et al. Clinical effectiveness and cost-effectiveness of endobronchial and endoscopic ultrasound relative to surgical staging in potentially resectable lung cancer: results from the ASTER randomized controlled trial. Health Technol Assess . 2012;16:1–75, iii–iv. doi: 10.3310/hta16180. [DOI] [PubMed] [Google Scholar]
- 4. Bellinger CR, Chatterjee AB, Adair N, Houle T, Khan I, Haponik E. Training in and experience with endobronchial ultrasound. Respiration . 2014;88:478–483. doi: 10.1159/000368366. [DOI] [PubMed] [Google Scholar]
- 5. Medford ARL. Learning curve for EBUS-TBNA: longer than we may think. Respiration . 2015;90:173. doi: 10.1159/000381554. [DOI] [PubMed] [Google Scholar]
- 6. Stather DR, Chee A, MacEachern P, Dumoulin E, Hergott CA, Gelberg J, et al. Endobronchial ultrasound learning curve in interventional pulmonary fellows. Respirology . 2015;20:333–339. doi: 10.1111/resp.12450. [DOI] [PubMed] [Google Scholar]
- 7. Rodríguez de Castro F, Díaz López F, Serdà GJ, López AR, Gilart JF, Cabrera Navarro P. Relevance of training in transbronchial fine-needle aspiration technique. Chest . 1997;111:103–105. doi: 10.1378/chest.111.1.103. [DOI] [PubMed] [Google Scholar]
- 8. Boonsarngsuk V, Pongtippan A. Self-learning experience in transbronchial needle aspiration in diagnosis of intrathoracic lymphadenopathy. J Med Assoc Thai . 2009;92:175–181. [PubMed] [Google Scholar]
- 9. Küpeli E. Conventional transbronchial needle aspiration in community practice. J Thorac Dis . 2015;7:S256–S265. doi: 10.3978/j.issn.2072-1439.2015.11.54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Kemp SV, El Batrawy SH, Harrison RN, Skwarski K, Munavvar M, Rosell A, et al. Learning curves for endobronchial ultrasound using cusum analysis. Thorax . 2010;65:534–538. doi: 10.1136/thx.2009.127274. [DOI] [PubMed] [Google Scholar]
- 11. Fielding D, Phillips M, Robinson P, Irving L, Garske L, Hopkins P, Thoracic Society of Australia and New Zealand Advanced interventional pulmonology procedures: training guidelines from the Thoracic Society of Australia and New Zealand. Respirology . 2012;17:1176–1189. doi: 10.1111/j.1440-1843.2012.02253.x. [DOI] [PubMed] [Google Scholar]
- 12. Du Rand IA, Barber PV, Goldring J, Lewis RA, Mandal S, Munavvar M, et al. BTS Interventional Bronchoscopy Guideline Group Summary of the British Thoracic Society guidelines for advanced diagnostic and therapeutic flexible bronchoscopy in adults. Thorax . 2011;66:1014–1015. doi: 10.1136/thoraxjnl-2011-201052. [DOI] [PubMed] [Google Scholar]
- 13.Naur TMH, Konge L, Nayahangan LJ, Clementsen PF. Training and certification in endobronchial ultrasound-guided transbronchial needle aspiration. J Thorac Dis. 2017;9:2118–2123. doi: 10.21037/jtd.2017.06.89. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Ernst A, Silvestri GA, Johnstone D, American College of Chest Physicians Interventional pulmonary procedures: guidelines from the American College of Chest Physicians. Chest . 2003;123:1693–1717. doi: 10.1378/chest.123.5.1693. [DOI] [PubMed] [Google Scholar]
- 15. Ernst A, Wahidi MM, Read CA, Buckley JD, Addrizzo-Harris DJ, Shah PL, et al. Adult bronchoscopy training: current state and suggestions for the future. CHEST expert panel report. Chest . 2015;148:321–332. doi: 10.1378/chest.14-0678. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Tanner NT, Pastis NJ, Silvestri GA. Training for linear endobronchial ultrasound among US pulmonary/critical care fellowships: a survey of fellowship directors. Chest . 2013;143:423–428. doi: 10.1378/chest.12-0212. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Stather DR, Maceachern P, Chee A, Dumoulin E, Tremblay A. Trainee impact on advanced diagnostic bronchoscopy: an analysis of 607 consecutive procedures in an interventional pulmonary practice. Respirology . 2013;18:179–184. doi: 10.1111/j.1440-1843.2012.02270.x. [DOI] [PubMed] [Google Scholar]
- 18. Nguyen S, Ferland N, Beaudoin S, Martel S, Simon M, Laberge F, et al. Influence of trainee involvement on procedural characteristics for linear endobronchial ultrasound. Thorac Cancer . 2017;8:517–522. doi: 10.1111/1759-7714.12481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Wahidi MM, Hulett C, Pastis N, Shepherd RW, Shofer SL, Mahmood K, et al. Learning experience of linear endobronchial ultrasound among pulmonary trainees. Chest . 2014;145:574–578. doi: 10.1378/chest.13-0701. [DOI] [PubMed] [Google Scholar]
- 20. Stather DR, Maceachern P, Rimmer K, Hergott CA, Tremblay A. Validation of an endobronchial ultrasound simulator: differentiating operator skill level. Respiration . 2011;81:325–332. doi: 10.1159/000323520. [DOI] [PubMed] [Google Scholar]
- 21. Konge L, Annema J, Clementsen P, Minddal V, Vilmann P, Ringsted C. Using virtual-reality simulation to assess performance in endobronchial ultrasound. Respiration . 2013;86:59–65. doi: 10.1159/000350428. [DOI] [PubMed] [Google Scholar]
- 22. Konge L, Clementsen PF, Ringsted C, Minddal V, Larsen KR, Annema JT. Simulator training for endobronchial ultrasound: a randomised controlled trial. Eur Respir J . 2015;46:1140–1149. doi: 10.1183/13993003.02352-2015. [DOI] [PubMed] [Google Scholar]
- 23. Davoudi M, Colt HG, Osann KE, Lamb CR, Mullon JJ. Endobronchial ultrasound skills and tasks assessment tool: assessing the validity evidence for a test of endobronchial ultrasound-guided transbronchial needle aspiration operator skill. Am J Respir Crit Care Med . 2012;186:773–779. doi: 10.1164/rccm.201111-1968OC. [DOI] [PubMed] [Google Scholar]
- 24. Scarlata S, Palermo P, Candoli P, Tofani A, Petitti T, Corbetta L. EBUS-STAT subscore analysis to predict the efficacy and assess the validity of virtual reality simulation for EBUS-TBNA training among experienced bronchoscopists. J Bronchology Interv Pulmonol . 2017;24:110–116. doi: 10.1097/LBR.0000000000000349. [DOI] [PubMed] [Google Scholar]
- 25. Wahidi MM, Herth F, Yasufuku K, Shepherd RW, Yarmus L, Chawla M, et al. Technical aspects of endobronchial ultrasound-guided transbronchial needle aspiration: CHEST guideline and expert panel report. Chest . 2016;149:816–835. doi: 10.1378/chest.15-1216. [DOI] [PubMed] [Google Scholar]
- 26. Silvestri GA, Gonzalez AV, Jantz MA, Margolis ML, Gould MK, Tanoue LT, et al. Methods for staging non-small cell lung cancer: diagnosis and management of lung cancer, 3rd ed. American College of Chest Physicians evidence-based clinical practice guidelines. Chest . 2013;143:e211S–e250S. doi: 10.1378/chest.12-2355. [DOI] [PubMed] [Google Scholar]
- 27.Houston M, Lin L.2012. https://www.learntechlib.org/primary/p/39738/
- 28. Mehta NB, Hull AL, Young JB, Stoller JK. Just imagine: new paradigms for medical education. Acad Med . 2013;88:1418–1423. doi: 10.1097/ACM.0b013e3182a36a07. [DOI] [PubMed] [Google Scholar]
- 29. Periyakoil VS, Basaviah P. The flipped classroom paradigm for teaching palliative care skills. Virtual Mentor . 2013;15:1034–1037. doi: 10.1001/virtualmentor.2013.15.12.medu1-1312. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. McLaughlin JE, Roth MT, Glatt DM, Gharkholonarehe N, Davidson CA, Griffin LM, et al. The flipped classroom: a course redesign to foster learning and engagement in a health professions school. Acad Med . 2014;89:236–243. doi: 10.1097/ACM.0000000000000086. [DOI] [PubMed] [Google Scholar]
- 31. Savery JR, Duffy TM. Problem based learning: an instructional model and its constructivist framework. Educational Technology . 1995;35:31–38. [Google Scholar]
- 32. Mercier EM, Higgins SE. Collaborative learning with multi-touch technology: developing adaptive expertise. Learn Instr . 2013;25:13–23. [Google Scholar]
- 33.McKagan S, Sayre E, Madsen A.2016. https://www.physport.org/recommendations/Entry.cfm?ID=93334
- 34. Jørgensen M, Konge L, Subhi Y. Contrasting groups’ standard setting for consequences analysis in validity studies: reporting considerations. Advances in Simulation (London, England) . 2018;3:5. doi: 10.1186/s41077-018-0064-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Jantz MA, McGaghie WC. It’s time for a STAT assessment of bronchoscopy skills: the endobronchial ultrasound bronchoscopy (EBUS)-STAT and EBUS-transbronchial needle aspiration skill evaluation. Am J Respir Crit Care Med . 2012;186:703–705. doi: 10.1164/rccm.201208-1398ED. [DOI] [PubMed] [Google Scholar]
- 36. Farr A, Clementsen P, Herth F, Konge L, Rohde G, Dowsland S, et al. Endobronchial ultrasound: launch of an ERS structured training programme. Breathe (Sheff) . 2016;12:217–220. doi: 10.1183/20734735.013116. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Konge L, Clementsen PF, Annema JT. Diagnostic yield of EBUS-TBNA during the learning curve. J Bronchology Interv Pulmonol . 2018;25:e26–e27. doi: 10.1097/LBR.0000000000000455. [DOI] [PubMed] [Google Scholar]
- 38. Miller GE. The assessment of clinical skills/competence/performance. Acad Med . 1990;65:S63–S67. doi: 10.1097/00001888-199009000-00045. [DOI] [PubMed] [Google Scholar]
- 39. Zhang WC, Chen W, Zhou JP, Lerner AD, Ni L, Shen JM, et al. A comparison of different training methods in the successful learning of endobronchial ultrasound-guided transbronchial needle aspiration. Respiration . 2017;93:319–326. doi: 10.1159/000462978. [DOI] [PubMed] [Google Scholar]
- 40. Stather DR, Maceachern P, Rimmer K, Hergott CA, Tremblay A. Assessment and learning curve evaluation of endobronchial ultrasound skills following simulation and clinical training. Respirology . 2011;16:698–704. doi: 10.1111/j.1440-1843.2011.01961.x. [DOI] [PubMed] [Google Scholar]
- 41. Stather DR, MacEachern P, Chee A, Dumoulin E, Tremblay A. Evaluation of clinical endobronchial ultrasound skills following clinical versus simulation training. Respirology . 2012;17:291–299. doi: 10.1111/j.1440-1843.2011.02068.x. [DOI] [PubMed] [Google Scholar]
- 42. Ost D, DeRosiers A, Britt EJ, Fein AM, Lesser ML, Mehta AC. Assessment of a bronchoscopy simulator. Am J Respir Crit Care Med . 2001;164:2248–2255. doi: 10.1164/ajrccm.164.12.2102087. [DOI] [PubMed] [Google Scholar]
- 43. Blum MG, Powers TW, Sundaresan S. Bronchoscopy simulator effectively prepares junior residents to competently perform basic clinical bronchoscopy. Ann Thorac Surg . 2004;78:287–291. doi: 10.1016/j.athoracsur.2003.11.058. [DOI] [PubMed] [Google Scholar]
- 44. Sehgal IS, Dhooria S, Aggarwal AN, Agarwal R. Training and proficiency in endobronchial ultrasound-guided transbronchial needle aspiration: a systematic review. Respirology . 2017;22:1547–1557. doi: 10.1111/resp.13121. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.