Advanced Endoscopy Trainee Involvement Early in EUS Training May Be Associated with an Increased Risk of Adverse Events

Abstract

Background

The quality of endoscopic ultrasound (EUS) involving advanced endoscopy trainees (AETs) is not well understood. In this study, we aimed to examine adverse events (AE) risk and diagnostic yield of EUS procedures involving AETs.

Methods

We conducted a retrospective single-centre review from September 2009 to August 2015. Clinical, procedural, cytological, and hospital visit data within 30 days of the EUS procedure was collected. Primary outcomes were occurrence of an AE and a diagnostic specimen on cytopathology. Each AE was classified as “definitely related,” “possibly related,” or “not related” to the EUS procedure based on a previously defined consensus approach. Advanced endoscopy trainee involvement was established through the operative report.

Results

Our study included 1657 EUS procedures, of which 27% (451 of 1657) involved AETs. Endoscopic ultrasound was most commonly performed to evaluate pancreatic pathology (46% of cases). Overall AE incidence was 3.4%; it was 4.9% when an AET was involved and 2.8% when the EUS was performed without an AET (P = 0.04). The risk of an AE when AETs were involved was greatest in the first three months of training (7.9% versus 2.7%, P = 0.04). Multivariate analysis limited to the first three months of training demonstrated AET involvement to be associated with an increased AE risk after adjusting for patient and procedural factors (adjusted OR 3.2; 95% CI, 1.1–8.7; P = 0.03). The overall diagnostic yield was 76%. This was not compromised by AET involvement for any quartile of training.

Conclusions

We observed an increased risk of EUS-related AEs when procedures involved AETs during the first three months of training.

INTRODUCTION

Endoscopic ultrasound (EUS) is an established modality used in the evaluation and management of gastrointestinal diseases (1). While EUS is regarded as a safe procedure (2, 3), it is operator-dependent, requiring technical and cognitive skills beyond that of standard endoscopy (4). Quality of EUS procedures is related to the training, skill and experience of the endoscopist (5).

To date, no study has specifically examined the quality of EUS performed by trainees under the supervision of a staff endoscopist, trained and credentialed to perform this procedure. A potential procedure quality and safety gap may exist when trainees are involved in endoscopy (6); however, the evidence to support this notion is scarce and generally favours no excess risk to patients (7–9).

Diagnostic yield is an important indicator of EUS quality (10). High yields prevent repeat procedures, reduce costs and ultimately minimize risk to patients (11). A prompt diagnosis also facilitates definitive management strategies (11). There has only been one study to date by Cote et al. (7) who examined this outcome. Although they found no difference between trainee and staff diagnostic yield, their study was limited by a small sample size and the number of trainees in their study (7).

With increasing advanced endoscopy training programs and limited data on trainee participation in EUS, we aimed to describe the quality of EUS procedures involving advanced endoscopy trainees (AETs) by examining adverse event (AE) details and diagnostic yield in a large cohort of patients.

MATERIAL AND METHODS

Study Design and Population

A total of 1723 consecutive EUS patient records between September 1, 2009, and August 31, 2015, were obtained from the Ottawa Hospital Data Warehouse (Ottawa, Ontario, Canada) with the approval of our Research Ethics Board.

Retrospective data regarding patient, procedural, and cytological details were recorded into a database. Cytology records were evaluated for diagnostic yield. Emergency room visits and hospitalizations in the Greater Ottawa Area (The Ottawa Hospital, Montfort Hospital, and Queensway Carleton Hospital) within 30 days of the EUS procedure were also reviewed separately to determine the relation of the visit to the procedure. Details of each AE were collected to describe outcomes.

All patients who underwent an EUS procedure at The Ottawa Hospital during the study period were considered for inclusion in the study. Patients who underwent therapeutic interventions (EUS-guided celiac plexus blocks or neurolysis, cyst-gastrostomy, fiducial placement or single-incision, needle-knife biopsy) were excluded because of the known higher adverse events risk associated with these procedures. They were also only performed toward the end of the study period (Figure 1). Cases where the endoscopy report did not provide sufficient information for data collection were also excluded.

Figure 1.

Flow diagram of included and excluded cases.

Outcomes and Variable Definitions

Two primary outcome measures were considered in our study: the occurrence of an AE within 30 days of the procedure and a diagnostic result on cytopathology. We defined AEs a priori as all main presentation diagnoses that have been reported to be EUS-related, including abdominal pain, fever, pancreatitis, bleeding, infection, perforation and death. Using a previously reported approach to evaluating adverse events related to endoscopic procedures (12), four of the investigators (UK, MA, AC and PJ) determined the relation of each hospital encounter to the procedure applying predefined criteria: 1) no other exposure was more likely than the EUS procedure to be related to the event; 2) this was a new clinical presentation for the patient; and 3) the presentation was a previously reported complication of EUS (12). If all criteria were met, the event was classified as “definitely related.” If at least one but not all criteria were met, the event was “possibly related.” If none were met, the event was “not related.” Adverse events were included in the analysis if they were either definitely or possibly related to the EUS procedure. The investigators were blinded to trainee participation, and disagreements were resolved based on consensus.

Records indicating “negative for malignancy” or a malignant diagnosis on cytology were considered diagnostic. “Highly suspicious” reports were also considered diagnostic (13). “Atypical,” “indeterminate” or “nondiagnostic” samples were categorized as nondiagnostic. To examine changes in AE risk and diagnostic yield during the one-year fellowship period, we categorized training into quartiles by month (July to September, October to December, January to March and April to June).

The active involvement of an AET during the procedure is routinely indicated in the operative report. Advanced endoscopy trainee involvement at our institution includes preprocedure patient evaluation, indication confirmation, consent, and logistical preparation. During the procedure, the trainee is supervised performing the EUS, including intubating the esophagus, maneuvering the echoendoscope, identifying landmarks, and noting normal and abnormal findings. If an endoscopic ultrasound-guided fine needle aspiration (EUS-FNA) is planned, the trainee is asked to identify the lesion of interest. If an EUS-FNA is performed, the AET will attempt at least the first pass. If this pass is performed safely and effectively, the trainee will go on to perform a second and any additional passes as needed. If there is concern that the AET is not performing the aspirations safely or effectively, the remainder of the procedure is completed by the supervising endoscopist.

Statistical Analysis

Categorical and continuous variables were reported using proportions and medians with interquartile range, respectively. Continuous variables did not follow a normal distribution based on the Shapiro-Wilk test of normality. Pearson chi-square and Fisher exact tests were used to compare categorical variables as indicated. Given the asymmetrical distribution of clinical data, the Mann-Whitney U test was performed for comparison of continuous variables. Variables that demonstrated an association in the univariate analysis (P < 0.10) were included in the multivariate binary logistic regression model. The forced entry method was used for the variables of interest. Statistical significance was set at P < 0.05. All statistical analyses were previously planned and completed using the PASW 18.0 Statistical Package (SPSS Inc, Chicago, 2009).

RESULTS

Patient and Procedural Details

Of the 1723 EUS procedures identified, 1657 met the study inclusion criteria. Five EUS procedures had insufficient data captured in their procedure report. Patient and procedural details are presented in Table 1. The median age was 64 years (interquartile range [IQR] 53 to 73), with 50% males. Of note, AETs were more likely involved in EUS procedures involving anesthesia-guided sedation (P < 0.01), EUS-FNA procedures (P < 0.01) and lesions greater than 3 cm (P < 0.01). Three trainees completed a one-year advanced endoscopy fellowship over the six-year study period. The EUS experience involving a fourth trainee was also captured for the two months of their fellowship (from July 1 to August 31 of the final year of the study period). Overall, trainees were involved in 27% (451 of 1657) of the EUS procedures. Among the three trainees completely captured, the median number of cases per AET per training cycle was 136 (IQR 108–161). Within their respective study periods, the median trainee involvement was 54% (IQR 46% to 63%) of cases. Two experienced staff endoscopists were captured in the entire study period. Both had received formal training in EUS and had performed over 1000 EUS procedures before the study date.

Table 1.

Baseline characteristics of patients who underwent endoscopic ultrasound

Variable		Trainee Absent (n=1206) N (%)*	Trainee Present (n=451) N (%)*	P value
Age in years, median (IQR)		63 (54–73)	64 (52–73)	0.48
Male sex		592 (49)	237 (52)	0.21
Smoker		419 (35)	137 (30)	0.06
Charlson Comorbidity Index		3 (1–4)	3 (1–4)	0.81
EUS indication				<0.01
Abdominal pain		68 (5.6)	13 (2.9)
Abnormal lymph node(s)		42 (3.5)	20 (4.4)
Duodenal tumor		23 (1.9)	8 (1.8)
Esophageal cancer staging		30 (2.5)	6 (1.3)
Esophageal tumor		30 (2.5)	5 (1.1)
Gallstone disease		48 (4.0)	10 (2.2)
Gastric cancer staging		27 (2.2)	9 (2.0)
Gastric tumor		91 (7.5)	36 (8.0)
Pancreatic cystic lesion		148 (12)	50 (11)
Pancreatic solid tumor		274 (23)	144 (32)
Pancreatitis		115 (9.5)	35 (7.8)
Rectal cancer staging		20 (1.7)	10 (2.2)
Subepithelial lesion		61 (5.0)	41 (9.1)
Other		241 (20)	73 (16)
Procedure details
Anesthesia-guided sedation		44 (4)	34 (8)	<0.01
Cytotechnologist present		431 (36)	168 (37)	0.57
Upper EUS		1169 (97)	434 (96)	0.53
Type of endoscope used				<0.01
Linear		851 (72)	375 (84)
Radial		225 (19)	64 (14)
Linear and radial		111 (9)	10 (2)
Additional procedure performed on same day as EUS				<0.01
Gastroscopy		150 (12)	28 (6)
Colonoscopy		17 (1)	7 (2)
Flexible sigmoidoscopy		10 (1)	4 (1)
ERCP		49 (4)	40 (9)
FNA details†
FNA performed		651 (54)	279 (62)	<0.01
Number of FNA passes, median (IQR)		3 (3–4)	3 (3–4)	0.16
FNA needle gauge‡	25 g	50 (8)	9 (3)	<0.01
	22 g	502 (78)	244 (86)	<0.01
	19 g	85 (13)	29 (10)	0.19
FNA approach‡	Transesophageal	55 (9)	6 (2)	<0.01
	Transgastric	249 (41)	114 (46)	0.14
	Transduodenal	291 (48)	120 (49)	0.79
Lesion size (≥ 3 cm)		206 (36)	125 (51)	<0.01
Type of lesion				0.20
Predominantly cystic		119 (28)	38 (20)
Predominantly solid		194 (45)	88 (47)
Solid and cystic		44 (10)	24 (13)
Other		74 (17)	39 (21)

^*Unless otherwise indicated.

†Limited to EUS-FNA procedures only.

^‡Global P value < 0.01.

Adverse Events

One hundred three patients presented to an emergency department (ED) or were hospitalized within 30 days of their EUS procedure (Table 2). Thirty-two (1.9%) and 24 (1.4%) cases were definitely and possibly related, respectively, to the EUS procedure. Forty-seven (2.8%) cases were not related, placing the overall AE risk at 3.4% (56 of 1657) when including definitely and possibly related cases. Of the 56 cases classified as definitely and possibly related, nine had a tandem endoscopic retrograde cholangiopancreatography [ERCP] performed. All ERCP cases were classified as possibly related. Six of these cases involved AETs, while three did not.

Table 2.

Adverse events risk within 30 days of endoscopic ultrasound, relation to the procedure, and outcomes

Variable	Trainee Absent (n=1206) N (%)*	Trainee Present (n=451) N (%)*	P value
Emergency room visit or hospitalization	64 (5)	39 (9)	0.01
Relation to EUS procedure			0.69
Definitely-related	18 (2)	14 (3)
Possibly-related	16 (1)	8 (2)
Not-related	30 (2)	17 (4)
Adverse event details†
Number of adverse events	34 (3)	22 (5)	0.04
Number requiring hospitalization	18 (2)	9 (2)	0.47
Median number of days in hospital (IQR)	6 (3–15)	4 (2–13)	0.23
Number requiring ICU care	3 (0.2)	4 (1)	0.09
Management†
Antibiotics	11 (1)	5 (1)	0.78
Blood transfusion (units)	1 (0.1)	2 (0.4)	0.18
Endoscopic management	1 (0.1)	1 (0.2)	0.47
Surgical management	2 (0.2)	2 (0.4)	0.30
Risk of adverse event†
Infection	3 (0.3)	3 (1)
Gastrointestinal bleeding	2 (0.2)	1 (0.2)	1.00
Pancreatitis	6 (1)	3 (1)	0.71
Perforation	0	2 (0.4)	0.64
Pulmonary embolism	1 (0.2)	0	1.00

^*Unless otherwise indicated.

†Cases that were definitely- and possibly-related to the EUS procedure were included.

ICU: intensive care unit

Adverse Event Outcomes

The overall AE hospitalization incidence was 48% (27 of 56). Intensive care (ICU) was required by 26% (7 of 27) of cases requiring admission. Abdominal pain (41%), nausea or vomiting (29%) and gastrointestinal bleeding (11%) were the most common symptoms on presentation. Antibiotic management was required by 29% of cases, while 4% required endoscopic and 7% surgical for those presenting to hospital (Table 2).

There were two cases of perforation observed. The first occurred during a rectal EUS in a patient known to have rectal cancer. The event was observed during EUS and subsequently required surgical management. The second was an iatrogenic duodenal perforation from an EUS-FNA and tandem ERCP. Laparotomy was performed twice to correct the perforation, but the patient ultimately died intraoperatively from the development of fecal peritonitis causing septic shock. An AET was involved during both cases; however, they were not performing the procedure at the time the perforations were believed to have occurred.

Diagnostic Yield

There were 947 EUS-FNA cases identified, and 900 cases had sufficient data for analysis. Forty-seven cases were excluded because the cytopathology report was either missing or incomplete. The overall EUS-FNA diagnostic yield was 76%. This was not affected by AET involvement (75% versus 78%, P = 0.26).

Overall Adverse Events Risk

Risk of adverse event was 4.9% when AETs were involved and 2.8% when they were not (P = 0.04). Univariate analysis revealed four factors that demonstrated association with increased risk and met our P value < 0.10 cutoff: EUS evaluation of solid pancreatic lesions (P = 0.09), presence of an AET during the procedure (P = 0.04), performance of an FNA (P < 0.01) and same-day ERCP (P < 0.01; Supplemental Table S1). In multivariate analysis, the relationship between AET involvement and AE risk was no longer significant (OR 1.53; 95% confidence interval [CI], 0.87–2.68; P = 0.14). Same-day ERCP (OR 2.8; 95% CI, 1.3–6.3; P = 0.01) and performance of FNA (OR 2.0; 95% CI, 1.1–3.9; P = 0.03) were related to an increased risk of an AE.

Adverse Events Risk by Training Period

Changes in AE risk and diagnostic yield stratified by time is presented in Table 3. An increased risk of an AE related to AET involvement during the EUS procedure was observed in the first three months of training (7.9% versus 3.3%, P = 0.04) but not for the other training periods. Adjusting for type of pancreatic lesion, same-day ERCP, and performance of FNA, the association between AE risk and involvement of a fellow during the first three months of their fellowship remained significant (OR 3.2; 95% CI, 1.1–8.7; P = 0.03) (Table 4). Fine needle aspiration performance was the only other predictor of increased risk (OR 4.07; 95% CI, 1.14–8.65; P = 0.03) during this period. Diagnostic yield was similar for all of the time periods whether an AET was involved in the EUS procedure or not.

Table 3.

Endoscopic ultrasound related adverse event risk and diagnostic yield over a one-year advanced endoscopy training period*

Training Quartile‡	Adverse Event Risk				Diagnostic Yield†
	n	Trainee Absent (n=1206) N (%)	Trainee Present (n=451) N (%)	P value	n	Trainee Absent (n=631) N (%)	Trainee Present (n=269) N (%)	P value
1	433	9 (3)	8 (8)	0.04	243	128 (72)	42 (76)	0.55
2	352	5 (2)	0	0.59	185	106 (78)	44 (90)	0.07
3	412	11 (4)	6 (4)	1.00	250	120 (78)	63 (75)	0.61
4	458	9 (3)	8 (6)	0.11	252	119 (73)	62 (77)	0.50
Overall	1657	34 (3)	22 (5)	0.04	900	473 (75)	211 (78)	0.26

*Cases that were definitely and possibly related to the EUS procedure were included.

^†Limited to EUS-FNA procedures only.

^‡1: July to September, 2: October to December, 3: January to March, 4: April to June.

Table 4.

Factors associated with endoscopic ultrasound related adverse events risk, multivariate regression analysis*

Training Period	Risk Factor	Multivariate, OR (95% CI)	P value
First quartile†	Fellow present during procedure	3.15 (1.14–8.65)	0.03
	EUS indication: pancreatic solid lesion	0.75 (0.22–2.57)	0.65
	Same-day ERCP	1.14 (0.12–10.49)	0.91
	FNA performed	4.07 (1.14–8.65)	0.03
Overall	Fellow present during procedure	1.53 (0.87–2.68)	0.14
	EUS indication: pancreatic solid lesion	1.13 (0.60–2.11)	0.70
	Same-day ERCP	2.84 (1.28–6.30)	0.01
	FNA performed	2.04 (1.08–3.86)	0.03

*Cases that were definitely- and possibly-related to the EUS procedure were included.

^†July to September.

OR: odds ratio

DISCUSSION

This is the first study characterizing the quality of EUS procedures involving AETs using the occurrence of well-defined AEs and diagnostic yield as quality indicators. We found an increased risk of AEs when EUS procedures involved AETs in the first three months of training. Diagnostic yield was not affected by trainee involvement and did not change throughout the training period.

To examine the incidence of AEs over time, we categorized AET training into quartiles. We observed an increased risk for procedures involving AETs during their first three months of training after controlling for patient and procedural factors. This is likely related to AETs’ lack of familiarity with EUS procedures. One study by Sharma et al. (6) examined AE risk in endoscopy when trainees participated in procedures. They demonstrated that involvement of trainees in endoscopic procedures was related to an increased risk of cardiopulmonary events (6). This study grouped multiple endoscopic procedures together, including gastroscopy, endoscopic ultrasound, endoscopic retrograde cholangiopancreatography and colonoscopy. The impact of trainee involvement on EUS alone was not evaluated. Further, their AE evaluation was limited to the immediate postoperative recovery period (6).

There was a trend toward an increased risk during the final three months of training observed in our univariate analysis, but the difference compared with procedures not involving AETs in the same period was not significant. This trend may be real and the result of trainees attempting more complex EUS procedures in their final stages of training. Our study may be underpowered to detect a significant risk trend toward the end of training, and further studies using large prospective cohorts are warranted.

Our overall AE incidence was 3.4%, which is greater than what some previous retrospective studies have reported. Adverse event risk estimates associated with EUS-FNA range between 1% and 4% (12, 14–16). Sources of AE risk variation between studies include how the adverse outcomes are defined, the study design, the data quality, the patient population being studied, and the expertise of the endoscopist(s) performing the EUS procedures. This is the first to examine AE risk among EUS procedures involving AETs using clearly defined criteria.

Diagnostic yield was not found to be compromised by AET involvement in EUS-FNA procedures. This is in keeping with what previous studies have demonstrated (7, 17). However, we found that trainees were more often involved in larger, solid pancreatic or submucosal lesions which have previously been shown to be associated with higher diagnostic yields (18, 19). A true difference may exist for more difficult lesions such as those that are smaller and cystic (20). Future studies may wish to examine the impact of trainee involvement on diagnostic yield while controlling for potential confounders such as procedure difficulty and duration.

Our study has some notable limitations. First, the retrospective design of our study makes it susceptible to bias. Second, while we attempted to capture the majority of AEs occurring within 30 days of the EUS procedures by collecting data from the three hospitals serving the Ottawa area (The Ottawa Hospital, Montfort Hospital and Queensway Carleton Hospital), information regarding patients who presented to emergency departments elsewhere could not be included in our analyses, and this may have introduced ascertainment bias. Given the centralized nature of the EUS service for Ottawa’s health region, AET involvement in the EUS procedure was unlikely to be related to where patients live or the health care facility they would visit for a possible AE. Thus, we do not believe that this limitation introduced significant bias. We did attempt to reduce selection bias by capturing all EUS procedures performed at our centre. Third, there is a risk of referral bias, but this is mitigated by the fact that only our centre offers EUS services for the health region. Fourth, the endoscopy training experience may differ between training sites, which would limit the generalizability of our findings. For example, the median number of cases performed by each AET was 136 in this study. The American Society of Gastrointestinal Endoscopy (ASGE) guidelines currently recommend a minimum of 150 supervised procedures, 50 of which should be FNA cases (21). However, the most recent survey conducted of EUS training programs in the United States indicated that, as of 2006, 52% did not meet the ASGE recommendations (22). Fifth, assessing the relationship of each AE to EUS is subjective. Unlike previous studies, we attempted to address this issue by using a standardized, well-described, reproducible and previously reported approach in assigning AE relation to the endoscopic procedure. Sixth, although we could identify procedures involving AETs, we could not precisely describe the extent to which the AET participated in the procedure. Finally, despite the relatively large number of cases we examined, our study may be underpowered to detect a significant EUS-related AE risk related to each individual AET or how this risk may change over the study period. Larger, well-designed and prospective studies examining how AET involvement can impact AE risk during EUS procedures are warranted.

In conclusion, we observed an increased risk of EUS-related AEs during the first three months of AET training. The diagnostic yield did not appear to be affected by AET involvement for EUS-FNA procedures. Advanced endoscopy training programs should consider ways to mitigate the increased risk to patients when AETs are involved in EUS procedures during their first few months of training.