Effect of the time of day and queue position in the endoscopic schedule on the performance characteristics of endoscopic ultrasound-guided fine-needle aspiration for diagnosing pancreatic malignancies

Abstract

Background and Objectives:

Recent reports have indicated that the time of day may impact the detection rate of abnormal cytology on gynecologic cytology samples. The aim of this study was to determine if procedure time or queue position affected the performance characteristics of endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) for diagnosing solid pancreatic malignancies.

Patients and Methods:

We conducted a retrospective study evaluating patients with solid pancreatic lesions in whom EUS-FNA was performed. Three timing variables were evaluated as surrogate markers for endoscopist fatigue: Procedure start times, morning versus afternoon procedures, and endoscopy queue position. Statistical analyses were performed to determine whether the timing variables predicted performance characteristics of EUS-FNA.

Results:

We identified 609 patients (mean age: 65.8 years, 52.1% males) with solid pancreatic lesions who underwent EUS-FNA. The sensitivity of EUS-FNA was 100% for procedures that started at 7 AM while cases that started at 4 PM had a sensitivity of 81%. Using start time on a continuous scale, each elapsed hour was associated with a 1.9% decrease in EUS-FNA sensitivity (P = 0.003). Similarly, a 10% reduction in EUS-FNA sensitivity was detected between morning and afternoon procedures (92% vs. 82% respectively, P = 0.0006). A linear regression comparing the procedure start time and diagnostic accuracy revealed a decrease of approximately 1.7% in procedure accuracy for every hour later a procedure was started. A 16% reduction in EUS-FNA accuracy was detected between morning and afternoon procedures (100% vs. 84% respectively, P = 0.0009). When the queue position was assessed, a 2.4% reduction in accuracy was noted for each increase in the queue position (P = 0.013).

Conclusion:

Sensitivity and diagnostic accuracy of EUS-FNA for solid pancreatic lesions decline with progressively later EUS starting times and increasing numbers of procedures before a given EUS, potentially from endoscopist fatigue and cytotechnologist fatigue.

INTRODUCTION

Pancreatic adenocarcinoma is the fourth leading cause of the cancer deaths in the United States.[1] In 2012, it was estimated that over 43,900 patients developed pancreatic cancer and almost 37,390 died from this disease.[2] Endoscopic ultrasound-guided fine-needle aspiration (EUS-FNA) is recognized as a safe and accurate technique to obtain cytologic and pathologic specimens for tissue diagnosis of solid pancreatic lesions. Previous studies of EUS-FNA of pancreatic lesions reveal sensitivities ranging 80%-90%, specificities of 90%-100%, and diagnostic accuracies of 85% to greater than 90%.[3,4,5,6]

Advances in cross-sectional imaging technologies have led to an increased detection of pancreatic lesions and placed further demand on the endoscopist to perform a large volume of endoscopic procedures. The shrinking availability of endoscopy time may require endoscopists to perform a number of complex diagnostic and therapeutic procedures on a given day. Previous studies have examined the impact of operator fatigue on colonoscopy success and endoscopic retrograde cholangiopancreatography (ERCP) outcomes.[7,8,9,10,11] While multiple studies have shown that the time of day of a colonoscopy affects adenoma detection rate and procedure completion, recent data for ERCP demonstrate no impact on procedure completion rates, length of procedures, or adverse events despite the obvious procedure complexity and the operator-dependent nature of highly technical biliary endoscopy.[11,12]

Endoscopic ultrasound (EUS) combines the technical skills of endoscopy with the added challenge of real-time interpretation of ultrasound imaging. The addition of fine-needle aspiration (FNA) sampling and the processing and immediate analysis of cytology samples add to the complexity and duration of the process.[13] Recent reports have indicated that the time of day and the workload of the cytotechnologist impact the detection rate of abnormal cytology on gynecologic cytology samples, with more abnormal results being detected from morning samples as compared to evening samples and from samples taken later in the week when compared to those obtained earlier in the week.[14,15] Operator fatigue has not been previously examined in relation to the diagnostic accuracy of EUS-FNA. The objective in this study was to explore our institution's experience with EUS-FNA of solid pancreatic masses to elucidate if the procedure time or queue position has an impact on the overall performance characteristics.

PATIENTS AND METHODS

Patients

We conducted a retrospective analysis of all patients with solid pancreatic mass lesions found on computed tomography (CT) or magnetic resonance imaging (MRI) imaging at Thomas Jefferson University Hospital from June 2000 to February 2012. Our inclusion criteria were those patients who underwent EUS-FNA of a pancreatic mass seen on imaging studies (CT scan or MRI). We excluded those patients who had contraindication for undergoing EUS/FNA procedure (i.e., unwilling or medically unstable patients or patients with severe coagulopathy). All EUS-FNA procedures were performed by three experienced faculty endoscopists with greater than 6 years of experience who had performed more than 500 EUS procedures. Patients were identified using interrogation of the electronic medical records and our endoscopy database. The data recorded from EUS records included the location, type, size, the number of passes made and type of needles used, sample adequacy, cytology results, final diagnoses, time of procedure, and queue position of the procedure for a given endoscopist.

Procedure

A curvilinear echoendoscope (GF UCT 140 or UCT 160; Olympus America Inc., Melville, NY, USA) was used to perform the EUS-FNA in the standard manner. EUS-FNA was performed using a 22G needle (EchoTip^®, Cook Medical Inc. Winston Salem, NC, USA). Rapid on-site evaluation (ROSE) was present for a majority of the procedures. A total of four cytotechonologists were involved in preparation and interpretation of the cytology samples. The cytotechnologist and cytopathologists were informed about the imaging studies and clinical history by the endoscopist at the time of EUS-FNA. Cytology aspirates were placed onto glass slides and preserved with a Diff-Quik stain (American Scientific Products, McGraw Park, IL, USA). In addition, a smear was placed in alcohol for Papanicolaou staining. Any additional material was sprayed into Hanks’ solution and sent for cell block processing. The cytology technician on site verified the adequacy of specimens. A specimen was considered adequate by ROSE if there were an adequate number of representative cells from the lesion. At least six passes were obtained from the pancreatic lesion unless the technician established the presence of malignant-appearing cells as per institutional protocol. The cytopathologist evaluated the material collected within 24 h after which a final report was submitted. A specimen was considered adequate if there were an adequate number of representative cells from the lesion. Interobserver agreement of the diagnosis by two cytopathologists as well of double viewing of the specimen was required before final sign-out of the specimen.

Study design

The timing of the EUS procedure was evaluated as the timing in the day, in 1-h intervals. In addition, we evaluated the “queue position” of the EUS procedure that we defined as the number of preceding endoscopic procedures performed by the endoscopist on that particular day.[16] The primary study outcome was the performance characteristics of EUS-FNA for the diagnosis of solid pancreatic lesions.

Patients in the current study were classified as having a benign lesion versus malignant lesion. A final diagnosis of pancreatic malignancy was based on:

1. Cytologic or histologic evidence of malignancy based on the material obtained by EUS-FNA, ERCP, or surgical biopsy or

2. Clinical course based on follow-up in which that patient developed radiographic evidence of local or distant metastasis, or death attributed to a malignant pancreatic lesion based on clinical records if the cytology or surgical pathology results were inconclusive.

A lesion was defined as being benign based on EUS-FNA results, clinical follow-up, and lack of disease progression over a period of at least 6 months.

EUS-FNA cytology samples considered to be adequate were interpreted as malignant, suspicious of malignancy, atypical cells, benign, and nondiagnostic. Immunohistochemical analysis of the cell block samples was performed in patients suspected to have lymphoma or neuroendocrine tumor (NET).

Patients with suspicious or malignant cytology were classified as “true positive” if the final diagnosis was malignancy, and those considered benign on the final surgical pathology were classified as “false positive.”[17] Similarly, lesions classified as benign by EUS-FNA with a final surgical pathology diagnosis of benign were considered as true negative and those considered malignant on surgical pathology were false negative. Atypical FNA cytology results were considered as a negative for malignancy in our study analysis. Ki-67 staining was used as a proliferation marker on cytology samples for NETs. The NET was considered to be malignant if Ki-67 index was greater than 3% and/or if the mitotic count was greater than 20 per 10 high power fields.

For the purpose of analysis, the timing of the EUS procedure was evaluated as the timing in the day in 1-h intervals from 7 AM to 4 PM. The patients were also stratified into two groups:

1. EUS procedures starting before 12 PM (the AM group) and

2. EUS procedures starting after 12 PM (the PM group).

The primary study outcomes were the performance characteristics of EUS-FNA for the diagnosis of solid pancreatic lesions.

Statistics

Performance characteristics of EUS-FNA for the diagnosis of pancreatic malignancies including the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy were compared between the two groups using Fisher's exact test. These were determined by comparing EUS-FNA results with the final diagnoses of the lesions, based on the above mentioned criteria. Diagnostic accuracy was defined as the ratio of the sum of true positive and true negative values divided by the number of lesions. In addition, the queue position (defined as the number of preceding procedures performed by the endoscopist on that particular day) was analyzed to determine the relationship of each of these variables to the performance characteristics of EUS-FNA for the diagnosis of pancreatic malignancies with a linear regression.

Continuous variables were presented as means ranges. Categorical variables were reported as frequencies. Intragroup comparison of performance characteristics was done using the Fisher's exact test where appropriate. Statistical significance was determined a priori at P ≤ 0.05.

Analyses were performed using Statistical Analysis System (SAS) V9.1 (SAS Institute, Cary, NC, USA).

RESULTS

Baseline characteristics

A total of 609 patients with solid pancreatic lesions who underwent EUS-FNA were included in the analysis. Twenty patients with solid pancreatic lesions on imaging did not undergo EUS-FNA as a result of being medical unstable or having severe coagulopathy and were excluded from the final analysis. General patient characteristics are summarized in Table 1.

Table 1.

General patient characteristics of patients undergoing EUS/FNA. Data are shown as mean and range or frequencies

Performance characteristics of endoscopic ultrasound-guided fine-needle aspiration

The EUS-FNA cytology results were compared to the final diagnosis of the pancreatic lesion as defined in the Methods section.

The overall performance characteristics of all EUS-FNAs in solid pancreatic lesions were as follows: Sensitivity 87.6% [95% confidence interval (CI) = 84.3%-90.2%; n = 451/513], specificity 93.7% (95% CI = 86.3%-97.4%; n = 90/96), PPV 98.6% (95% CI = 97.1%-99.4%; n = 451/457), and NPV 59.2% (95% CI = 50.9%-67.0%; n = 90/152). The overall diagnostic accuracy of EUS-FNA was 89%. There were a total of 6/609 (1.0%) false positive cases and 62/609 (10.2%) false negative cases. Overall, 507 (83.2%) lesions were pancreatic adenocarcinomas, 30 (4.9%) were neuroendocrine tumors, 6 (0.9%) were lymphomas, 18 (2.9%) were metastases, and 48 (7.9%) were consistent with chronic focal pancreatitis.

The performance characteristics of EUS-FNA cytology results were analyzed based on the timing of the procedure [Table 2]. The sensitivity of EUS-FNA to diagnose a pancreatic malignancy was higher in procedures performed in the morning and it decreased as the day progressed. The sensitivity of EUS-FNA was 100% for procedures that started at 7 AM while cases that started at 4 PM had a sensitivity of 81.2%. A linear regression [Figure 1] comparing the procedure start time and sensitivity revealed a decrease of approximately 1.9% in procedure sensitivity for every hour later a procedure was started and a R² of 0.697; this was found to be statistically significant (P = 0.003). Similarly, comparison of EUS-FNA sensitivity of morning (7-11 AM) start times versus all afternoon start times (12-4 PM) revealed a sensitivity of 92.4% versus 81.6%, which is a statistically significant difference (P = 0.0006). There was no statistically significant difference in the specificity of EUS-FNA to diagnose pancreatic malignancies based on the time when the procedure was performed. Diagnostic accuracy was lower for procedures started later in the day. Procedures early in the morning (7 AM) had 100% accuracy, whereas later cases (4 PM) had an accuracy of only 83.3%. A linear regression [Figure 2] comparing the procedure start time and diagnostic accuracy revealed a decreased of approximately 1.7% in procedure accuracy for every hour later a procedure was started and a R² of 0.667. Comparing all morning (7-11 AM) start times with all afternoon start times (12-4 PM) revealed a diagnostic accuracy of 100% versus 83.5%, a statistically significant difference (P = 0.0009). The size and location of tumor were similar for the procedures performed in the morning and at afternoon. Similarly, there was no difference in the mean number of needle passes in the morning versus afternoon (4.8 versus 5 respectively, P = 0.18).

Table 2.

EUS-FNA performance characteristics and procedure start time

Figure 1.

Linear regression analysis comparing procedure start time and EUS-FNA sensitivity

Figure 2.

Linear regression analysis comparing procedure start time and EUS-FNA diagnostic accuracy

The diagnostic accuracy of FNA cytology results based on position in an endoscopist's queue was also analyzed. Diagnostic accuracy was lower for procedures that were later in an endoscopist's queue. A procedure that was first and second in the overall procedure queue had accuracy of 89.7% and 86.6%, respectively while cases that were seventh or eighth in the queue position had a diagnostic accuracy of only 66.7%. A linear regression comparing the procedure queue position and diagnostic accuracy revealed a drop of 2.4% diagnostic accuracy for each increase of one procedure in the queue prior to the index procedure, with a R² of 0.86 (P = 0.01) [Figure 3]. Comparing all procedures in the first half of the endoscopist's queue (first through fourth) with all procedures in the second half of the endoscopist's queue (fifth through eighth) revealed a diagnostic accuracy of 94% versus 87%, which was a statistically significant difference (P = 0.01). There was no difference in the procedure queue position and number of needle passes by linear regression (r² = 0.03, P = 0.69) [Table 3]. Individual performance characteristics of EUS-FNA cytology results for each of the three endoscopists were compared and no statistically significant difference was found between the three. Similarly, no difference was found in the individual outcomes of the two cytopathologists involved in this study [Table 4].

Figure 3.

Linear regression analysis comparing endoscopy queue position and EUS-FNA diagnostic accuracy

Table 3.

EUS-FNA diagnostic accuracy and procedure queue position

Table 4.

EUS-FNA performance characteristics by individual physician

DISCUSSION

In this current study, we demonstrated that progressively later EUS starting times and increasing numbers of procedures before a given EUS significantly decrease the sensitivity and diagnostic accuracy of EUS-FNA for solid pancreatic lesions, potentially from endoscopist fatigue and cytotechnologist fatigue. To our knowledge, this is the first report demonstrating this correlation. The reduction of diagnostic accuracy based on queue position was substantial (4% for each added procedure). Additionally, the reduction in sensitivity and diagnostic accuracy for a procedure that was started each hour later was 1.9% and 1.7%, respectively. The latest start time for a procedure was 4 PM, giving a 17% lower diagnostic accuracy than one started at 7AM.

Although we evaluated both the procedure start time and procedure queue position as they affected diagnostic accuracy, it is likely that queue position is ultimately the more powerful determinant. Studies utilizing time-based variables are challenging but this is the most utilized surrogate for physician fatigue short of direct physician query. The procedure start time is likely to be affected by other variables and a late start time does not necessarily mean that a physician has performed a multitude of cases that day; it may actually be the first or second endoscopic procedure for that endoscopist and the late start time is due to scheduling. Queue position, however, is not subject to this limitation, and the 4% loss of diagnostic accuracy for each subsequent procedure is highly significant.

Similar to previous endoscopic trials, we utilized time-related variables and queue position was the surrogate marker for the endoscopist's fatigue. These trials have demonstrated that reduction in colon polyp detection rates over the day has traditionally been presumed to be due to operator fatigue.[8,9,16] Effective EUS imaging combines both endoscopic and radiologic skills plus the difficulty of precision targeting of potentially small lesions with an FNA needle. It follows that a more complicated and labor-intensive procedure such as EUS-FNA should be similarly affected.

An added layer of complexity accounting for a lower diagnostic yield of EUS-FNA as the day progresses may be from fatigue of the cytotechnologist who processes and interprets the initial specimen at our institution. Although a majority of the literature illuminating the effects of cytotechnologist fatigue on the processing of cytopathology specimens is in the setting of cervical smear samples, the processing of FNA samples is similar and may hold true.[14,15] Some studies have reported cytology failure rates of as high as 31% due to cytotechnologist fatigue.[18] It is difficult to separate the endoscopist's fatigue from cytological technician's failure since the two are so intertwined in EUS-FNA.

In this study, the mean number of EUS-FNA passes performed was 4.9 passes, which is comparatively high when compared to previous trials where the mean number of passes was 3. We speculate that the higher number of FNA passes required to make a diagnosis was because our cytopathologists recommend that we perform one pass that is submitted exclusively for cell block preparation. This extra pass on most patients then increases the overall mean number of passes performed.

Our study had several limitations. This was a single-center experience. Although all procedures were performed by endoscopists with multiple years of experience and at least 500 EUS-FNA examinations prior to the collection of this data set, it is possible that techniques and skill levels changed over this long time period. In addition, cytopathology technician experience and potential fatigue were not controlled and may have resulted in some bias. An on-site cytopathology technician was also not available for every procedure, particularly for procedures that started later in the day and thus, the samples were not processed until the next day, and some bias may have been introduced. Additionally, there were fewer procedures performed later in the day, which might have introduced an element of bias in the analysis of EUS-FNA performance characteristics. We were unable to assess other workloads such as outpatient clinic as well as the allocated time for each procedure, which could be possible confounding factors.

As endoscopists become busier and there is pressure to perform more procedures per session due to scheduling demands, it is important not to compromise patient care. This is especially true of a cognitively and physically demanding procedure such as EUS-FNA.

CONCLUSION

Sensitivity and diagnostic accuracy of EUS-FNA for solid pancreatic lesions declines with progressively later EUS starting times and increasing numbers of procedures before a given EUS, potentially from endoscopist fatigue and cytotechnologist fatigue.

Financial support and sponsorship

This study was funded entirely by Thomas Jefferson University Hospital and University of Utah through existing intramural funds and salary support.

Conflicts of interest

The authors attest that they have no commercial associations (e.g., equity ownership or interest, consultancy, patent and licensing agreement, or institutional and corporate associations) that might be a conflict of interest in relation to the submitted manuscript.