Accuracy of endoscopic ultrasound-fine needle aspiration of solid lesions over time: Experience from a new endoscopic ultrasound program at a Canadian community hospital

Abstract

Background and Objectives:

A Canadian Community Hospital launched a new Endoscopic Ultrasound (EUS) Program in 2011. The aim of this study was to report the accuracy of EUS-fine needle aspiration (EUS-FNA) of solid lesions over time as it pertains to cytotechnologists’ involvement and learning curves.

Methods:

The electronic medical records of patients that had a EUS from July 2011 to January 2014 were retrospectively reviewed. Only solid lesions with FNA sampling were included in the study. The primary outcome assessed was the accuracy of specimen acquisition for pathological review. The secondary outcome was diagnostic accuracy. Cases were separated by chronological order into thirds for the assessment of learning curves. Cytotechnologists’ involvement was correlated to determine its impact on accuracy.

Results:

Two hundred and seventy-one EUS-FNA procedures were completed for solid lesions. Cytotechnologists’ involvement resulted in a specimen acquisition accuracy of 82.6%, compared with 68.8% without a cytotechnologist (P = 0.009; 95% confidence interval [CI] 3.2%–25.0%). Diagnostic accuracy was 74.2% with a cytotechnologist while 62.4% without a cytotechnologist (P = 0.038; 95% CI 0.3%–23.7%). The specimen acquisition accuracy increased from 73.2% from the first third of cases to 92.3% for the last third with a cytotechnologist (P = 0.004; 95% CI 6%–33.0%). Without a cytotechnologist, the specimen accuracy was 67.6% for the first third while 57.7% for the last third of cases (P = 0.434; 95% CI − 33.9–14.4%). In the multivariable regression analysis, after adjusting for other predictors, a present cytotechnologist (P = 0.022) and lesion size 21 mm–30 mm (P = 0.039) and >30 mm (P = 0.001) were significantly associated with increased specimen acquisition accuracy. Only a present cytotechnologist (P = 0.046) was significantly associated with increased diagnostic accuracy.

Interpretation:

Cytotechnologists’ involvement significantly improved the accuracy of specimen acquisition. Although accuracy was impacted by a cytotechnologist learning curve, our results highlight the importance of a cytotechnologist being present for EUS-FNA sampling of solid lesions.

INTRODUCTION

Pancreatic cancer is the fourth leading cause of cancer-related deaths in both men and women in the United States and Canada.[1,2] It is responsible for 5.5% and 6.0% of all male and female Canadian cancer-related deaths, respectively. Based on 2007 figures, the Canadian Cancer Society estimates a 1 in 71 lifetime risk of pancreatic cancer in males and a 1 in 69 lifetime risk in females. Among all cancers in Canada, it has the lowest 5-year relative survival ratio at 8%.

Endoscopic ultrasound (EUS) is a useful tool for the diagnosis and staging of upper gastrointestinal malignancies and rectal cancers. The use of EUS has been well established since the 1990's in the investigation of solid pancreatic lesions. Although computed tomography (CT) scans are considered a first-line imaging modality for suspected pancreatic masses, CT scans may not detect pancreatic lesions <20 mm.[3] Given the ominous prognosis associated with pancreatic cancer, early identification and tissue confirmation are crucial.

EUS in comparison to other imaging modalities is the most sensitive and arguably the most specific tool available for pancreatic cancer. Historically, sensitivities of 93%–100% and specificities of 33%–100% for pancreatic cancer have been quoted.[4] A recent meta-analysis of 33 studies identified that EUS-fine needle aspiration (FNA) has a pooled sensitivity and specificity, and a positive and negative predictive value of 85%–91%, 94%–98%, 98%–99%, 65%–72%, respectively, for malignant cytology.[5] These figures were subsequently confirmed by Chen et al.'s meta-analysis of 31 studies in 2013.[6] A pooled sensitivity and specificity and a positive and negative likelihood ratio of 89%, 96%, 16.88 %, and 0.13% for pancreatic cancer were found, respectively. Diagnostic accuracy rates from 78% to 95% for solid pancreatic masses and complication rates from 0% to 2% have been quoted in the literature.[7] Given its diagnostic yield, minimal complication rate, cost-effectiveness, and theoretical lower risk of needle-tract seeding in comparison to transcutaneous aspirations, EUS is the preferred sampling technique for pancreatic masses when available.[7]

Despite these figures, numerous factors including lesion and technical characteristics can influence yield. Lesion size, location, and consistency are important considerations as larger nonindurated lesions in accessible locations can improve yield.[8] Sensitivity decreases, however, for lesions >4 cm in diameter due to a propensity for a central area of necrosis.[9] Sensitivity may also be compromised for lesions <1 cm. For larger lesions, the sampling of multiple areas with each pass, referred as the “fanning” technique, has resulted in a first-pass diagnosis in 85.7% of patients versus 57.7% with standard techniques.[10] Historically, needle size was thought to affect accuracy; however, a recent meta-analysis of needle size did not find a difference in accuracy between gauges.[11] Ideally, EUS-FNA should be completed with the least number of passes possible to minimize the complication risk and associated procedural time. Seven passes for solid pancreatic lesions have been associated with a sensitivity of 83% and specificity of 100%.[12]

The diagnostic yield has been substantially improved with rapid on-site cytopathological evaluation (ROSE). Immediate feedback and commentary by a cytopathologist on the adequacy of specimens has increased diagnostic sensitivity from 78.2% to 96.2% and has reduced the number of inadequate samples from 12.6% to 1%.[13] A recent meta-analysis found that ROSE significantly improved sample adequacy rates.[14] Due to cost, ROSE performed by a cytopathologist may not be a viable option for all centers. A reasonable alternative is a trained cytotechnologist. With additional training and experience for cytotechnologists, specimen adequacy rates were not statistically different between cytotechnologists and cytopathologists.[15] Despite efforts to improve yield and accuracy, false-negative results for malignancy may occur in up to 20%–40% of cases, and in such settings, repeat EUS-FNA should be considered if there is a strong suspicion for malignancy.[16,17]

Despite EUS training and achieving the minimum standards for competence, the learning curve is likely to continue well into practice.[18,19] Endosonographers performing a high volume of FNAs are more likely to be successful and overcome operator-dependent limitations and variability.[20] Unfortunately, published Canadian data on center-based EUS performance characteristics are limited. In addition, as the scope of EUS-based practice continues to expand in Canada, the experiences from community hospitals have not been reported. The aim of our study is to report our specimen acquisition and diagnostic accuracy rates in relation to learning curves and cytotechnologist involvement.

METHODS

Following the Research Ethics Board approval by Halton Health Sciences, the electronic medical records of patients that were evaluated by EUS at the Oakville Trafalgar Memorial Hospital in Oakville, Ontario, were retrospectively reviewed. The inclusion criteria comprised procedural completion from July 2011 to January 2014. Only solid lesions were included in the analysis.

Within the defined period, all procedures were completed by a single endosonographer who had completed a formal 6-month EUS Training Program in Marseille France. Procedures commencing in July 2011 represented the launch of a new EUS program at the Oakville Trafalgar Memorial Hospital. A PENTAX EG-3870UTK echoendoscope was used for all procedures. EUS-FNA was completed with a 19-, 22-, or 25-gauge Boston Scientific^® aspiration needle with the stylet inserted. The number of passes and immediate complications were recorded.

A cytotechnologist was present to confirm cellular contents for the majority of cases. In total, two cytotechnologists were responsible for interpretation. There were an equal number of cases completed by each cytotechnologist. Slides were prepared and stained using the Diff-Quik method. Samples were obtained until cellular adequacy was confirmed by the cytotechnologist. Additional samples were obtained for cell block preparation. The absence of a cytotechnologist resulted in specimen acquisition by the endosonographer. Samples were collected until adequate cellular contents were thought to be obtained. All specimens were processed and reviewed for adequacy and diagnosis by a general pathologist.

Baseline demographic information in addition to lesion, technical, and operator characteristics was collected. Cystic lesions and patients with incomplete medical records were excluded from the study.

The primary outcome assessed was the accuracy of adequate specimen acquisition for pathological review. Specimen acquisition accuracy was compared with and without a cytotechnologist present. Endosonographer and cytotechnologist learning curves were assessed. Cases were separated into three groups of approximately 90 cases each based on chronological order for interpretation of learning curves. Diagnostic accuracy as defined by the frequency of cases with a definitive diagnosis following pathological confirmation was the secondary outcome. Diagnostic accuracy with and without a cytotechnologist and its respective learning curves were assessed.

IBM SPSS^® Statistics for Windows, Version 22.0 (Armonk, NY, USA) and StatsDirect^®, Version 3.0 (Cheshire, UK) were used for statistical analysis. Categorical data were compared using Fisher's exact test. When two proportions were compared, the difference between the proportions and the 95% confidence interval (CI) of the difference, as well as an exact two-sided P value, was calculated using a mid-P approach to Fisher's exact test. P<0.05 was considered statistically significant.

Potential predictors for specimen acquisition accuracy and diagnostic accuracy were preselected and then analyzed by univariable analyses as well as multivariable logistic regression. A priori determined potential predictors tested in the model were cytotechnologists’ involvement, location of lesion (pancreatic head or neck, body, tail, or other), size of lesion in cm (0–1.0, 1.1–2.0, 2.1–3.0, and >3.0), size of FNA needle gauge (19, 22, and 25), and number of passes (1, 2, and 3).

RESULTS

A total of 339 EUS-FNA procedures were completed within the defined time period. Two hundred and seventy-one of these were for solid lesions. The remaining 68 procedures were for cystic lesions and were excluded from the study. Baseline demographic information on patient, lesion, and technical characteristics is outlined in Table 1.

Table 1.

Baseline demographic information of patients, lesions, and technical characteristics

The pooled specimen acquisition accuracy was 77.9%. With a cytotechnologist present, the specimen accuracy was 82.6%, and without a cytotechnologist, it was 68.8% [P = 0.009; 95% CI 3.2%–25.0%; Table 2]. The pooled diagnostic accuracy was 70.1%. The presence of a cytotechnologist resulted in a diagnostic accuracy of 74.2%. The absence of a cytotechnologist reduced the diagnostic accuracy to 62.4% (P = 0.038; 95% CI 0.3%–23.7%). If samples with an inadequate specimen for pathological interpretation are excluded, the diagnostic accuracy was identical at 89.1% with and without a cytotechnologist (P = 1.00; 95% CI 8.3%–10.9%), respectively.

Table 2.

Specimen acquisition and diagnostic accuracy rates with and without a cytotechnologist present

The specimen acquisition accuracy with a cytotechnologist present increased from 73.2% for the first third of cases to 92.3% for the last third [P = 0.004; 95% CI 6%–33.0%; Table 3]. Although a significant difference was not found from the first third to the second third of cases (73.2% vs. 80.7%; P = 0.282; 95% CI 8.2%–23.1%), the increase in specimen accuracy from the second third to the last third of cases was significant (P = 0.042; 95% CI 0.5%–24.8%). Without a cytotechnologist, the specimen accuracy was 67.6% for the first third and 57.7% for the last third of cases (P = 0.434; 95% CI 33.9%–14.4%). We did not find a statistically significant incremental increase in specimen acquisition accuracy from the first to the second (78.8%) and from the second to the third sets of cases without a cytotechnologist (P = 0.289, 95% CI 10.4%–31.9%, and P = 0.061; 95% CI 43.8%–2.8%, respectively).

Table 3.

Specimen acquisition accuracy separated by chronological order of cases

The diagnostic accuracy with a cytotechnologist present increased from 67.9% for the first third of cases to 78.5% for the last third [P = 0.159; 95% CI 5.2%–26.4%; Table 4]. No significant difference was found between the first and second third of cases (67.9% vs. 75.4%; P = 0.312; 95% CI 9.1%–24%) or between the second and last third of cases [P = 0.674; 95% CI 18.3–12.0%; Table 4]. Without a cytotechnologist, the diagnostic accuracy was 70.6% for the first third and 42.3% for the last third of cases (P = 0.023; 95% CI 50.6%–−2.9%). Diagnostic accuracy was not different between the first third to the second third of cases without a cytotechnologist (70.6% vs. 69.7%, respectively; P = 1.00, 95% CI 22.8%–21%) but significantly decreased in the last third to 42.3% (P = 0.004; 95% CI − 49.9%–−1.8%).

Table 4.

Diagnostic accuracy separated by chronological order of cases

In the univariable analysis, a present cytotechnologist (P = 0.009 and P = 0.044) and the size of the largest lesion (P < 0.001 and P = 0.002) were significantly associated with increased specimen acquisition and diagnostic accuracy, respectively [Tables 5 and 6]. In the multivariable regression analysis, after adjusting for other predictors, a present cytotechnologist (P = 0.022) and lesion size measuring 21 mm–30 mm (P = 0.039) and >30 mm (P = 0.001) were significantly associated with increased specimen acquisition accuracy [Table 7]. Multivariable regression analysis for diagnostic accuracy identified a significant association with a present cytotechnologist [P = 0.046; Table 8].

Table 5.

Potential predictors for specimen acquisition accuracy analyzed by univariable

Table 6.

Potential predictors for diagnostic accuracy analyzed by univariable analysis

Table 7.

Potential predictors for specimen acquisition accuracy analyzed by multivariable regression analysis

Table 8.

Potential predictors for diagnostic accuracy by multivariable regression analysis

DISCUSSION

Specimen acquisition and diagnostic accuracy can be improved by ROSE. In our study, due to resource limitation, ROSE was performed by a trained cytotechnologist rather than a cytopathologist. Specimen acquisition accuracy significantly increased from 68.8% to 82.6% with a cytotechnologist present for ROSE. Our findings paralleled the Canadian figures reported by Alsohaibani et al. in 2009 where a cytotechnologist lead ROSE resulted in an increase in diagnostic accuracy from 53% to 77%.[21] A recent meta-analysis supported these findings and identified ROSE as an important determinant in EUS-FNA accuracy.[22]

We reported a significant increase in diagnostic accuracy from 62.4% to 74.2% with a cytotechnologist present for ROSE. Nondiagnostic specimens often result in repeat procedures or alternative modes of diagnostic investigations and tissue acquisition. This increases healthcare-associated costs and patient burden and risk. Our rates of diagnostic accuracy were primarily driven by adequate tissue acquisition. After correcting for this, there was no difference in diagnostic accuracy with or without a cytotechnologist. Conversely, however, the number of nondiagnostic specimens increases without a cytotechnologist. Varadarajulu et al. reported that ROSE performed by a cytopathologist was 100% diagnostic.[23] This was in comparison to two or four passes with a 25-gauge needle for cell block preparation which was 81% diagnostic.

Specimen acquisition and diagnostic accuracy increased significantly from the first third to the last third of cases with a cytotechnologist present for ROSE. Without a cytotechnologist, specimen acquisition accuracy did not change and diagnostic accuracy decreased from the first third to the last third of cases. This suggests that the statistically significant increase in accuracy is primarily driven by the cytotechnologist's interpretation. Conversely, a learning curve was not identified for the endosonographer alone. A study assessing the training of a cytotechnician and the influence on accuracy revealed an in-room adequacy of 68.2% over a 12-month pretraining period where 107 patients were assessed. This was in comparison to a blind-review pathologist whose accuracy was 93.4%.[15] Following a 12-month training period, the cytotechnicians in-room adequacy increased to 87.4% over 95 cases. This was not statistically different from the blinded pathologist whose adequacy rate was 95.8%. The statistically significant increase in specimen acquisition accuracy from the second third (80.7%) to the last third (92.3%) of cases demonstrates the learning curve for the cytotechnologist. Further, the lack of a statistically significant difference in accuracy rates with and without a cytotechnologist until the last third of cases may suggest that a cytotechnologist requires 150–200 cases to achieve adequate rates of accuracy.

There was a 5-year delay from the end of endosonographer EUS training to the launch of our hospital's EUS program. Despite this, we were not able to identify an endosonographer learning curve for accuracy. This was consistent with what was reported by Eloubeidi and Tamhane in 2005.[18] However, Eloubeidi and Tamhane reported a learning curve for complications and number of passes with fewer of each at the end of training. Given the completion of EUS training, this may explain the lack of a learning curve for complications and number of passes.

Although the literature reports a reduction in the number of passes, complications, and the potential need for repeat or additional investigations, ROSE may not be a viable, cost-effective strategy for many centers or institutions. It was estimated based on the US figures that ROSE performed by a cytopathologist accrued an additional 40–50 US dollars per procedure.[24] This in addition to the 476 Canadian Dollars required for a diagnostic EUS.[25] Despite accessibility, coordination is required by the endosonographer and cytopathologist or cytotechnologist to ensure timely interpretation of samples. In our study, the 93 procedures not attended by a cytotechnologist were due to lack of availability or inadequate coordination between the endoscopy department, endosonographer, and cytotechnologist. The number of procedures not attended by a cytotechnologist did not differ between the first, second, and last third of cases. Nonetheless, our results including univariable and multivariable analyses highlight the importance of the cytotechnologist being present for ROSE.

Hypothesis to rationalize the increase in specimen and diagnostic accuracy from the first third to the last third of cases include increased cytotechnologist experience and familiarity with the processing and interpretation of samples. New EUS programs should expect accuracy rates to improve over time, as cytotechnologist, endosonographers and pathologists gain experience. Continual coordination and discussions between all parties is paramount to improve EUS programs. The decrease in endosonographer alone specimen and diagnostic accuracy from the second to the last third of cases may represent duplicate nondiagnostic and technically challenging procedures, endosonographer selection bias of difficult cases given increased experience and volume. A referral bias of more technically challenging cases, given the widespread dissemination and acceptance of a EUS program, may be a potential explanation. Ultimately, one can surmise that EUS-FNA should not be done in the absence of a cytotechnologist or cytopathologist.

Alternative means to increase accuracy without a cytotechnologist include endosonographer training in basic cytopathology and self-assessment of specimen adequacy, cell block preparation of multiple samples, and dynamic telecytology.[19] Although the two former options may be cost-effective, telecytological interpretation may accrue larger upfront costs for equipment. Ultimately, for a hospital or institution launching a new EUS program, investing in a cytotechnologist may be a cost-effective measure to increasing accuracy rates. Our data suggest that this investment should be considered earlier rather than later to account for learning curves. This should minimize the need for duplicate or alternative procedures or diagnostics and should reduce patient burden and complications.

CONCLUSION

For new or existing EUS programs, ROSE may be in the best interests of the endosonographer, patient, hospital/institution and governing bodies, although further data are required to confirm its cost-effectiveness in Canada.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.