Abstract
Background and study aims In this study, we evaluated the performance of community hospitals involved in the Dutch quality in endosonography team regarding yield of endoscopic ultrasound (EUS)-guided tissue acquisition (TA) of solid pancreatic lesions using cumulative sum (CUSUM) learning curves. The aims were to assess trends in quality over time and explore potential benefits of CUSUM as a feedback-tool.
Patients and methods All consecutive EUS-guided TA procedures for solid pancreatic lesions were registered in five community hospitals between 2015 and 2018. CUSUM learning curves were plotted for overall performance and for performance per center. The American Society of Gastrointestinal Endoscopy-defined key performance indicators, rate of adequate sample (RAS), and diagnostic yield of malignancy (DYM) were used for this purpose. Feedback regarding performance was provided on multiple occasions at regional interest group meetings during the study period.
Results A total of 431 EUS-guided TA procedures in 403 patients were included in this study. The overall and per center CUSUM curves for RAS improved over time. CUSUM curves for DYM revealed gradual improvement, reaching the predefined performance target (70 %) overall, and in three of five contributing centers in 2018. Analysis of a sudden downslope development in the CUSUM curve of DYM in one center revealed temporary absence of a senior cytopathologist to have had a temporary negative impact on performance.
Conclusions CUSUM-derived learning curves allow for assessment of best practices by comparison among peers in a multidisciplinary multicenter quality improvement initiative and proved to be a valuable and easy-to-interpret means to evaluate EUS performance over time.
Introduction
Endoscopic ultrasound (EUS)-guided tissue acquisition (TA) is first choice for establishing a tissue diagnosis in suspected pancreatic cancer 1 . The increasing use of neoadjuvant chemotherapy for pancreatic carcinoma, and the fact that neoadjuvant treatments require pathological confirmation of the diagnosis, have rendered quality of EUS-guided TA of solid pancreatic lesions ever more important 2 3 . Proficiency in EUS-guided TA can only be reached in centers in which all its aspects, including TA, tissue handling, microscopic assessment and reporting, are safeguarded. Feedback on performance is key to improving quality 4 .
In 2015, the American Society of Gastrointestinal Endoscopy (ASGE) defined the following key performance indicators (KPIs) for EUS-guided TA in solid pancreatic lesions: rate of adequate sample (RAS) with a performance target of 85 %, diagnostic yield of malignancy (DYM) with a performance target of 70 %, and sensitivity for malignancy (SFM) with a performance target of 85 % 5 . RAS mainly reflects the quality of the process within the endoscopy suite (TA, preparation of smears, including transport to the cytopathology lab), whereas DYM and SFM reflect the quality of the entire process, including patient selection, specimen preparation, microscopic assessment and reporting.
Currently, quality control for the yield of EUS-guided TA is not customary or required for centers performing EUS-guided TA. Quality measurements for EUS-guided TA procedures were previously described as a monitoring tool during the development of academic or regional EUS programs 6 7 8 . Wani et al. used CUSUM curves to describe the development of competence in advanced endoscopy trainees performing both EUS and ERCP 9 10 11 12 13 . CUSUM curves reflect development of quality delivered in time relative to predefined performance targets.
In 2015 the Dutch Quality in Endosonography Team (QUEST) was founded. This is a regional EUS interest group, consisting of endosonographers and pathologists from five community hospitals in the Netherlands. QUEST aims to improve performance of EUS-guided TA by providing feedback on KPIs of individual centers based on a prospective registration of consecutive EUS-guided TA procedures of solid pancreatic lesions. This has led to improvements in RAS (80 % to 95 %), DYM (28 % to 64 %), and SFM (63 % to 84 %) comparing the results of an initial retrospective analysis of yield of EUS-guided TA to the first 21 months of prospective registration 14 .
This study evaluated the use of CUSUM curves to monitor performance of contributing centers regarding the yield of EUS-guided TA of solid pancreatic lesions. Using this tool, we aimed to assess trends in KPIs over time, and explore potential benefits of CUSUM curves as a feedback-tool.
Patients and methods
This was a prospective, multicenter, quality improvement study of consecutive EUS-guided TA procedures on solid pancreatic lesions conducted in five community hospitals in the Netherlands. The local medical ethics committee (METC Zuidwest Holland 17–038) approved the study protocol. Informed consent was obtained from all patients. The study is registered in the Dutch trial registry (NTR) with trial number NL9470.
Study population and data collection
All patients aged 18 and older with a solid pancreatic lesion with high suspicion of malignancy who underwent an EUS-guided TA procedure were eligible for this study. Primary outcome parameters were CUSUM-derived learning curves with RAS and DYM as input parameters. RAS was defined as proportion of procedures yielding specimen sufficient for cytopathological and/or histopathological analysis. DYM was defined as the proportion of procedures yielding a “suspicious for malignancy” or a “malignant” diagnosis. The secondary outcome parameter was SFM. SFM was defined as the total of true positives (“suspected malignancy” or “malignancy” based on EUS-guided TA with a malignancy as final diagnosis) divided by all patients with a final diagnosis of malignancy.
Collected data on EUS-guided TA procedures included: patient demographics, localization of the pancreatic mass, hospital, endosonographer, pathologist, needle diameter ( < 22-gauge or 22-gauge), type of needle (fine-needle aspiration [FNA]/fine-needle biopsy [FNB]), number of passes, use of suction (slow withdrawal of stylet or vacuum suction), availability of rapid on-site specimen evaluation (ROSE), and the result of the cytopathological and/or histopathological evaluation of the EUS-guided TA specimen. Based on current practice guidelines and previous experience of our group, endosonographers were advised to perform at least three passes with FNA needles or at least two passes with FNB needles (unless ROSE detected sufficient material for diagnosis earlier), and to use vacuum suction 14 15 . All other techniques and materials used were at the discretion of the local clinicians and according to local availability of equipment and hospital standards.
The results of cytopathological and/or histopathological evaluation were classified as follows: non-diagnostic, benign, atypical, suspicious for malignancy, and malignant. Neuroendocrine tumors were classified as malignant. For the purpose of this study “suspicious for malignancy” and “malignant” were both considered malignant. All types of pancreatic and periampullary malignancies were considered a malignant reference standard. The gold standard for a malignant diagnosis was based on either histopathological diagnosis after surgical resection or progression of disease compatible with malignancy during a minimum of 12 months of follow-up.
Feedback on performance
Regional interest group meetings were organized three times a year. Prior to meetings, all contributors received data regarding the performance of their individual center accompanied by anonymized benchmark data from the other centers. At the regional interest group meetings, the results of prospective registration, best practices, guidelines, and difficult cases were discussed. Until 2017, feedback on performance overall and per center was provided as RAS, DYM, and SFM (proportions). From 2018 onward, visual feedback by means of CUSUM curves of RAS and DYM was also provided. At meetings all data (numbers and CUSUM curves) were presented (in an anonymized fashion) and subsequently discussed. Participating endosonographers and pathologists were invited to reflect on changes in directions of the curves provided. Significant changes in the direction of the curve were subjected to further analysis, of which, the results were discussed separately with the practitioners from the centers involved, prior to the next general meeting. At a subsequent meeting, the results of these analyses were presented and discussed, with emphasis on potential learning opportunities for all participants. All gastroenterologists and pathologists involved had completed their training at least 3 years before the start of this study 14 .
Statistics
Cumulative sum analysis (CUSUM)
Each EUS procedure was scored as a success (adequate sample/malignant outcome) or failure (inadequate sample/non-malignant outcome). Each success is rewarded with adding score s, each failure results in subtraction of (1 – s). Each procedure is a dot in the learning curve that is created by a plot of the cumulative sum of all cases in chronological order.
The acceptable rates (P0) and unacceptable rates (P1) were defined based on the ASGE KPIs and a previous publication by Eltoum et al. 16 . For inadequate samples, we designated 10% as acceptable (P0) and 15 % as unacceptable (P1) rates. For a nonmalignant outcome of the EUS, the P0 was defined as 25 % and the P1 as 30 %.
Decision limits
Two decision limits (h1 and h0) were calculated. The decision limits are calculated based on type I (α) and type II (β) errors. A type I error is the risk of rejection of a true null hypothesis and a type II error is the risk of non-rejection of a false null hypothesis. The formulas that are used to calculate h0 and h1 were previously described 16 . The meaning of the decision limits in relation to the curve can be explained as follows: 17 18
If the learning curve crosses the upper decision limit, the failure rate is within the preset acceptable range and it reflects high quality.
If the learning curve crosses the lower decision limit, the failure rate is above the preset unacceptable rates and an intervention is needed.
If the learning curve remains between the two decision limits, the performance is within the preset acceptable range.
CUSUM charts
CUSUM charts were constructed using Excel. Each success (adequate sample/malignant outcome) contributes to an upward slope of the CUSUM curve. Each inadequate sample will contribute to a downward slope of the CUSUM curve. A downslope curve means that the key performance indicator is not met. A horizontal curve indicates that quality is up to standards. An upslope curve signifies quality is above the predefined key performance indicator threshold.
Multivariable analysis
To investigate the association of RAS and DYM with procedure characteristics, we fitted logistic mixed models. Given the limited number of inadequate samples, only two parameters (suction: yes/no and ROSE: yes/no) could be included in the RAS model.
The model for the DYM included the variables suction type (no, slow withdrawal of stylet or vacuum), ROSE, number of passes (continuous), needle size (< 22-gauge, 22-gauge) and needle type (FNA or FNB). In both models we used endoscopist specific (random) intercepts to take into account that samples obtained by the same endoscopist may not be independent. The model for DYM also included a pathologist specific (random) intercept. Both models were fitted in the Bayesian framework, which allowed us to include observations for which some of the covariates were missing. We used normal priors with mean 0 and standard deviation 100 for all regression coefficients. The Bayesian models were fitted using Markov chain Monte Carlo, with the help of the freely available and widely used “JAGS” software 19 that uses Gibbs sampling and provides a wide range of samplers to sample from full-conditional distributions that do not have a closed form. Results are presented as posterior mean and 95 % confidence interval (CI). Calculations were performed in R version 4.0.2 (2020–06–22) (R Core Team 2020) and the package JointAI 1.0.0.9000 20 . Missing observations were imputed during the analysis.
Results
From January 2015 until December 2018, 431 EUS-guided TA procedures on solid pancreatic lesions in 403 individual patients were included. The median age of the patients was 68 years (range 27–88), and 51 % were men. During follow-up, a pancreatic or periampullary malignancy (reference standard) was diagnosed in 87 % of all cases. Per hospital, two to four endosonographers were involved in these procedures. A wide range of eight to sixteen pathologists per hospital were involved ( Table 1 ).
Table 1. Characteristics of the participating patients and hospitals.
| Total cohort (n = 403) |
A (n = 79) | B (n = 88) | C (n = 81) | D (n = 94) | E (n = 61) | |
| Sex male, n (%) | 206 (51 %) | 43 (54 %) | 42 (48 %) | 40 (49 %) | 54 (57 %) | 27 (44 %) |
| Median age in years (range) | 68 (27–88) | 70 (42–86) | 68 (43–86) | 68 (27–87) | 67 (33–88) | 68 (35–88) |
| Reference standard malignant, n (%) | 351 (87 %) | 69 (87 %) | 77 (88 %) | 68 (84 %) | 81 (86 %) | 56 (92 %) |
| Number of endoscopists involved | 15 | 2 | 4 | 2 | 3 | 4 |
| Number of pathologists involved | 39 | 16 | 8 | 8 | 8 | 14 |
Rate of adequate sample overall and per hospital
A total of 399 of 431 procedures yielded an adequate sample. Hence, RAS was 93 % for the complete cohort (range 86 %–99% among individual hospitals). The ASGE-defined KPI of RAS ≥ 85 % was met overall and in each of the individual hospitals ( Table 2 ). This can also be appreciated from the upslope direction of the overall learning curve drawn for this parameter ( Supplementary Fig. 1 ). The RAS learning curves of the individual hospitals indicate adequate and stable quality (curves between the decision limits) in Hospitals A, B, and E, and adequate and improving quality in Hospitals C and D ( Supplementary Fig. 2, Supplementary Fig. 3, Supplementary Fig. 4, Supplementary Fig. 5, Supplementary Fig. 6 ).
Table 2. Values of RAS, DYM and SFM for the complete cohort and per hospital.
| Hospital | No. of procedures | RAS | DYM | SFM |
| A | 87 | 75 (86 %) | 53 (61 %) | 68 % |
| B | 91 | 82 (90 %) | 57 (63 %) | 71 % |
| C | 90 | 87 (97 %) | 59 (66 %) | 79 % |
| D | 100 | 99 (99 %) | 75 (75 %) | 87 % |
| E | 63 | 56 (89 %) | 41 (65 %) | 73 % |
| Total cohort | 431 | 399 (93 %) | 285 (66 %) | 76 % |
Italics: equal or above ASGE performance target.
RAS, rate of adequate sample; DYM, diagnostic yield of malignancy; SFM, sensitivity for malignancy.
Diagnostic yield of malignancy overall and per hospital
A total of 285 of 431 procedures yielded a malignant diagnosis. Therefore, the overall DYM was 66 % (ranging from 61 %–75 % in the individual hospitals). This is below the KPI of DYM ≥ 70 % ( Table 2 ). The overall learning curve of this parameter has a downslope direction (crossing the lower decision limit) until January 2018 ( Fig. 1a ). From this point onward, the curve has a more horizontal direction between the newly constructed decision limits, indicating an adequate and stable quality throughout 2018 ( Fig. 1a and Fig. 1b ).
Fig. 1 .
DYM CUSUM learning curve of the complete cohort. a January 2015 to December 2018. b January 2018 to December 2018. DYM, diagnostic yield of malignancy.
In only one of the contributing hospitals (Hospital D) the KPI of DYM ≥ 70 % was met overall ( Table 2 ). However, the learning curves of the individual hospitals for this parameter developed from an initial downslope (Hospitals B and E) or horizontal direction (Hospitals C and D) into a horizontal (Hospitals B, C, and E) or an upslope direction (Hospital D) ( Fig. 2a , Fig. 3a , Supplementary Fig. 7a, Supplementary Fig. 8a, Supplementary Fig. 9a ). This indicates a gradual improvement in these centers up to an adequate quality level in 2018.
Fig. 2.
DYM CUSUM curve of hospital a January 2015 to December 2018. b January 2018 to December 2018. Black arrow marks the decrease in evaluating pathologists from nine to three. DYM, diagnostic yield of malignancy.
Fig. 3.
DYM CUSUM curve of hospital C. a January 2015 to December 2018. b October 2017 to December 2018. Black arrows mark the temporarily absence of one experienced cytopathologist. DYM, diagnostic yield of malignancy.
The CUSUM curve for Hospital B started with a downward slope and in January 2018, the curve suddenly improved to a horizontal slope ( Fig. 2a and Fig. 2b ).
The curve of Hospital C initially showed a stable and adequate quality until May 2017. From this point onward there was a remarkable short and sharp downslope development of the curve, which again developed in a more horizontal direction from September 2017 onward ( Fig. 3a and Fig. 3b ). This indicates a 4-month episode during which a significantly lower number of malignant diagnoses were made. During these 4 months, a high proportion of specimens with atypia (40 %) was graded in comparison to the episodes prior to May 2017 (4 %) and from September 2017 onward (11 %) ( Supplementary Table 1 ). The 4-month episode coincided with the temporary absence of the most experienced cytopathologist in this center, who had been involved in all cytopathological evaluations of pancreatic lesions in the previous years in this hospital.
Sensitivity for malignancy overall and per hospital
The overall SFM for the contributing hospitals throughout the 4 years of this study was 76 %, ranging from 68 % to 87 % among different hospitals. The KPI of SFM ≥ 85 % was not met in four of five contributing hospitals. The developments in the learning curves regarding DYM suggest improvement in quality in the majority of these centers. In 2018, the final year of this study, the overall SFM was 85 %, ranging from 69 % to 96 % among the centers. In this year, the KPI of SFM ≥ 85 % was met in three of five centers ( Supplementary Table 2 ).
FNB versus FNA needles
A total of 282 FNA procedures and 127 FNB procedures were performed. The outcome of FNA and FNB procedures was similar ( Supplementary Table 3 ) The use of FNB needles did not increase over time.
Multivariable analysis
Nine observations for which all covariates were missing were excluded from the analysis. Missing values in the remaining 422 observations were imputed (missing values: suction type 4.7 %, needle brand 2.8 %, number of passes 2.1 %, needle size 1.7 %, needle type 0.9 %, ROSE 0.2 %, and suction 0.2 %). The use of any type of suction and the presence of ROSE were positively associated with RAS, with odds ratios of 3.2, 95 % CI (1.1–7.8) and 2.8, 95 % CI (1.1–8.4), respectively ( Table 3 ). There was no clear evidence that any of the covariates considered was associated with DYM ( Table 3 ).
Table 3. Odds ratios and corresponding 95 % CIs for the logistic mixed models for RAS and DYM.
| RAS | DYM | ||||
| Covariate | OR | 95 % CI | Covariate | OR | 95 % CI |
| Use of suction (vacuum and/or slow-withdrawal of stylet) | 3.2 | 1.1 – 7.8 | No suction | 0.7 | 0.3 – 1.6 |
| ROSE | 2.8 | 1.1 – 8.4 | Vacuumsuction | 1.1 | 0.5 – 2.3 |
| ROSE | 1.5 | 0.9 – 2.4 | |||
| Number of passes | 1 | 0.8 – 1.4 | |||
| < 22G needle (FNA and/or FNB) | 1.5 | 0.4 – 4.9 | |||
| 22G needle (FNA and/or FNB) | 0.9 | 0.6 – 1.5 | |||
| FNB | 1.1 | 0.7 – 2.1 | |||
There were missing values in seven covariates, with a percentage of missing observations
per variable ranging from 0 % to 5 %. These missing observations were imputed during the analysis.
RAS, rate of adequate sample; DYM, diagnostic yield of malignancy; OR, odds ratio; CI, confidence
interval; FNA, fine needle aspiration; FNB, fine needle biopsy; ROSE, rapid on-site evaluation.
Feedback and interpretation of curve deflections
During the 4 years of prospective registration, the following changes were reported by contributing practitioners. Hospitals A, D and E requested ROSE on a regular basis, which they did not do before. Hospital A started with ROSE halfway into 2016, Hospital D from January 2018 onward, and Hospital E at beginning of 2016. In Hospitals B and C, there were changes in the number of pathologists involved in EUS-guided TA procedures of the pancreas. In Hospital B, the group of pathologists that reviewed pancreatic samples collected with EUS was downsized from eight to three in January 2018. The most experienced cytopathologist from Hospital C was temporarily absent during a 4-month period in 2017.
The time that the events previously described took place are marked with an arrow in Fig. 2a , Fig. 3a , Supplementary Fig. 6, Supplementary Fig. 7a, Supplementary Fig. 8, Supplementary Fig. 8a , and Supplementary Fig. 9a .
Discussion
This study evaluated the performance of five community hospitals regarding the yield of EUS-guided TA of solid pancreatic lesions using CUSUM curves to assess trends in quality over time and explored potential benefits of CUSUM curves as a feedback tool. Throughout the 4 years of this study, all three ASGE defined KPIs improved. A KPI for RAS ≥ 85 % was met consistently in most of the centers and overall (93 %). A KPI of DYM ≥ 70 % was not met overall throughout the study between 2015 and 2018, but eventually yielded 75 % overall in 2018. Similarly, the KPI for SFM ≥ 85 % was not met overall from 2015 to 2018, but improved to 85 % in 2018. Because not all ASGE-defined KPIs are consistently met in each center, feedback on performance and analyses for potential improvements are indicated and ongoing.
The diagnostic yield of EUS-guided TA for solid pancreatic lesions is considered a benchmark for quality measurements in EUS 1 . However, the majority of studies in which the ASGE-defined KPI are based were performed in tertiary care facilities 21 . Moreover, the majority of publications on EUS-guided TA in solid pancreatic lesions were controlled trials focusing on discrete factors influencing the yield, i. e. different types and diameters of needles, use of suction, the use of ROSE, or the optimal number of passes to perform 22 23 24 25 26 27 28 29 30 31 32 33 34 . Therefore, when comparing the current study to these previous publications, it cannot be ruled out that differences regarding patient selection may have influenced yield of EUS-guided TA. Nevertheless, questioning the generalizability of the benchmark data may never be an excuse to stop monitoring and improving your performance.
To improve quality of EUS-guided TA, it is necessary to provide feedback on performance. For providing feedback, CUSUM-derived learning curves have several advantages over tables with numbers. First, their interpretation is easy and does not require any knowledge about specific KPI values (a downward trend is not good, a horizontal line is good, and an upward trend is better). Second, they allow determination of best practices and comparison among peers. Third, they provide a more detailed picture of development over time, allowing for focused analysis of performance within specific timeframes 35 . The analysis of the sudden downslope deflection in the DYM curve of Hospital C, coinciding with the 4-month absence of a senior cytopathologist, is an excellent example of this. Analysis of this specific example teaches us how vulnerable the multistep process of EUS-guided TA is, being dependent on each factor or operator involved. Therefore, the discriminating advantage of learning curves for feedback over tables with numbers is that they provide additional learning opportunities.
RAS and DYM are obviously related. However, because CUSUM curves of these variables reflect quality relative to a predefined quality target, they do not necessarily develop in the same direction. An upward RAS curve, therefore, does not mean the DYM curve has to be upward as well. In other words: Having a sample that contains at least a couple of cells from the target organ (adequate sample) does not automatically mean that a pathologist will be confident about the malignant origin of the lesion. This can lead to a RAS above the performance target and a DYM and SFM below the performance target.
Supported by feedback provided by CUSUM analyses, several changes regarding protocols and/or staff involved were made in individual hospitals. In Hospital C today, a pathology report regarding pancreatic cytology or histopathology can only be finalized after consent of a dedicated cytopathologist. Several hospitals implemented routine use of ROSE and the number of pathologists involved was reduced in one of the centers. Although multivariable analysis supports the use of suction and ROSE to be beneficiary to RAS, an overall positive effect of these changes can be assumed. After all, with a RAS of 85 %, the lowest acceptable level according to ASGE definitions, the SFM can never exceed 85 %, and makes DYM ≥ 70 % in patients with solid pancreatic lesions difficult to achieve.
To our knowledge, this is the largest prospective multicenter study of EUS-guided TA of solid pancreatic lesions from community hospitals and the first to implement CUSUM-derived learning curves as a tool for monitoring and improving KPI of these procedures. Previous publications on the use of CUSUM curves in EUS-guided TA investigated performance of either cytopathologists or endoscopy trainees 9 10 11 12 13 16 . In contrast to these studies, we used CUSUM curves to evaluate the entire process defining quality and yield of these procedures, including the work of both endosonographers and cytopathologists. Some of the data presented in this study (133 procedures, performed from January 2015 to September 2016) were previously described in the initial publication about this community hospital quality initiative 14 . The current study shows ongoing and persistent improvement in performance and introduces learning curves as a feedback and monitoring tool.
The main limitation of this study is the fact that feedback, either in tables with numbers or as learning curves, was not provided real time. Ideally, CUSUM curves would have been drawn three times a year, enabling contributing centers to respond more quickly to changes in curve directions. Because of logistic challenges and the time-consuming nature of data collection, this could not be realized in the current study. Another limitation is the fact that in the current study, no subtypes of FNB needles were recorded. Recent publications indicate improved outcome with a subtype of FNB needles over FNA needles 36 . The fact that no difference between FNA and FNB was detected in our study may be related to the unclear mix of subtypes of FNB needles used. However, other confounders such as the endosonographer learning curve for a new type of needle or pathologist learning curve for evaluating tissue cores may have been involved.
Future directions
Performing EUS-guided TA comes with the responsibility to measure KPI regarding these procedures. To facilitate this, an automated system is needed allowing EUS-procedural parameters and concomitant pathology reports to be added on regular basis. Subsequently CUSUM curves can be constructed based on KPI data at any point in time, allowing for constant trend analysis thereby providing the fundament for quality improvement. We believe that feedback on KPI is an essential first step for quality improvement. If KPIs are not up to par, this should be followed by a cycle of protocol changes and continued KPI measurements and evaluations (plan-do-check-act cycle), aiming for continuous improvement of quality and life-long learning opportunities for all collaborators.
Changes in protocol are to be tailored and center-specific, depending on KPI measurements and available resources. A measure aiming to increase a low adequate sample rate in a center using 22-gauge FNA needles, three passes and suction, for example could be: 1. The introduction of ROSE; or 2. The introduction of an FNB needle. If the hospital involved does not have its own cytopathology lab, implementation of FNB needles could solve their problem. A measure aiming to increase DYM, with current adequate RAS and high proportions of atypia diagnoses, for example, might be: 1. Reorganization of the workflow in the pathology lab to have all samples evaluated by two cytopathologists instead of seven; 2. Introducing liquid based cytology instead of smears only; or 3. Introducing the use of FNB needles. There is evidence to support that changes made “bottom-up” are more likely to be sustained in comparison to changes implemented “top-down” 37 .
Conclusions
In conclusion, this prospective multicenter study using CUSUM-derived learning curves for both quality monitoring and feedback demonstrates consistent improvement of KPIs RAS, DYM, and SFM over time. It illustrates the benefits of using learning curves with easy-to-interpret feedback regarding performance of a whole process or its individual components while also allowing comparison with peers. Use of CUSUM curves is an excellent way for responsible staff to monitor and scrutinize their performance and improve the outcome of KPI up to the desired level.
Footnotes
Competing interests The authors declare that they have no conflict of interest.
Supplementary material :
References
- 1.Wani S, Wallace M B, Cohen J et al. Quality indicators for EUS. Am J Gastroenterol. 2015;110:102–113. doi: 10.1038/ajg.2014.387. [DOI] [PubMed] [Google Scholar]
- 2.Kitano M, Yoshida T, Itonaga M et al. Impact of endoscopic ultrasonography on diagnosis of pancreatic cancer. J Gastroenterol. 2019;54:19–32. doi: 10.1007/s00535-018-1519-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Tempero M A, Malafa M P, Al-Hawary M et al. Pancreatic Adenocarcinoma, Version 2.2017, NCCN Clinical Practice Guidelines in Oncology. J Natl Compr Canc Netw. 2017;15:1028–1061. doi: 10.6004/jnccn.2017.0131. [DOI] [PubMed] [Google Scholar]
- 4.Kaye A D, Okanlawon O J, Urman R D. Clinical performance feedback and quality improvement opportunities for perioperative physicians. Adv Med Educ Pract. 2014;5:115–123. doi: 10.2147/AMEP.S62165. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Wani S, Wallace M B, Cohen J et al. Quality indicators for EUS. Gastrointest Endosc. 2015;81:67–80. doi: 10.1016/j.gie.2014.07.054. [DOI] [PubMed] [Google Scholar]
- 6.Eloubeidi M A.Developing an academic EUS program: the University of Alabama at Birmingham experience Gastrointest Endosc 2007651039–1041.; discussion 1039 [DOI] [PubMed] [Google Scholar]
- 7.Gordon H M, Lloyd D AJ, Higginson A et al. A regional EUS service using a collaborative network. Frontline Gastroenterol. 2017;8:26–28. doi: 10.1136/flgastro-2016-100716. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Oppong K W, Richardson D L, Charnley R M et al. The development and evolution of a tertiary pancreaticobiliary endoscopic ultrasound service: lessons learned. Frontline Gastroenterol. 2011;2:66–70. doi: 10.1136/fg.2010.003814. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Wani S, Keswani R N, Han S.Competence in endoscopic ultrasound and endoscopic retrograde cholangiopancreatography, from training through independent practice Gastroenterology 20181551483–1494.e1487 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Wani S, Cote G A, Keswani R et al. Learning curves for EUS by using cumulative sum analysis: implications for American Society for Gastrointestinal Endoscopy recommendations for training. Gastrointest Endosc. 2013;77:558–565. doi: 10.1016/j.gie.2012.10.012. [DOI] [PubMed] [Google Scholar]
- 11.Lee L S, Andersen D K, Ashida R et al. EUS and related technologies for the diagnosis and treatment of pancreatic disease: research gaps and opportunities-Summary of a National Institute of Diabetes and Digestive and Kidney Diseases workshop. Gastrointest Endosc. 2017;86:768–778. doi: 10.1016/j.gie.2017.08.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Wani S, Hall M, Keswani R N.Variation in aptitude of trainees in endoscopic ultrasonography, based on cumulative sum analysis Clin Gastroenterol Hepatol 2015131318–1325.e1312 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Wani S, Hall M, Wang A Y et al. Variation in learning curves and competence for ERCP among advanced endoscopy trainees by using cumulative sum analysis. Gastrointest Endosc. 2016;83:711–719 e711. doi: 10.1016/j.gie.2015.10.022. [DOI] [PubMed] [Google Scholar]
- 14.Quispel R, van Driel L, Honkoop P et al. Collaboration of community hospital endosonographers improves diagnostic yield of endoscopic ultrasonography guided tissue acquisition of solid pancreatic lesions. Endosc Int Open. 2019;7:E800–E807. doi: 10.1055/a-0898-3389. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Dumonceau J M, Deprez P H, Jenssen C et al. Indications, results, and clinical impact of endoscopic ultrasound (EUS)-guided sampling in gastroenterology: European Society of Gastrointestinal Endoscopy (ESGE) Clinical Guideline – Updated January 2017. Endoscopy. 2017;49:695–714. doi: 10.1055/s-0043-109021. [DOI] [PubMed] [Google Scholar]
- 16.Eltoum I A, Chhieng D C, Jhala D et al. Cumulative sum procedure in evaluation of EUS-guided FNA cytology: the learning curve and diagnostic performance beyond sensitivity and specificity. Cytopathology. 2007;18:143–150. doi: 10.1111/j.1365-2303.2007.00433.x. [DOI] [PubMed] [Google Scholar]
- 17.Williams S M, Parry B R, Schlup M M. Quality control: an application of the cusum. BMJ. 1992;304:1359–1361. doi: 10.1136/bmj.304.6838.1359. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Davies O L. Malden, USA: Wiley-Blackwell Publishing; 1955. Design and analysis of industrial experiments: Statistica Neerlandica. [Google Scholar]
- 19.[Anonymous]. JAGS 4.3.0. In: Source Forge
- 20.Erler N S, Rizopoulos D, Lesaffre E MEH. California, USA: Foundation for Open Access Statistics; 2020. JointAI: Joint Analysis and imputation of incomplete data in R. [Google Scholar]
- 21.Hewitt M J, McPhail M J, Possamai L et al. EUS-guided FNA for diagnosis of solid pancreatic neoplasms: a meta-analysis. Gastrointest Endosc. 2012;75:319–331. doi: 10.1016/j.gie.2011.08.049. [DOI] [PubMed] [Google Scholar]
- 22.Vanbiervliet G, Napoleon B, Saint Paul MC et al. Core needle versus standard needle for endoscopic ultrasound-guided biopsy of solid pancreatic masses: a randomized crossover study. Endoscopy. 2014;46:1063–1070. doi: 10.1055/s-0034-1377559. [DOI] [PubMed] [Google Scholar]
- 23.Tarantino I, Di Mitri R, Fabbri C et al. Is diagnostic accuracy of fine needle aspiration on solid pancreatic lesions aspiration-related? A multicentre randomised trial. Dig Liver Dis. 2014;46:523–526. doi: 10.1016/j.dld.2014.02.023. [DOI] [PubMed] [Google Scholar]
- 24.Laquiere A, Lefort C, Maire F et al. 19 G nitinol needle versus 22 G needle for transduodenal endoscopic ultrasound-guided sampling of pancreatic solid masses: a randomized study. Endoscopy. 2019;51:436–443. doi: 10.1055/a-0757-7714. [DOI] [PubMed] [Google Scholar]
- 25.Lee J K, Lee K T, Choi E R et al. A prospective, randomized trial comparing 25-gauge and 22-gauge needles for endoscopic ultrasound-guided fine needle aspiration of pancreatic masses. Scand J Gastroenterol. 2013;48:752–757. doi: 10.3109/00365521.2013.786127. [DOI] [PubMed] [Google Scholar]
- 26.Noh D H, Choi K, Gu S et al. Comparison of 22-gauge standard fine needle versus core biopsy needle for endoscopic ultrasound-guided sampling of suspected pancreatic cancer: a randomized crossover trial. Scand J Gastroenterol. 2018;53:94–99. doi: 10.1080/00365521.2017.1390597. [DOI] [PubMed] [Google Scholar]
- 27.Bang J Y, Hebert-Magee S, Trevino J et al. Randomized trial comparing the 22-gauge aspiration and 22-gauge biopsy needles for EUS-guided sampling of solid pancreatic mass lesions. Gastrointest Endosc. 2012;76:321–327. doi: 10.1016/j.gie.2012.03.1392. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Woo Y S, Lee K H, Noh D H et al. 22G versus 25G biopsy needles for EUS-guided tissue sampling of solid pancreatic masses: a randomized controlled study. Scand J Gastroenterol. 2017;52:1435–1441. doi: 10.1080/00365521.2017.1322136. [DOI] [PubMed] [Google Scholar]
- 29.Crino S F, Le Grazie M, Manfrin E et al. Randomized trial comparing fork-tip and side-fenestrated needles for EUS-guided fine-needle biopsy of solid pancreatic lesions. Gastrointest Endosc. 2020;92:648–658 e642. doi: 10.1016/j.gie.2020.05.016. [DOI] [PubMed] [Google Scholar]
- 30.Capurso G, Archibugi L, Petrone M C et al. Slow-pull compared to suction technique for EUS-guided sampling of pancreatic solid lesions: a meta-analysis of randomized controlled trials. Endosc Int Open. 2020;8:E636–E643. doi: 10.1055/a-1120-8428. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Kudo T, Kawakami H, Hayashi T et al. High and low negative pressure suction techniques in EUS-guided fine-needle tissue acquisition by using 25-gauge needles: a multicenter, prospective, randomized, controlled trial. Gastrointest Endosc. 2014;80:1030–1037 e1031. doi: 10.1016/j.gie.2014.04.012. [DOI] [PubMed] [Google Scholar]
- 32.Wani S, Early D, Kunkel J et al. Diagnostic yield of malignancy during EUS-guided FNA of solid lesions with and without a stylet: a prospective, single blind, randomized, controlled trial. Gastrointest Endosc. 2012;76:328–335. doi: 10.1016/j.gie.2012.03.1395. [DOI] [PubMed] [Google Scholar]
- 33.Wang Y, Wang R H, Ding Z et al. Wet- versus dry-suction techniques for endoscopic ultrasound-guided fine-needle aspiration of solid lesions: a multicenter randomized controlled trial. Endoscopy. 2020;52:995–1003. doi: 10.1055/a-1167-2214. [DOI] [PubMed] [Google Scholar]
- 34.Abe Y, Kawakami H, Oba K et al. Effect of a stylet on a histological specimen in EUS-guided fine-needle tissue acquisition by using 22-gauge needles: a multicenter, prospective, randomized, controlled trial. Gastrointest Endosc. 2015;82:837–844 e831. doi: 10.1016/j.gie.2015.03.1898. [DOI] [PubMed] [Google Scholar]
- 35.Noyez L. Control charts, Cusum techniques and funnel plots. A review of methods for monitoring performance in healthcare. Interact Cardiovasc Thorac Surg. 2009;9:494–499. doi: 10.1510/icvts.2009.204768. [DOI] [PubMed] [Google Scholar]
- 36.Kovacevic B, Vilmann P. EUS tissue acquisition: From A to B. Endosc Ultrasound. 2020;9:225–231. doi: 10.4103/eus.eus_21_20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Hut-Mossel L, Ahaus K, Welker G et al. Understanding how and why audits work in improving the quality of hospital care: A systematic realist review. PLoS One. 2021;16:e0248677. doi: 10.1371/journal.pone.0248677. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.