Establishing a benchmark for patient comfort during colonoscopy and esophagogastroduodenoscopy using the St. Paul’s Endoscopic Comfort Scale (SPECS)

Abstract

Background and study aims

Patient comfort during endoscopy is an important performance measure. However, performance benchmarks for patient comfort have not been established. The aim of this study is to establish benchmarks for patient comfort during colonoscopy and esophagogastroduodenoscopy (EGD) using the St. Paul’s Endoscopic Comfort Scale (SPECS).

Patients and methods

Patient, nurse, and physician focus groups were conducted to establish an acceptable threshold of comfort using SPECS. SPECS for colonoscopy and EGD at St. Paul’s Hospital from January 1, 2021, to December 31, 2023, were collected. Using the SPECS threshold, we calculated benchmarks using 3 approaches: (1) 75th percentile, (2) the mean of the top 25% of physicians, and (3) Achievable Benchmark of Care (ABC), which adjusts performance for procedural volume.

Results

Four patients, 6 endoscopy nurses, and 10 physicians participated in separate focus groups: a SPECS ≤ 3 was the comfort threshold determined. SPECS from 17 405 colonoscopies conducted by 18 physicians and 6180 EGDs performed by 13 physicians were used to calculate a benchmark for patient comfort. Using the 75th percentile approach, the benchmark was 96.3% of colonoscopies and 95.9% of EGDs. The mean proportion of comfortable colonoscopies and EGDs among the top 25% of endoscopists was 96.8% and 96.8%, respectively. The ABC benchmark values were 97.1% for colonoscopy and 96.9% for EGD.

Conclusion

SPECS of ≤3 is an acceptable level of comfort. All 3 benchmarking methods generated comparable results, suggesting that physicians should aim for 96% of their procedures have a SPECS ≤3.

Introduction

The patient experience before, during, and after an endoscopic procedure is considered an indicator of procedure quality and a measure of performance at an endoscopy unit level and an individual physician level.^1–3 The patient experience includes patient comfort during the procedure. Guidelines recommend that patient comfort be assessed, using a validated instrument by endoscopy nurses as well as self-reported by the patient through a survey.¹ Patient comfort during endoscopic procedures is associated with overall patient satisfaction^4,5 and patient intention to undergo procedures, if recommended, in the future.⁵

Quality assurance and improvement initiatives such as the Global Rating Scale (GRS) for endoscopy strive to capture the patient experience, including comfort. The GRS incorporates routine documentation of quality indicators with regular audit of unit and individual physician performance, measured against a target benchmark.^6,7 There are several instruments available to assess patient comfort during endoscopic procedures^8–12; however, none have established a performance benchmark.

Originating in the field of economics, benchmarking is now widely utilized in healthcare, particularly in surgery.^13–16 Benchmarking may refer to a comparison of physicians within an endoscopy unit, comparing different units, or measuring the performance of a physician or a unit to an established level. A benchmark for a particular performance indicator may reflect the minimum acceptable performance or may be aspirational, based on the achievements of top performers.

We have previously validated the St. Paul’s Endoscopic Comfort Scale (SPECS) for colonoscopy and esophagogastroduodenoscopy (EGD).^8,9 SPECS is measured and recorded by a nurse for every endoscopic procedure at our site that is performed using conscious sedation under the direction of the endoscopist or without sedation, at the patient’s request.

In this study, we aim to establish an aspirational and achievable benchmark for patient comfort during colonoscopy and EGD using SPECS.

Methods

A mixed methods design incorporating qualitative and quantitative data was used to determine a benchmark for SPECS. Colonoscopy and EGD were evaluated separately throughout the study.

First, during the qualitative phase, focus groups were conducted to explore stakeholders’ perception of patient discomfort during colonoscopy and EGD, and how this aligns with a numerical value of SPECS. Second, during the quantitative phase, a retrospective cohort study assessed individual physician SPECS scores during colonoscopy and EGD and applied benchmarking analysis.

SPECS measures patient comfort during endoscopy using both verbal and non-verbal cues across 3 domains: (1) positioning and motor expression of discomfort, (2) verbal expression of discomfort, and (3) anxiety. The lowest score of 0 reflects no discomfort and the highest possible score is 9. SPECS has been described in detail in previous publications (Table 1).^8,9

Table 1.

St. Paul’s Endoscopy Comfort Scale (SPECS) score sheet.

Vocalization: whimpering, moaning, grunting, vocalized pain complaint
Frequency	None/relaxed	1-4	5-9	10 or more
Score	0	1	2	3
Positioning/body language: tensing, guarding due to pain, clutching, leg movements
Frequency	None/relaxed	1-4	5-9	10 or more
Score	0	1	2	3
Anxiety/emotion
Frequency	None/relaxed	Slightly agitated, minimal anxiety	Visibly upset, calmed by vocal reassurance	Visibly upset, not calmed by vocal reassurance
Score	0	1	2	3

Setting

St. Paul’s Hospital is a tertiary care hospital in Vancouver, Canada. The patient population undergoing endoscopy at St. Paul’s Hospital consists of individuals who reside in the local community as well as individuals referred for subspecialty care from throughout British Columbia and the Yukon. Subspecialty patients undergoing colonoscopy include those with inflammatory bowel disease and those referred for complex tissue resection or third space procedures. Endoscopic procedures are performed by gastroenterologists, colorectal surgeons, and learners including gastroenterology and colorectal surgery trainees. At the time of procedure booking, the patient receives written information describing the procedure, preparation, potential benefits and harms. The pre-procedure information discusses the use of conscious sedation. The admitting nurse counsels the patient that they may experience some discomfort during the procedure.

The hospital has an electronic medical record system into which a SPECS score is entered for every endoscopic procedure performed under conscious sedation or without sedation. The SPECS score is determined by the nurse assistant present during the procedure. The endoscopy nurses received group training in SPECS completion by J.J.T. This consisted of an interactive review of the SPECS elements and practical patient examples. The initial training was completed in 2014 with subsequent nurses trained by the experienced nurses during the training period at the endoscopy unit. Individual nurse SPECS scoring was recorded during the validation studies for SPECS in colonoscopy and EGD with only minor variability amongst the participating nurses.

Procedures at St. Paul’s Hospital are conducted using conscious sedation with intravenous midazolam and fentanyl. The current dataset did not include medications administered; however, the average doses for colonoscopy and EGD have been previously reported: midazolam 3.7 mg (SD 1.3) and 3.3 mg (SD 1.6) for colonoscopy and EGD, respectively; fentanyl 64.5 mcg (SD 28.0) and 51.4 mcg (29.7 mcg) for colonoscopy and EGD, respectively.^8,9 The average doses of sedation included patients who elected not to receive sedation for their EGD or colonoscopy, approximately 12% and 3%, respectively.

Qualitative phase

Three separate focus groups were convened with stakeholders: patients, endoscopy nurses and physicians who perform endoscopy at St. Paul’s Hospital. Questions to guide the focus groups were used (Supplementary Material) and each focus group was 15-30 min in length.

The patient group included English-speaking adults over 18 years of age who had undergone at least one outpatient endoscopy in the preceding 6 months. Patients were recruited through study flyers distributed to patients at the time of their discharge from the St. Paul’s Hospital endoscopy unit and posters hung within the unit. All interested patients were included with a pre-determined maximum number of 7. The interviewer had no prior relationship with the focus group participants and communicated with the participants via email to arrange a time for a virtual, video focus group with all 4 participants. The interviewer shared his healthcare background and interests in the research topic with the participants. The first participant had undergone a single colonoscopy, the second participant had undergone over 10 colonoscopies and 1 EGD, the third participant had undergone 3 or 4 colonoscopies, and the fourth participant had undergone 2 colonoscopies and 3 EGDs. The endoscopy history was self-reported by each patient.

Endoscopy nurses and physicians had worked a minimum of 6 months at St. Paul’s Hospital and were familiar with SPECS. The interviewer was well-known to the nurse and physician participants and explained the aim of the study. Endoscopy nurses were invited to participate in the in-person focus group during time dedicated to medical education. All interested nurses were included. The physician focus group was conducted during a regularly scheduled meeting by telephone. No nurses or physicians refused to participate.

Focus groups were facilitated by 1 of 2 investigators (A.Z. a male internal medicine resident, J.J.T. a female gastroenterologist) using prompts as needed (Supplementary Material). Only the focus group participants and one of the interviewers were present. Only one interview per focus group was conducted. Discussions were transcribed using NVIVO (Lumivero, Denver, United States) or Otter AI (Otter.ai, Mountainview, United States) software. Transcripts were not shared with the focus group participants. A thematic framework was developed to capture participants’ perceptions of procedure-related discomfort in relation to SPECS scoring by the 2 interviewers (A.Z. and J.J.T.).¹⁷ This process informed the identification of a SPECS score that stakeholders felt represented a level of discomfort that a reasonable patient would expect during a colonoscopy or EGD.

Quantitative phase

All patients who underwent colonoscopy or EGD at St. Paul’s Hospital between January 1, 2021, and December 31, 2023, were included. The patient cohort consisted of inpatients and outpatients with all procedure indications included. Patients who received deep sedation under the direction of anaesthesia were excluded; this accounts for approximately 10% of the total colonoscopies and EGDs performed at St. Paul’s Hospital. Colonoscopy was performed by 13 gastroenterologists and 5 colorectal surgeons, while EGD was performed by 13 gastroenterologists. Learners may have participated in endoscopic procedures. Physicians may not have worked at St. Paul’s Hospital for the entire study period. Data were collected from the hospital electronic medical record system: procedure performed, physician performing the procedure, and SPECS score. Cases with missing SPECS scores were excluded from the analysis.

Benchmark development

Each colonoscopy and EGD were categorized as either comfortable or uncomfortable using the threshold established from the focus groups. For each physician, we calculated the proportion of comfortable procedures by dividing the number of comfortable procedures by the total number of procedures. Benchmarking was then established using 3 different approaches with the goal of setting a high but attainable benchmark:

1. Percentile-based benchmark: Physicians were ranked by the proportion of comfortable procedures. The physician at the 75th percentile set the benchmark, with their percentage serving as the benchmark value.¹³

2. Average of high performers: Physicians were ranked by the proportion of comfortable procedures. The mean proportion of comfortable procedures for all physicians above the 75th percentile was calculated.^13,14

3. Achievable benchmark of care (ABC): The adjusted performance fraction (APF) was calculated for each physician. The APF weights the proportion of comfortable procedures by the total number of procedures for each physician. APFs were ranked in descending order, and the subset of physicians collectively accounting for at least 10% of the total number of procedures was used to establish the benchmark. The ABC reflects the mean proportion of comfortable procedures for the top 10% of physicians.¹⁶

After calculating the benchmark for comfort during colonoscopy and EGD using the above methods, we examined individual physician performance using the 3 benchmarks.

The analyses were conducted in SAS 9.4 and R 4.3.

The protocol was approved by the University of British Columbia’s Research Ethics Board (UBC REB Numbers H24-03982 and H24-02043).

Results

Qualitative phase

A total of 4 patients, 6 endoscopy nurses, and 10 physicians participated in the focus groups. The nurses and physicians provided short and exact responses to questions regarding SPECS and more descriptive narratives to prompts regarding causes of patient discomfort and actions to prevent or resolve discomfort during endoscopic procedures.

Patients consistently reported that both colonoscopy and EGD were generally well-tolerated. Several participants expressed that lower SPECS scores did not reflect meaningful discomfort. For example, 1 patient noted: “The procedure was actually fine; I had minimal pain and could have tolerated it even with less sedation.” Another described their discomfort as “less than expected.”

Clinicians echoed these views, emphasizing that although most patients scored between 0 and 2 on SPECS, discomfort at these levels was usually minimal, and patients seldom required additional sedation unless scores exceeded 3. The endoscopy nurses described discomfort during colonoscopy as “situational” (examples such as diverticulosis, prior abdominal surgeries were mentioned) and “sporadic” and “depended on the doc(tor)”. Discomfort during EGD was “usually due to anxiety” and “initial discomfort with gagging”. Physicians stated that “mild discomfort in the sigmoid colon and at flexures” was “expected during colonoscopy”. “A little bit of gagging during intubation” and “traversing the pylorus” was also acceptable. Nurses and physicians independently reported that significant discomfort during EGD or colonoscopy was “uncommon,” and a SPECS score of >3 represented an unacceptably uncomfortable procedure.

The 3 groups separately reached consensus that a SPECS of >3 represented an unacceptable amount of discomfort for either procedure. Based on this, a threshold of SPECS ≤ 3 was adopted to categorize colonoscopy or EGD as comfortable in subsequent analyses.

Quantitative phase

A total of 17 405 colonoscopies performed by 18 physicians and 6180 EGDs performed by 13 physicians between January 1, 2021, and December 31, 2023, were included. The median number of colonoscopies per physician was 895 (range 355 to 1804, IQR 716) and the median number of EGDs per physician was 368 (range 184-782, IQR 372).

The distribution of SPECS scores as a continuous variable for colonoscopy and EGD is reported in Table 2. Most procedures were associated with low scores, SPECS 0 or 1. Figure 1 illustrates the distribution of SPECS scores for colonoscopy, in blue,and EGD, in orange, demonstrating that 94.4% of colonoscopies and 94.6% of EGDs had SPECS ≤ 3. Amongst physicians, the proportion of comfortable colonoscopies ranged from 91.8% to 97.9%. For EGD, the proportion of comfortable procedures by physician ranged from 91.8% to 96.9%.

Table 2.

Total SPECS for colonoscopy and EGD.

	Colonoscopy	EGD
Number of procedures	17 405	6180
Mean SPECS (SD)	0.94 (0.94)	1 (1)
Median SPECS (range; IQR)	0 (0-9; 0,2)	0 (0-9; 0,2)
SPECS >3 (%)	5.56	5.44

Abbreviations: EGD, esophagogastroduodenoscopy; IQR, interquartile range; SD, Standard deviation; SPECS, St. Paul’s Endoscopy Comfort Scale.

Figure 1.

SPECS distribution for colonoscopy and EGD. The distribution of SPECS scores for all physicians for colonoscopy and EGD. EGD, esophagogastroduodenoscopy; SPECS, St. Paul’s Endoscopy Comfort Scale.

Benchmark development

Table 3 summarizes the results of the 3 methods of benchmark determination: all yielded similar results for both colonoscopy and EGD. Using the 75th percentile method, the benchmark was 96.3% for colonoscopies and 95.9% for EGDs. The mean proportion of comfortable colonoscopies and EGDs among the top 25% of endoscopists was 96.8% and 96.8%, respectively. The ABC values were 97.1% for colonoscopy and 96.9% for EGD.

Table 3.

Comparison of 3 benchmarking methods for SPECS.

Method of benchmark determination	Colonoscopy benchmark	EGD benchmark
75th percentile	96.3%	95.9%
Average 75th percentile	96.8%	96.8%
Achievable Benchmark of Care	97.1%	96.9%

Abbreviations: EGD, esophagogastroduodenoscopy; SPECS, St. Paul’s Endoscopy Comfort Scale.

Figure 2 represents the proportion of colonoscopies achieving a SPECS < 3 at an individual physician level. For colonoscopy, 1 of the 18 physicians met all 3 benchmarks while 5 physicians had a SPECS ≤ 3 on 96.3% of their colonoscopies, the lowest benchmark calculated using the 75th percentile method (Figure 2). Figure 3 represents the proportion of EGDs achieving a SPECS < 3 at an individual physician level. For EGD, 3 of 13 physicians met all 3 benchmarks and 4 physicians had a SPECS ≤ 3 on 95.9% of their EGDs, the lowest benchmark calculated using the 75th percentile method (Figure 3). Physicians achieving the benchmarks for comfort in colonoscopy included gastroenterologists and colorectal surgeons across a range of colonoscopy volumes. The 3 gastroenterologists achieving the benchmarks for comfort in EGD had higher EGD volumes than those not achieving the benchmark.

Figure 2.

Proportion of colonoscopy SPECS ≤3 by individual physicians compared to benchmarks. ABC, Achievable Benchmark of Care; SPECS, St. Paul’s Endoscopy Comfort Scale.

Figure 3.

Proportion of EGD SPECS ≤3 by individual physicians compared to benchmarks. ABC, Achievable Benchmark of Care; EGD, esophagogastroduodenoscopy; SPECS, St. Paul’s Endoscopy Comfort Scale.

Discussion

This study aimed to establish a high and achievable benchmark for patient comfort during colonoscopy and EGD using SPECS, a comfort score developed and validated by our group.^8,9 Through stakeholder focus groups, we determined that an SPECS ≤ 3 is considered a comfortable procedure. Benchmarking analysis supports that 96% of EGDs and colonoscopies have a SPECS ≤ 3, at a physician level as an aspirational and achievable benchmark.

The GRS for endoscopy was developed in England and adopted, with some modifications, by the Canadian Association of Gastroenterology.^6,18 The GRS is a patient-centered quality assurance and improvement tool completed biannually that drives quality improvement initiatives. St. Paul’s Hospital has participated in the GRS since 2008. Last year, the British Columbia Ministry of Health mandated that all endoscopy units in the province use the GRS as a platform for quality assurance and improvement. Patient comfort is one component of the GRS and incorporates nursing assessment and recording of patient comfort, audit of patient comfort with feedback to individual endoscopists, and action if an individual physician’s patient comfort levels fall below acceptable levels. While there are several patient comfort scores available to assess and record comfort,^8–12 performance targets have not been established. A benchmark is required to identify any physician not performing to acceptable standards, so support is provided to help them improve.

Our findings indicate that colonoscopy and EGD are generally well-tolerated. The median SPECS score was 0 for both colonoscopy and EGD. The low SPECS scores align with our thematic analysis from the focus groups, where patients, as well as endoscopy nurses and physicians, reported tolerable experiences with both procedures for most patients. Prior studies using SPECS have higher median scores of 2.0 (IQR 0,3) for colonoscopy⁸ and 1.0 (IQR 0,1) for EGD.⁹ These discrepancies may be explained by the smaller sample sizes of the earlier studies. Nonetheless, across both the previous and current studies, most patients did not experience significant discomfort, as determined by the current study, as SPECS ≤3. Our results are supported by a study measuring comfort using a nurse-led 5-point scale that reported 7.7% of patients experienced significant discomfort during colonoscopy compared to 5.6% in the current study.¹⁹

Another key finding of this investigation is that different approaches to benchmarking comfort produced relatively similar results. All 3 methods of benchmarking—75th percentile, mean 75th percentile, and ABC—yielded nearly identical results for both colonoscopy and EGD. While the ABC method is more resource-intensive, it accounts for differences in endoscopist case volume and may be particularly useful in centers with greater variability in practice. It was developed by investigators at the University of Alabama at Birmingham to identify performance targets based on data from peer groups that were objective and reproducible under the premise that a benchmark should reflect a level of excellence and be clinically attainable.¹⁶ The 75th percentile method, though least resource-intensive, may be susceptible to outliers, an issue partially mitigated by calculating the mean 75th percentile. Based on these findings, we conclude that all 3 approaches are feasible for benchmarking in high-volume centers; however, in settings with greater heterogeneity among endoscopists, the ABC method may provide the most reliable results.

Comfort scores are related to and must be interpreted in the context of other quality indicators. A physician may achieve excellent comfort scores at the expense of safe sedation practices or cecal intubation. A composite outcome combining these 3 indicators has been proposed and found to be associated with polyp detection rates.²⁰ Essentially, the investigators found that the high performers had comfortable patients, safe sedation practices, high cecal intubation rates, and high polyp detection rates.²⁰ To account for this, our physicians will receive annual reports with the proportion of comfortable procedures achieved for colonoscopy and EGD, with a target benchmark of 96%. The reports will also contain information of mean sedation practices, use of reversal agents, and adverse events. All physicians at St. Paul’s Hospital receive an annual report from the British Columbia Colon Screening Program, which contains their cecal intubation rate, adenoma detection rate, and sessile serrated lesion detection rate; we cannot provide this information for non-program colonoscopies yet, but hope to do so in the future.

The aim of this study was to determine a high and achievable benchmark for patient comfort. However, it is also important to identify physicians who are not meeting acceptable standards. At St. Paul’s Hospital, individuals who do not achieving standards meet with a physician peer to review their practice. This has been initiated for physicians not achieving cecal intubation rate or adenoma detection rate benchmarks and could lead to the physician re-taking the Colonoscopy Skills Improvement course, undergoing Direct Observation of Skills for colonoscopy at a shorter interval, observing a high performer, or undertaking changes in their sedation practice. There are no minimum standards for comfort scores; however, the 25th percentile of participating physicians would yield a data-driven threshold.

The main limitation of this study is generalizability. First, SPECS determination by endoscopy nurses may vary between and within sites. While nurses at St. Paul’s are trained in SPECS assessment and typically confer with the other nurse in the procedure room, this may not be standardized at other sites. Second, the benchmark produced in this analysis may not be appropriate for different patient populations or physician groups, particularly as it relates to sedation practices. Although there are conflicting data, increased sedation dosing is intended to result in a more comfortable patient.^11,19 While this is a desired outcome, it must be balanced against the potential harms of sedation, including adverse events and a reduction in dynamic position change during colonoscopy withdrawal, which improves adenoma detection rate²¹ and optimizes lesion resection. Third, our study was limited to EGD and colonoscopy. Other endoscopic procedures, such as endoscopic retrograde cholangiopancreatography and endoscopic ultrasound, which are typically longer and more technically complex, should be included in future benchmarking analyses. Our group is currently in the process of reviewing SPECS scores for endoscopic retrograde cholangiopancreatography. Finally, our focus group consisted of 4 patient stakeholders rather than our target of 7, which risks unrepresentative data. However, given the narrow focus of the study, the interviewers noted recurring comments from within the individual focus groups and across the 3 focus groups, indicating data saturation.

In conclusion, our findings support the use of SPECS ≤3 as a meaningful cut-off for distinguishing a comfortable colonoscopy or EGD. SPECS benchmarking analysis indicated that each physician should target a SPECS of ≤3 in 96% of colonoscopies and EGDs they perform. Future validation of these findings at other sites will help determine if this benchmark is applicable to a wider population of patients, endoscopy nurses, and physicians.

Author contributions

Conception and design: All authors. Analysis and interpretation of data: Aein Zarrin, Hong Qian, Jennifer Telford. Critical revision of manuscript for important intellectual content: All authors. Final manuscript approval: Jennifer Telford.

Supplementary material

Supplementary material is available at Journal of the Canadian Association of Gastroenterology online.

Funding

None declared.

Conflicts of interest

Conflict of interest disclosure forms (ICMJE) have been collected for all co-authors and can be accessed as supplementary material.

Ethics clearance

This investigation has obtained ethics clearance by the University of British Columbia’s Research Ethics Board (UBC REB Numbers H24-03982 and H24-02043).

Data availability

The data are not available for sharing as per our Research Ethics Board application.