A 10-item checklist improves reporting of critical procedural elements during transurethral resection of bladder tumor

Abstract

Purpose

Previous studies have demonstrated significant variation in recurrence rates after transurethral resection of bladder tumor (TURBT), likely due to differences in surgical quality. We sought to create a framework to define, measure and improve the quality of TURBT using a surgical checklist.

Materials and Methods

We formed a multi-institutional group of urologists with expertise in bladder cancer and identified 10 critical items that should be performed during every high-quality TURBT. We prospectively implemented a 10-item checklist into practice and reviewed the operative reports of TURBTs performed before and after implementation. Results from all institutions were combined using a meta-analysis to estimate the overall change in the mean number of items documented.

Results

The operative notes for 325 TURBTs during checklist use were compared to 428 TURBTs performed prior to checklist implementation. Checklist use increased the mean number of items reported from 4.8 to 8.0 per TURBT, resulting in a mean increase of 3.3 (95% CI 1.9, 4.7) items on meta-analysis. The percentage of reports that included all 10 items increased from 0.5% to 27% (p<0.0001) with the checklist. Surgeons who reported more checklist items tended to have a slightly higher proportion of biopsies containing muscle, though not at conventional significance (p=0.062).

Conclusions

The use of a 10-item checklist during TURBT improved reporting of critical procedural elements. Although there was no clear impact on the inclusion of muscle in the specimen, checklist use may enhance surgeon attention to important aspects of the procedure and be a lever for quality improvement.

Introduction

Transurethral resection of bladder tumor (TURBT) is a common diagnostic and therapeutic procedure for patients with non-muscle invasive bladder cancer (NIMBC). Of the 74,000 patients with newly diagnosed bladder cancer in 2015, approximately 75% presented with NIMBC.^{1, 2} Although patients with NIMBC generally have a low cancer-specific mortality, up to half experience an intravesical recurrence and require additional TURBTs.² Most of the 600,000 bladder cancer survivors today have their native bladders and are at risk for intravesical recurrences and multiple TURBTs, resulting in added patient morbidity and increased healthcare costs.^{3, 4}

The quality of surgical resection may have a significant impact on the risk of intravesical recurrence. According to data from European intravesical chemotherapy trials of nearly 2,500 NIMBC patients treated at 63 hospitals, early intravesical recurrence rates varied from 0–43% depending on where patients were treated.⁵ These differences persisted after accounting for disease- and treatment-related factors, and were thought to be explained by variations in TURBT quality. There is strong evidence that a more a complete resection is associated with improved NMIBC outcomes,^5–8 and that some patients receive grossly incomplete TURBTs.⁹

Improving the quality of care for patients with NMIBC may be possible through modification of TURBT surgical technique.^{10, 11} When attempting to improve surgical outcomes it is critical to measure and compare processes of care;¹² however there is no structure within which to measure TURBT quality due to lack of procedural standardization and reporting methods. Whereas successful efforts have been made to standardize reporting of colonoscopy and diagnostic radiology as a means of quality improvement, relatively little has been done for TURBT.^{13, 14} We sought to define TURBT quality and investigate the use of a checklist to improve surgical quality.

Methods

We formed a multi-institutional group of urologists with expertise in bladder cancer and an interest in surgical quality. Our first objectives were to define a high-quality TURBT and identify the operative steps necessary to achieve a successful operation. The group used an iterative process to identify key TURBT elements associated with oncologic and safety endpoints from an extensive list of potentially important factors identified from literature review, current guidelines and expert opinion. These results were compiled into a set of 10 critical and 3 optional items that should be performed, at minimum, during every high-quality TURBT. These items included steps necessary to assign disease risk (tumor number, size, multifocality, characteristics, concern for presence of carcinoma in situ (CIS), recurrent versus primary tumor), clinical stage (exam under anesthesia, assignment of clinical tumor stage), adequacy of the resection (visually complete resection, visualization of muscle at the resection base), and presence of complications (assessment for perforation). These items were assembled into a user-friendly list with suggestions on how each item can be documented (table 1).

Table 1.

TURBT Quality Audit
A high quality TURBT includes
1. Obtaining the information necessary for accurate classification of clinical stage and cancer risk.
2. Complete resection of all visible tumors and suspicious areas when safe, feasible and bladder preservation is planned.
3. Careful assessment of bladder integrity after tumor resection
Procedure Checklist
Assessment of prognostic factors	Acceptable responses
1. Describe number of tumors	1, 2–5, >5, diffuse
2. Describe size of largest tumor	For reference: end of cutting loop is approximately 1 cm wide
3. Describe characteristics of tumors	Sessile, nodular, papillary, flat
4. Describe recurrent versus primary tumors	Recurrent, primary
5. Assess for presence of carcinoma in situ	Suspicious, not suspicious
6. Report 2010 AJCC clinical tumor stage	cTis, cTa, cT1, cT2, cT3, cT4
Intraoperative processes
7. Bimanual exam under anesthesia	Yes, no
8. Visually complete resection	Yes, no
9. Visualization of detrusor muscle in resection base	Yes, no
10. Visual evaluation for perforation	Yes, no
Options
11. Photographic documentation of resection bed	Yes, no
12. Drawing or description of tumor location	Yes, no
13. Separate deep biopsy sent from resection bed	Yes, no

Our second objective was to determine if checklist utilization at the time of surgery improved operative reporting, which may be a proxy for surgical quality. We first evaluated the quality of TURBT reporting at each institution by retrospectively counting the number of critical elements in each operative report for consecutive TURBTs. In evaluating these reports, we considered a description of tumor characteristics as any effort to describe the visual appearance of the tumors (flat, papillary, sessile, etc.). Likewise, we considered any description of tumor size (1 cm, large, extensive, etc.) and number (solitary, multiple, 3, etc.) adequate. We considered the items related to adequacy of resection, visualization of muscle at the resection base, presence of CIS, and evaluation for perforation to have been addressed if they were mentioned in the operative report whether or not they occurred. For instance, surgeons were credited for documenting completeness of resection even if a tumor was incompletely resected, and for documenting presence of CIS if there was no concern for CIS. Each institution then prospectively implemented the checklist during TURBT. Surgeons were directed to consult the checklist prior to TURBT and when entering the operative report. We included all TURBTs in which a cutting loop was used to resect tissue concerning for carcinoma, regardless of final pathology. Institutional Review Board approval was obtained at each institution when required.

Outcome

The primary outcome was number of critical procedural elements included in the TURBT operative report. To determine if improved reporting was associated with a clinically meaningful outcome, our secondary outcome was the percentage of TURBT specimens that contained detrusor.

Statistical analysis

We compared the absolute difference in the mean number of checklist items documented per operative report for each institution before and after checklist implementation. The results for each institution were combined using a meta-analysis with random effects to obtain an estimate of the overall change in the mean number of items documented. We used a chi-squared test to compare the proportion of operative notes with documentation of all 10 steps before and after checklist implementation, as well as reporting rates for each checklist item. The relationship between the probability of muscle in the TURBT specimen and the number of documented checklist items was estimated using locally-weighted polynomial regression. A t-test was used to determine whether there was a significant difference in the mean number of documented items between the procedures that included muscle in the biopsy and those that did not. We did not include the three optional checklist items in our analyses. Seventeen reports had incomplete documentation on a single item, which we considered undocumented for the purposes of our analyses. The four cases with missing documentation on muscle were excluded when performing the analyses of muscle inclusion. All analyses were conducted in Stata 13 (Stata Corp., College Station, TX).

Results

We compared 428 and 325 operative reports before and after checklist implementation, respectively. Most patients had non-muscle invasive disease and approximately half had high-grade tumors (table 2).

Table 2.

Characteristics of the cohort. Statistics presented are median (IQR) or frequency (percent).

	Before checklist implementation N (%)	After checklist implementation N (%)
Number of cases	428 (57%)	325 (43%)
Median age at surgery (years) (N=699)	68 (60, 76)	70 (63, 78)
Male (N=644)	307 (81%)	207 (78%)
Repeat TURBT for high risk patients (N=752)	147 (34%)	107 (33%)
Cases by institution
Institution 1	39 (9%)	56 (17%)
Institution 2	116 (27%)	99 (30%)
Institution 3	47 (11%)	19 (6%)
Institution 4	70 (16%)	35 (11%)
Institution 5	56 (13%)	25 (8%)
Institution 6	50 (12%)	32 (10%)
Institution 7	50 (12%)	59 (18%)
Pathologic tumor stage
pT0	110 (26%)	51 (16%)
Pta	155 (36%)	133 (41%)
pTis	33 (8%)	47 (14%)
pT1	87 (20%)	47 (14%)
pT2–pT4	41 (10%)	35 (11%)
Unknown	2 (1%)	12 (4%)
High grade tumor	225 (53%)	166 (51%)

We observed an increase in the mean number of documented items from 4.8 to 8.0 per operative report after checklist implementation. Meta-analysis estimated a mean of 3.3 (95% CI 1.9, 4.7) more items documented per surgery after checklist implementation across all institutions (figure 1). There was significant heterogeneity of this effect across institutions (p<0.001). Four of the seven institutions had similar mean increases in the number of items per report (figure 1). The institution with the largest mean increase (6.2, 95% CI 5.8, 6.5) initially had the second lowest mean number of items per report prior to checklist implementation. In contrast, the two institutions with the lowest mean increases had the two highest mean numbers of documented items prior to checklist implementation, and less opportunity for overall improvement.

Figure 1.

Absolute difference in the mean number of documented checklist items by institution after checklist implementation and weighted overall estimate by meta-analysis.

The proportion of operative reports that included all 10 checklist items increased from 0.5% to 27% (absolute increase 26.5%; 95% CI 22%, 32%; p<0.0001) after checklist implementation. We also observed a significant increase in the frequency of reporting for each of the individual checklist items (all p<0.0001; table 3) after checklist implementation. Notably, there was a significant increase in description of completeness of resection after checklist implementation (63% vs. 82%, p<0.0001). After checklist implementation, there was no significant change in the percentage of specimens that contained detrusor (65% vs. 64%, p=0.8), including among the three institutions (1, 4 and 7) with the greatest reporting increases (64% vs. 62%, p=0.9).

Table 3.

Documentation of each checklist item before and after checklist implementation. Statistics presented are frequency (percent).

Checklist item	Before checklist implementation	After checklist implementation*
Describe tumor number	332 (78%)	303 (93%)
Describe tumor size	259 (61%)	286 (88%)
Describe tumor characteristics	292 (68%)	298 (92%)
Describe recurrent vs. primary tumor	192 (45%)	257 (79%)
Assess for presence of CIS	160 (37%)	259 (80%)
Report 2010 AJCC clinical tumor stage	77 (18%)	250 (77%)
Bimanual exam under anesthesia	194 (45%)	226 (70%)
Visually complete resection	270 (63%)	268 (82%)
Visualization of detrusor muscle in resection base	126 (29%)	222 (68%)
Visual evaluation for perforation	171 (40%)	237 (73%)

^*All changes are statistically significant (p<0.0001)

When considering TURBTs both before and after checklist implementation, procedures in which the operative report included more checklist items were modestly more likely to have muscle in the biopsy. For instance, an increase in documentation from 5 to 8 items was associated with a 2.9% increase in the probability of muscle inclusion in the biopsy (figure 2), though this result did not meet conventional levels of significance (p=0.062). The impact of documentation on muscle inclusion did not vary according to tumor grade (p=0.6 for the interaction coefficient of items documented and high grade tumor).

Figure 2.

Probability of detrusor muscle by number of items documented. Dashed lines represent 95% confidence interval.

Discussion

Although TURBT is one of the most common urologic procedures, it can be difficult to perform well. The goals of TURBT are to risk-stratify disease based on physical and cystoscopic examination, obtain tissue for pathologic staging, and completely resect all visible and microscopic disease. Because of the link between resection quality and recurrence rates, this procedure has become a target for quality improvement for patients with NMIBC.^{10, 11, 15, 16}

There have been other efforts to measure and improve the quality of care for patients with NIBMC. Using population-level data, Chamie et. al. observed significant variation in adherence to recommended perioperative processes of care for patients with NIMBC, including the adequacy of clinical and pathologic staging and use of treatment and surveillance strategies.^{17, 18} The appropriate use of immediate intravesical chemotherapy has specifically been proposed as a quality measure of NMIBC treatment, as it has been shown to decrease intravesical recurrences.¹⁵ These important studies highlight the need for adherence to guideline-recommended perioperative processes, but do not specifically assess the quality of TURBT or compensate for an incomplete surgery.^{6, 10}

The presence of muscle in biopsy specimens of high risk tumors has been proposed as an indicator of TURBT quality.^{17, 19} While highly measurable, the presence of muscle in the biopsy does not necessarily lead to actionable changes in the quality of the operation, is recognized until after surgery, and does not apply to all tumors.

In order to evaluate and improve the quality of surgery, it is necessary to create a framework within which it can be measured. This framework may also help standardize how the procedure is performed and documented, potentially allowing for better communication of intraoperative findings between physicians and improved patient outcomes. Such an effort was made to improve the quality of screening colonoscopy: a procedure performed on millions of patients annually, and historically associated with substantial variation in procedural quality.²⁰ A specialty Task Force convened to define what was required for a high-quality procedure²⁰ upon which a standardized colonoscopy data collection and reporting system was created to measure quality.¹³ Variation in reporting was observed, and a quality improvement effort through physician feedback and education, including use of a list of key procedural elements, led to improved outcomes.²¹

We propose that TURBT quality may be improved in a similar fashion. Two other TURBT checklists have been proposed.^{15, 16} One is a comprehensive list of over 60 perioperative elements of care, which seems prohibitively long.¹⁶ The second checklist is substantially more concise, but fails to include several important intraoperative processes.¹⁵ To our knowledge, neither has been implemented or evaluated in clinical practice.

Our checklist is based on several evidence-based and guideline-recommended steps that are important to accomplish the diagnostic and therapeutic goals of TURBT. Tumor characteristics, such as number of tumors, tumor size and tumor appearance, are strongly associated with oncologic outcomes and are only discoverable during cystoscopy.^{22, 23} Likewise, the surgeon’s assessment of clinical stage helps inform the aggressiveness of resection and need for further therapy. All professional guidelines recommend a complete tumor resection with sampling of surrounding tissue for accurate pathologic staging.^{2, 23, 24} Complete resection is not only therapeutically important for patients with NMIBC, but may also be so for patients with invasive disease.^{25, 26} Guidelines also recommend an examination under anesthesia, and stress the importance of describing the steps of the operation.²⁴

Despite the importance of these critical steps during TURBT, we found underreporting of multiple items, such as examination under anesthesia and completeness of resection. These findings are in agreement with a previous observation of significant variation in TURBT operative and pathologic reporting.¹⁷ Use of a simple checklist that includes these critical procedural elements is associated with improved reporting of every key step and across all institutions. Most members of our group felt the checklist had strong face validity and was useful in performing and teaching TURBT.

Although we were unable to evaluate if more accurate reporting was associated with lower recurrence rates, there was evidence that surgeons who produced a more detailed report tended to obtain muscle in the biopsy more often. If this finding were confirmed by an appropriately powered study designed to investigate the impact of this checklist on surgical quality, our checklist would represent a worthwhile addition to clinical practice given its ease of implementation and low cost. Improved TURBT reporting will also be useful in the clinical trial setting where clinical and cystoscopic assessment of disease often dictates eligibility and treatment response.

The operative note is meant to be a useful document with a complete description of all clinically relevant procedural elements. It can be used as a reference to recall intraoperative findings, and the information may be important for directing future treatment and communicating with other physicians. Whether more detailed reporting is necessarily associated with better surgical outcomes is unclear. However, Stewart et. al. observed that laparoscopic cholecystectomy operative reports that lacked specific key elements of the operation were associated with higher rates of common bile duct injuries than reports with more complete reporting.²⁷ The authors suggest that the steps documented in an operative report reflect their importance to the surgeon, and that “[a detailed operative note] is expected to enhance the chances of the desired result” by “framing the thinking of the surgeon.”

Our study is not without limitations. The checklist was created through group consensus on several potentially important intraoperative steps; therefore, may not include certain parts of the procedure that some surgeons consider important. We relied solely on operative and pathology reports for all of our data abstraction, thus we risk underreporting certain elements that may have been included elsewhere in the medical record, such as whether the procedure was a repeat TURBT for a high-risk patient. While we demonstrate that use of a checklist can improve how a TURBT is reported, we were not able to clearly show an impact on meaningful outcomes, such as intravesical recurrence rates. We propose to measure and improve TURBT quality via operative reporting, but other strategies for surgical quality improvement have also been described.^{28, 29} Still, there is currently no accepted structure within which to evaluate and compare the process of TURBT and we propose this may be a reasonable first step. New technologies, such as narrow-band imaging or fluorescence cystoscopy, may allow for improved visualization and resection of bladder tumors, although we believe the steps outlined in the checklist still apply.

Quality improvement across a population of providers is difficult without an established continuity of practice. This checklist begins to establish this continuity of practice from which one might envision more specific quality improvement efforts using plan-do-check-adjust methodology. This has been foreign to most of our field. Some may question whether quality of process reporting is necessarily associated with quality of the process. However, other established quality improvement efforts are rooted on adequacy of documentation, which can be considered a proxy for quality of care.³⁰

Conclusion

The use of a 10-item checklist during TURBT improved reporting of critical procedural elements, and enhanced surgeon attention to important aspects of the procedure. Although we did not observe a statistically significant effect of documentation on the probability of muscle inclusion in the specimen, our findings suggest that further research is warranted to investigate this checklist as a tool for measuring and improving TURBT quality.

Abbreviations

CI

confidence interval

CIS

carcinoma in situ

NMIBC

non-muscle invasive bladder cancer

TURBT

transurethral resection of bladder tumor