Appropriateness Criteria for Ureteral Stent Omission following Ureteroscopy for Urinary Stone Disease

Abstract

Objective:

To bridge the gap between evidence and clinical judgement, we defined scenarios appropriate for ureteral stent omission after uncomplicated ureteroscopy (URS) using the RAND/UCLA Appropriateness Method (RAM). We retrospectively assessed rates of appropriate stent omission, with the goal to implement these criteria in clinical practice.

Methods:

A panel of 15 urologists from the Michigan Urological Surgery Improvement Collaborative (MUSIC) met to define uncomplicated URS and the variables that influence stent omission decision-making. Over two rounds, they scored clinical scenarios for Appropriateness Criteria (AC) for stent omission based on a combination of variables. AC were defined by median scores: 1 to 3 (inappropriate), 4 to 6 (uncertain), and 7 to 9 (appropriate). Multivariable analysis determined the association of each variable with AC scores. Uncomplicated URS cases in the MUSIC registry were assigned AC scores and stenting rates assessed.

Results:

Seven variables affecting stent decision-making were identified. Of the 144 scenarios, 26 (18%) were appropriate, 88 (61%) inappropriate, and 30 (21%) uncertain for stent omission. Most scenarios appropriate for omission were pre-stented (81%). Scenarios with ureteral access sheath or stones >10mm were only appropriate if pre-stented. Stenting rates of 5,181 URS cases correlated with AC scores. Stents were placed in 61% of cases appropriate for omission (practice range, 25% to 98%).

Conclusion:

We defined objective variables and AC for stent omission following uncomplicated URS. AC scores correlated with stenting rates but there was substantial practice variation. Our findings demonstrate that the appropriate use of stent omission is underutilized.

Introduction

Guidelines from the European Association of Urology (EAU) and American Urological Association (AUA) state ureteral stents may be omitted following uncomplicated ureteroscopy (URS) in some situations.^{1, 2} However, stenting remains a common practice, occurring in over two-thirds URS for urinary stone disease.^{3, 4} Stents are associated with pain and urinary symptoms,⁵ as well as increased risk of an emergency department (ED) visit after URS.^{4, 6, 7} These unplanned encounters substantially increase the cost of URS.⁸ Importantly, overuse of stenting can lead to unnecessary patient suffering and loss of income due to work incapacity.⁹ Therefore, efforts to decrease stenting rates may improve health-related quality of life and reduce healthcare costs.

Idiosyncratic physician practice patterns are recognized as one of the strongest determinants of treatment variation.¹⁰ In a recent publication, significant surgeon variation in stenting after URS was observed, ranging from 10% to 100%.⁴ This variation may be an indication of uncertainty regarding the criteria outlined by the AUA and EAU guidelines. Specifically, the definition of uncomplicated URS or which scenarios are most suitable for stent omission.^{1, 2} One intervention that addresses variation and bridges the gap between evidence and clinical judgement is the RAND/UCLA Appropriateness Method (RAM) and creation of Appropriateness Criteria (AC).^{11, 12} This method has been used to reduce the inappropriate use of procedures, including percutaneous coronary interventions.^{12, 13} In urology, the RAM has been used to develop AC for active surveillance of prostate cancer.¹⁴ Dissemination of AC helped decrease surgery for low-risk prostate cancer in the Michigan Urological Surgery Improvement Collaborative (MUSIC).^{15, 16}

In this context, we undertook a quality improvement project following the RAM with a diverse panel of urologists in MUSIC to 1) review the evidence supporting stent omission following URS for urinary stone disease and define uncomplicated URS, 2) develop a list of variables that determine eligibility for stent omission, 3) assign AC scores for all combinations of these variables, and 4) retrospectively measure stenting rates within the MUSIC registry for each AC score. Our long-term goals are to reduce uncertainty regarding stent omission decision-making, increase appropriate use of stent omission, improve patient outcomes and reduce avoidable healthcare utilization.

Materials and Methods

Michigan Urological Surgery Improvement Collaborative:

MUSIC was established in 2011, in partnership with Blue Cross Blue Shield of Michigan and is a statewide quality improvement collaborative consisting of over 90% of urologists in the State of Michigan. Details on the Reducing Operative Complications from Kidney Stones (ROCKS) initiative and clinical registry have been previously described.^{4, 17, 18} Each MUSIC practice has obtained an exemption or approval by the local institutional review board for participation in the collaborative. This study was conducted by the MUSIC coordinating center and, thus, participants were limited to those within the State of Michigan.

RAND/UCLA Appropriateness Method:

The RAM is a multi-step process that requires a panel of experts in the field to score and, unlike the traditional Delphi method, discuss clinical scenarios for appropriateness a chosen intervention.¹¹ We began the process by inviting all MUSIC urologists to participate in the panel. In accordance with the RAM manual, we included only urologists on our panel because no other specialty is involved in the decision-making process for stent omission.¹¹ We ultimately selected 15 urologists from 25 respondents in order to include a diverse group of panelists from a variety of practice types and sizes across Michigan (Suppl. Fig. 1). Since our findings were to be implemented in Michigan, we did not include outside urologists. At the first meeting (Round 1), panelists were provided a synthesis of the available evidence, including a literature review conducted by the MUSIC coordinating center (Suppl. Fig. 2). Guideline statements from the AUA, EAU, and UK National Institute for Health and Care Excellence were discussed by an expert in the field (JSW) (Suppl. Fig. 3).^{1, 2, 19} At this meeting the panelists reached consensus on the definition of uncomplicated URS, for which ureteral stent omission was being considered (Fig. 1). Following this, the panel had to decide on patient and surgical variables that determined stent omission decision-making. At the conclusion of the meeting, consensus was reached on seven variables: stone size ($\le$ 5mm, >5mm – $\le$ 10mm, or >10mm – $\le$ 15mm), stone location (kidney or ureter), pre-stent (yes or no), urinalysis or urine culture result (treated positive culture or negative), non-balloon ureteral dilation performed (yes or no), ureteral access sheath (UAS) use (yes or no), and presence of “basketable” sized residual stone fragments (yes or no) (Fig. 2). Based on all combinations of variables, this resulted in 192 clinical scenarios to be scored for stent omission appropriateness (Suppl. Fig. 4).

Figure 1.

MUSIC Appropriateness Criteria panelists consensus definition of uncomplicated ureteroscopy.

Figure 2.

Panel consensus definitions of the seven variables that determine appropriateness for ureteral stent omission following uncomplicated ureteroscopy.

The panelists then individually scored these scenarios for stent omission using AC scoring of 1 (highly inappropriate) to 9 (highly appropriate) over a 3-month period. Scores were collated by the MUSIC coordinating center and presented to panelists at a second-round meeting, where they reviewed the distribution of all the panelists scores for each scenario (Suppl. Fig. 5). Panelists were given the opportunity to discuss each scenario and to change the variables. At this meeting, there was unanimous consensus to remove the ‘ureteral dilation’ variable from scenarios with pre-stenting, decreasing the total scenarios to 144. The panel decided any pre-stented scenario that still required ureteral dilation during URS should not be considered uncomplicated. A MUSIC patient advocate was present and contributed to the discussion. At the conclusion of the meeting, panelists again individually scored each clinical scenario. We created a color-coded heatmap based on the median Round 2 scores for appropriateness of stent omission. We also categorized scenarios on agreement, determined by the quintile of standard deviation of the Round 2 scores for a given scenario. Any scenario with $\ge$ 5 panelists assigning an AC score between 1 to 3 and 7 to 9, were reclassified as uncertain, regardless of the actual median, per the RAM manual definition of disagreement.¹¹

Statistical Methods:

To assess the strength of the association of the panel variables with AC score, a logistic regression model was used with stent omission AC as the dependent variable and the seven decision-making parameters as independent variables. In only this model, median Round 2 scores for each scenario were dichotomized into appropriate (scores 7 to 9) versus not appropriate (scores 1 to 6) for stent omission.

To provide context to the results, we identified all URS cases in the MUSIC registry between 2016 and 2019 that met criteria for uncomplicated URS (Suppl. Fig. 6). These were assigned a corresponding AC score and the proportion with stent placement were determined. The association between AC scores and stenting rates was calculated with Spearman’s correlation coefficient (rs). Only practices with greater than or equal to 10 uncomplicated URS procedures appropriate or inappropriate for stent omission in the MUSIC registry were included. Scenarios with disagreement were excluded from the registry analysis. Statistical analysis was completed using SAS 9.4 (SAS Institute, Inc).

Results

Of the 144 clinical scenarios, 38 (26%) were scored highly inappropriate, 50 (36%) inappropriate, 30 (21%) uncertain, 20 (14%) appropriate, and 6 (4%) as highly appropriate for ureteral stent omission. Fig. 3 displays a heatmap of median appropriateness scores by each scenario. Most of the scenarios appropriate for stent omission can be found at the top of the heatmap, within the pre-stented scenarios. Of the 26 scenarios appropriate for stent omission, 21 (81%) were pre-stented. If a UAS was employed, only pre-stented scenarios were appropriate for stent omission. There were no scenarios with ureteral dilation performed that were appropriate for stent omission. Additionally, no scenarios with stones greater than 10mm and without pre-stenting were appropriate for stent omission.

Figure 3.

Heatmap of Appropriateness Criteria for ureteral stent omission after URS: 1 - highly inappropriate, to 9 - highly appropriate. Displayed are median scores for all 144 clinical scenarios. UAS +, ureteral access sheath used. UAS -, no ureteral access sheath used. Frag +, yes fragments left behind. Frag -, no fragments left behind.

Each of the chosen variables significantly impacted stent omission decision-making (Fig. 4). The parameter with the greatest association was the absence of ureteral dilation (OR 14.6; 95% CI 8.08–26.49). Conversely, stone location had the least impact on decision-making, evidenced by ureteral (vs renal) stones having the lowest odds of being scored appropriate for omission (OR 1.5; 95% CI 1.16–2.05).

Figure 4.

Forrest plot of odds ratios of panelist scoring clinical scenario appropriate for stent omission by variable.

We observed a wide distribution of agreement amongst panelists for individual scenarios (Fig. 5). A total of 11 scenarios met RAM criteria for disagreement and are shaded in Fig. 5. In general, the least agreement (greatest standard deviation) was seen in scenarios with pre-stenting and a treated positive urine culture. The greatest agreement was seen in scenarios highly inappropriate for stent omission such as those without pre-stenting, stones greater than 10mm, a treated positive urine culture and ureteral dilation performed.

Figure 5.

Heatmap of standard deviation of round 2 scores among panelists for all clinical scenarios. Darker blue represents less agreement (higher standard deviation). Shaded squares meet RAM manual criteria for disagreement. UAS +, ureteral access sheath used. UAS -, no ureteral access sheath used. Frag +, yes fragments left behind. Frag -, no fragments left behind.

We identified 5,181 cases that met AC for uncomplicated URS. A stent was placed in 3654 (70.5%). AC scores correlated with stenting rates in these cases (rs=−0.967) (Fig. 6A). Fig. 6 also displays the variation in stenting rates between practices among cases appropriate (Fig. 6B) and inappropriate (Fig. 6C) for stent omission. Of the 2735 cases appropriate for stent omission, a stent was placed in 1659 (60.6%), and practice-level rates varied from 25.0% to 98.3%. Of the 1881 cases inappropriate for omission, a stent was placed in 1608 (85.5%), and practice-level rates varied from 35.7% to 100%.

Figure 6.

A). Ureteral stent placement rates following uncomplicated ureteroscopy in the MUSIC registry by assigned stent omission appropriateness criteria score. Number of stented cases for each score and Spearman’s correlation coefficient (rs) are displayed. B). Rates of ureteral stent placement among MUSIC practices for cases determined to be appropriate. C). Rates of ureteral stent placement among MUSIC practices for cases determined to be inappropriate. Data shown for practices with 10 or more cases in each category.

Discussion

We sought to develop AC for stent omission following uncomplicated URS by using the RAM. Our study has several key findings. First, we created a consensus definition of uncomplicated URS to identify patients suitable for consideration of stent omission. Second, we identified seven objective variables that affect decision-making for stent omission. Third, we created AC for stent omission following URS. Lastly, we found that stenting rates correlated with our AC. However, there was variation in practice patterns and a high rate of stenting, even among cases highly appropriate for stent omission. Collectively, these findings demonstrate that appropriate use of stent omission after uncomplicated URS is currently underutilized.

Few studies have investigated decision-making about stent placement or omission following URS. A recent Cochrane review assessing outcomes associated with stent placement following URS acknowledged the need for higher quality evidence.²⁰ AUA guidelines state clinicians may omit ureteral stenting if all the following criteria are met: normal contralateral kidney, normal renal function, no impediments to stone fragment clearance, no ureteric injury, and no planned secondary URS.¹ EAU guidelines similarly advocate that routine stenting after uncomplicated URS is unnecessary; however, no criteria are provided.² In our study, the RAM panel created a consensus definition of uncomplicated URS based on clinical experience. Given the subjective nature of the topic, some considerations remain unclear. Although non-balloon ureteral dilation was chosen as a variable, none of the scenarios with dilation performed met AC for omission and its absence was the strongest predictor being scored appropriate for omission. There is limited evidence regarding the safety of stent omission following ureteral dilation.^{21, 22} Future research should investigate the type and size of ureteral dilators used in order to provide additional clarity.

The majority of scenarios appropriate for stent omission occurred in pre-stented patients however, many pre-stented scenarios were either inappropriate or uncertain. While some may advocate for stent omission in all pre-stented uncomplicated URS,²³ this sentiment was not shared by the panel. Additionally, previous studies have demonstrated the safety of stent omission following UAS use in select pre-stented patients.^{24, 25} Our panel found that stent omission was appropriate in pre-stented scenarios with UAS use if the stone was less than or equal to 10mm, across a variety of other parameters.

Surprisingly, stone location had little effect on stent omission decision-making. The safety of stent omission following uncomplicated URS for ureteral stones is well established.^{3, 22, 25–27} However, no randomized trial has assessed the safety of stent omission for patients with renal stones.^{20, 22, 28, 29} Despite having the least impact on decision-making of our chosen variables, we found that scenarios with ureteral stones were significantly more likely to be appropriate for stent omission than scenarios with renal stones. Lastly, we found that stone size was associated with decision-making, and contrary to AUA guidelines,¹ stones >10mm were considered inappropriate for stent omission if not pre-stented.

Our study has several limitations. Our criteria for uncomplicated URS should be validated in future research. It was not possible to capture every variable under consideration during stent omission decision-making. Several factors cannot be objectively quantified and were not included as variables. The panel spent considerable time discussing “tightness” of the ureter while passing the ureteroscope, degree of stone impaction, operative technique (dusting versus fragmentation/extraction) and size criteria for fragments however it was ultimately decided that variables such as these were too subjective for inclusion. The amount of time needed to score scenarios is a constraint of the RAM, and it was not feasible to include every proposed variable. Specifically, the panel discussed including pre-operative hydronephrosis and stratification of the ureteral location but concluded these were not of sufficient importance. In the retrospective analysis, the MUSIC registry does not capture every factor included in the uncomplicated URS definition such as operative time, neurogenic bladder or incomplete bladder emptying. Practice patterns in Michigan may not be representative of international trends, where stenting rates may be much higher.³ Lastly, the term “inappropriate” in the RAM is not meant to provide judgment from a patient safety or medico-legal perspective, but rather expert opinion in the absence of strong scientific evidence.¹¹ In future publications, it is possible to combine the nine categories into three broad categories of stent omission such as “consider “, “indeterminate” and “not consider” which may be easier for clinicians to implement.

Limitations notwithstanding, the implications of our work are substantial and we have laid the foundation for future efforts to address variation in stenting practices. In the section on future research in the AUA surgical management of stone guidelines, it is recommended that future efforts should better identify patients safe for stent omission.¹ Our study is the first instance of a standardized data-driven method being used to understand stenting after URS. As a result of this project, we have created a clinical decision-aid for dissemination to all practices in Michigan (Suppl. Fig. 7). All resources are freely available at www.musicurology.com/rocks. We also aim to develop target rates of stent omission for different scenarios which can be tracked as a quality measure in the future however we expect wide-spread adoption will take time and, thus, a prospective analysis of our results did not occur in the current study.

Future research should focus on validation of the safety of stent omission in the aforementioned appropriate scenarios, as this should address the heterogeneity amongst panelists. The AC developed provide a framework for clinical trials, especially in scenarios found to be uncertain, such as the setting of an appropriately treated positive urine culture or non pre-stented patients. Importantly, investigation into the linkage between patient-reported outcomes and stenting practices has not been previously done but we have begun to capture this. Finally, if appropriate stent omission reduces healthcare utilization and improves health-related quality of life, an increase in payment by payors for procedural services that includes stent omission may be an important lever for reducing utilization and changing physician behavior thereby minimizing variation.³⁰

Conclusions

We identified seven objective variables that impact decision-making for stent placement after uncomplicated URS. Through the development of AC we defined scenarios that were appropriate and inappropriate for stent omission. Data from the MUSIC registry confirmed decreasing stenting rates with increasing AC for stent omission scores, however there was significant practice variation. Implementation of these AC into practice could help address variation, reduce stenting rates and improve the quality of care.