Furthering the capacity of urologists to shape the field of dissemination and implementation science and enhance clinical practice

The underlying premise of the field of Dissemination and Implementation Science (D&I), is that even therapies that are 100% efficacious are 0% effective, if they don’t reach the patients who need them.¹ Urologists were early adopters of D&I, with representation in early training programs (e.g., National Cancer Institute’s Mentored Training in Dissemination and Implementation Research in Cancer). These programs propelled the careers of urological scientists, leading to funding, publication, and development of dense research networks. D&I is now firmly established not only in urology, but as a global science, well supported with >18 consecutive years of NIH funding across almost all institutes, years of international investment (e.g., Canada, U.K., Australia, China), and contributions from low- and middle-income countries.

Continuing urology’s pioneering role in D&I, this year’s AUA Urologic Oncology Research Symposium, “Implementation Science in Urologic Oncology: Translating Evidence into Practice” highlighted efforts to improve evidence-to-practice gaps in urological care, including implementing evidence-based practice (EBP) and de-implementing practices no longer evidence-based. Investigators illustrated D&I principles in efforts to improve intravesical chemotherapy administration for non-muscle invasive bladder cancer, increase risk-aligned bladder cancer surveillance, widen access to cancer clinical trials, reduce low-value androgen deprivation therapy and imaging for prostate cancer, implement geriatric assessments in bladder cancer, and de-intensify imaging for microscopic hematuria.

Why We Need the Science of Implementation

Symposium speakers distinguished implementation practice, the process of applying and adapting implementation approaches in different settings to achieve positive outcomes, from implementation science, the rigorous study of implementation strategies to inform effective, sustained, and embedded adoption of evidence-based practice (EBP) interventions and de-adoption of harmful, wasteful, or ineffective practices. Symposium panelists illustrated the need for implementation science by diving into evidence supporting one of the most studied implementation strategies: Audit and Feedback (A&F). Despite being a cornerstone of most implementation bundles, A&F has modest effects on clinical practice and mixed results in urology. A D&I approach helps explain these mixed results in order to optimize the strategy itself. A&F is thought to operate through systematic measurement of practice, offering individual urologists insight into the true delivery of an EBP. In turn, this creates a platform for goal setting and behavior monitoring. In theory, bringing attention to inconsistencies between desired and actual practice, in combination with peer comparisons, motivates health professionals to improve. However, how A&F is defined, specified, and applied is critical to its effectiveness as highlighted in this session.

Accurate measurement of a modifiable outcome (i.e., the audit) is critical to effective A&F. Furthermore, different types of outcomes are better suited to driving behavior change. A Veterans Health Administration study, using process outcomes demonstrated A&F was effective in decreasing unnecessary urine culture orders and antimicrobial prescribing by 40% in patients with catheter-associated asymptomatic bacteriuria.² Another use of A&F, unlikely to engender improvement, is the US News and World Report Rankings.³ Rather than focusing on process outcomes directly related to the desired behavior change (e.g., prescribing), the rankings focus on 30-day mortality—an outcome measure. Outcome measures are typically complex constructs with multifactorial causes, some of which are not in direct control of the providers to which they are attributed, diluting the ability of A&F to drive behavior change.

The rankings also introduce a co-intervention to A&F: public reporting. Use of other co-interventions such as financial incentives and improvement collaboratives should be conceptualized and measured separately from A&F, if possible, as their influence could amplify or detract from the A&F process and impacts. For example, financial incentives might increase the amount of attention providers pay to feedback but may not lead to behavior change if no group deliberation occurs about what to change, the group does not have quality improvement competency, or no resources are available to support change. Measuring the effects of A&F and co-interventions is challenging. In fact, the effects of A&F could not be disentangled from the effects of an improvement collaborative in a study of intravesical therapy use in bladder cancer conducted across a large international urology network,⁴ from the financial incentives and improvement collaborative used in a study promoting prostate cancer active surveillance in the Michigan Urological Surgery Improvement Collaborative (MUSIC),⁵ or from the financial incentives used in a large urology group practice study conducted by Genesis Healthcare Partners.⁶ More sophisticated study designs are needed to shed light on how to effectively structure both A&F and other co-interventions.

Moreover, implementation strategy mapping helps ensure A&F is an appropriate strategy to produce a given outcome. In one study, A&F was not shown to be effective in improving prostatectomy margins.⁷ In this case, the rationale for selecting an A&F intervention was that it worked to drive down margin rates in the past. However, as the authors acknowledge, rates had steadily improved over several years with A&F. The residual poor performance was likely due to other factors that behavioral regulation alone could not address. Ideally, the remaining determinants of behaviors leading to positive margins would be thoroughly assessed and those determinants mapped to an implementation strategy known to address them.

In summary, the degree of improvement needed, how feedback is delivered, how often, by whom, with what co-interventions, with what consequence, and the kinds of problems we are providing feedback about may impact the amount of attention paid to the feedback and the amount of effort devoted to improvement—all of which would impact effectiveness of A&F. It is important to reiterate that receiving feedback may not be sufficient to change quality. Quality improvement requires skills and resources to plan and follow through on initiatives. Without strong managerial support to facilitate feedback and navigate positive action, A&F can decrease physicians’ job satisfaction and increase burnout.⁸ Measure development, cohort selection, and data systems needed to support A&F can be costly,⁶ representing a high implementation burden for practices. Because A&F is not without costs and unintended consequences, it should not be selected as an implementation strategy without fully understanding how it is anticipated to work and designing for optimal effect.

Next Steps for the Global Urology Community

Urology’s stake in D&I is well-established, but there remains more to do. Symposium participants identified additional competencies needed to fulfill the promise of implementation science. Training in implementation study designs, particularly pragmatic trials; implementation outcome measurement; and implementation strategy mapping were requested. The kinds of practice-based research designs, well suited to evaluating and optimizing implementation strategies, such as cluster-based randomized trials (used in the imaging de-implementation study) sequential multiple assignment randomized trials (SMART), and multiphase optimization strategy (MOST), which could enhance studies of A&F, are not often taught in clinical research training programs but are essential for moving D&I and urology practice forward. The ability of these designs to efficiently make multiple comparison and to allow for interventions that can be tailored to heterogeneous providers more closely resemble real world practice conditions and may yield results more rapidly. Likewise, the development and use of psychometrically-validated measures across implementation studies is also a priority of the D&I field: accurate measurement of implementation outcomes is critical to understanding if implementation efforts are ineffective or if the measures simply cannot detect change when it in fact occurs. Thus attendees’ requests for additional training should be used by the AUA Research Education, Conferences, and Communications Committee in planning future training symposia that not only meet AUA members’ needs, but also advance D&I and scientific impact.

Many clinics are needed to power D&I studies. Research-practice partnerships have been instrumental in these studies (e.g., MUSIC), but other clinical networks can be cultivated to join in these initiatives (e.g., AUA Quality Registry network). MUSIC organizes ~40 urology practices to participate in quality improvement by engaging urologists in designing improvement strategies, providing clinically credible, rigorous measurement infrastructure, and offering forums for moving performance feedback into actionable improvement steps. The AQUA registry already unites more than 200 urology practices around the collection of common data measures, but data could be collected about the practices to better understand the contexts in which improvement initiatives work and about the barriers these practices face in implementing evidence-based urological care. AQUA needs additional members to propel its research capacity and members willing to participate in care delivery research.

Finally, the field of D&I has embraced health equity and global health in recent years. Going forward, urology D&I should focus on ensuring that implementation interventions address disparities, rather than exacerbate them. D&I already has a foothold in global urology efforts: IMAGINE, the IMpact Assessment of Guidelines Implementation and Education, is currently trying to harmonize urological practice across Europe.⁹ Efforts to apply D&I to bladder outlet obstruction in Uganda¹⁰ further model processes to help identify, quantify, prioritize, and implement translation of modern care methods to meet specific global needs. These efforts can impact millions of patients in need of urological intervention. Maintaining a focus on D&I—locally and globally—and continuing to support the development of urological D&I researchers is imperative to achieve these goals.