Clinical Decision Support for Surgery: A Mixed Methods Study on Design and Implementation Perspectives from Urologists

Abstract

Objectives:

To assess urologist attitudes toward clinical decision support embedded into the electronic health record and define design needs to facilitate implementation and impact. With recent advances in big data and artificial intelligence, enthusiasm for personalized, data-driven tools to improve surgical decision-making has grown, but the impact of current tools remains limited.

Methods:

A sequential explanatory mixed methods study from 2019–2020 was performed. First, survey responses from the 2019 American Urological Association annual census evaluated attitudes toward an automatic clinical decision support tool that would display risk/benefit data. This was followed by the purposeful sampling of 25 urologists and qualitative interviews assessing perspectives on clinical decision support impact and design needs. Bivariable, multivariable, and coding-based thematic analysis were applied and integrated.

Results:

Among a weighted sample of 12,366 practicing urologists, the majority agreed clinical decision support would help decision-making (70.9%, 95% CI 68.7–73.2%), aid patient counseling (78.5%, 95% CI 76.5–80.5%), save time (58.1%, 95% CI 55.7–60.5%), and improve patient outcomes (42.9%, 95% CI 40.5–45.4%). More years in practice was negatively associated with agreement (p<0.001). Urologists described how clinical decision support could bolster evidence-based care, personalized medicine, resource utilization, and patient experience. They also identified multiple implementation barriers and provided suggestions on form, functionality, and visual design to improve usefulness and ease of use.

Conclusions:

Urologists have favorable attitudes toward the potential for clinical decision support in the electronic health record. Smart design will be critical to ensure effective implementation and impact.

Introduction

Urologists make highly complex decisions about surgery that consider potential benefits and harms for patients. A significant driver of surgical decision-making is individual risk perception and judgement, which can differ greatly between surgeons and contributes to significant variation in treatment use and patient outcomes.¹ The near universal adoption of electronic health records (EHRs) has created multiple avenues to improve healthcare efficiency, quality, and outcomes. EHR-embedded clinical decision support (CDS) offers one approach to enhance surgical decision-making by providing well-curated, patient-specific information to users (e.g., surgeons, patients) in real-time.² CDS can take on many forms including safety alerts, guideline reminders, risk stratification/prediction, decision aids, and patient-reported outcomes. Enthusiasm for EHR-embedded CDS has grown tremendously with recent advances in big data and artificial intelligence.³

Despite some early successes, the broader impact of CDS remains limited. In a meta-analysis of over 100 randomized and quasi-experimental CDS trials, the absolute percent increase in recommended care was 5.8% while the improvement in clinical endpoints was even lower at 0.3%.⁴ Furthermore, while numerous risk prediction tools (RPTs) exist in urology and surgery,^5–7 ongoing issues related to usability and implementation limit their impact as CDS within the clinical workflow.⁸ In fact, less than half of urologists routinely use life expectancy calculators, cancer nomograms, decision aids, and quality of life instruments in practice.⁹ Concurrently, there has been growing recognition for the need to design health information technology around clinician needs and workflow.² However, even if offered in an unobtrusive manner, risk data may not alter behavior. In a study testing the impact of the ACS-NSQIP surgical risk calculator, the output altered risk perception but not surgery recommendations.^10,11 Finally, the EHR itself serves as a major source of dissatisfaction and burnout for surgeons, in part due to the significant time spent entrenched in documentation.^12,13 So even as the field makes strides in automated data collection and prediction models, barriers remain on how surgeons interface with, interpret, and use risk data in decision-making.

For the next generation of EHR-based CDS to meaningfully impact surgical care, equal attention needs to be given to the design and implementation as to the data and analytic methods behind them.¹⁴ Accordingly, we performed a mixed method study of urologists to assess their attitudes toward EHR-based CDS tools and explore user needs and design preferences that would promote usability and implementation. In doing so, we can develop user centered CDS to impact decision-making, affect surgical care, and improve outcomes.

Materials and Methods

Study Design and Participants

We performed a sequential explanatory mixed methods study that connected a national survey with qualitative interviews from practicing urologists. The American Urological Association (AUA) conducts an Annual Census of the urologic workforce. It maintains a urologist master file, derived from National Provider Identifier file, the American Board of Urology certification records, and AUA membership files. From here, the AUA opens the Annual Census each year in May at the Annual Meeting followed by bi-weekly email invitations until the end of September.¹³ The AUA cross-references respondents with the master file to verify they are practicing urologists and uses sampling weights to create a probabilistic sample of the whole to account for nonresponse bias.

In the 2019 Annual Census, 2,081 of 2,159 (96.4%) practicing urologists in the US reported use of the EHR and completed supplemental questions on the EHR, RPTs, and CDS (https://www.med.unc.edu/urology/wp-content/uploads/sites/637/2024/03/2019-AUA-Census-Module.pdf).¹² Specific to CDS, the Census asked respondents to rate their agreement with four statements on a proposed CDS using a 5-point Likert scale (1: Strongly Disagree to 5: Strongly Agree): if the EHR system could automatically calculate and display validated information on surgery risks and benefits at the point of care, it would (1) help my decision-making, (2) aid my counseling of patients, (3) save me time; and (4) improve patient outcomes. These questions underwent cognitive interviewing and pilot testing among 10 volunteer urologists.

Connection and Qualitative Interviews

Building upon the survey, we developed an interview guide based on conceptual frameworks in medical decision-making, clinical informatics, and implementation science to explore two primary areas: 1) surgical decision-making; and 2) EHR-based CDS. The study team iteratively refined the interview guide (https://www.med.unc.edu/urology/wp-content/uploads/sites/637/2022/12/Interview-Guide.pdf) through two pilot interviews, so it would elicit substantive information from interviewees in the allotted time. Trained qualitative researchers conducted the interviews. Specific to this analysis, interviewers asked the following: If you were building a CDS tool for the EHR to support surgical decision-making, how would you design it? Interviewers asked additional questions on content, visual design, implementation needs, and environmental barriers. Interviewers concluded the interview by asking how CDS could impact decision-making, patient counseling, clinical efficiency, and patient outcomes if well-designed and successfully implemented.

We recruited interviewees among Census respondents who consented to follow-up contact (62.9%). An unweighted analysis of the survey data indicated links between years in practice and weekly patient encounters with attitudes toward the EHR, RPTs, and CDS. Connecting these preliminary results, we sampled urologists stratified on these attributes (<18 vs. 18+ years in practice, <75 vs. 75+ patient encounters/week). After verbal informed consent was obtained, the qualitative team conducted interviews by telephone for approximately 45 minutes. Interviewees received a $100 gift card upon completion. Nearing thematic saturation (i.e., the point at which little or no relevant new themes were discovered in the data) by interview 18,¹⁵ we purposefully sampled urologists with negative attitudes toward EHR, RPTs, and CDS with secondary attention given to gender, geography, and practice type. Thematic saturation was achieved by interview 25.

Data Analysis and Integration

To be representative of the US urologist workforce and adjust for non-response bias, we applied the AUA Annual Census’ poststratification weighting technique based on gender, geographic location, certification status, and years since initial certification.¹³ The survey sample of 2,081 is weighted to represent 12,366 of 13,044 practicing urologists in the US as captured in the AUA master file. Within the weighted sample, we calculated descriptive statistics of the survey questions with 95% confidence intervals. Next, we created binary measures for each CDS statement (i.e., strongly agree and agree vs. not) then fitted multivariable weighted logistic regression models to identify key drivers. Covariates included: years in practice, gender, race (white vs. all other races), fellowship training, AUA section, rurality, scope of practice (general vs. subspeciality), practice setting (solo, urology group, multispecialty, private hospital, academic medical center, public, other), ownership status, number of urologists in practice, patient visits/week, major inpatient cases/month, clinical hours/week, and non-clinical hours/week. To further contextualize attitudes toward the proposed CDS, we dichotomized responses as agree (i.e., 4 and 5) versus not agree (i.e., 1–3) and performed bivariable analysis to assess how CDS responses differed by EHR and RPT use and attitudes.¹² Statistical analyses were performed using SAS v9.4 (Cary, NC) with significance set at 0.05.

For the qualitative analysis, all interviews were recorded, transcribed, de-identified, and imported into Dedoose, a qualitative research software management tool. Based on the interview guide and field notes, we developed a codebook then pilot tested it by independently coding several transcripts, which led to fine-tuning concept definitions and decision rules. The qualitative team then applied the resulting codebook to the remaining interview transcripts, capturing emerging themes and reconciling discrepancies through discussion and consensus, using standard consensus coding procedures.¹⁶ The qualitative team generated code reports for each code and narrative summaries describing themes and subthemes along with illustrative quotes. Finally, the research team integrated quantitative and qualitative findings to gain deeper insight into how receptive urologists are toward CDS and how future EHR-based CDS should be designed and implemented to affect clinical care. This was completed iteratively through use of a weaving narrative and joint displays linking survey responses to qualitative themes.

This study received approval from the UNC Institutional Review Board (IRB# 18–3166). The survey questions underwent additional clearance through AUA statistical services.

Results

Characteristics of the survey respondents and interviewees are reported in Appendix 1. Among the weighted sample of 12,366, 70.9% (95% CI 68.7–73.2%) agreed the proposed CDS would help decision-making, 78.5% (95% CI 76.5–80.5%) agreed it would aid patient counseling, 58.1% (95% CI 55.7–60.5%) agreed it would save time, and 42.9% (95% CI 40.5–45.4%) agreed it would improve patient outcomes (Figure 1). From the multivariable regression analysis, years in practice emerged as the key driver across all four survey items, with urologists in practice longer less likely to agree that CDS can improve the 4 queried facets of clinical care (Table 1). Practice owners less often felt CDS could improve patient counseling compared to non-owners (p=0.034) while those practicing in rural areas less often felt CDS could save time and improve patient outcomes compared to those in urban areas (Table 1).

Figure 1:

Urologist attitudes toward proposed CDS integrated with qualitative themes explaining their views.

Table 1:

Determinants of CDS Attitudes (MVA Results)

		CDS: Help Decision-Making		CDS: Aid Patient Counseling		CDS: Save Me Time		CDS: Improve Patient Outcomes
		OR (95% CI)	P-value	OR (95% CI)	P-value	OR (95% CI)	P-value	OR (95% CI)	P-value
Years in Practice (5-year increments)		0.89 (0.85–0.93)	<.001	0.88 (0.84–0.93)	<.001	0.89 (0.86–0.94)	<.001	0.93 (0.88–0.97)	<.001
Female		0.79 (0.56–1.13)	0.196	1.26 (0.84–1.89)	0.268	0.99 (0.72–1.37)	0.961	0.93 (0.68–1.28)	0.649
Caucasian		0.82 (0.62–1.10)	0.192	0.83 (0.61–1.15)	0.267	0.82 (0.63–1.07)	0.143	0.69 (0.54–0.89)	0.004
AUA Section
	North Central	1.30 (0.91–1.85)	0.358	1.32 (0.90–1.93)	0.961	1.34 (0.96–1.87)	0.236	1.07 (0.77–1.49)	0.959
	South Central	1.30 (0.90–1.89)	0.355	1.74 (1.16–2.61)	0.089	1.29 (0.92–1.82)	0.398	1.22 (0.86–1.72)	0.345
	Mid Atlantic	1.01 (0.67–1.54)	0.499	1.47 (0.90–2.39)	0.570	1.40 (0.94–2.11)	0.238	1.14 (0.76–1.70)	0.727
	Northeastern	1.11 (0.63–1.96)	0.921	1.21 (0.66–2.22)	0.751	0.97 (0.57–1.66)	0.441	1.31 (0.77–2.23)	0.371
	New England	1.11 (0.68–1.80)	0.891	1.27 (0.73–2.20)	0.887	1.27 (0.80–2.03)	0.605	0.96 (0.60–1.52)	0.536
	Western	1.47 (1.02–2.11)	0.083	1.76 (1.16–2.65)	0.083	1.12 (0.81–1.57)	0.839	1.07 (0.77–1.49)	0.977
	Southeast	Ref		Ref		Ref		Ref
	New York	0.91 (0.57–1.45)	0.248	0.95 (0.58–1.56)	0.117	0.93 (0.60–1.44)	0.223	0.89 (0.58–1.39)	0.300
Fellowship Trained		1.01 (0.72–1.42)	0.956	0.97 (0.67–1.40)	0.883	1.09 (0.81–1.48)	0.575	1.19 (0.87–1.63)	0.269
Scope - Subspecialty		0.97 (0.68–1.37)	0.845	0.79 (0.54–1.16)	0.228	0.72 (0.53–0.98)	0.036	0.83 (0.60–1.14)	0.244
Non-Metropolitan		0.72 (0.50–1.06)	0.093	0.74 (0.49–1.12)	0.151	0.64 (0.45–0.91)	0.012	0.68 (0.47–0.98)	0.038
Ownership - Any		0.76 (0.57–1.03)	0.077	0.70 (0.51–0.97)	0.034	0.78 (0.59–1.02)	0.071	0.80 (0.61–1.05)	0.110
Type
	Academic	1.10 (0.74–1.65)	0.959	1.02 (0.66–1.56)	0.949	1.01 (0.70–1.46)	0.321	1.21 (0.83–1.76)	0.326
	Multi-Specialty	0.96 (0.67–1.38)	0.297	0.86 (0.58–1.27)	0.257	0.99 (0.70–1.38)	0.228	0.79 (0.56–1.12)	0.025
	Private Hospital	1.36 (0.81–2.27)	0.338	0.88 (0.51–1.51)	0.472	1.19 (0.74–1.91)	0.873	1.20 (0.75–1.92)	0.503
	Urology Group	Ref		Ref		Ref		Ref
	Solo	1.25 (0.76–2.07)	0.597	1.44 (0.83–2.50)	0.172	1.25 (0.77–2.03)	0.705	1.10 (0.66–1.83)	0.861
	Public	1.19 (0.71–2.00)	0.732	1.02 (0.57–1.82)	0.967	1.16 (0.72–1.87)	0.980	1.03 (0.64–1.63)	0.840
	Other	0.98 (0.49–1.95)	0.663	1.07 (0.44–2.59)	0.911	1.61 (0.79–3.25)	0.271	1.18 (0.66–2.12)	0.667
Number of Urologists		1.01 (1.00–1.02)	0.009	1.01 (1.00–1.02)	0.163	1.01 (1.00–1.02)	0.062	1.00 (0.99–1.01)	0.376
Major Surgeries/mo		0.99 (0.98–1.01)	0.221	0.99 (0.98–1.01)	0.175	1.00 (0.99–1.02)	0.681	1.00 (0.98–1.01)	0.449
Patient Visits/wk		1.00 (0.99–1.00)	0.384	1.00 (0.99–1.00)	0.175	1.00 (0.99–1.00)	0.949	1.00 (0.99–1.00)	0.463
Clinical Hrs/wk		1.00 (0.99–1.00)	0.644	1.00 (0.99–1.01)	0.469	0.99 (0.99–0.99)	0.014	1.00 (0.99–1.00)	0.600
Non-Clinical Hrs/wk		1.00 (0.98–1.01)	0.772	1.00 (0.99–1.02)	0.928	1.01 (1.00–1.02)	0.162	1.00 (0.99–1.01)	0.926

^*CDS survey responses dichotomized as agree (i.e., 4 and 5) versus not agree (i.e., 1–3)

As exemplified jointly in Figure 1, interviewees described several means by which an EHR-based CDS could impact the four queried facets of clinical care. Thematically, these coalesced into 4 domains: evidence-based care, personalized medicine, resource utilization, and patient experience. In having surgical risk/benefit data at point of care, interviewees felt it would bring more objectivity to the decision-making process, making it more evidence-based and potentially more equitable. Interviewees also commented that CDS could bring attention to risks that may be overlooked or underappreciated by gestalt or knowledge alone. Next, interviewees spoke about how these data could facilitate more personalized medicine by calculating specific risks and enabling more tailored conversations. Some interviewees also noted that CDS could help direct services and resources more efficiently, with regards to both time internally and resources externally. As one example, CDS could support advanced practice providers in clinic, freeing up surgeon time to perform more procedures. Finally, interviewees remarked that these data could enhance the patient experience by directing conversations, supporting recommendations, and alleviating fears related to undesirable outcomes, ultimately leading to more confidence in the surgeon and the final treatment decision.

Figure 2 further contextualizes attitudes for CDS with current use and attitudes toward the EHR and RPTs. As shown, less than half of urologists agreed that the EHR improves clinical efficiency (35.8%, 95% CI 33.5–38.2%) or improves patient care (43.1%, 95% 40.6–45.5%), use RPTs often or always (30.4%, 95% CI 28.2–32.6%), or find RPTs to be helpful often or always (34.3%, 95% CI 31.9–36.6%). However, many of those with mixed/negative views still responded favorably to CDS. Specifically, 74.1% (95% CI 71.5–76.7%) of those who reported using RPTs never, rarely, or sometimes in clinical practice agreed that CDS would aid in patient counseling, and 63.1% (95% CI 60.1–66.0%) of those who found RPTs helpful never, rarely, or sometimes agreed that CDS would help decision-making. Additionally, 51.8% (95% CI 48.7–54.9%) of those who did not agree that the EHR increases clinical efficiency agreed CDS would save them time, and 32.4% (95% CI 29.3–35.4%) of those who did not agree that the EHR helps them deliver better patient care agreed that CDS would improve patient outcomes.

Figure 2:

Movement in attitudes toward EHR and existing RPTs versus the potential benefits of CDS. Blue indicates agreement with benefits of CDS, gray indicates neutral view, and red indicates disagreement with benefits of CDS.

Qualitatively, these positive attitudes appear conditional on specific user needs, design preferences, and implementation considerations (Table 2). First and foremost, interviewees described the form and function of the CDS, so it can be easy to use and useful. Specifically, CDS should be embedded within the EHR and easily accessible with minimal clicks. Additionally, the tool should automatically populate patient information stored in the EHR. Interviewees also raised interoperability and connectivity issues, highlighting that a tool must be compatible across different EHRs and devices (i.e., desktop, laptop, iPad, smartphone). Interviewees desired a degree of customizability and interactivity, including the ability to finetune or adjust risk estimates, consider hypotheticals, see predictions as part of broader trends and trajectories, and share information with patients and family. Critical content included life expectancy and surgical complications. Interviewees also desired data on cardiovascular/renal outcomes, resource use (e.g., readmissions), and cancer outcomes. Visually, interviewees wanted a simple design that conveyed information in multiple ways such as numeric data, infographics, and color-coding all within a single “snapshot.” Finally, participants shared multiple workflow considerations that would need to be addressed to facilitate implementation. Beyond auto-population of patient data, interviewees stressed the need to have information communicated quickly early in the encounter before they see the patient. They also emphasized the importance of the information being relevant and conveyed in a manner respectful of surgeon autonomy, in contrast to how most Best Practice Alerts currently function.

Table 2.

Surgeon Needs, Design Preferences, and Implementation Considerations

	Ease of Use	Usefulness
Form and Function	■ Embedded into EHR  “You wouldn’t have to leave and go to an internet site” ■ Easy to Locate - Dedicated Space or Visual Cues  “You’d have to find a space, a living space, for it” ■ Automated Data Pull/Feed  “It has to be this auto-populating feed that is absolutely brainless. It has to be—we don’t have to think about it” ■ One or Two Clicks  “To have a button that you just click on, and you have that information in front of you” ■ Simple/intuitive Set-Up  “I want it to be something that’s easy to customize and set-up…there’s so much there [in the EHR] and it’s so complicated” ■ Interoperability across Systems & Devices  “The ideal system would be a system that we could access at any point, from any device, that would be always available 24-hours a day from anywhere”	■ Surgery or Condition Specific  “It would have to be risk-prediction tools for the kind of surgery that I do” ■ Customizable Information  “Can pick and choose what they want to use or see within that programmed area” ■ Finetune/Adjust Inputs  “I could make some adjustments, if there’s some things that didn’t come through appropriately” ■ Patient-Facing Capability  “It might be nice if there was something where it had an interface where you could show it to the patient” or “have patient-friendly handouts” ■ Depict Trends & Trajectories  “Have all the information flow… can follow the trends.” ■ Show Hypothetical Choices/Results  “It might be worth capturing when exceptions are made and seeing what the outcome is from that exception.”
	General	Specific
Outputs	■ Mortality/Life Expectancy/Cancer Survival  “Everybody wants to know what their chances are of dying.” ■ Risk of Surgical Complications  “I think the two most helpful objective parameters… are life expectancy and risk of complications.” ■ Hospital Readmission & Disposition  “I think they’d want to know if they’d be readmitted…admission to skilled nursing facility or to nursing home would be helpful” ■ Frailty/Global Function  “It would have sort of a global wellness piece….”	■ Renal Function  “How many years ‘til they go on dialysis, and should we not operate on that kidney to give ‘em a few more years off dialysis.” ■ Margin Status  “What the data shows by the patient’s risk is for, I guess, negative margins for a partial nephrectomy.” ■ Cardiovascular Risk  “You wanna make sure that it takes into consideration atrial fibrillation and anticoagulation and things like that and cardiac risks assessment.” ■ Key Drivers of Risk (data points/medical history)  “Knowing about their medical history that’s important.”
Visual Design	■ Single Snapshot, Multi-Dimensional, not Long Form  “I would love something—they have a function where it’s like a snapshot where you can see all the information. I’d love to be able to just see a few things of information that are important to me.” ■ Infographics and Charts  “I think visual interfaces are hugely important for speed and interpretation. It gives you ability to share with patients.” ■ Color-Coded Information  “Maybe color-coding things based on their risk, like green to red, or somehow making it a visual for people who can’t appreciate numbers as much.” ■ Familiarity/Mimics Existing Tools  “I envision them looking the same as some of the Web-based ones that are out there already, at least the output.”
Implementation Considerations	■ Time limitations  “In clinic, I don’t slow down for anything…If something slows me down, then I’m unlikely to use it.” ■ Must be relevant and appropriate  “The best practice advisories most of the time are annoying and not relevant.” ■ Respecting surgeon autonomy  “As long as it’s not something that absolutely has to be clicked on and has to be ordered, I think a visual reminder that these are things at this age that should be checked or tested, is a good reminder” ■ Workflow integration  “I think having that information ahead of time so that you can digest it and figure out how you’re gonna talk to that patient about those results and about the plan, would be helpful.”

Discussion

Surgical decision-making relies significantly on the risk perception and judgment of individual surgeons,¹⁰ which is susceptible to bias and may contribute to highly variable delivery of surgical care. The development of patient-specific surgical risk tools and their deployment through EHRs offers one approach to reduce uncertainty and variation.¹⁷ In this mixed methods study, we found that urologists—despite limited use of RPTs and somewhat negative views of the EHR—generally held favorable views on the potential for EHR-embedded CDS to improve aspects of surgical care. These views centered on themes of evidence-based and personalized medicine, resource utilization, and patient experience but were conditioned upon numerous user needs and preferences.

With the advent of big data in the EHR and recent advances in automated data collection and artificial intelligence (AI), enthusiasm for CDS has grown.³ Previously, CDS faced multiple barriers to use, including urologists’ time constraints, lack of confidence in the predicted outcomes, and belief they could outperform model-derived estimates.^9,18,19 When combined with ongoing challenges with the EHR,¹² CDS has yet to reach its full potential. However, more recent risk tools have addressed some of the issues that have plagued earlier iterations. As a prime example, the MySurgeryRisk platform uses EHR data from over 100 variables, automatically collected in real-time, to generate AI models that predict postoperative complications and mortality with accuracy that surpasses physician estimates.²⁰ Consistent with user design preferences identified in this study, MySurgeryRisk platform can be adjusted based on physician feedback and generates interoperable output that can be sent quickly to a mobile device.²¹ Its successful implementation at a large, academic medical center showcases recent advances in CDS and the opportunity to take advantage of persistent optimism for this technology as identified here.²²

While providing urologists and surgeons with accurate, patient-specific risk estimates in real-time appears increasingly feasible, the question remains whether this will affect surgical care and outcomes. Despite its successful implementation, researchers found that MySurgeryRisk tool did not actually change surgeon risk prediction when used after an initial assessment was already complete, illustrating the power of anchoring and the importance of delivering the right information at the right time.²³ This is further substantiated by studies indicating that exposure to the ACS-NSQIP risk calculator altered risk perception but not surgery recommendations.^10,11 These findings suggest that providing complication and mortality data alone may be insufficient to change decision-making in a meaningful manner. Part of this discrepancy may be driven by surgeon gestalt, which carries a huge role in surgical decision-making along with other cognitive biases and heuristics.¹ In this study, urologists shared multiple design features that would enable them to contextualize, customize, and process risk information (i.e., multiple domains, interactive features, visual cues) in a manner that would be easier and more helpful to them. The design of CDS, in addition to the statistical piece, may be the key for EHR-embedded CDS to truly enhance surgical decision-making.

These findings should be considered in the context of several limitations. First, the interviews may be subject to response bias and not capture the full breadth of held views. In using purposeful sampling though, we recruited a diverse set of urologists including those with negative views.¹² Second, there exists potential for non-response bias in the survey responses. To address this concern, the AUA administers the Annual Census through mixed modes and provides sampling weights to generate nationally representative estimate. Additionally, the high absolute number of respondents ensures that the analysis captures practicing urologists with different experience levels, training experiences, and practice settings. Third, our primary outcomes represent hypotheticals as opposed to a specific tool. However, this frees respondents from reacting to predetermined features and may yield more generalizable results. Fourth, the design needs and preferences come from qualitative findings, so we are unable to quantify their importance based on the current study design. This could be the topic of future research to streamline design process. Fifth, the survey outcomes are not validated, nor have they undergone psychometric testing. However, they have undergone cognitive interviewing and pilot testing. Furthermore, our aim is not to measure a specific attribute but to guide efforts to optimally design CDS.

Overall, these findings have important implications on the development and implementation of CDS to enhance surgical decision-making. The 5 “Rights” of CDS serve as the prevailing framework for design: the right information, to the right person, in the right format, through the right channel, at the right time in the workflow.²⁴ Whereas the right person (i.e., surgeon) and right channel (i.e., EHR) are set and tremendous progress has been made in generating data, there remains significant need to further define the key information, effective formats, and optimal insertions into the workflow, which our study confirms as essential considerations. To solve these challenges, clinicians and developers will need to partner together.² User-centered design offers one approach to navigate the development of meaningful CDS. This strategy spans the entire development process and focuses on user engagement, including focus sessions, pilot testing, iterative development, multidisciplinary teams, usability and satisfaction assessments, and impact analyses.²⁵ Studies have demonstrated that CDS developed through user-centered design demonstrates increased adoption and effectiveness with respect to change in treatment decision.²⁶ Already, user-centered design has been used in urology to develop patient urostomy resources, patient-reported outcomes, and design aids.^27–30 By using this approach and incorporating iterative feedback, tools may achieve higher levels of usability and ultimately impact.

Conclusion

Overall, urologists have favorable attitudes toward the potential of EHR-embedded CDS to improve aspects of surgical care, including decision-making, patient counseling, efficiency, and patient outcomes. Previous CDS has faced multiple barriers to use. In this study, urologists identified multiple design features that would promote usability and usefulness (i.e., multiple domains, interactive features, visual cues). The design of CDS may be the key for HER-embedded CDS to truly improve surgical decision-making.