Analysis of Online Urologist Ratings: Are Rating Differences Associated With Subspecialty?

Abstract

Patients are increasingly using online rating websites to obtain information about physicians and to provide feedback. We performed an analysis of urologist online ratings, with specific focus on the relationship between overall rating and urologist subspecialty. We conducted an analysis of urologist ratings on Healthgrades.com. Ratings were sampled across 4 US geographical regions, with focus across 3 practice types (large and small private practice, academic) and 7 urologic subspecialties. Statistical analysis was performed to assess for differences among subgroup ratings. Data were analyzed for 954 urologists with a mean age of 53 (±10) years. The median overall urologist rating was 4.0 [3.4-4.7]. Providers in an academic practice type or robotics/oncology subspecialty had statistically significantly higher ratings when compared to other practice settings or subspecialties (P < 0.001). All other comparisons between practice types, specialties, regions, and sexes failed to demonstrate statistically significant differences. In our study of online urologist ratings, robotics/oncology subspecialty and academic practice setting were associated with higher overall ratings. Further study is needed to assess reasons underlying this difference.

Introduction

Over the past decade, increasing focus has been placed on patient experience as an important marker of care quality. Included in this larger initiative are programs developed by the Center for Medicare and Medicaid Services that provide financial incentives for quality health care (eg, value-based purchasing) (1). As part of this program, patient experience is assessed through patient-reported surveys (Hospital Consumer Assessment of Healthcare Providers and Systems [HCAHPS]) and used as a metric of health-care quality (2).

At the same time, patients are increasingly using commercial physician rating websites (PRW) to provide reviews about their care experience. Recent data demonstrate that the majority of Americans use the Internet to obtain health information and that nearly 60% report that PRW are somewhat or very important in choosing a health-care provider (3 -5). As measures of patient experience become increasingly used as a metric for care quality, shape reimbursement structure, and also guide physician choice by patients, it is critical that physicians understand factors that may influence these ratings.

The influence of interpersonal factors on patient ratings and satisfaction is well described. Accordingly, physician communication skills and time spent with patients are positively associated with patient satisfaction and ratings (6). Similarly, satisfaction with practice personnel correlates with higher physician ratings (7). As such, authors have suggested active steps that can be taken to optimize these interpersonal variables and manage online digital reputation, such as personalized emails and other online content (8,9).

In contrast to interpersonal factors, the potential effect of fixed physician demographics on patient ratings is less understood. Prior investigation has shown variable correlation of patient ratings with demographics such as practice setting, physician age, and gender (10 -12). In contrast, there is a paucity of research to help understand the influence of specialty or subspecialty on patient ratings. In the limited investigation available, differences in patients’ ratings and satisfaction have been shown when comparing various medical specialties (13,14). Even less is known about the effect of subspecialty on ratings. Some insight is available through studies focused on subspecialties within otolaryngology and spine surgery and demonstrate that certain related subspecialties are associated with higher ratings (11,15). We are aware of no prior study investigating the possible influence of urologic subspecialties on patient ratings or satisfaction.

Although fixed characteristics such as subspecialty cannot be changed, it is nonetheless important to understand whether they influence patient ratings as part of a larger familiarity by physicians with their online reputation. Despite this importance, study demonstrates that a large percentage of physicians have little familiarity with PRW, do not commonly check their own reviews, and spend minimal time managing their digital reputation (8). As such, understanding all variables that influence ratings and developing methods to help improve digital reputation is important.

Urology is a diverse field that comprises numerous subspecialties with quite different characteristics. For example, certain urologic subspecialties specifically focus on life-saving interventions (eg, oncology), whereas others on quality-of-life (eg, female pelvic medicine) interventions. Given these significant inherent differences across subspecialties, urology represents an ideal field to assess for rating differences across subspecialty. Indeed, given prior study demonstrating that surgical oncologists had higher satisfaction that most other surgical subspecialties, we hypothesized that urologic oncology may have higher ratings than other urologic subspecialties. In contrast, we hypothesized that lower ratings may be associated with female pelvic medicine given its focus on surgeries often characterized by partial improvement (eg, incontinence surgery) or treatments for chronic and refractory pelvic pain.

Accordingly, we developed the present study with the aim to compare online ratings in a large sample of urologists, with specific focus on potential differences across subspecialties. A secondary study objective was to assess for rating differences based on demographic factors including practice location, practice type, and physician gender.

Methods

We conducted an analysis of urologic physician ratings on the website Healthgrades.com (Healthgrades). Healthgrades is a commercial PRW that asks patients to rate their physicians and office experiences. Ratings are provided on 5-point Likert scales, including an overall likelihood of recommending the physician to a family or friend (1 = “not at all likely” to 5 = “completely likely”). We chose Healthgrades because it is the most frequently used PRW by patients and the most frequently used PRW in published investigation (16,17). University of Virginia IRB review determined that this study met exemption criteria for nonhuman research (IRB # 20592).

Physician Cohort Selection

Our physician cohort selection and data collection methodology is summarized in Figure 1. To identify a representative cohort of practicing United States urologists, we first selected 20 states of various sizes and geographic regions. In order to provide a representation of urologists nationally, states were chosen across 4 regions corresponding to those of the American Urological Association sections. In addition, given a focus on academic versus private practice physicians, states without medical schools and related academic centers were excluded. Using online searches, we next selected 1 academic center, 1 large private practice, and 3 small practices (<5 physicians) within each state. When possible, the largest academic and private practices were chosen. One of the selected states, Nevada, did not have an academic center with a urology program, so nearby University of Utah was chosen instead. These practices’ official websites and Healthgrades.com pages were then manually reviewed to generate a list of urologists and their corresponding Healthgrades ID (last alphanumeric phrase on providers’ Healthgrades.com page URL). Although many urology practices include advanced practice providers, radiologists, and radiation oncologists, these providers were excluded to ensure focus on urologic subspecialties only.

Figure 1.

Urologic physician cohort selection and data extraction.

Data Extraction

Using Java (version 8) programing, we then extracted data on the selected cohort of urologists from Healthgrades.com. Data included overall physician rating, number and distribution of ratings, and physician characteristics including age and sex. At the time of this manual online review, providers were categorized into 1 of 7 subspecialty groups: general, female, infertility/men’s sexual heath, pediatrics, reconstruction, robotics/oncology, and stones/endourology. If a subspecialty was not apparent through available online data, a category of general was applied.

Statistical Analysis

Descriptive statistics were first performed to assess overall physician ratings distribution, practice, and physician characteristics (subspecialty, practice type, sex, and geographic region). Kruskal-Wallis tests were used to assess for differences in physician ratings based on these characteristics. For subgroups with significant ratings differences, post hoc analysis was performed using Dunn’s tests with a Bonferroni correction. All statistical analysis was performed in R (version 3.4.1). Data are summarized as mean (± standard deviation) or median [interquartile range] as appropriate. All tests were performed with α = .05. University of Virginia IRB review determined that this study met criteria for nonhuman research.

Results

We identified 954 urologists via the cohort selection process described above. Of these, 872 (91%) had at least one Healthgrades rating and were included in our analysis. There were 10 376 ratings across the cohort, with a median of 9 (5,15) ratings per physician. The cohort’s stratification by practice type, subspecialty, region, and sex, along with median ratings is shown in Table 1. Mean provider age was 53 (±10) years, and 90% of physicians were male. Physicians classified as having a subspecialty of reconstruction accounted for the fewest profile represented (n = 12 [1%]).

Table 1.

Urologist Demographics and Practice Characteristics.

	n (%)	Median rating [IQR]	P value
Overall	872 (100)	4.0 [3.4-4.7]
Practice type
Large	424 (49)	3.9 [3.4-4.4]	<.001
Academic	282 (32)	4.4 [3.8-5.0]
Small	166 (19)	3.8 [3.2-4.3]
Subspecialty
General	426 (49)	3.9 [3.3-4.4]	<.001
Robotics/oncology	195 (22)	4.5 [3.8-5.0]
Female	81 (9)	4.0 [3.3-4.5]
Stones/endourology	65 (7)	4.3 [3.5-5.0]
Infertility/men’s sexual health	48 (6)	3.8 [3.5-4.5]
Pediatrics	45 (5)	4.1 [3.4-5.0]
Reconstruction	12 (1)	4.0 [3.4-4.8]
Region
Central	289 (33)	4.2 [3.4-4.8]	.13
South	247 (28)	4.0 [3.4-4.5]
Northeast	203 (23)	4.1 [3.5-4.7]
West	133 (15)	4.0 [3.3-4.4]
Sex
Male	784 (90)	4.0 [3.4-4.7]	.97
Female	88 (10)	4.0 [3.4-4.6]

Abbreviation: IQR, interquartile range.

The median overall urologist rating was 4.0 [3.4-4.7]. Figure 2 demonstrates ratings stratified by urologist subspecialty. Kruskal-Wallis analysis and post hoc Dunn’s test demonstrated that robotics/oncology subspecialty ratings (4.5 [3.9-5.0]) were significantly higher when compared to female (4.0 [3.3-4.5], P = .002), general (3.9 [3.3-4.4], P < .001), and infertility/men’s sexual health (3.9 [3.5-4.5], P = .02). Figure 3 demonstrates ratings distribution stratified by practice type. Kruskal-Wallis analysis and post hoc Dunn’s test demonstrated that academic practice type ratings were significantly higher when compared to remaining practice types (median 4.4 vs 3.8 and 3.9, P < .001 for both). None of the other rating distributions for regions, practice types, subspecialties, or sexes were statistically different.

Figure 2.

Overall physician ratings stratified by urologist subspecialty. *p < 0.05, **p < 0.01, ***p < 0.001 for comparison to Robotics/Oncology.

Figure 3.

Overall physician ratings stratified by practice type. ***p < 0.001 for comparison to Academic.

Discussion

As patient experience becomes increasing used to assess care quality and to guide reimbursement structure, understanding characteristics that influence patient ratings becomes increasingly important. Our study demonstrated that, in a diverse cohort of urologists, higher patient rating was associated with oncology/robotic subspecialty. There is limited prior study to provide insight into the impact of subspecialty on online patient ratings. In analysis of otolaryngology ratings on Healthgrades, Sobin and Goyal found that facial plastic subspecialty was associated with lower ratings than both laryngology and head and neck surgery subspecialties (11). The authors suggest that patient expectations may play a role in this finding. Further, in a cohort of 25 neurosurgeons, Agarwal and colleagues reported that higher HCAHPS scores were associated with spine as compared to cranial subspecialty (15). Our study is the first to assess this question in the field of urology and supports the findings of these prior studies suggesting that certain subspecialties are associated with higher patient ratings.

The reason underlying the finding of higher ratings associated with oncology/robotic subspecialty is not clear. However, during our study design, we hypothesized that urologic oncologists may have higher ratings because of their focus on life-saving interventions and the close relationships that patients often develop with their physicians in the course of addressing oncologic illness. This hypothesis was based on not only anecdotal experience but supporting literature. Accordingly, Daskivich et al found that surgical oncologists had the second highest mean overall satisfaction scores in their analysis of Healthgrades ratings across 15 surgical subspecialties (14). Similarly, in comparison of Canadian PRW across numerous surgical and nonsurgical specialties, Liu and colleagues found that oncologists were more likely to be rated in the top 50th percentile when compared many other specialties (10).

In addition, prior study has shown that oncologists demonstrated the highest level of patient-centered behavior when compared with other specialties (obstetrics, primary care, surgery) (18). Physicians have also been shown to use increased patient-centered behaviors when treating patients with more severe health conditions (19). Although it is unclear whether patient-centered behavior directly translates to higher online physician ratings, these behaviors have been associated with higher patient satisfaction (20). Our findings support the value of further research to understand the positive ratings effect associated with oncologic subspecialties.

Second, based on previously described literature, we theorized that patients might not associate the same positive significance with the subspecialty of female pelvic medicine, given its focus on quality-of-life diseases (eg, incontinence). Further, given that related interventions (eg, suburethral sling) frequently result in improvement rather than cure, patient frustration regarding incomplete symptom resolution is common. Prior literature indeed demonstrates that a significant percentage of patients undergoing anti-incontinence repair are dissatisfied if improvement, but not cure, is achieved (21). Such data underscores the significant and often challenging role of managing patient expectations preoperatively in this subspecialty. Finally, this subspecialty also includes focus on diseases characterized by chronic and often refractory pain (eg, interstitial cystitis, chronic pelvic pain syndrome) that could be associated with lower ratings related to patient frustration. Indeed, in their analysis of PRW for chronic pain medicine physicians, Orhurhu and colleagues comment that the 20% rate of negative reviews observed is generally higher than previous analyses of other medical specialties (22). Despite this background, we found no statistically significant differences in ratings when comparing female pelvic medicine subspecialty with the remaining subspecialties.

A second study aim was to assess for rating differences based on other demographic characteristics given the paucity of related research. As opposed to demographic characteristics, previous investigation has elucidated many interpersonal factors that influence online ratings. Accordingly, physician qualities, such as communication skills and time spent with patients, have been shown to correlate with online ratings (16). In addition to physician characteristics, satisfaction with office staff, facility variables (eg, cleanliness), and ease of care access have also demonstrated correlations with online physician ratings (7). However, less research is available to understand the effect of fixed characteristics, such as practice location, practice type, and physician gender. In our analysis, urologists in an academic practice setting had higher ratings when compared to physicians in a large or small private practice. Remaining demographic characteristics analyzed demonstrated no association with ratings.

The reason underlying higher ratings associated with academic setting is unclear. Generally, teaching hospitals are commonly associated with the perception of high-quality care and this fact may underlie positive satisfaction and higher ratings of academic physicians. Supporting this perception are prior investigations showing that teaching hospitals demonstrate higher performance on metrics of care quality and safety when compared to nonacademic centers (23 -25). At the same time, several characteristics of academic hospitals may be associated with negative patient perceptions. These characteristics include the involvement of trainees in the care process, the lack of physician continuity, and prolonged appointment wait times (25,26). In contrast, private practices may offer qualities that appeal to patients, including ease of access and continuity of care.

Data evaluating the impact of academic versus nonacademic setting on patient satisfaction of individual physician or overall care experience is conflicting. Prior study shows that large and high-volume hospitals, such as academic medical centers, are associated with improved satisfaction measured through HCAHPS surveys (27). Specific to online PRW, prior investigation of spine surgeons has demonstrated that academic practice is associated with higher ratings (28). In contrast, Clark et al reported that patient satisfaction was significantly lower in academic when compared to nonacademic medical centers (12).

Our investigation has several limitations. First, we were unable to assess the numerous additional contributors to patient satisfaction described above. Second, a sample bias is possible given the random sampling of regions and practice types. Additionally, this study focused on ratings from a single PRW and may not be representative of all PRW. Healthgrades was selected given that its wide use by consumers and in published investigation (16,17). Our study is strengthened by the large sample that provided a representative cohort of urologists throughout various regions, practice types, and subspecialties. As such, we believe these results are helpful to the urologic community as a descriptive analysis of online trends across urologic subspecialties and practice setting. Further study would be helpful to determine how each of these variables contributes to overall physician ratings.

Despite the increasing use of PRW by patients, physician criticism is frequent given prior study demonstrating that PRW may not correlate with care quality (29,30). In contrast, other investigation suggests that online ratings may predict hospital outcomes, such as mortality and readmission rates (31). Further, the significance of the higher ratings is unclear. Although our data demonstrated higher ratings associated with robotic/oncology subspecialty and academic setting (absolute difference of 8% to 14%), it is unknown whether this difference correlates with other meaningful outcomes. Further research is necessary to determine how differences in PRW ratings specifically influence or relate to patient decision-making and patient outcomes. Despite these inconsistencies, as patient reviews are increasingly used as surrogates for care quality and to shape reimbursement structures. As such, it is critical that physicians understand the many factors that may influence their online ratings.

Conclusion

In our study, robotics/oncology subspecialty and academic practice setting were associated with higher ratings when compared with remaining subspecialties and practice settings. Further research is necessary to determine how various physician and practice characteristics contribute to overall satisfaction with and online ratings of urologists.

Author Biographies

Jacqueline Zillioux recently completed her urology residency at the University of Virginia and is now pursuing a fellowship in Female Pelvic Medicine & Reconstructive Medicine at the Cleveland Clinic.

C William Pike is a third year medical student at Georgetown University School of Medicine.

Devang Sharma is a fellowship-trained urologist practicing in the Maryland suburbs of Washington, DC. He specializes in male infertility, sexual medicine, and reconstruction.

David E Rapp is associate professor of urology at the University of Virginia (UVA) School of Medicine. Prior to his appointment at UVA, Dr Rapp served as clinical associate professor at the Virginia Commonwealth University School of Medicine and for nearly 10 years as the co-director of the Virginia Urology Center for Continence and Pelvic Floor Reconstruction. He is also founding member and president of Global Surgical Expedition, a charity that sends surgical teams to underserved populations internationally to provide life changing surgeries.