A Large Number of Reviews on Physician Rating Websites May Reflect Reputation Management

Abstract

Background

Physicians with a large number of reviews and a high rating may be employing reputation management strategies. Specialists may be more likely than non-specialists to employ such strategies. This should be apparent in a study of online physician reviews on physician rating websites (PRW).

Methods

Using one physician rating website, we gathered orthopedic surgeon and family physician reviews. We measured Spearman correlations between the number of reviews and average numerical rating and used chi-squared to test threshold relationships.

Results

There were very small negative Spear-man correlations between the number of online reviews and the average numerical rating for orthopedic surgeons (p= -0.097, p-value=<0.001) family medicine physicians (p= -0.170, p-value=<0.001; Figure 2). Physicians with more than 100 reviews had a greater average numerical rating than physicians with fewer than 50 reviews. Orthopedic surgeons are more likely than family medicine physicians to have a large number of reviews and average numerical rating greater than 3.

Conclusion

The small fraction of physician with a high number of reviews may be utilizing reputation management strategies, and this seems relatively specific to specialists rather than non-specialists.

Level of Evidence: III

Introduction

Websites where patients can post reviews of physicians are intended to help people choose a physician. The percentage of people who use such websites to help make a choice is low, but the people who do use them indicate that they find the information helpful¹. It has been reported that patients who are highly educated, younger, women, and those with a long-term or permanent disease are more likely to review physician-rating websites (PRW) when selecting a physician.⁹

Reputation management services have emerged as a way to help physicians defend their online reputation.¹⁰ These services request favorable reviews and highlight positive messaging⁷ however are costly.⁸ An alternative to reputation management services is for physicians to encourage satisfied patients to write online reviews.³

Most physicians tend to have a small number of online reviews making their average rating relatively susceptible to one or two bad ratings.² One study in 2017 reported a median of 7 total online reviews per physician⁵ and in 2018 it was between 8-10 reviews.⁴ Much larger numbers of reviews might reflect a physician’s attempt to manage their reputation. However, anecdotally, it seems unusual for a physician to have more than 100 reviews when they are not performing some form of reputation management. Additionally, referral specialty practices seem more likely to invest in reputation management.

Online reviews were studied to address the following questions: (1) Is there a correlation between the number of online reviews about individual surgeons and their average numerical rating? (2) Is there a correlation between the number of online reviews for a family medicine physician and their average numerical rating? (3) Do physicians with an average numerical rating of 3 or below have a relatively low number of reviews (less than 50) or a relatively high number of reviews (greater than 100)? (4) Do orthopedic surgeons or family medicine physicians have an average numerical rating of 3 or below?

Methods

This study is exempt from institutional review board approval as it uses open source data from anonymous online users.

On Vitals.com, orthopedic surgeons and family medicine physicians in the 20 most populated cities in the United States were searched. Orthopedic surgeons and family medicine physicians were selected because we wanted to contrast specialty and non-specialty care. The number of reviews for internal medicine physicians is too large for the version of the software that was utilized to collect the text from the website. The following cities were included in the study: New York City, Los Angeles, Chicago, Houston, Phoenix, San Antonio, Philadelphia, San Diego, Dallas, San Jose, Austin, Jacksonville, Fort Worth, San Francisco, Charlotte, Columbus, Indianapolis, Seattle, Denver, and Washington DC. These cities were chosen as an accurate representation of the US population due to their geographic distribution and population size. The terms “orthopedic surgeon” and “family medicine” were entered into the search bar as we found these terms resulted in the highest number of reviews. Scrapestorm® was used to collect the specialty, location, average numerical rating, and number of reviews for each physician. Extracted data was exported and stored in Excel. This resulted in 6597 family medicine physician reviews and 6607 orthopedic surgeon reviews.

A Spearman rank correlation coefficient was used to determine whether there is a correlation in the number of reviews for individual orthopedic surgeons and their average numerical rating. This was performed for family medicine physicians as well. A chi-square test was used to compare: 1) the proportion of physicians with an average numerical rating of 3 or below between physicians with more than 100 reviews and physicians with 50 or fewer reviews; and 2) the proportion of physicians with an average numerical rating of 3 or below between orthopedic surgeons and family medicine physicians.

Results

There was a very small negative Spearman correlation between the number of online reviews for an orthopedic surgeon and the average numerical rating (p= -0.097, p-value=<0.001) that we felt was likely spurious given the scatter plot (Figure 1). A histogram shows that a few surgeons have a very high number of reviews (Figure 3). There was a small negative Spearman correlation between the number of online reviews for a family medicine physician and the average numerical rating (p= -0.170, p-value=<0.001; Figure 2). A histogram shows that very few family physicians have more than 100 reviews with just one outlier at more than 200 reviews (Figure 4). Physicians with more than 100 reviews had a greater average numerical rating than physicians with fewer than 50 reviews (Table 1). Orthopedic surgeons are more likely than family medicine physicians to have an average numerical rating greater than 3 (Table 2; Figures 5 and 6).

Figure 1.

Orthopedic surgeons average rating plotted against number of reviews.

Figure 3.

Histogram of number of online reviews for orthopedic surgeons.

Figure 2.

Family medicine physicians average rating plotted against number of reviews.

Figure 4.

Histogram of number of online reviews for family physicians.

Table 1.

Difference in Average Numerical Rating Between Physicians With More Than 100 Reviews and Physicians with 50 or Fewer Reviews

	≤3	>3	P-value
Physicians with 50 or fewer reviews	1546	10768	<0.001
Physicians with more than 100 reviews	2	190

We found a difference in average numerical rating of 3 or below between physicians with more than 100 reviews and people with 50 or fewer.

Table 2.

Difference in average Numerical Rating Between Orthopedic Surgeons and Family Physicians

	≤3	≤3	Totals	P-value
Orthopedic Surgeons	545	6062	6607	<0.001
Family Physicians	1020	5577	6597

We found a significant difference in the average numerical rating of 3 or below between orthopedic surgeons and family physicians.

Figure 5.

Histogram of average ratings for orthopedic surgeons.

Figure 6.

Histogram of average ratings for family physicians.

Discussion

Physicians can manage their reputation on physician rating websites by either using reputation management services or by encouraging reviews from satisfied patients. Prior studies found that most physicians have a small number of online reviews. We speculated that physicians with a very high number of reviews were using reputation management resulting in a relatively high mean rating. Overall, this study demonstrated signs of reputation management and non-specialists received more negative reviews than specialists.

There were limitations within this study. First, the software for retrieving data for numerous doctors is only usable with one rating site (Vitals.com). On the other hand, the ratings sites seem to have similar data and consistent number and character of reviews. Second, we had a limited version of ScrapeStorm® that limited the amount of data that could be exported. The number of family practitioners was easier to manage than the number of internal medicine doctors. The findings may be different if this study compared to internal medicine physicians.

The finding of a very small negative Spearman’s rank correlation coefficient between number of physician reviews and average numerical rating for both orthopedic surgeons and family medicine physicians is difficult to interpret and may be unauthentic. The correlation was so small and the sample so large that it’s probably best to interpret this as showing no correlation over the continuum of number of ratings. On the other hand, it may be that, when people do not participate in reputation management, a single bad rating is more likely to have an effect on the average numerical rating. According to one study, physicians making sound medical decisions that are discordant with the patient can lead patients to express their anger through a bad rating.⁶

The finding that both surgeons and family practitioners with a large number of reviews (100+) have a more favorable rating almost certainly represents a concerted effort at reputation management. The histograms show that the vast majority of both orthopedic surgeons and family practitioners have fewer than 25 reviews (Figures 3 and 4). And most raters give favorable reviews. A 5-star rating, the highest rating possible, was the most popular rating for both family medicine physicians and orthopedic surgeons (Figures 5 and 6). We believe that the very small fraction of surgeons with more than 100 reviews are using reputation management strategies.

The finding that orthopedic surgeons are more likely to have a higher rating suggests that orthopedic surgeons may be employing reputation management strategies more often than family physicians. Out of the 13204 reviews collected there were 192 physicians with more than 100 reviews, 178 of these reviews belonged to orthopedic surgeons and only 14 belonged to family medicine physicians. This confirms our previous finding that physicians with a large number of reviews (100+) may be utilizing reputation management.

Patients using physician rating websites to choose a physician can use the total number of reviews as a marker of reputation management. Our data shows that having more than 100 reviews is unusual and may suggest reputation management. Reputation management seems more common for specialists than non-specialists. This information can help people make more informed use of these web sites.