Assessment of laryngologists' ratings on physician review websites

Abstract

Objective

To assess and characterize online ratings and comments on laryngologists and determine factors that correlate with higher ratings.

Methods

All the American Laryngological Association (ALA) members were queried across several online platforms. Ratings were normalized for comparison on a five‐point Likert scale. Ratings were categorized based on context and for positive/negative aspects.

Results

Of the 331 ALA members, 256 (77%) were rated on at least one online platform. Across all platforms, the average overall rating was 4.39 ± 0.61 (range: 1.00–5.00). Specific positive ratings including “bedside manners,” “diagnostic accuracy,” “adequate time spent with patient,” “appropriate follow‐up,” and “physician timeliness” had significant positive correlations to overall ratings, by Pearson's correlation (P < 0.001). Long wait times had significant negative correlations to overall ratings (P < 0.001).

Conclusion

Online ratings and comments for laryngologists are significantly influenced by patient perceptions of bedside manner, physician competence, and time spent with the patient.

Key Points

• Ratings of laryngologists on online platforms are overall high and consistent with otolaryngologists.

• Minimizing the number of negative comments regarding physicians' professionalism and clinical outcome leads to higher overall scores and online perception.

• Patient experience as reflected in online ratings for laryngologists are significantly influenced by patient perceptions of bedside manner, physician competence, and time spent with the patient.

INTRODUCTION

Physician rating websites have become commonly utilized tools for patients to provide feedback on physician performance. ¹ It has been reported that approximately 60% of people use web‐based physician rating systems in choosing a physician and that the number of online ratings has been increasing in recent years. ¹ , ² In response to the increase in the number of patients who search the internet for health information, multiple platforms for web‐based physician rating systems have been developed. Some of these platforms are specific to the field of medicine such as “RateMD” or “WebMD,” while others like “Yelp” or “Google Reviews” are all‐encompassing review websites documenting customer satisfaction. ² , ³ These websites contain information regarding physician demographics, such as education information, hospital affiliations, years of experience, and languages spoken in addition to ratings and reviews. ¹ , ³

A recent national survey in the United States revealed that 59% of participants reported that patient online reviews were very important or somewhat important when choosing a physician, though patient online reviews were endorsed less frequently than other factors such as word of mouth from family and friends and whether the physician accepted one's insurance. ⁴ In a more recent survey done by Press Ganey, 83% of respondents said they went online to read reviews of a physician after receiving a referral from another provider. ⁵ In choosing a new primary care doctor, 51.1% check online first, 23.8% seek a referral from another healthcare provider, and 4.4% get information from an insurer or a benefits manager, according to the survey. ⁵

Online physician ratings have also been investigated by specialty in various fields, including orthopedic surgery, ⁶ , ⁷ plastic surgery, ⁸ , ⁹ pediatrics, ¹⁰ and otolaryngology. ¹¹ , ¹² , ¹³ , ¹⁴ To date, no study has investigated online ratings or comments exclusively for laryngologists to our knowledge. We aimed to assess and characterize online ratings and comments on laryngologists and determine factors that correlate with higher overall physician ratings. Analysis of online physician website ratings can help guide important aspects of patient care and treatment.

METHODS

All American Laryngological Association (ALA) members were queried on Healthgrades, WebMD, Vitals, RateMDs, Health US News, and Google Reviews from September to December 2021. Information extracted included sex, years in practice, medical school and residency program attended, state of practice, and rating criteria provided by the websites. Ratings were normalized for comparison on a five‐point Likert scale. Furthermore, a weighted overall rating for each laryngologist was calculated via the following formula: ([Healthgrades rating × number of Healthgrades votes] + [WebMD rating × number of WebMD votes] + [Vitals rating × number of Vitals votes] + [RateMDs rating × number of RateMDs votes] + [Health US News rating × number of Health US News votes] + [Google Reviews rating × number of Google Reviews votes])/(Total number of votes across the six platforms). All comments were placed into thematic categories derived from factors listed within the websites and were aligned with a positive or negative aspect. States of practice were categorized into four geographical regions: Northeast, Midwest, South, and West. U.S. News and World Report Rankings for 2022 were used to rank the programs. Statistical analysis was performed using PASW Statistics 28.0 software (SPSS Inc.) with P < 0.05 considered statistically significant. Pearson correlation, independent sample t‐tests, and one‐way analysis of variance (ANOVA) were used to compare continuous and/or ordinal variables.

RESULTS

Of the 331 ALA members, 256 (77%) were rated on at least one online platform. Of those rated, 228 (89%) were on Healthgrades, 217 (85%) on Vitals, 183 (71%) on WebMD, 109 (43%) on RateMDs, 157 (61%) on Health US News, and 99 (39%) on Google Reviews. Across all platforms, the average overall rating was 4.39 ± 0.61 (range: 1.00–5.00) (Table 1). The average number of ratings per laryngologist was 76.41 ± 125.17, while the average number of comments per rated laryngologist was 6.51 ± 17.09 (Table 1). Specific positive ratings including “bedside manners,” “diagnostic accuracy,” “adequate time spent with patient,” “appropriate follow‐up,” and “physician timeliness” had significant positive correlations to overall ratings, by Pearson's correlation (P < 0.001). Long wait times had significant negative correlations to overall ratings (P < 0.001) (Table 2). Overall rating by geographic region was as follows with number and percentage of physicians in parenthesis: Northeast (102, 39.8%): 4.43 ± 0.59, Midwest (56, 21.9%): 4.35 ± 0.63, South (45, 17.6%): 4.48 ± 0.52 and West (53, 20.7%): 4.29 ± 0.70. One‐way ANOVA showed no significant difference in overall rating and region of practice (P = 0.399).

Table 1.

Online ratings of the American laryngological association members across various rating platforms.

Rating website	No. of rated laryngologists	Average overall rating score	Average No. of raters per laryngologist with rating (SD)	Average No. of comments per rated laryngologist (SD)
Healthgrades	228	4.22 (0.91)	13.85 (29.31)	7.16 (23.43)
Vitals	217	4.27 (0.65)	16.49 (22.19)	6.19 (14.11)
WebMD	183	4.30 (0.65)	16.56 (22.26)	1.26 (9.87)
RateMDs	109	3.95 (1.02)	7.88 (14.08)	7.80 (14.03)
Health US News	157	4.42 (0.81)	48.67 (73.29)	–
Google Reviews	99	4.59 (0.77)	15.58 (44.42)	10.14 (24.01)

Abbreviation: SD, standard deviation.

Table 2.

Association between specific factors underlying overall laryngologist ratings and the weighted overall rating.

Criteria	Average rating (SD)	P Value	r a
Healthgrades	4.22 (0.91)	<0.001	0.719
Vitals
Ease of scheduling appointments	4.03 (0.85)	<0.001	0.497
Timeliness	3.99 (0.86)	<0.001	0.604
Friendly demeanor	4.24 (0.82)	<0.001	0.548
Diagnostic accuracy	4.22 (0.90)	<0.001	0.685
Bedside manners	4.10 (0.96)	<0.001	0.705
Adequate time spent with patient	4.09 (0.95)	<0.001	0.684
Appropriate follow‐up	3.99 (1.02)	<0.001	0.689
Wait‐time in minutes	17.34 (10.53)	<0.001	−0.248
WebMD
Counseling of diagnosis and treatment	4.35 (0.59)	0.013	0.236
Addresses questions thoroughly	4.48 (0.72)	<0.001	0.722
Appropriate follow‐up	4.43 (0.79)	<0.001	0.613
Office cleanliness	4.28 (1.04)	0.025	0.353
Helpful and friendly staff	4.17 (1.20)	0.005	0.430
Scheduling flexibility	4.10 (1.20)	0.041	0.321
RateMDs	3.95 (1.02)	<0.001	0.518
Health US News
Comprehensive examination	4.34 (1.08)	<0.001	0.526
Ability to address questions	4.35 (1.12)	0.008	0.278
Clear instructions	4.47 (0.98)	<0.001	0.461
Appropriate follow‐up	3.99 (1.14)	<0.001	0.731
Time spent with patient	4.06 (1.10)	<0.001	0.704
Provider's demeanor	4.06 (1.12)	<0.001	0.648
Perceived outcomes	3.91 (1.08)	<0.001	0.636
Patient loyalty	4.10 (1.10)	<0.001	0.644
Patient‐centered decision making	4.44 (1.10)	0.694	0.100
General feedback	4.24 (1.07)	<0.001	0.508
Google Reviews	4.59 (0.77)	0.003	0.296

Abbreviation: SD, standard deviation.

^a Pearson's bivariate correlation was performed comparing each specific criterion with the weighted overall rating.

There was a significant negative correlation between years of experience (mean: 29.50 ± 15.74, range: 3–68) and rating (Spearman's P < 0.001, R = −0.268). Independent samples t‐test, however, demonstrated that laryngologists who attended a top‐50 medical school (both research [P = 0.364] and primary care [P = 0.947] or residency [P = 0.250]) did not differ in overall ratings compared with those who did not attend a top‐50 program. Similarly, laryngologists who attended a top‐25 medical school (both research [P = 0.982] and primary care [P = 0.221] or residency [P = 0.848]) did not differ in overall ratings compared with those who did not attend a top‐25 program (Table 3).

Table 3.

Association between attendance at top‐ranked medical school and residency programs and physicians' weighted overall rating.

Criteria	Mean score of those meeting criteria (n)	Mean score of those not meeting criteria (n)	P Valuea
Top‐50 medical school (research)	4.33 (55)	4.41 (202)	0.364
Top‐50 medical school (primary care)	4.40 (52)	4.37 (205)	0.947
Top‐50 otolaryngology residency program	4.50 (34)	4.38 (223)	0.250
Top‐25 medical school (research)	4.39 (96)	4.38 (161)	0.982
Top‐25 medical school (primary care)	4.32 (67)	4.41 (190)	0.221
Top‐25 otolaryngology residency program	4.38 (102)	4.44 (155)	0.848

^a Results were calculated via independent sample t‐test.

A total of 5060 comments across five platforms (1632 on Healthgrades; 231 on WebMD; 1343 on Vitals; 850 on RateMDs; and 1004 on Google Reviews) were analyzed and categorized (Table 4). For example, a comment that states “incredibly unprofessional physician” is straightforward to include in the professionalism category with a negative aspect. A more challenging comment would be “the best doctor for my whole family” which we ultimately included in the friendliness category with a positive aspect. Since the categories were not mutually exclusive (one comment could meet multiple categories' criteria), this yielded a total of 11,334 category entries. A comment that mentions both friendly staff and long wait times would therefore be represented in both categories. Of these, 9186 (81.0%) had positive and 2148 (19.0%) had negative connotations.

Table 4.

Comment categorization and the respective number of comments containing the underlying theme (not mutually exclusive).

	Positive comments		Negative comments
Comment category	n (mean ± SD per physician)	P Value (r)a	n (mean ± SD per physician)	P Value (r)a
Professionalism, communication, and counseling	2 798 (12.11 ± 22.22)	0.036 (0.138)*	562 (2.43 ± 4.29)	<0.001 (−0.409)*
Clinical outcome	2 474 (10.71 ± 19.46)	0.075 (0.117)	417 (1.81 ± 3.34)	<0.001 (−0.450)*
Friendliness, compassionate, and comfortability	1 728 (7.48 ± 16.53)	0.029 (0.144)*	348 (1.51 ± 3.10)	<0.001 (−0.429)*
Time spent with patient	779 (3.37 ± 8.04)	0.039 (0.136)*	188 (0.81 ± 1.69)	<0.001 (−0.350)*
Wait time	120 (0.52 ± 2.03)	0.070 (0.119)	133 (0.58 ± 1.31)	0.276 (−0.072)
Courteous staff	694 (3.00 ± 7.29)	0.080 (0.115)	139 (0.60 ± 1.47)	0.030 (−0.143)*
Ease of scheduling appointments and follow‐ups	289 (1.25 ± 2.50)	0.168 (0.091)	145 (0.63 ± 1.30)	0.067 (−0.121)
Cost and insurance difficulties	16 (0.07 ± 0.33)	0.116 (0.104)	97 (0.42 ± 1.22)	<0.001 (−0.310)*
Office environment	288 (1.25 ± 3.35)	0.085 (0.114)	119 (0.51 ± 1.36)	0.002 (−0.205)*

Abbreviation: SD, standard deviation.

^a Pearson's bivariate correlation was performed to show the effect of comment categories on the physician's weighted overall rating.

^* P < 0.05.

DISCUSSION

The majority of ALA members (77%) have one or more online reviews, with most physicians having ratings on Healthgrades (89.1%) or Vitals (85.8%). Overall ratings are generally high with an average of 4.39 ± 0.61 out of 5. These findings are largely consistent with results reported by Sobin and Goyal ¹² and Chua et al. ¹³ Sobin and Goyal ¹² queried 281 academic otolaryngologists in 2013 and found that 186 (69.9%) physicians rated on Healthgrades and 202 (81.8%) rated on Vitals had average ratings of 4.4 and 4.25, respectively. Chua et al. ¹³ studied 489 American Society of Pediatric Otolaryngology members in 2021 and found that 84% were on Healthgrades and 88% on Vitals, with an average rating of 4.08 and 4.19, respectively. The higher average ratings of our cohort (4.22 and 4.27 for Healthgrades and Vitals, respectively) compared to those of Chua et al. ¹³ could be attributed to the subspecialty studied, which is supported by Sobin and Goyal's ¹² findings since laryngology had the highest mean score (4.76) of subspecialties they studied. The lower average ratings of our cohort as compared with Sobin and Goyal's cohort could be attributed to the inclusion of more physician reviews since their findings are from 9 years ago.

It was previously shown that there was an average of 4.00 and 3.18 ratings per laryngologist on Healthgrades and Vitals, respectively. ¹² This contrasts with our findings of 13.85 and 16.49 average ratings per practitioner profile on Healthgrades and Vitals, respectively. This difference on average ratings may be due to varying physician inclusion criteria, since we queried all ALA members regardless of professional affiliations, whereas Sobin and Goyal only identified academic faculty members in otolaryngology programs in the Northeast US region.

Medical school ranking and residency ranking categorized by top‐25 or top‐50 institutions did not influence ratings or comments, a finding consistent with other studies. ¹¹ , ¹² , ¹³ States of residence and practice also did not correlate with average rating, similar to findings by Chua et al. ¹³ and Sobin and Goyal. ¹² In contrast to prior studies, we found that years in practice negatively correlated with overall rating. This may be a result of physicians who have been in practice for longer having more patients, and thus a higher possibility of receiving more negative ratings and comments. Additionally, patients may favor younger physicians who tend to have an increased online presence. Previous studies have also found inverse relationships between years of experience and performance on quality measures. ¹⁵ , ¹⁶ , ¹⁷ A systemic review done by Choudary et al. ¹⁵ found that physicians who have been in practice longer possess less factual knowledge and are less likely to adhere to appropriate standards of care.

We found statistically significant correlations between overall average ratings and ratings on all individual subcategories but one. The factors with the strongest correlation to overall rating (r ≥ 0.6, P < 0.001) were the physician's timeliness, diagnostic accuracy, bedside manners, adequate time spent with the patient, appropriate follow‐up, ability to address questions, and his/her demeanor. Chua et al. and Goshtasbi et al. also found most notable predictors of higher ratings to be a physician's accurate diagnosis, bedside manner, and adequate time spent with the patient. ¹¹ , ¹³ Among the 5060 narrative comments evaluated, the highest number of both positive and negative comments left by raters across all platforms was related to the physician's professionalism, communication, and ability to answer questions. Prior studies have also found similar results, which highlight that high‐quality, patient‐centered health care delivery requires not only the surgeon's ability, but his/her affability and availability. ¹⁸ , ¹⁹

Although the number of positive comments outnumbered the negative ones, more negative comment categories showed statistically significant correlations with the physicians' overall ratings. A negative correlation was observed between the laryngologists' overall score and the number of negative comments about the physician's professionalism, communication, counseling, clinical outcome, friendliness, adequate time spent with patient, courteous staff, cost and insurance difficulties, and office environment. The majority of the categorized positive comments did not provide any statistically significant correlation, suggesting that negative comments across all platforms had the greatest impact on laryngologists' overall scores. However, both positive and negative comments related to the physician's professionalism, communication, counseling, friendliness, and adequate time spent with patient showed a significant correlation with the overall physician rating. The greatest number of negative comments were related to physician professionalism, communication and counseling skills (26.2%), and clinical outcome (19.4%), indicating that the most influential negative ratings were related to the physician's bedside manners and ability to create a measurable change in the patient's health.

One of the main limitations of online platforms is the biased selection of patients, as those who have either had remarkably positive or negative experiences may be more likely to rate or comment on their physician online. Furthermore, due to the nature of online review websites, raters, and commenters share their opinions under full anonymity, so there is a level of uncertainty about the validity and accuracy. Additionally, some ratings and comments may not be from verified patients, or one patient may be rating the physician multiple times. While there was no association to negative reviews between region of practice or academic training institution, we were unable to analyze differences across individual institutions which could account for variation in institutional response to negative reviewers. Despite the biases and limitations, online physician ratings should be regarded as useful tools for patients when selecting healthcare providers.

CONCLUSIONS

Online rating websites for physicians have increased in use and provide opportunities for patients to direct their medical care. ¹⁸ , ²⁰ Our study emphasizes that online ratings for laryngologists are significantly influenced by patient perceptions of bedside manner, physician competence, and time spent with the patient. Although these perceptions influence both ratings and positive/negative comments, we show that minimizing the number of negative comments regarding physicians' professionalism and clinical outcome leads to higher overall scores and online perception. Our study highlights the factors that contribute most to the overall online ratings of laryngologists and identified specific targets to improve the patient experience.

AUTHOR CONTRIBUTIONS

Meleeka Akbarpour: Conceptualization, methodology, data collection, data analysis, writing—original draft, writing—review and editing; Karen Tawk: Conceptualization, methodology, data collection, data interpretation, writing—review and editing; Madelyn Frank: Methodology, data collection, writing—review and editing; Alizah S. Gomez: Methodology, data collection, writing—review and editing; Navid Mostaghni: Methodology, writing—review and editing; Mehdi Abouzari: Conceptualization, supervision, writing—review and editing.

CONFLICT OF INTEREST STATEMENT

The authors declare no conflict of interest.

ETHICS STATEMENT

Given the deidentified and publicly available nature of this database, this study was exempted from Institutional Review Board approval.

Akbarpour M, Tawk K, Frank M, Gomez AS, Mostaghni N, Abouzari M. Assessment of laryngologists' ratings on physician review websites. World J Otorhinolaryngol Head Neck Surg. 2024;10:1‐6. 10.1002/wjo2.95

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are openly available in Healthgrades, WebMD, Vitals, RateMDs, Health US News, and Google Reviews.