Introduction
Patients are increasingly seeking information about physicians online. Nearly 60% report that online reviews are important when choosing a physician.1 Since publicly reported quality data are not reported at the physician level, patients must consult physician-rating websites to find such reviews.2 The purpose of this cross-sectional study was to describe the structure of commercial physician-rating websites and the quantity of physician reviews on these sites.
Methods
During September, 2016, we searched Google for websites that allowed patients to review physicians in the United States, using search terms such as “rate my doctor.” We included active sites that were written in English, available to the public, allowed patients to leave reviews, did not require a subscription, and allowed searching by physician name. We excluded websites that were affiliated with an insurance company, health system, or were limited to a single specialty. We cross-referenced search results against a published list,3 added any additional websites that met inclusion criteria, and recorded website characteristics. We then used publicly available lists of registered and active physicians to identify a random sample of 600 physicians from three metropolitan areas (Boston, Portland, and Dallas). We searched each website for reviews and calculated mean and median number of reviews per physician per site. (Statistical Analysis System, SAS Institute, Inc. Version 9.3).
Results
We identified 66 potential sites, of which 28 met inclusion criteria (Table 1). We excluded 5 nonfunctional sites, 8 without reviews, 9 that required subscriptions, 7 that did not allow searching by name, and 9 that were limited to a single specialty or insurance plan. Most sites (n=26, 93%) included an overall “star” rating (hereafter “quantitative reviews”) and collected narrative comments (n=28, 100%). Notably, two sites that collected narratives did not post them. Few sites allowed the user to search by clinical condition (n=5, 18%), gender of physician (n=4, 14%), hospital affiliation (n=15, 54%), languages spoken (n=3, 11%) or insurance accepted (n=9, 32%).
Table 1.
Website characteristics
| N | (%) | |
|---|---|---|
| 28 | 100 | |
| General characteristics | ||
| Advertising, health related* | 17 | (61) |
| Advertising, not health related* | 20 | (71) |
| Fees for physician to promote profile | 9 | (32) |
| Registration required to review physician | 12 | (43) |
| Search functions | ||
| Physician name | 28 | (100) |
| Specialty | 27 | (96) |
| Physician area of clinical expertise | 25 | (89) |
| Clinical condition | 5 | (18) |
| State | 27 | (96) |
| City | 27 | (96) |
| Gender | 4 | (14) |
| Hospital | 15 | (54) |
| Language(s) spoken | 3 | (11) |
| Only physicians with reviews | 8 | (29) |
| Insurance accepted | 9 | (32) |
| Accommodations for disabilities | 0 | (0) |
| Review Structure | ||
| General | ||
| Included an overall rating | 26 | (93) |
| Allowed side-by-side comparisons | 10 | (36) |
| Reviews approved by administrator** | 6 | (21) |
| Solicited narratives | 28 | (100) |
| Posted narratives | 26 | (93) |
| Physician demographics | ||
| Name | 27 | (96) |
| Address | 27 | (96) |
| Phone | 27 | (96) |
| Sub-specialty | 23 | (82) |
| Hospital affiliation | 21 | (75) |
| Education | 11 | (39) |
| Residency | 9 | (32) |
| Years experience | 9 | (32) |
| Licenses/Certifications | 7 | (25) |
| Publications | 3 | (11) |
| Languages spoken | 3 | (11) |
| Professional affiliations | 11 | (39) |
| Questions Asked as Part of Structured Reviews | ||
| Availability | 10 | (36) |
| Punctuality | 10 | (36) |
| Staff characteristics | 10 | (36) |
| Costs/Billing | 7 | (25) |
| Facility/Location | 10 | (36) |
| Bedside manner | 11 | (39) |
| Time with doctor | 9 | (32) |
| Technical aspects of care | 8 | (29) |
| Likelihood of Recommending to Others | 11 | (39) |
| Communication/Education | 11 | (39) |
Advertising refers to advertisements for products that are posted on the website
All reviews are examined by an administrator before they are posted
Across 28 websites, there were 8,133 quantitative reviews (1,784 with narrative comments) for the 600 physicians. Among physicians with at least one review on any site, the median number was 7 (Interquartile Range [IQR] 2– 20) reviews per physician across all sites. We report median number of reviews per physician per site in Table 2. One-third (34%) of sampled physicians did not have a review on any site.
Table 2.
Number of reviews for 600 physicians across 28 websites
| Characteristic | Quantitative | Narrative | |||||
|---|---|---|---|---|---|---|---|
| Number of Physicians Queried |
Number of Physicians with Reviews (%) |
Number of Reviews |
Median (IQR)* Reviews/physician (Among physicians with reviews) |
Number of Physicians with Reviews (%) |
Number of Reviews |
Median (IQR)* Reviews/physician (Among physicians with reviews) |
|
| Overall | 600 | 397 (66) | 8133 | 7 (2 – 20) | 250 (42) | 1784 | 3 (1 – 6) |
| Site** | |||||||
| healthgrades.com | 600 | 295 (49) | 2175 | 4 (2 – 10) | n/a | n/a | n/a |
| vitals.com*** | 600 | 295 (49) | 3314 | 4 (1 – 8) | 197 (33) | 832 | 2 (1 – 4) |
| ucomparehealthcare.com | 600 | 153 (25) | 522 | 2 (1 – 4) | 27 (4) | 49 | 1 (1 – 2) |
| ratemds.com | 600 | 125 (21) | 397 | 2 (1 – 4) | 120 (20) | 386 | 2 (1 – 4) |
| lifescript.com | 600 | 73 (12) | 752 | 3 (1 – 6) | 6 (1) | 7 | 1 (1 – 1) |
| drscore.com | 600 | 64 (11) | 112 | 1 (1 – 2) | 0 (0) | 0 | |
| wellness.com | 600 | 52 (9) | 220 | 2 (1 – 4.5) | 38 (6) | 163 | 2 (2 – 4) |
| yelp.com | 600 | 25 (4) | 53 | 1 (1 – 3) | 25 (4) | 53 | 1 (1 – 3) |
| plus.google.com | 600 | 16 (3) | 21 | 1 (1 – 1) | 13 (2) | 17 | 1 (1 – 1) |
| Specialty | |||||||
| Generalists | 300 | 202 (67) | 4916 | 6 (2 – 21) | 124 (41) | 1124 | 3 (1 – 8) |
| Subspecialists | 300 | 195 (65) | 3217 | 8 (3 – 19) | 126 (42) | 660 | 3 (1 – 5) |
| Type of specialty | |||||||
| Internal Medicine or Family Medicine-general |
230 | 152 (66) | 4283 | 2 (7 – 22) | 95 (41) | 948 | 3 (1 – 8) |
| Internal medicine – subspecialty |
75 | 58 (77) | 1014 | 7.5 (4 – 16) | 41 (55) | 181 | 2 (1 – 4) |
| Pediatrics – general | 70 | 50 (71) | 633 | 4.5 (2 – 16) | 29 (41) | 176 | 3 (1 – 8) |
| Surgery | 57 | 44 (77) | 1124 | 17.5 (8 – 35.5) | 36 (63) | 188 | 3 (2 – 6.5) |
| Psychiatry | 33 | 25 (76) | 299 | 4 (3 – 14) | 11 (33) | 78 | 3 (1 – 10) |
| Obstetrics/Gynecology | 20 | 15 (75) | 359 | 12 (3 – 24) | 12 (60) | 112 | 2.5 (1 – 8) |
| Radiology | 22 | 12 (54) | 87 | 3 (1 – 4) | 6 (27) | 22 | 2.5 (2 – 6) |
| Emergency Medicine | 22 | 11 (50) | 39 | 4 (1 – 6) | 5 (23) | 8 | 1 (1 – 2) |
| Neurology | 12 | 10 (83) | 183 | 10 (4 – 18) | 9 (75) | 56 | 3 (1 – 8) |
| Anesthesia | 32 | 8 (25) | 41 | 4 (1 – 9.5) | 2 (6) | 3 | 1.5 (1 – 2) |
| Pediatrics – subspecialty | 12 | 6 (50) | 58 | 6 (1 – 21) | 3 (25) | 11 | 4 (2 – 5) |
| Pathology | 13 | 5 (38) | 11 | 2 (1 – 2) | 1 (8) | 1 | 1 (1 – 1) |
| Dermatology | 2 | 1 (50) | 2 | 2 (2 – 2) | 0 (0) | 0 | |
| State | |||||||
| Texas | 200 | 150 (75) | 4653 | 7.5 (3 – 22) | 96 (48) | 983 | 4 (2 – 8) |
| Massachusetts | 200 | 129 (64) | 1632 | 6 (2 – 12) | 70 (35) | 306 | 2 (1 – 5) |
| Oregon | 200 | 118 (59) | 1848 | 8.5 (3 – 21) | 84 (42) | 495 | 3 (1.5 – 6) |
IQR=Interquartile range; Median and IQR are calculated among physicians who had at least one review on that site
List of sites limited to sites with 5 or more physicians with reviews
One physician occupied 1389 quantitative reviews and 144 narrative reviews on vitals.com, representing 42% of all quantitative reviews on vitals.com
Discussion
Commercial physician-rating websites have significant limitations. Search mechanisms are cumbersome and reviews scarce. The number of physician reviews online appears to be increasing4 (a similar 2009 study revealed only 190 reviews for 300 physicians across 33 sites, with 27% of physicians having no review on any site).2 However, the increase in number of reviews we observed was not meaningful; most physicians in 2016 still had no more than one review on any site. This study is limited by small sample size, inability to limit the denominator to practicing physicians, and inability to assess the effect of reviews on patients or physicians. We did not attempt to analyze quantitative ratings because of the paucity of reviews per physician and variation in scales across sites. Despite these limitations, these results demonstrate that it is difficult for a prospective patient to find (for any given physician on any commercial physician-rating website) a quantity of reviews that would accurately relay the experience of care with that physician.
Given the demand by consumers for information about physicians, other methods for publishing patient feedback are being developed,5 and some health systems are beginning to report quantitative reviews and narratives drawn from patient experience surveys.6 Because of the scarcity of reviews on commercial sites, one of these other methods of publishing patient feedback may emerge as the dominant route by which patients seek reviews about physicians. Methods that use systematic data collection (e.g., surveys) may have a greater chance of amassing a sufficient quantity and quality of reviews to allow patients to make inferences about patient experience of care.
Acknowledgments
Acknowledgements & Funding Sources
Dr. Lagu is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number K01HL114745. Dr. Goff is supported by the National Institute of Child Health and Development of the National Institutes of Health under Award Number K23HD080870-01A1. Dr. Lindenauer is supported by the National Heart, Lung, and Blood Institute of the National Institutes of Health under Award Number 1K24HL132008. The funders had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and no role in the decision to submit the manuscript for publication. We would like to acknowledge, for help with manuscript preparation and submission, three students from the University of Massachusetts, Amherst: Caroline Norton, Lindsey Russo, and, Jessica Meyers. All three received compensation for their work on this manuscript. We have obtained written permission to include the names of Caroline, Lindsey, and Jessica in the Acknowledgment section of this manuscript. Dr. Lagu had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Footnotes
Disclosures
The authors have no potential conflicts of interest.
Bibliography
- 1.Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. PUblic awareness, perception, and use of online physician rating sites. JAMA. 2014;311(7):734–735. doi: 10.1001/jama.2013.283194. [DOI] [PubMed] [Google Scholar]
- 2.Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients’ evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med. 2010;25(9):942–946. doi: 10.1007/s11606-010-1383-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Which Sites Matter? | Review Concierge. [Accessed September 22, 2016]; http://reviewconcierge.com/docs/which-sites-matter. [Google Scholar]
- 4.Gao GG, McCullough JS, Agarwal R, Jha AK. A changing landscape of physician quality reporting: analysis of patients’ online ratings of their physicians over a 5-year period. J Med Internet Res. 2012;14(1):e38. doi: 10.2196/jmir.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Schlesinger M, Grob R, Shaller D, et al. Taking Patients’ Narratives about Clinicians from Anecdote to Science. N Engl J Med. 2015;373(7):675–679. doi: 10.1056/NEJMsb1502361. [DOI] [PubMed] [Google Scholar]
- 6.Lee TH, M D. Online Reviews Could Help Fix Medicine. [Accessed November 25, 2014];Harvard Business Review. https://hbr.org/2014/06/online-reviews-could-help-fix-medicine. [Google Scholar]