Abstract
BACKGROUND
Internet-based social networking tools that allow users to share content have enabled a new form of public reporting of physician performance: the physician-rating website.
OBJECTIVE
To describe the structure and content of physician-rating websites and to assess the extent to which a patient might find them valuable.
METHODS
We searched Google for websites that allowed patients to review physicians in the US. We included websites that met predetermined criteria, identified common elements of these websites, and recorded website characteristics. We then searched the websites for reviews of a random sample of 300 Boston physicians. Finally, we separately analyzed quantitative and narrative reviews.
RESULTS
We identified 33 physician-rating websites, which contained 190 reviews for 81 physicians. Most reviews were positive (88%). Six percent were negative, and six percent were neutral. Generalists and subspecialists did not significantly differ in number or nature of reviews. We identified several narrative reviews that appeared to be written by the physicians themselves.
CONCLUSION
Physician-rating websites offer patients a novel way to provide feedback and obtain information about physician performance. Despite controversy surrounding these sites, their use by patients has been limited to date, and a majority of reviews appear to be positive.
KEY WORDS: physician-rating websites, social networking, public reporting
INTRODUCTION
Public reporting is a key strategy in the effort to improve health care quality. Current public reporting initiatives generally assess physician performance by measuring adherence to clinical guidelines, and some also include data on patients’ experiences of care obtained through surveys.1–3 However, patients remain unfamiliar with public reporting websites4 and do not routinely use publicly reported data to guide their choice of providers.5
“Physician-rating websites” are a novel, alternative method for patients to provide feedback and obtain information about physician performance.6–8 While these sites contain information about physician licensure, office location, and disciplinary records, their primary goal is to allow patients to rate and discuss physician quality. Their structure is similar to other Internet-based rating systems that combine public reporting with social networking, such as travel websites that allow guests to discuss the quality of hotels and restaurants.
The British National Health Service (NHS) has encouraged patients to review their physicians and hospitals, suggesting that this new form of public reporting is growing in popularity and gaining respect.9,10 In the US, however, physician organizations [including the American Medical Association (AMA)] have opposed the development of physician-rating websites, arguing that the identity of patients cannot be confirmed, physicians’ responses will be hampered by confidentiality issues, and reviews will be excessively negative.11 Despite this controversy, little is known about physician-rating websites. The aim of this study was to describe the structure and content of these websites and to assess the extent to which a patient might find them valuable.
METHODS
Search Strategy
We conducted a systematic Google search for all websites that stated they allow patients to rate physicians. We individually searched and also combined the following terms: “rate doctors,” “MD review,” “physician ranking,” “doctor rating,” “find doctors,” and “best doctors.” We then consulted reference sections of lay press articles and blog posts to supplement our list of websites further. Finally, we had an expert review our list to ensure completeness.
Inclusion Criteria
Of the websites identified using the above strategies, we only included websites that were: (1) active between March 1, 2009–June 30, 2009; (2) written in English; (3) available to the public; (4) structured to allow patients to leave quantitative ratings in pre-specified domains and/or to read and write reviews about physicians; (5) available to patients across the US (in order to make our results as generalizable as possible); and (6) not restricted by medical subspecialty (because we were interested in the relationship between subspecialty and number of reviews). Two authors (NH, TL) reviewed each site to ascertain which sites met inclusion criteria. Disagreements were resolved by consensus.
Examination of Website Structure
Two authors (NH, TL) identified elements common to many websites: fees for users and/or physicians, presence of advertising, requirement for user registration, search characteristics, and review structure. We discussed these findings with a third investigator (PKL) and created classification criteria based on group consensus. One author (NH) then re-examined all websites to extract the presence of characteristics. A second investigator (TL) confirmed these findings. At the end of the study period, we re-examined all websites for any changes.
Presence of Physician Information
In order to assess the prevalence of reviews, we searched websites for a random sample of physicians practicing in Boston, Massachusetts. We chose Boston because it is a large metropolitan area with many physicians and a technologically savvy population. The Massachusetts Board of Registration in Medicine provided us with a list of practicing physicians free of charge. We hypothesized that the number and quality of reviews might differ between generalists (family medicine, internal medicine, pediatrics, obstetrics and gynecology) and subspecialists, so we generated random numbers to select 150 physicians from each category. On each website, we confirmed physician identity (using first name and last name plus one additional identifier: middle initial, subspecialty, or medical school) and recorded number of reviews (including both written narratives and quantitative reviews).
Evaluation of Reviews
Because review domains and scales differed across sites, we created a single summary score for each quantitative review. We then classified summary scores as “positive, negative, or neutral.” (For sites with a 1–4 scale, we scored 1–2 as negative and 3–4 as positive. For sites with a 1–5 scale, we scored 1–2 as “negative,” 3 as “neutral,” and 4–5 as “positive,” etc.) We compared generalists’ and subspecialists’ number of reviews using a Wilcoxon rank sum test and review favorability using a two-sided t-test weighted by number of reviews per physician. All other statistics were descriptive. Finally, we conducted a qualitative analysis of narrative reviews using a method that has been described in detail elsewhere.12
RESULTS
Website Identification
Our search mechanisms revealed 66 websites. Of these, 33 met inclusion criteria. We eliminated sites for the following reasons: they did not allow patients to review physicians (16); they were not publicly available (6); they were not active (4); they were limited to a single medical subspecialty (3); they were not in use in the US (2); or they were limited to a single state or city (2).
Website Characteristics
Eleven were local search sites that included physician reviews among many other review categories (e.g., restaurants); 8 were health sites that also allowed physician reviews; 2 were operated by insurance companies; and 12 were solely dedicated to rating physicians. Most sites (61%) required that patients register with an e-mail address to leave a review (Table 1). One site (3%) required credit-card information for patients to review physicians or read reviews. Fourteen sites (42%) allowed physicians to pay a fee to make their profiles more complete or visible, but only 5 of these 14 sites indicated that these “featured” physician profiles were paid advertisements. One site offered a free gift ($200 value) in exchange for reviewing at least eight physicians.
Table 1.
Website Characteristics
| N | (%) | |
|---|---|---|
| 33 | 100 | |
| General characteristics | ||
| Reviewer information required for review | 20 | (60.6) |
| Fees for patient use | 1 | (3.0) |
| Fees for physician listing | 1 | (3.0) |
| Physician may edit own profile | 27 | (81.8) |
| Patient may edit physician profile | 16 | (48.5) |
| Physician may purchase "enhanced profile" | 14 | (42.4) |
| “Enhanced profile” differentiated | 5 | (15.2) |
| Advertising | 31 | (93.9) |
| Health advertising | 25 | (75.8) |
| Search functions | ||
| Physician name | 31 | (93.9) |
| Sub-specialty | 25 | (75.8) |
| Physician area of clinical expertise | 7 | (21.2) |
| Ailment | 4 | (12.1) |
| State | 31 | (93.9) |
| City | 29 | (87.9) |
| Gender | 3 | (9.1) |
| Language(s) spoken | 2 | (6.1) |
| Geographically limited | 20 | (60.6) |
| Review structure | ||
| General | ||
| Overall rating | 27 | (81.8) |
| Side-by-side comparison | 3 | (9.1) |
| Structured questions available | 29 | (87.9) |
| Written comments available | 30 | (90.9) |
| Physician demographics | ||
| Name | 33 | (100) |
| Address | 29 | (87.9) |
| Phone | 29 | (87.9) |
| Sub-specialty | 28 | (84.8) |
| Physician area of clinical expertise | 9 | (27.3) |
| Hospital affiliation | 9 | (27.3) |
| Education | 13 | (39.4) |
| Residency | 9 | (27.3) |
| Years experience | 10 | (30.3) |
| Licenses/certifications | 12 | (36.4) |
| Publications | 2 | (6.1) |
| Languages spoken | 8 | (24.2) |
| Structured reviews | ||
| Non-clinical | ||
| Availability | 6 | (18.2) |
| Punctuality | 16 | (48.5) |
| Staff characteristics | 12 | (36.4) |
| Costs/billing | 5 | (15.2) |
| Location | 6 | (18.2) |
| Clinical | ||
| Bedside manner | 13 | (39.4) |
| Time with doctor | 6 | (18.2) |
| Correct diagnosis/knowledge | 16 | (48.5) |
| Follow-up | 3 | (9.1) |
| Recommend to others | 7 | (21.2) |
| Communication/education | 10 | (30.3) |
| Shared decision making | 1 | (3.0) |
Physician Information
Most sites (88%) reported physician address and phone number in addition to physician names and most allowed patients to search by either physician name (94%) or location (93%). Two sites (6%) allowed users to search for physicians by language spoken. Patients were asked to provide feedback on both clinical [e.g., bedside manner (39%), physician’s knowledge (49%), communication skills (30%)] and non-clinical [e.g., punctuality (49%), staff friendliness (36%)] aspects of care. One site (3%) asked whether the physician involved the patient in medical decision-making. Two sites featured very complete physician lists (about 90% of the 300 were listed on these sites), but the rest of the sites ranged from 0–60% in listing sampled physicians (Table 2). On average, a given physician could be found on about 30% of sites.
Table 2.
Presence, Reviews, and Written Comments Across 33 Websitesa
| Physicians with name listed | Any review | Patient narrative | ||||
|---|---|---|---|---|---|---|
| (N) | (%) | (N) | (%) | (N) | (%) | |
| 300 | 100 | 300 | 100 | 300 | 100 | |
| Local search | ||||||
| Citysearch.com | 63 | (21.0) | 0 | (0) | 0 | (0) |
| Yelp.com | 43 | (14.3) | 1 | (0.3) | 1 | (0.3) |
| Yellowpages.com | 16 | (5.3) | 0 | (0) | 0 | (0) |
| Local.yahoo.com | 107 | (35.7) | 1 | (0.3) | 1 | (0.3) |
| Local.com | 39 | (13.0) | 0 | (0) | 0 | (0) |
| Insiderpages.com | 106 | (35.3) | 2 | (0.7) | 2 | (0.7) |
| Yellowbook.com | 19 | (6.3) | 0 | (0) | 0 | (0) |
| Yellowbot.com | 25 | (8.3) | 0 | (0) | 0 | (0) |
| Kudzu.com | 238 | (79.3) | 0 | (0) | 0 | (0) |
| Angieslist.com | 13 | (4.3) | 13 | (4.3) | 13 | (4.3) |
| Openlist.com | 101 | (33.7) | 8 | (2.7) | 0 | (0) |
| General health | ||||||
| Steadyhealth.com | 77 | (25.7) | 0 | (0) | 0 | (0) |
| Wellness.com | 57 | (19.0) | 0 | (0) | 0 | (0) |
| Revolutionhealth.com | 210 | (70.0) | 3 | (1.0) | 1 | (0.3) |
| Checkmd.com | 85 | (28.3) | 0 | (0) | 0 | (0) |
| Findadoc.com | 175 | (58.3) | 0 | (0) | 0 | (0) |
| Mydochub.com | 150 | (50.0) | 0 | (0) | 0 | (0) |
| Healthworldweb.com | 196 | (65.3) | 0 | (0) | 0 | (0) |
| Healthcarescoop.com | 0 | (0) | 0 | (0) | 0 | (0) |
| Insurance portal | ||||||
| Healthcare.com | 80 | (26.7) | 0 | (0) | 0 | (0) |
| Vimo.com | 86 | (28.7) | 3 | (1.0) | 3 | (1.0) |
| Physician-rating specific | ||||||
| Healthgrades.com | 272 | (90.7) | 43 | (14.3) | 0 | (0) |
| Vitals.com | 264 | (88.0) | 24 | (8.0) | 11 | (3.7) |
| Ratemds.com | 173 | (57.7) | 21 | (7.0) | 19 | (6.3) |
| Drscore.com | 182 | (60.7) | 15 | (5.0) | 0 | (0) |
| Doctortree.org | 33 | (11.0) | 0 | (0) | 0 | (0) |
| Suggestadoctor.com | 11 | (3.7) | 0 | (0) | 0 | (0) |
| Doctorscorecard.com | 2 | (0.7) | 1 | (0.3) | 1 | (0.3) |
| Bookofdoctors.com | 0 | (0) | 0 | (0) | 0 | (0) |
| Healthcarereviews.com | 0 | (0) | 0 | (0) | 0 | (0) |
| Doctordecision.com | 116 | (38.7) | 2 | (0.7) | 0 | (0) |
| Yourcity.md | 146 | (48.7) | 0 | (0) | 0 | (0) |
| Mdrating.com | 0 | (0) | 0 | (0) | 0 | (0) |
| Average (without zeros) | 93.5 | (31.2) | 4.2 | (1.4) | 1.6 | (0.5) |
aThis table reports physician-level, website-specific data. Across all websites, there were a total of 190 reviews written for 81 physicians. Patients included narrative comments for 66 of these reviews (among 52 physicians)
Quantity and Content of Reviews
More than 70% of our physician sample did not have a review on any of the 33 sites. There were 190 reviews posted for 81 physicians; of these, 170 were quantitative reviews (some with and others without additional narrative comments). The number of generalists (37) and subspecialists (38) with any quantitative review was similar. Among physicians with reviews, there were six generalists and one subspecialist with five or more quantitative reviews, but the number of reviews per person did not differ significantly (p = 0.47) between generalists and subspecialists. The vast majority (88%) of quantitative reviews were positive. Six percent were negative, and six percent were neutral. Overall, generalists and subspecialists had a similar percentage of positive, negative, and neutral quantitative reviews. After accounting for varying number of reviews per physician, generalists tended to have more positive reviews than subspecialists, but this did not reach statistical significance (p = 0.06).
Sixty-six reviews included written patient narratives about 52 physicians. The vast majority (89%) of patient narratives were positive: “Dr ____ genuinely cares about her patients…She returns calls in an expeditious manner, apologizing if she couldn't explain the reason for the delay. She has made special arrangements to suit her treatment plan even if they are an obvious inconvenience for her…She follows up on everything with you and she does not treat you like a number…She's a 10 on this 1–5 scale.”
The minority (11%) of patient narrative comments were negative: “I was always kept waiting almost 45 min to see him at each appointment, and when he finally got me, he had the nerve several times to just glance at the chart and try to practically walk out without giving me a chance to discuss my concerns.”
Half (33) of the patient narrative comments included information intended to help other patients take actions to improve their care experience: “Helpful, will listen. But, be persistent. They are very busy over there. Go past the secretary if there is something you’re not getting and e-mail the doctor directly. Ask for his business card. Be heard and you will be taken care of.”
A few narratives appeared to have been written by the physician or by an agent of that physician: “Dr. ____ is a wonderful, caring physician who actually listens…Dr. _____ is a faculty member of the Reproductive Endocrine Unit, an Assistant Professor in Medicine at ______ Medical School ….She not only sees patients but also does research and edits a professional journal, so she is definitely up-to-date on all the latest developments in reproductive endocrinology. Highly recommend her.”
DISCUSSION
Recent efforts to measure and report physician performance have had only limited success in engaging patients.4 Physician-rating websites, a new addition to the “Social Web,” or “Web 2.0” represent an alternative route for patients to express satisfaction (or lack of satisfaction) with their physicians.13 The sites’ format, already familiar to travelers who read and write reviews about restaurants or hotels, may be easier for patients to interpret than other forms of public reporting. In what we believe to be the first formal study of physician-rating websites, we identified 33 such sites.
Explaining the addition of physician reviews to the NHS’ “Choices” website, the former health minister for the UK said, “I wouldn’t think of going on holiday without cross referencing two guide books and using Trip Advisor. We need to do something similar for the modern generation in health care.”10 In contrast, physician groups in the US have expressed opposition to physician rating websites on the grounds that reviews will be mostly negative.11
We found that neither of these viewpoints accurately reflects the state of physician rating in the US. Instead, we found that reviews were scarce, and when present, most were positive. For 300 physicians, we found only 66 written patient narratives across 33 sites. In contrast, an Internet search of restaurants in the Beacon Hill area of Boston, where several of the physician offices included in our study were located, turned up 38 narrative reviews for a single Lebanese restaurant (also mostly positive) on six separate travel and restaurant review sites.
Another of the AMA’s concerns, that physicians will be unable to respond to negative views because of confidentiality issues, was also not confirmed. We found that many of the patient’s complaints (e.g., “not enough parking,” “didn’t spend enough time,” “waited too long”) could be addressed without violating patient confidentiality. Also notable, but not predicted by physician groups, was the presence of several reviews that may have been written by the physician him/herself. These reviews were qualitatively different from typical reviews, including information not known to most patients, such as academic titles and services offered by the practice (e.g., on-site laboratory and radiology). Recent posts by respected medical bloggers have suggested that physicians should encourage patients to post positive comments.14,15 A few physicians have admitted to posting reviews about themselves.15,16 Said one, "Every anonymous review I've written on myself has been glowing."15
Overall, we found that most sites were neither user-friendly nor patient-centered. Patients searching for specific physicians will find that search mechanisms are cumbersome, advertising is prevalent, and information about physicians is incomplete. Although several recent studies have found that patients place a high value on shared decision making,17,18 only one site included questions about patient involvement in decision making. Similar to other forms of public reporting,19 there was enough variation in structured questions that it was difficult to compare physicians across sites, making side-by-side comparisons difficult.
This study has several limitations. We made extensive efforts to identify all physician-rating sites, but we may have missed some. We limited our examination to sites accessible to the general public, and did not include sites that were limited to one insurance company or a single geographic area. However, given the uniform paucity of information, it is unlikely that other sites have more ratings. We studied a sample of Boston physicians, and our findings may not be generalizable to other locations.
Physician-rating websites are a novel method for patients to share information about medical care they receive. Although these websites have the potential to empower patients looking for a physician and to offer a new route for providing physicians with constructive feedback, use by patients has been limited to date and reviews are mostly positive. Further study would help to better highlight the potential benefits and pitfalls of these sites for both physicians and patients.
Acknowledgements
The authors thank David A. Asch of the Leonard Davis Institute of Health Economics at the University of Pennsylvania for his help in editing this manuscript, Ms. Aruna Priya for her help with statistical analysis, and Ms. Long-Chau Van and Ms. Jill Avrunin for their assistance with manuscript preparation.
Financial or material support The study was conducted with funding from the Center for Quality of Care Research at Baystate Medical Center.
Conflict of Interest Dr. Lagu, Mr. Hannon, Dr. Rothberg, and Dr. Lindenauer have no potential conflicts of interest. Dr. Rothberg is the recipient of a clinical scientist development award from the Doris Duke Charitable Foundation.
References
- 1.Aligning forces for quality: Local efforts to transform American health care. Robert Wood Johnson Foundation; 2010. Available at: http://www.forces4quality.org/about-af4q [Accessed April 12, 2010].
- 2.2009 Healthcare Quality Report. Minnesota Community Measurement; 2009. Available at: http://www.mncm.org/site/ [Accessed April 12, 2010].
- 3.Jha AK, Orav EJ, Zheng J, Epstein AM. Patients' perception of hospital care in the United States. N Engl J Med. 2008;359(18):1921–31. doi: 10.1056/NEJMsa0804116. [DOI] [PubMed] [Google Scholar]
- 4.2008 update on consumers' views of patient safety and quality information. The Kaiser Family Foundation; 2008. Available at: http://www.kff.org/kaiserpolls/posr101508pkg.cfm [Accessed April 12, 2010].
- 5.Fung CH, Elliott MN, Hays RD, et al. Patients' preferences for technical versus interpersonal quality when selecting a primary care physician. Health Serv Res. 2005;40(4):957–77. doi: 10.1111/j.1475-6773.2005.00395.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Sarasohn-Kahn J. The Wisdom of Patients: Health Care Meets Online Social Media-CHCF.org. Available at: http://www.chcf.org/topics/chronicdisease/index.cfm?itemID=133631 [Accessed April 12, 2010].
- 7.Kamel Boulos MN, Wheeler S. The emerging Web 2.0 social software: an enabling suite of sociable technologies in health and health care education. Health Info Libr J. 2007;24(1):2–23. doi: 10.1111/j.1471-1842.2007.00701.x. [DOI] [PubMed] [Google Scholar]
- 8.Given R. MD Rating Sites: Current State of the Space and Future Prospects. The Health Care Blog. 2008. Available at: http://www.thehealthcareblog.com/the_health_care_blog/2008/11/md-rating-sites.html [Accessed April 12, 2010].
- 9.Bacon N. Will doctor rating sites improve standards of care? Yes. BMJ. 2009;338:b1030. doi: 10.1136/bmj.b1030. [DOI] [PubMed] [Google Scholar]
- 10.McCartney M. Will doctor rating sites improve the quality of care? No. BMJ. 2009;338:b1033. doi: 10.1136/bmj.b1033. [DOI] [PubMed] [Google Scholar]
- 11.Dolan P. Patients rarely use online ratings to pick physicians. June 23/30, 2008. AMedNews.com. 2008. Available at: http://www.ama-assn.org/amednews/2008/06/23/bil10623.htm [Accessed April 12, 2010].
- 12.Lagu T, Kaufman EJ, Asch DA, Armstrong K. Content of weblogs written by health professionals. J Gen Intern Med. 2008;23(10):1642–6. doi: 10.1007/s11606-008-0726-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Just Looking: Consumer Use of the Internet to Manage Care. California HealthCare Foundation; 2007. Available at: http://www.chcf.org/topics/view.cfm?itemID=133641 [Accessed April 12, 2010].
- 14.Pho K. How doctors should deal with physician rating sites. KevinMD.com. 2009. Available at: http://www.kevinmd.com/blog/2009/05/how-doctors-should-deal-with-physician-rating-sites.html [Accessed April 12, 2010].
- 15.Donnel R. Rate MDs and other internet physician rating sites. Notes from Dr. RW. 2009. Available at: http://doctorrw.blogspot.com/2009/05/rate-mds-and-other-internet-physician.html [Accessed April 12, 2010].
- 16.Solomon S. Doc's RateMDs battle turns ugly. National Review of Medicine. 2007. Available at: http://www.nationalreviewofmedicine.com/issue/2007/05_15/4_patients_practice09_9.html [Accessed April 12, 2010].
- 17.Schattner A, Bronstein A, Jellin N. Information and shared decision-making are top patients' priorities. BMC Health Serv Res. 2006;6:21. doi: 10.1186/1472-6963-6-21. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Cheraghi-Sohi S, Hole AR, Mead N, et al. What patients want from primary care consultations: a discrete choice experiment to identify patients' priorities. Ann Fam Med. 2008;6(2):107–15. doi: 10.1370/afm.816. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Rothberg MB, Morsi E, Benjamin EM, Pekow PS, Lindenauer PK. Choosing the best hospital: the limitations of public quality reporting. Health Aff (Millwood) 2008;27(6):1680–7. doi: 10.1377/hlthaff.27.6.1680. [DOI] [PubMed] [Google Scholar]