Abstract
Background
Patients use Web-based platforms to review general surgeons. However, little is known about the free-form text and structured content of the reviews or how they relate to the physicians’ characteristics or their practices.
Objective
This observational study aimed to analyze the Web-based reviews of general surgeons on the west side of Los Angeles.
Methods
Demographics, practice characteristics, and Web-based presence were recorded. We evaluated frequency and types of Yelp reviews and assigned negative remarks to 5 categories. Tabulated results were evaluated using independent t test, one-way analysis of variance, and Pearson correlation analysis to determine associations between the number of total and negative reviews with respect to practice structure and physician characteristics.
Results
Of the 146 general surgeons, 51 (35%) had at least 1 review and 29 (20%) had at least 1 negative review. There were 806 total reviews, 679 (84.2%) positive reviews and 127 (15.8%) negative reviews. The negative reviews contained a total of 376 negative remarks, categorized into physician demeanor (124/376, 32.9%), clinical outcomes (81/376, 22%), office or staff (83/376, 22%), scheduling (44/376, 12%), and billing (44/376, 12%). Surgeons with a professional website had significantly more reviews than those without (P=.003). Surgeons in private practice had significantly more reviews (P=.002) and more negative reviews (P=.03) than surgeons who were institution employed. A strong and direct correlation was found between a surgeon’s number of reviews and number of negative reviews (P<.001).
Conclusions
As the most common category of complaints was about physician demeanor, surgeons may optimize their Web-based reputation by improving their bedside manner. A surgeon’s Web presence, private practice, and the total number of reviews are significantly associated with both positive and negative reviews.
Keywords: patient satisfaction, general surgery, Los Angeles, Web-based ratings, digital health, Yelp
Introduction
One of the most ubiquitous business review websites, Yelp was established because the founder was unable to find recommendations for local physicians on the Web [1]. As customer feedback has become increasingly accessible, Web-based rating sites now have an influential impact on the impression and decisions of patients, with as many as 68% of patients turning to these resources to research or review physicians [2]. Physicians are beginning to realize that reactions and ratings detailed on these websites may impact which and how many patients visit them, as well as their overall reputation [3-5].
There are at least 33 websites where patients can describe their experience at hospitals, clinics, or clinical practices [6]. These websites range from general consumer rating websites to websites geared specifically toward the medical field. The structure of the all-purpose websites tends to afford more freedom to the commenters, whereas medical-based websites generally have a more structured format. In addition to premade surveys, several medical-based rating websites also allow reviewers to make unique remarks [6]. Yelp is one of the most used Web-based resources to review physicians [7-12].
An analysis of the content of Web-based reviews of general surgeons, including free-form content, has not been systematically described. We investigated the Yelp reviews of general surgeons in a defined region to categorize the content of the negative reviews and determine whether the number of reviews and the number of negative reviews correlated with the characteristics of the physicians. Los Angeles was chosen as the site of this study because it is home to a large variety of practices and institutions.
Methods
We identified general surgeons practicing on the west side of Greater Los Angeles using The Medical Board of California Web-based database [13] and InfoUSA (Papillion, Nebraska), a marketing company that provides contact databases and mailing lists. The physicians practicing in the 31 zip codes on the west side of Los Angeles were examined. The active practice status of the physician was determined using The Medical Board of California’s License verification Web-based tool [13]. Those surgeons who were still in training, retired, and those without an active medical license were eliminated. Physician gender, years since graduating from medical school, and medical school attended were identified. A Google search was performed to determine if the physician had a medical practice website.
A physician was considered to have a Yelp page if they could be found and reviewed on a page designated for an institution, clinical practice, or the physician. The following was documented: presence of a Yelp page, number of reviews, number of positive reviews, and number of negative reviews. Yelp users rate an institution or physician on a 5-star system, with 5 stars defined as “Woohoo! As good as it gets!” and 1 star defined as “Eek! Methinks not.” Yelp defines 4 stars as “Yay! I’m a fan.” As 4 stars was less than a perfect star rating, a negative review was defined as a review that contained at least 1 negative remark and had a rating of 4 stars or fewer. Yelp defines 3 stars as A-OK, and because it is not a 4- or 5-star review, a level of mediocrity is implied [14]. A positive review was defined as any 5-star or 4-star review that contained no negative remarks.
Negative reviews were further characterized. Each commenter’s username, date of review, and star rating was noted. Free-form reviews were manually tabulated, categorized, and resolved by 2 independent reviewers and were empirically divided into 5 categories modified from previously defined categories [15]: Scheduling (doctor availability and punctuality), billing, office and staff (staff friendliness or professionalism, staff presence, and office décor or location), clinical outcome (correct diagnosis or treatment, technical skill, treatment of unforeseen complications, if additional treatment was needed, and follow-up care), and physician demeanor (education, empathy, bedside manner, professionalism, preparedness or organization, time with doctor, communication skills, shared decision making, and general impression). Remarks within negative reviews were noted to be negative, neutral, or positive.
Once the raw data were collected, summary statistics were calculated using univariable analyses. An independent t test, which determines whether there is a statistically significant difference between the means in 2 unrelated groups, was run to determine whether or not certain factors (possession of professional website, private practice or institution employed, gender, and medical school outside of the United States) impacted the number of total reviews or negative reviews of a surgeon. One-way analysis of variance analysis, which determines whether there is a statistically significant difference between the means in more than 2 unrelated groups, was used to determine if zip code or the number of years since graduating from medical school impacted a surgeon’s number of total reviews or negative reviews. A Pearson correlation, which measures the linear correlation between 2 variables, was performed to determine if there was any relationship between a give surgeon’s total number of reviews and quantity of negative reviews. Statistical Package for the Social Sciences statistical software was used.
Results
We identified 146 practicing general surgeons on the west side of Greater Los Angeles. A total of 33 (22.6%, 33/146) surgeons were female and 113 (77.4%, 113/146) were male. Moreover, 55 (37.7%, 55/146) surgeons were in private practice and 91 (69.2%, 91/146) were institution employed. In addition, 19 (13.0%, 19/146) surgeons went to medical school outside of the United States. Furthermore, 42 (28.8%, 42/146) physicians had a professional website (Table 1).
Table 1.
Demographics (N=146).
| Characteristics | Statistics, n (%) |
| Male | 113 (77.4) |
| Private practice | 55 (38) |
| Institutional employed | 91 (69.2) |
| Physicians with website | 42 (28.7) |
| Foreign medical school graduate | 19 (13.0) |
| Physicians with a Yelp page | 59 (40.4) |
| Physicians reviewed on Yelp | 51 (34.9) |
| Physicians with negative reviews | 29 (19.8) |
A total of 59 (40.4%, 59/146) surgeons had a Yelp page. Moreover, 51 physicians (34.9% of all surgeons and 86% of those with a Yelp) had at least 1 review. Of the 59 physicians who had a Yelp page, 29 (49%, 29/59) had at least 1 negative review and 48 (81%, 48/59) had at least 1 positive review (Figure 1).
Figure 1.
Distribution of reviews. The chart on the left shows the breakdown of positive reviews among the 59 physicians who had a Yelp page, whereas the chart on the right shows the breakdown of negative reviews. The number of negative and positive reviews is shown along with the number of physicians who had that number of negative and positive reviews, respectively.
There was a total of 806 documented reviews. Of these reviews, 679 (84.2%) were positive and 126 (15.6%) were negative. Within the 126 negative reviews, there were 376 negative remarks: 124 (32.9%) concerning physician demeanor, 81 (21.5%) on clinical outcomes, 83 (22.0%) regarding the office or staff, 44 (11.7%) about scheduling, and 44 (11.7%) in relation to billing (Table 1).
The existence of a professional website as well as the type of employment had a significant impact on the total number of reviews (Tables 2-4). Those with a professional website had significantly more overall reviews compared with those without (mean 16.36, SD 31.0 vs mean 1.14, SD 3.23; P=.003). Two-thirds of physicians in private practice had reviews, compared with one-third of physicians employed by an institution (mean 18.0, SD 31.4 vs mean 0.96, SD 3.18; P=.002). Being in private practice was also significantly associated with the number of negative reviews (mean 3.11, SD 8.13 vs mean 0.09, SD 0.35; P=.03). As such, private practice was significantly associated with both more overall reviews and more negative reviews. Furthermore, there was a trend toward a correlation between a greater number of reviews and a greater number of years since graduation from medical school (P=.05).
Table 2.
Statistical analysis via 2-way Pearson correlation analysis.
| Reviews | Number of reviews, mean (SD) | P value |
| Total reviews (n=806) | 5.6 (18.08) | .001 |
| Negative reviews (n=126) | 0.86 (4.33) | .001 |
Table 4.
Statistical analysis via 1-way analysis of variance analysis.
| Variable | P value | |
|
|
Comparing total number of reviews | Comparing number of negative reviews |
| Number of years since graduating from medical school | .05 | .85 |
| Zip code | .20 | .7 |
A 2-tailed Pearson correlation was performed to evaluate the association between the total number of reviews and the number of negative reviews. The correlation between the total number of reviews and negative reviews was strong (r=.862; P<.001).
The total number of reviews was not impacted by a surgeon’s gender (male: mean 4.2, SD 11.6 vs female: mean 9.9, SD 31.3; P=.32), whether they attended medical school outside the United States (non-United States: mean 1.6, SD 3.20 vs United States: mean 6.1, SD 19.3; P=.32), or the practice zip code (P=.20).
The likelihood of having negative reviews did not significantly differ between surgeons with a professional website versus surgeons without a professional website (mean 2.3, SD 7.80 vs mean 0.28, SD 0.98; P=.10). Similarly, the number of negative reviews was not significantly affected by a surgeon’s gender (male: mean 0.48, SD 1.22 vs female: mean 2.18, SD 8.79; P=.28), whether or not they attended medical school outside the United States (non-United States: mean 0.6, SD 1.54 vs United States: mean 0.91, SD 4.6; P=.76), the zip code they practiced in (P=.77), or the number of years since the surgeon graduated from medical school (P=.85).
Table 3.
Statistical analysis via independent t test.
| Variable | Number of physicians with any reviews | Reviews | Negative reviews | ||||
|
|
Total, N | Mean (SD) | P value | Total, N | Mean (SD) | P value | |
| Private practice physicians (n=55) | 34 | 720 | 18 (31.4) | .002 | 118 | 3.11 (8.13) | .03 |
| Institution employed physicians (n=91) | 17 | 86 | 0.96 (3.18) | .002 | 8 | 0.09 (0.35) | .03 |
| Medical school out of United States (n=19) | 5 | 31 | 1.6 (3.20) | .32 | 11 | 0.6 (1.54) | .76 |
| Medical school in United States (n=127) | 46 | 775 | 6.1 (19.3) | .32 | 115 | 0.91 (4.60) | .76 |
| Male (n=113) | 38 | 480 | 4.2 (11.6) | .32 | 54 | 0.48 (1.22) | .28 |
| Female (n=33) | 13 | 326 | 9.9 (31.3) | .32 | 72 | 2.18 (8.79) | .28 |
| Possess professional website (n=42) | 28 | 687 | 16 (31.0) | .003a | 97 | 2.3 (7.80) | .10 |
| Do not possess professional website (n=104) | 23 | 119 | 1.1 (3.23) | .003 | 29 | 0.28 (0.98) | .10 |
Discussion
Our finding that the majority of Web-based reviews were positive supports prior findings [10,16]. Unfortunately, negative reviews do exist and can have a damaging effect on a physician’s reputation and practice [3,5,9,17,18]. The most common category of Yelp complaints was physician demeanor. As such, surgeons may optimize their Web-based reputation by improving their bedside manner. A surgeon’s type of practice (ie, private practice or institution employed) and the total number of reviews were significantly associated with more negative reviews. A surgeon’s type of practice and the possession of a personal website were significantly associated with more total Yelp reviews.
Of the negative reviews, the most common category was criticism of physician demeanor. Patient perception of physician demeanor is a factor about which physicians may have some control. Thus, it might be possible to improve one’s reviews by enhancing patient-physician interactions. This includes being better prepared for each consultation, spending sufficient time with patients, clearly communicating the plan of care and disease processes affecting each patient, and displaying empathy. Over one-fifth of the complaints centered around clinical outcomes and another one-fifth concentrated on the office or staff. Although physicians are not always in control of clinical outcomes, keeping one’s skills up to date and practicing evidence-based medicine may improve clinical outcomes. To reduce the number of negative Web-based reviews, it is probably important to treat unforeseen complications quickly and empathically. In addition, hiring courteous office staff and optimizing the aesthetic appearance of one’s practice environment may enhance the quality of Web-based reviews.
Our study found that Web presence significantly impacted a physician’s total number of reviews. Presence of a website, personal or private practice, was shown to have a significant effect on the total number of reviews a physician received. The mean number of reviews for those with a professional website was significantly greater than the number of reviews for surgeons without a website, indicating that an increased Web presence leads to more Web-based reviews. General surgeons in private practice had significantly more reviews overall (P=.002) and significantly more negative reviews (P=.03) than those who were institution employed. In addition, the total number of reviews was strongly correlated with the number of negative reviews. This suggests that although a Web-based presence may be important in enhancing a surgeon’s reputation, it may also be detrimental, depending on the content of the individual reviews.
There are several limitations to this study. Restricting physicians examined to a single geographic area is descriptive for the region but is also limiting in size and scope. Due to the presence of large health institutions in the west side of Los Angeles, the number of surgeons with reviews may be artificially low because institution-based physicians may have a much more limited Web presence and thus fewer Web-based reviews. Yelp was the only Web-based rating website examined, excluding any reviews and Web-based presence surgeons may have had on other rating websites [7]. There is inherent response bias as patients make the active decision whether to write a review or not. This study only aimed to examine patient perception of physicians as well as any perceived problems they encountered in their experiences, and as such, we cannot comment on how this reflects on the quality of the physicians themselves [17,19].
Future studies are needed to determine if the trends and correlations found are applicable to surgeons practicing in other and larger geographic regions. Other specialties need to be examined to see if there is a difference in the number and type of reviews and Web presence among other specialties. Most importantly, the impact of Yelp reviews on a surgeon’s practice needs to be assessed.
Acknowledgments
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Footnotes
Conflicts of Interest: None declared.
References
- 1.Yelp Official Blog: Yelp, Inc; 2013. [2019-03-17]. Study: Yelp ratings work for hospitals, too https://www.yelpblog.com/2013/03/study-yelp-ratings-work-for-hospitals-too . [Google Scholar]
- 2.Rastegar-Mojarad M, Ye Z, Wall D, Murali N, Lin S. Collecting and analyzing patient experiences of health care from social media. JMIR Res Protoc. 2015;4(3):e78. doi: 10.2196/resprot.3433. http://www.researchprotocols.org/2015/3/e78/ v4i3e78 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Emmert M, Meszmer N, Sander U. Do health care providers use online patient ratings to improve the quality of care? Results from an online-based cross-sectional study. J Med Internet Res. 2016 Sep 19;18(9):e254. doi: 10.2196/jmir.5889. http://www.jmir.org/2016/9/e254/ v18i9e254 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Emmert M, Sauter L, Jablonski L, Sander U, Taheri-Zadeh F. Do physicians respond to web-based patient ratings? An analysis of physicians' responses to more than one million web-based ratings over a six-year period. J Med Internet Res. 2017 Jul 26;19(7):e275. doi: 10.2196/jmir.7538. http://www.jmir.org/2017/7/e275/ v19i7e275 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Holliday AM, Kachalia A, Meyer GS, Sequist TD. Physician and patient views on public physician rating websites: a cross-sectional study. J Gen Intern Med. 2017 Jun;32(6):626–31. doi: 10.1007/s11606-017-3982-5.10.1007/s11606-017-3982-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients' evaluations of health care providers in the era of social networking: an analysis of physician-rating websites. J Gen Intern Med. 2010 Sep;25(9):942–6. doi: 10.1007/s11606-010-1383-0. http://europepmc.org/abstract/MED/20464523 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Lagu T, Metayer K, Moran M, Ortiz L, Priya A, Goff SL, Lindenauer PK. Website characteristics and physician reviews on commercial physician-rating websites. J Am Med Assoc. 2017 Dec 21;317(7):766–768. doi: 10.1001/jama.2016.18553. http://europepmc.org/abstract/MED/28241346 .2603919 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Geletta S. Measuring patient satisfaction with medical services using social media generated data. Int J Health Care Qual Assur. 2018 Mar 12;31(2):96–105. doi: 10.1108/IJHCQA-12-2016-0183. [DOI] [PubMed] [Google Scholar]
- 9.Dorfman RG, Purnell C, Qiu C, Ellis MF, Basu CB, Kim JYS. Happy and unhappy patients: a quantitative analysis of online plastic surgeon reviews for breast augmentation. Plast Reconstr Surg. 2018 Dec;141(5):663e–673e. doi: 10.1097/PRS.0000000000004268.00006534-201805000-00013 [DOI] [PubMed] [Google Scholar]
- 10.Ellimoottil C, Hart A, Greco K, Quek ML, Farooq A. Online reviews of 500 urologists. J Urol. 2013 Jun;189(6):2269–73. doi: 10.1016/j.juro.2012.12.013.S0022-5347(12)05837-5 [DOI] [PubMed] [Google Scholar]
- 11.Ramkumar PN, Navarro SM, Chughtai M, La T, Fisch E, Mont MA. The patient experience: an analysis of orthopedic surgeon quality on physician-rating sites. J Arthroplasty. 2017 Dec;32(9):2905–2910. doi: 10.1016/j.arth.2017.03.053.S0883-5403(17)30290-5 [DOI] [PubMed] [Google Scholar]
- 12.Kadry B, Chu LF, Kadry B, Gammas D, Macario A. Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res. 2011;13(4):e95. doi: 10.2196/jmir.1960. http://www.jmir.org/2011/4/e95/ v13i4e95 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Medical Board of California. 2018. [2019-03-14]. Medical Board of California Breeze License Verification http://www.mbc.ca.gov/Breeze/License_Verification.aspx .
- 14.Woolf M. minimaxir. [2019-03-14]. The Statistical Difference Between 1-Star5-Star Reviews on Yelp https://minimaxir.com/2014/09/one-star-five-stars/
- 15.Asanad K, Parameshwar PS, Houman J, Spiegel BM, Daskivich TJ, Anger JT. Online physician reviews in female pelvic medicine and reconstructive surgery: what do patients really want? Female Pelvic Med Reconstr Surg. 2018;24(2):109–114. doi: 10.1097/SPV.0000000000000503.01436319-201803000-00010 [DOI] [PubMed] [Google Scholar]
- 16.Cloney M, Hopkins B, Shlobin N, Dahdaleh NS. Online ratings of neurosurgeons: an examination of web data and its implications. Neurosurgery. 2018 Apr 03;83(6):1143–52. doi: 10.1093/neuros/nyy064.4959195 [DOI] [PubMed] [Google Scholar]
- 17.Chen J, Presson A, Zhang C, Ray D, Finlayson S, Glasgow R. Online physician review websites poorly correlate to a validated metric of patient satisfaction. J Surg Res. 2018 Jul;227:1–6. doi: 10.1016/j.jss.2018.01.037.S0022-4804(18)30041-6 [DOI] [PubMed] [Google Scholar]
- 18.Emmert M, Meier F, Pisch F, Sander U. Physician choice making and characteristics associated with using physician-rating websites: cross-sectional study. J Med Internet Res. 2013;15(8):e187. doi: 10.2196/jmir.2702. http://www.jmir.org/2013/8/e187/ v15i8e187 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Haskins IN, Krpata DM, Rosen MJ, Perez AJ, Tastaldi L, Butler RS, Rosenblatt S, Prabhu AS. Online surgeon ratings and outcomes in hernia surgery: an Americas Hernia Society quality collaborative analysis. J Am Coll Surg. 2017 Nov;225(5):582–9. doi: 10.1016/j.jamcollsurg.2017.08.007.S1072-7515(17)31769-6 [DOI] [PubMed] [Google Scholar]