Abstract
Background
Recent evidence suggests that patients prefer subjective and crowd-sourced information over data-driven or quality-based outcomes when choosing a surgeon. Online physician rating and review websites continue to increase in popularity, and over half of patients use them to research physicians. Specifically, Yelp.com is the most frequently utilized online resource by patients. Data regarding the characteristics of negative reviews for spine surgeons and practices is lacking.
Methods
Orthopedic Spine surgeons and practices in 8 major US metropolitan regions were surveyed for one-star reviews on Yelp.com. The factors noted in the reviews were recorded and they were classified according to their clinical or nonclinical nature. Reviews were also subclassified into nonsurgical or surgical episodes of care.
Results
A total of 6,286 Yelp reviews were discovered, 671 (10.6%) of which were rated one-star. The majority of negative reviews (76.4%) were from patients who did not report surgery by the surgeon or practice. Of all comments, 491 (77.6%) related to nonclinical complaints. The most common factors noted in negative reviews were related to bedside manner, rude or unprofessional staff, and wait time.
Conclusion
Choosing a surgeon is a complex process for patients. The large majority of negative reviews were related to nonclinical issues such as poor bedside manner or rude staff and most of these were written by patients that did not undergo a surgical procedure. This may explain the large discrepancy that has been observed between quality metrics and online crowd-sourced reviews. Paying attention to these nonclinical factors may represent the most feasible and valuable targets to improve a surgeon's practice and attract future patients.
Keywords: Online, Physician ratings, Physician reviews, Social media, Quality, Reimbursement
Introduction
Low back pain is the leading cause of disability worldwide [1]. It is estimated that 1-2% of the population is disabled because of low back pain and up to 80% of the population will be affected by it [2]. Pain originating from the cervical or thoracic regions can also be disabling and present significant burdens to the individual [3], [4], [5]. Not surprisingly, much attention has been focused on the appropriate management of this pain.
Although surgical intervention is not a first-line treatment, it remains an option after failure of conservative measures such as anti-inflammatory medications, activity modification, physical therapy, and steroid injections. The role of surgical management in these patients continues to significantly increase in the United States. The rate of spinal fusions alone have showed a 137% increase since 1998, a rate that surpasses that of other common orthopedic procedures such as hip and knee arthroplasty [6,7] Other spinal procedures, including discectomies and laminectomies, have increased during this period as well [8]. Total health expenditures for all adults with chronic back pain have been estimated to be as high as $102 billion annually [9].
Considering the rising cost associated with spine surgery as well as healthcare delivery as a whole, it is no surprise that there has been a heightened interest in maximizing the value of health care in recent years. Although provider quality data had been publicly available for years, the Affordable Care Act prompted more transparency of previously confidential Medicare utilization and payment data. In doing so, it has increased the transparency of provider comparison and patient-centered outcomes. This was followed in 2014 by the Centers for Medicare and Medicaid Services (CMS) release of detailed utilization information for over $77 billion in payments from over 880,000 physicians [10]. As part of this collaborative initiative, the Consensus Core Set of Orthopedic Measures was released in 2016 in order to simplify the reporting of value-based payment and inform decision-making by patients [11]. This release has caused an increased reliance by payers and patients on online reviews and ratings related to comparing providers.
In addition to the quantifiable disease specific and data-driven outcomes available to compare providers, there is now also an additional focus on the subjective patient experience. The Consensus Core Set of Orthopedic Measures includes “patient experience” as one of the core 7 domains. This is also demonstrated by the Hospital Consumer Assessment of Healthcare Providers and Systems measurement of patient perceptions of their hospital experience [12]. In addition to these formal methods, there are many public and private websites that serve as resources to document patient perceptions and opinions of the their care. Specifically, it has been demonstrated that Yelp.com is the most frequently used online resource for patients to evaluate providers [13].
Online physician ratings and reviews continue to increase in number, and up to 60% of patients utilize them in choosing a physician [14,15]. There have been several previous reports characterizing online reviews of orthopedic surgeons. One such study demonstrated that although the majority of reviews were positive, over 30% were either negative or extremely negative [16]. Several other studies have also characterized the reviews or evaluated the factors associated with positive reviews [17], [18], [19]. To the authors’ knowledge, there has not been an investigation focused on spine surgeon reviews and only one study analyzing very negative reviews, which was limited to arthroplasty surgeons [20]. The purpose of this study was to characterize negative reviews for spine surgeons and practices.
Materials and methods
As all data utilized in the study was publicly available, the study was deemed IRB exempt. A search was performed using the keywords “orthopedic spine surgery” on yelp.com for 8 major metropolitan areas including New York, Boston, San Francisco, Los Angeles, Dallas, Phoenix, Seattle, and Washington DC. Although there exist several online physician review websites to review surgeons, Yelp was chosen as it has been reported as the most frequently visited by patients [13]. The practice or surgeon named in the review was then confirmed to be an accredited orthopedic surgeon that performs spine surgery through a secondary internet search using physician websites, hospital group websites, Angie's List, Doximity, and vitals.com. Reviews that did not name a particular surgeon of a practice that performs multiple specialties were excluded in order to focus particularly on confirmed orthopedic spine surgery reviews. All reviews other than single-star (out of a possible 5 stars) were excluded from the study (Fig. 1).
Fig. 1.
Online review selection.
Flowchart analysis of orthopedic spine surgery reviews analyzed from yelp.com.
One-star reviews were classified as directed toward a physician, practice, or both. Reviews were categorized as clinical if comments included a reference to treatment complications, reoperation, pain, perceived misdiagnosis, readmissions, delay in communication, unclear treatment plan, or delay in care. Reviews that acknowledged bedside manner, professionalism, wait time, cost issues, limited time with provider, insurance issues, facilities, and scheduling were categorized as nonclinical. Reviews were considered surgical or nonsurgical on the basis of whether or not a surgical episode of care was explicitly referenced. Comments referencing hospitalization were not considered surgical unless surgery was directly referenced. Reviews that were unable to be categorized per the above criteria were excluded. Reviews were initially classified and categorized as mentioned above by two authors separately (JS and KL). Inter-rater reliability was calculated (83.3% agreement and Kohens Kappa statistic of 0.667) and a third author (JP) resolved conflict when classifications and categories did not coincide between authors JS and KL.
Univariate analysis was performed to determine means and 95% confidence intervals. Continuous variables were compared using the student's t-test. Categorical variables were analyzed using the chi-square test with alpha set to 0.05. The rate ratio (the ratio of the rate for nonsurgical divided by surgical reviews) was determined for each category. Statistical analysis was performed using a commercially available software package (JMP Pro v.13; SAS Institute, INC.).
Results
A total of 6,286 Yelp reviews were discovered. Of these, 671 (10.6%) were identified as single-star reviews. These reviews were recorded on a total of 305 surgeons, averaging 2.2 negative reviews per surgeon included. A total of 374 reviews were excluded due to an inability to classify them or an unclear relation to spine surgery, and the remaining 297 reviews were included for analysis (Fig. 1). Nonsurgical patients were associated with 227 reviews and 70 were linked to surgical patients (Table 2). Of all comments within reviews, there were 491 comments relating to non-clinical complaints and 142 relating to clinical issues (Table 1).
Table 2.
Clinical focus of surgical vs. non-surgical patients.
| Surgical (n = 70) | Non-surgical (n = 227) | p-value | |
|---|---|---|---|
| Clinical | 99 (54.7%) | 43 (9.5%) | <0.001 |
| Non-clinical | 82 (45.3%) | 409 (90.5%) | <0.001 |
Table 1.
Negative online review clinical and surgical characterization.
| Focus of the Review | Number (percent) of complaints | Surgical Patients (n = 70) | Non-Surgical Patients (n = 227) | p-value |
|---|---|---|---|---|
| Total complaints, N (%) | Total complaints, N (%) | |||
| Clinical Complaints (N = 142) | ||||
| Complication | 48 (16%) | 45 (64%) | 3 (1%) | <0.001 |
| Readmission | 4 (1%) | 3 (4%) | 1 (0%) | <0.001 |
| Reoperation | 9 (3%) | 9 (13%) | 0 (0%) | N/A |
| Uncontrolled pain | 37 (12%) | 27 (39%) | 10 (4%) | <0.001 |
| Misdiagnosis | 18 (6%) | 6 (9%) | 12 (5%) | 0.232 |
| Unclear treatment plan | 8 (3%) | 1 (1%) | 7 (3%) | 0.430 |
| Delay in care | 13 (4%) | 7 (10%) | 6 (3%) | <0.001 |
| Clinical Other | 5 (2%) | 1 (1%) | 4 (2%) | 0.833 |
| Non-Clinical Complaints (N = 491) | 99 | 43 | ||
| Bedside Manner Doctor/Unprofessional | 146 (49%) | 32 (46%) | 114 (50%) | 0.595 |
| Bedside Manner Midlevel/Unprofessional | 21 (7%) | 5 (7%) | 16 (7%) | 0.976 |
| Rude/unprofessional Staff | 92 (31%) | 12 (17%) | 80 (35%) | 0.011 |
| Wait Time | 75 (25%) | 9 (13%) | 66 (29%) | 0.012 |
| Not enough time spent with provider | 49 (16%) | 4 (6%) | 45 (20%) | 0.008 |
| Cost/billing/insurance | 66 (22%) | 18 (26%) | 48 (21%) | 0.406 |
| Facilities | 10 (3%) | 1 (1%) | 9 (4%) | 0.287 |
| Scheduling issues | 30 (10%) | 1 (1%) | 29 (13%) | 0.008 |
| Non-clinical Other | 2 (1%) | 0 (0%) | 2 (1%) | 0.432 |
| 82 | 409 | |||
Table 1. Clinical and non-clinical review comments categorized by surgical and non-surgical patients. Bold denotes statistical significance (p<0.05) rate of complaint between surgical and non-surgical patients.
Clinical factors most commonly addressed included complication (48 reviews, 16%), uncontrolled pain (37 reviews, 12%), perceived misdiagnosis (18 reviews, 6%), delay in care (13 reviews, 4%), reoperation (9 reviews, 3%), and unclear treatment plan (8 reviews, 3%). The most common non-clinical complaints referenced bedside manner (146 reviews, 49%), rude or unprofessional misconduct (92 reviews, 31%), wait time (75 reviews, 25%), cost or insurance issues (66 reviews, 22%), limited time with provider (49 reviews, 16%), and scheduling issues (30 reviews, 10%).
A greater number of reviews were derived from patients that did not explicitly cite a surgical episode (227) when compared to those that reported surgery (70). When compared, nonsurgical reviews were more likely to contain nonclinical rather than clinical complaints (p<0.001). Of these, scheduling issues (p=0.008), limited time with provider (p=0.008), rude or unprofessional misconduct (p=0.011), and wait time (p=0.012) were more common for nonsurgical versus surgical reviews. Conversely, clinical complaints were proportionately more common in patients that reported a surgical episode (p<0.001) (Table 2). When compared to non-surgical patients, surgical patients demonstrated higher rates of clinical complaints related to uncontrolled pain (p<0.001), delayed care (p<0.001), complication (p<0.001) and readmission (p<0.001).
Discussion
Spine surgeons are experiencing an increased demand for their work as well as heightened use of data-driven metrics and evaluation measures. The accessibility of quality data has generated the rise in popularity of online evaluation and review websites, and it is predicted that patient utilization of these resources will continue to rise in orthopedic surgery [14,21,22]. Accordingly, it is important to understand how patients interpret and apply this data in choosing a surgeon. The purpose of this study was to characterize the extremely negative reviews of spine surgeons and practices. Single-star reviews comprised 10.6% of all reviews analyzed. Our results demonstrated that non-clinical factors such as bedside manner, rude or unprofessional staff, wait time, and cost or insurance issues were more likely to be associated with a negative review than pain, complications, or reoperation. A minority of these patients reported a surgical episode of care.
Crowd-sourced information relating to patient satisfaction of their healthcare experience is available through several public websites. It remains unclear whether the information on these websites accurately portrays the quality of care received. A study investigating the five busiest physician-rating websites and user-generated data found that none contained the 7 Consensus Core Set of Orthopedics Measures [17]. Additionally, numerous investigations have assessed the correlation between online reviews and various outcomes including malpractice claims, mortality, infection, and readmission. Results have revealed weak, if any, correlation between these outcomes and online ratings [15,23,24]. Similar findings have been reported for nursing home care, as Yelp.com reviews and a CMS-sponsored site had poor correlation in terms of overall star ratings and quality measures [25]. In our study, we found that 77.6% (491/633) of all one-star reviews reflected nonclinical rather than clinical issues. This suggests the large majority of negative reviews are exclusive of factors that reflect healthcare quality and coincides with previous investigations that report a poor correlation between online ratings and patient outcomes.
Several studies have evaluated the psychology and behavior of online physician reviews. It has been demonstrated that patients have higher comprehension when information is presented with low cognitive burden and prefer patient comments over quality metrics [13,26]. This may explain why, when compared to other online patient resources, CMS data ranks 8 out of 13 in terms of overall patient use [13]. Patients also demonstrate a preference toward more succinct reviews, which may promote user-generated data to trend toward either extremely positive or extremely negative [26]. The rate of such extreme reviews has been shown to be as high as 64% [16].
The motivation of a dissatisfied customer to retaliate against the business along with the idea that satisfied patients are less likely to leave reviews may lead one to expect a higher proportion of strongly negative reviews [27]. However, this was not borne out in our study as one-star reviews comprised only 10.6% (671/6286) of all reviews. It is difficult to discern why this was the case in our study but may reflect an increased awareness of the impact of online reviews and efforts by physicians to combat them. Investigations have reported that some physicians directly ask for online reviews as well as hand out comment cards in order to improve their online rating [28]. Further, there now exist companies that help improve physician ratings as well as prevent damaging online rating from being placed [29,30].
The majority of patients utilize online reviews to choose a physician, and the number of online ratings continue to increase [14]. Accordingly, it is beneficial for physicians to understand the nature of online reviews and how to maintain a favorable rating. In the presented study, the most common reasons for negative reviews were related to negative interactions with healthcare providers, whether it was poor bedside manner of the physician (49%) or rude/unprofessional staff (31%) (Table 1). Logistical issues such as scheduling issues (10%), insufficient time with provider (16%), and cost or insurance problems (22%) were less common (Table 1).
Non-surgical patients were found to be responsible for the vast majority of negative reviews and were also more likely to report non-clinical factors. This is consistent with the high degree of nonclinical factors cited in negative reviews for dermatologists and arthroplasty surgeons [20,31]. Contrary to what a surgeon commonly defines as successful practice (e.g., outcomes, quality), the public considers friendly demeanor and amicable staff interactions as most important in choosing a physician. Paying attention to these factors may represent the most feasible and valuable targets to improve a surgeon's clinical practice, maintain a favorable online presence, and attract future patients.
This study has several limitations. Our search was limited to 8 metropolitan areas which may limit the applicability of our findings, especially to smaller or more rural areas. Although we used multiple reviewers to mitigate error, it is possible that reviews were erroneously categorized. Another limitation is that only Yelp.com reviews were considered, which may not reflect all online review or rating sites.
Choosing a surgeon is a complex process for patients. To the authors’ knowledge, this is the first investigation to focus on negative reviews for spine surgeons. We found that the large majority of negative reviews were related to nonclinical issues such as poor bedside manner or rude staff and that most of these were written by patients that did not undergo a surgical procedure. These nonsurgical patients were more likely to complain about wait times, scheduling issues, and limited time with the provider when compared to surgical patients. Patients that underwent surgery were also more likely to complain about pain, delayed care, and readmission.
Source of Funding
None.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Editor: Dr J Grauer
Footnotes
Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.ejps.2020.105216.
Appendix. Supplementary materials
References
- 1.Deyo RA, Weinstein JN. Low back pain. N Engl J Med. 2001;344(5):363–370. doi: 10.1056/NEJM200102013440508. [DOI] [PubMed] [Google Scholar]
- 2.Andersson GBJ. Epidemiology of low back pain. Acta Orthop Scand. 1998;69(sup281):28–31. doi: 10.1080/17453674.1998.11744790. [DOI] [PubMed] [Google Scholar]
- 3.Bovim G, Schrader H, Sand T. Neck pain in the general population. Spine. 1994;19(12):1307–1309. doi: 10.1097/00007632-199406000-00001. [DOI] [PubMed] [Google Scholar]
- 4.Austin GP, Benesky WT. Thoracic pain in a collegiate runner. Man Ther. 2002;7(3):168–172. doi: 10.1054/math.2002.0461. [DOI] [PubMed] [Google Scholar]
- 5.Edmondston SJ, Singer KP. Thoracic spine: anatomical and biomechanical considerations for manual therapy. Man Ther. 1997;2(3):132–143. doi: 10.1054/math.1997.0293. [DOI] [PubMed] [Google Scholar]
- 6.Rajaee SS, Bae HW, Kanim LEA, Delamarter RB. Spinal fusion in the United States: analysis of trends from 1998 to 2008. Spine. 2012;37(1):67–76. doi: 10.1097/BRS.0b013e31820cccfb. [DOI] [PubMed] [Google Scholar]
- 7.Bernstein DN, Brodell D, Li Y, Rubery PT, Mesfin A. Impact of the economic downturn on elective lumbar spine surgery in the United States: a national trend analysis, 2003 to 2013. Glob Spine J. 2017;7(3):213–219. doi: 10.1177/2192568217694151. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Deyo RA, Mirza SK. Trends and variations in the use of spine surgery. Clin Orthop. 2006;443:139–146. doi: 10.1097/01.blo.0000198726.62514.75. [DOI] [PubMed] [Google Scholar]
- 9.Martin BI, Deyo RA, Mirza SK, et al. Expenditures and health status among adults with back and neck problems. JAMA. 2008;299(6):656–664. doi: 10.1001/jama.299.6.656. [DOI] [PubMed] [Google Scholar]
- 10.The Medicare Physician-Data Release — Context and Rationale | NEJM. Accessed August 23, 2020. https://www.nejm.org/doi/full/10.1056/NEJMp1405026 [DOI] [PubMed]
- 11.Newsroom Homepage | CMS. Accessed August 23, 2020. https://www.cms.gov/newsroom
- 12.HCAHPS: Patients’ Perspectives of Care Survey | CMS. Accessed August 23, 2020. https://www.cms.gov/Medicare/Quality-Initiatives-Patient-Assessment-Instruments/HospitalQualityInits/HospitalHCAHPS
- 13.Ziemba JB, Allaf ME, Haldeman D. Consumer preferences and online comparison tools used to select a surgeon. JAMA Surg. 2017;152(4):410–411. doi: 10.1001/jamasurg.2016.4993. [DOI] [PubMed] [Google Scholar]
- 14.Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. JAMA. 2014;311(7):734–735. doi: 10.1001/jama.2013.283194. [DOI] [PubMed] [Google Scholar]
- 15.Murphy GP, Radadia KD, Breyer BN. Online physician reviews: is there a place for them? Risk Manag Healthc Policy. 2019;12:85–89. doi: 10.2147/RMHP.S170381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Frost C, Mesfin A. Online reviews of orthopedic surgeons: an emerging trend. Orthopedics. 2015;38(4):e257–e262. doi: 10.3928/01477447-20150402-52. [DOI] [PubMed] [Google Scholar]
- 17.Ramkumar PN, Navarro SM, Chughtai M, La T, Fisch E, Mont MA. The patient experience: an analysis of orthopedic surgeon quality on physician-rating sites. J Arthroplasty. 2017;32(9):2905–2910. doi: 10.1016/j.arth.2017.03.053. [DOI] [PubMed] [Google Scholar]
- 18.Ramkumar PN, Navarro SM, Haeberle HS, Chughtai M, Flynn ME, Mont MA. Social media and total joint arthroplasty: an analysis of patient utilization on instagram. J Arthroplasty. 2017;32(9):2694–2700. doi: 10.1016/j.arth.2017.03.067. [DOI] [PubMed] [Google Scholar]
- 19.Trehan SK, DeFrancesco CJ, Nguyen JT, Charalel RA, Daluiski A. Online patient ratings of hand surgeons. J Hand Surg. 2016;41(1):98–103. doi: 10.1016/j.jhsa.2015.10.006. [DOI] [PubMed] [Google Scholar]
- 20.Arthur JR, Etzioni D, Schwartz AJ. Characterizing extremely negative reviews of total joint arthroplasty practices and surgeons on yelp.com. Arthroplasty Today. 2019;5(2):216–220. doi: 10.1016/j.artd.2019.02.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Sculco PK, McLawhorn AS, Fehring KA, De Martino I. The future of social media in orthopedic surgery. Curr Rev Musculoskelet Med. 2017;10(2):278–279. doi: 10.1007/s12178-017-9412-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.How do patients select a new physician? 77% say online reviews. Accessed September 13, 2020. https://www.beckershospitalreview.com/healthcare-information-technology/how-do-patients-select-a-new-physician-77-say-online-reviews.html
- 23.Trehan SK, Nguyen JT, Marx R, et al. Online patient ratings are not correlated with total knee replacement surgeon-specific outcomes. HSS J Musculoskelet J Hosp Spec Surg. 2018;14(2):177–180. doi: 10.1007/s11420-017-9600-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Murphy GP, Awad MA, Osterberg EC, et al. Web-based physician ratings for california physicians on probation. J Med Internet Res. 2017;19(8):e254. doi: 10.2196/jmir.7488. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Johari K, Kellogg C, Vazquez K, Irvine K, Rahman A, Enguidanos S. Ratings game: an analysis of Nursing Home Compare and Yelp ratings. BMJ Qual Saf. 2018;27(8):619–624. doi: 10.1136/bmjqs-2017-007301. [DOI] [PubMed] [Google Scholar]
- 26.Peters E, Dieckmann N, Dixon A, Hibbard JH, Mertz CK. Less is more in presenting quality information to consumers. Med Care Res Rev MCRR. 2007;64(2):169–190. doi: 10.1177/10775587070640020301. [DOI] [PubMed] [Google Scholar]
- 27.Bougie R, Pieters R, Zeelenberg M. Angry customers don't come back, they get back: THE experience and behavioral implications of anger and dissatisfaction in services. J Acad Mark Sci. 2003;31(4):377–393. doi: 10.1177/0092070303254412. [DOI] [Google Scholar]
- 28.Samora JB, Lifchez SD, Blazar PE. Physician-rating web sites: ethical implications. J Hand Surg. 2016;41(1):104–110. doi: 10.1016/j.jhsa.2015.05.034. .e1. [DOI] [PubMed] [Google Scholar]
- 29.Medical Justice | Medical Malpractice Insurance Is Not Enough. Medical Malpractice Insurance is Not Enough - Medical Justice. Accessed January 3, 2021. https://medicaljustice.com/
- 30.EMPATHIQ – Healthcare Reputation Management. Accessed January 3, 2021. https://empathiq.io/
- 31.Smith RJ, Lipoff JB. Evaluation of dermatology practice online reviews: lessons from qualitative analysis. JAMA Dermatol. 2016;152(2):153–157. doi: 10.1001/jamadermatol.2015.3950. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.