Abstract
Introduction:
Patients may express views about their orthodontic care by posting publicly available reviews on the Internet. This study analyzed online reviews of orthodontists with an emphasis on the types and frequency of complaints expressed in negative reviews.
Methods:
A random sample of 10.6% of the American Association of Orthodontists membership was evaluated to identify members practicing in a North American office that is limited to orthodontics and has an online presence. Information regarding those orthodontists and their offices was collected. For offices with Google and/or Yelp reviews, all negative (1- or 2- star) reviews were saved for content analysis. If an office posted a response to a negative review, those responses were also collected for analysis.
Results:
Of the 807 eligible orthodontists, 92.4% had reviews on Google and/or Yelp. Average ratings of orthodontists were very positive [i.e., 4- and 5-star reviews constituted >97% (Google) and >88% (Yelp) of reviews] yielding an average orthodontist rating of 4.72 on Google and 4.42 on Yelp. Yet, approximately half of those orthodontists (50.9%) also had at least one negative review. The three most frequently mentioned categories of complaints concerned Quality of Care/Service, Interpersonal interactions, and Money/Financial issues. An analysis of the specific kinds of complaints is described. Orthodontists posting responses to negative reviews had significantly better average ratings than those who did not, but this association does not demonstrate a causal relationship.
Conclusions:
Understanding the complaints orthodontic patients express in online reviews may suggest strategies to improve patient satisfaction and an orthodontist’s online reputation.
Keywords: Patient satisfaction, Online reviews, Online reputation, Orthodontics, Customer complaint management
Introduction
Since the advent of the consumer rights movement, patient satisfaction in health care has been increasingly emphasized with reports that higher satisfaction is associated with better quality health care, increased treatment compliance, better perceived treatment results, and fewer medical malpractice claims.1–3 Improving patient satisfaction has been recommended as a practice-building strategy.4,5 Consequently, patient satisfaction has received attention from medical and dental specialties,6–10 including orthodontics.11–18 In orthodontics, Bennet and colleagues concluded that the treatment process, psychosocial benefits, and overall treatment outcome were the three most valid and reliable factors related to patient satisfaction.13 A systematic review from 2015, found that the factors most commonly associated with satisfaction in orthodontics were perceived esthetic outcomes, psychological benefits, and quality of care, while dissatisfaction was related to treatment duration, pain/discomfort, and retention appliances.14
Social media provides a public platform to express opinions about one’s health care. Online patient reviews and ratings give prospective patients easily accessible information to consider when selecting healthcare practitioners.2,19,20 Survey results indicate that a majority of the US population is aware of online physician ratings and consider physician rating websites to be either “somewhat important” or “very important” when choosing a physician.21 Another survey found that 41% of consumers reported that information found on social media can affect their choice of a doctor.22 The number of individuals using online reviews to evaluate physicians is growing rapidly as is the number of physicians receiving online ratings and the number of ratings per provider.20,23,24 Research suggests that prospective patients conducting online searches are more likely to select a doctor with more favorable reviews and avoid doctors with negative reviews.20,21,25 Indeed, a 2016 survey found that 48% of respondents would be willing to go out of their insurer’s provider network to avoid a doctor with worse reviews.20
While much patient satisfaction research has relied on patient interviews, surveys or focus groups, publicly posted online reviews, ratings, or commentaries provide a new method to evaluate patient satisfaction.26–28 Research in the field of medicine suggests that patient satisfaction based on views expressed on social media can reflect the quality of health care. At the level of the practitioner, quality of care was better correlated with online review content than with a physician’s clinical experience, board certification, education, or lack of malpractice claims.23 At an institutional level, ratings posted on Yelp produce findings comparable to traditional hospital performance measures, such as the systematically collected Hospital Consumer Assessment of Healthcare Providers and Systems ratings.29
Despite the growing body of medical research on patient satisfaction using social media,19,26,27,30,31 there has been limited research evaluating the concerns expressed online about orthodontics.4,28 Although patient satisfaction in orthodontics is generally high,13,14 there are still dissatisfied patients, and social media allows these individuals to share their negative opinions publicly. This descriptive study analyzed online reviews posted about a random sample of orthodontists with an emphasis on the content of negative online reviews as well as the content of office responses to negative reviews. Negative reviews are usually unsolicited and should provide insight about the causes of patient dissatisfaction. In addition, little is known about how orthodontic offices respond online to negative reviews. Understanding the causes of patient online complaints is a prerequisite to devising strategies that will reduce complaints with the ultimate goal of increasing patient satisfaction and improving an orthodontic office’s online reputation.
Materials and Methods
This study evaluates the content, prevalence and distribution of online reviews of orthodontists obtained from two commonly used public websites that provide reviews of orthodontists, i.e., Google and Yelp. The University of Washington’s Human Subjects Research Determination Guide indicated that this study did not require approval by the Institutional Review Board because the information is publicly available and not considered private.
Sample Selection, Data Collection, and Content Analysis of Online Reviews
The American Association of Orthodontics (AAO) provided its membership list (in alphabetical order by last name) from which a random sample of orthodontists was selected by placing the individual members in a random order based on a set of random numbers generated using the RAND function in Excel. A large sample of orthodontists (~10.6% of the membership list) was selected to improve the accuracy of the descriptive statistical estimates being calculated. For example, based on the 10.6% sample of the total AAO membership, the margin of error for each of the estimated percentages in that sample is at most +/− 1.5%, based on the formula for the standard error of a sample proportion with adjustment for the finite population correction.32 Estimated percentages that are based on a subset of this sample will have larger margins of error that can be determined as described in Bondy and Zlot.32
Methodological details describing data collection, analysis and scoring of online reviews, and the reliability of review scoring are provided in Supplemental Appendix 1.
Statistical Analysis.
Descriptive statistics including counts, percentages, means, standard deviations, and medians were calculated for various variables. Several t-tests were performed to make comparisons of interest. These t-tests allowed for unequal variance, produced a 95% confidence interval (95% CI) that accompanies each test and did not account for multiple comparisons.
Results
The study sample consisted of 1,024 orthodontists randomly selected from the AAO’s list of 9,627 members as described in the flowchart (Figure 1). A subset of those selected orthodontists (953, 93.1%) were identified on the AAO’s online membership directory during the ~13 month data collection period (5/21/2017 – 6/24/2018) following receipt of the membership list. Of those confirmed AAO members, ~84.7% (807) were in private practice orthodontics settings with an average number of 1.53 offices per orthodontist (median = 1 office, range of offices 1 – 8). Online reviews could be found for the great majority of these orthodontists (92.4%; 746/807) with approximately half of them (50.9%; 380/746) having received at least one negative Google and/or Yelp review. These 380 orthodontists yielded a total of 956 negative textual reviews for analysis.
Figure 1.
Flow diagram describing the process used to identify negative online reviews related to orthodontists practicing in offices that provide only orthodontic care.
Table 1 describes the proportion of eligible orthodontists and their orthodontic offices that have Google and/or Yelp reviews as well as the distribution of those reviews according to the overall number of stars assigned. An important observation is that most reviews are positive (i.e., 5- and 4-star ratings constituted >97% of reviews on Google and >88% of reviews on Yelp). Negative (1- and 2-star) ratings constituted 2.1% of reviews on Google and 9.8% of reviews on Yelp. There were 134 negative 1- and 2-star ratings on Google that did not include a textual review.
Table 1:
Eligible Orthodontists and Orthodontics-Only Offices with an Online Presence
| Categories | Reviews on Google | Reviews on Yelp | Reviews on Google or Yelp |
No reviews on Google or Yelp |
|---|---|---|---|---|
| Orthodontists (N = 807) | ||||
| % with 1–5 star reviews (N) | 89.2% (720) | 62.1% (501) | 92.4% (746) | 7.6% (61) |
| % with 1- and 2-star reviews (N) | 36.7% (296) | 23.7% (191) | 47.1% (380) | |
| Orthodontic Offices (N = 1,234) | ||||
| % with 1–5 star reviews (N) | 80.9% (998) | 50.8% (627) | 85.3 (1,052) | 18.2% (182) |
| % with 1- and 2-star reviews (N) | 27.6% (340) | 18.2% (224) | 37.0% (457) | |
| Distribution of All Reviews | ||||
| Total number reviews | 29,133 | 5,279 | ||
| % 5-star reviews (N) | 95.2% (27,733) | 84.8% (4,476) | ||
| % 4-star reviews (N) | 2.4% (690) | 3.9% (205) | ||
| % 3-star reviews (N) | 0.4% (104) | 1.5% (79) | ||
| % 2-star reviews (N | 0.4% (106) | 2.0% (108) | ||
| % 1-star reviews (N) | 1.7% (500) | 7.8% (411) | ||
| Mean stars | 4.89 | 4.56 |
The average online rating per orthodontist was 4.72 stars on Google, and 4.42 stars on Yelp (Table 2A). Each individual orthodontist’s rating is an average of that individual’s mean ratings from his/her office(s) (calculated separately for Google and Yelp). All of the orthodontists’ individual mean ratings were then averaged to yield the overall mean Google and Yelp ratings for orthodontists. When orthodontist ratings are compared based on whether they have negative online reviews or not, having negative reviews lowers the overall mean rating for orthodontist as would be anticipated. Specifically, orthodontists without any negative reviews have higher average ratings (mean orthodontist-averaged Google rating = 4.96; mean orthodontist-averaged Yelp rating = 4.92) compared to orthodontists with negative reviews (mean orthodontist-averaged Google rating = 4.39; mean orthodontist-averaged Yelp rating = 3.61) (Table 2A). The distribution of ratings among orthodontists with negative reviews differs from the distribution of ratings among those without any negative reviews. For Google reviews, orthodontists with negative reviews have an average of 60.6 reviews distributed as averages of 1.7 reviews for 1-star ratings, 0.4 for 2-star ratings, 0.2 for 3-star ratings, 1.6 for 4-star ratings, and 56.7 for 5-star ratings, while those with no negative reviews have an average of 26.4 reviews distributed as no 1- and 2-star ratings and averages of 0.1 reviews for 3-star ratings, 0.5 for 4- star ratings, and 25.8 for 5-star ratings. For Yelp reviews, orthodontists with negative reviews have an average of 19.2 reviews distributed as averages of 2.2 reviews for 1-star ratings, 0.6 for 2-star ratings, 0.3 for 3-star ratings, 0.8 for 4-star ratings, and 15.3 for 5-star ratings, while those with no negative reviews have an average of 5.2 reviews distributed as no 1- and 2-star ratings and averages of 0.1 reviews for 3-star ratings, 0.2 for 4-star ratings, and 5.0 for 5-star ratings.
Table 2A:
Online Presence, Online Ratings and Years Since Graduating from Orthodontic Education
| Categories | Mean Years (SD, Range) After Ortho Graduation (2019 - graduation year) |
Mean Google Rating (N, SD) |
Mean Yelp Rating (N, SD) |
|---|---|---|---|
| Orthodontists in AAO directory (N = 953) | 23.1 (12.8, 0–69) | NA | NA |
| Eligible Orthodontists (N = 807) | |||
| Had online Google or Yelp reviews (N = 746) | 23.4 (12.1, 3–69) | 4.72 (720, 0.58) | 4.42 (501, 1.00) |
| No online Google or Yelp reviews (N = 61) | 27.6 (13.8, 4–60) | NA | NA |
| Orthodontists with online reviews (N = 746) | |||
| Had negative reviews on Google or Yelp (N = 380) | 24.0 (11.5, 3–67) | ||
| No negative reviews on either Google or Yelp (N = 366) | 22.7 (12.7, 3–69) | ||
| Had negative reviews on Google (N = 296) | 24.0 (11.5, 3–67) | 4.39 (296, 0.78) | |
| No negative reviews on Google (N = 424) | 22.8 (12.5, 3–69) | 4.96 (424, 0.13) | |
| Had negative reviews on Yelp (N = 191) | 24.2 (11.2, 4–61) | 3.61 (191, 1.20) | |
| No negative reviews on Yelp (N = 310) | 21.8 (12.0, 3–58) | 4.92 (310, 0.25) |
On average, orthodontists with no online reviews completed orthodontic training 4.2 years before those orthodontists with online reviews (t-test = 2.3, df = 67.7, p = 0.02, 95% CI for this difference range = 0.6 – 7.9 years before, Table 2A). However, whether orthodontists with online reviews do, or do not, have any negative reviews was not significantly related to years since completing orthodontics training. On average, orthodontists with no negative online reviews completed orthodontic training 1.3 years after orthodontists who had one or more negative reviews (t-test = −1.5, df=730.2, p=0.14, 95% CI for this difference range = 0.4 years before to 3.1 years after, Table 2A). Table 2B also presents the average number of reviews and average orthodontist rating (separately for Google and Yelp) according to decade of graduation.
Table 2B:
Online Ratings by Decade of Graduation from Orthodontic Education
| Orthodontist with Online Reviews by Graduation Decade |
Google Reviews Orthodontists (N) |
Yelp Reviews Orthodontists (N) |
Mean Google Rating (Mean Number Reviews) |
Mean Yelp Rating (Mean Number Reviews) |
|---|---|---|---|---|
| 1950s | 3 | 1 | 3.31 (3.3) | 1.00 (4.0) |
| 1960s | 7 | 5 | 4.67 (24.0) | 4.80 (3.6) |
| 1970s | 68 | 46 | 4.46 (29.5) | 4.19 (4.9) |
| 1980s | 142 | 87 | 4.67 (24.5) | 4.16 (8.4) |
| 1990s | 194 | 146 | 4.67 (36.8) | 4.46 (10.0) |
| 2000s | 214 | 152 | 4.84 (55.5) | 4.47 (15.6) |
| 2010s | 92 | 64 | 4.89 (48.3) | 4.78 (7.3) |
Almost half of all negative online reviews (454/956=47.5%) pertained to the period of “active orthodontic treatment.” The next most frequently cited periods of orthodontic care referred to the: 1) “No specified time / in general” (169/956=17.7%), 2) “Exam / Consult / Records” appointments (154/956=16.1%), 3) “Retention Phase or Post-Treatment Retrospective Evaluation” (121/956=12.7%), and 4) “Prior to the Initial Appointment” (62/956=6.5%). Other possible phases of orthodontic care such as “Recall” exams, “Second Opinion” exams, and “Emergency Appointments” occurred in less than 1% of reviews.
Supplemental Table S1 presents the detailed findings from the content analysis regarding the types of complaints expressed in at least 1% of the 956 negative online reviews. The three most common general categories or domains of complaints found in negative reviews were about: 1) Quality of Care/Service, which occurred in 53.2% of reviews, 2) Interpersonal interactions, which occurred in 52.0% of reviews, and 3) Money/Financial issues, which occurred in 46.9% of reviews. Significant but less frequent types of complaints had to do with: 4) Time, which occurred in 26.8% of reviews, 5) Communication, which occurred in 23.8% of reviews, and 6) Unprofessional behavior, which occurred in 10.7% of reviews. The top 5 specific complaints across all categories were “Impolite/Rude/Mean” (23.5% of reviews), “Bad outcome/Complications/Not finish” (21.3% of reviews), “Errors/Poor skill, ability, office organization” (18.1% of reviews), “Money before patients/Greedy” (15.6% of reviews), and “No time for patient/Factory environment” (14.1% of reviews).
Certain types of complaints are usually attributed to specific personnel in the orthodontic office (Supplemental Table S1). For example, orthodontists were typically cited for problems about excessive length of treatment while the front office staff were cited for scheduling errors / changes. Complaints regarding money/financial issues typically referred to the orthodontist although 82.1% of billing problems were attributed to the billing staff. Communication complaints were largely about the reception staff, interpersonal complaints were most frequently associated with the orthodontist and the reception staff, and quality of care/service was most frequently attributed to the orthodontist.
Frequently mentioned miscellaneous complaints were: 1) poor customer service (7.6% of reviews), 2) lost/broken/replace retainer (7.0% of reviews), and 3) refusal to treat (or continue to treat) (2.5% of reviews). Complaints of multiple providers/staff turnover occurred in 1.8% of reviews, while complaints of office layout/office design, office cleanliness/shabby, and office catered towards only children/adult each occurred in 1.2% of reviews. No identifiable complaint was found in 3.0% of all negative reviews.
Table 3 describes the 219 responses posted in reply to negative online reviews. A minority of orthodontic offices (27.4%, 93/340) that received a negative Google review posted a response and none of those negative reviews were updated following the office’s response. Similarly, a minority of orthodontic offices (31.3%, 70/224) that received a negative Yelp review posted a response. Out of those 70 offices, 17 (24.3%) had patients update their review following the office response. However, only 4 of those 17 offices had reviews that were updated to a more positive rating of 3-, 4-, or 5- stars while the updated reviews remained negative (either a 1- or 2- star rating) for the other 13 offices. Orthodontists who responded to negative reviews had a greater number of total reviews (mean of 72.0 Google reviews and 34.7 Yelp reviews) compared to those who did not post a response (mean of 59.1 Google reviews and 16.1 Yelp reviews) [Google: difference = 13.0, t-test = 1.2, df = 174.7, p = 0.24, 95% CI for this difference range = −9.0 – 34.9; Yelp: difference = 18.5, t-test = 3.0, df = 81.5, p < 0.01, 95% CI for this difference range = 6.2 – 30.9]. Additionally, orthodontists who responded to negative Google reviews also had a significantly higher average rating (4.54) when compared to orthodontists who did not respond to negative Google reviews (4.35) [difference=0.19, t-test = 2.1, df = 187.0, p = 0.03, 95% CI for this difference range = 0.01 – 0.36, Table 3]. Similarly, orthodontists who responded to negative Yelp reviews had a significantly higher average rating (4.15) than orthodontists who did not post a response (3.41) [difference=0.74, t-test = 5.2, df = 183.6, p < 0.01, 95% CI for this difference range = 0.46 – 1.02, Table 3].
Table 3:
Responses to Negative Online Reviews Posted by Orthodontists and Orthodontics-Only Offices with 1- and 2-star reviews
| Categories | Yelp | Google or Yelp | |
|---|---|---|---|
| Orthodontists (N) with 1- and 2-star reviews | 296 | 191 | 380 |
| % Orthodontists (N) posting a response | 29.1% (86) | 30.9% (59) | 34.2% (130) |
| Mean number of 1- and 2-star reviews | 2.5 | 4.5 | |
| Mean number of total reviews | 72.0 | 34.7 | |
| Mean rating for orthodontist | 4.54 | 4.15 | |
| % Orthodontists (N) not posting a response | 70.9% (210) | 69.1% (132) | 65.8% (250) |
| Mean number of 1- and 2-star reviews | 2.0 | 2.5 | |
| Mean number of total reviews | 59.1 | 16.1 | |
| Mean rating for orthodontist | 4.35 | 3.41 | |
| Orthodontic Offices (N) with 1- and 2-star reviews | 340 | 224 | 457 |
| % Orthodontic offices (N) posting a response | 27.4% (93/340) | 31.3% (70/224) | 33.0% (151/457) |
| Offices with updated reviews after reply: % (N) | 0.0% (0) | 24.3% (17/70) | |
| Review updated to 3-, 4-, or 5-stars: % (N) | NA | 23.5% (4/17) | |
| Review remained 1- or 2-stars: % (N) | NA | 76.5% (13/17) | |
| Responses to online negative reviews (N) | 110 | 109 | 219 |
| Content type included in response* % (N) | |||
| Request Contact About Problem: | 60.9% (67) | 44.0% (48) | 52.5% (115) |
| Apologetic: | 48.2% (53) | 54.1% (59) | 51.1% (112) |
| Explanatory / Informative: | 38.2% (42) | 59.6% (65) | 48.9% (107) |
| Defensive: | 26.4% (29) | 30.3% (33) | 28.3% (62) |
| Thanking: | 26.4% (29) | 25.7% (28) | 26.0% (57) |
| Questioning Validity of Review: | 20.9% (23) | 8.3% (9) | 14.6% (32) |
| Argumentative: | 6.4% (7) | 11.0% (12) | 8.7% (19) |
| Cite HIPAA Concerns: | 5.5% (6) | 0.9% (1) | 3.2% (7) |
[A reply to an online review can contain more than one content type and so the total can exceed 100%.]
In order from most to least frequent, the content of responses posted by orthodontic offices to negative reviews was to: 1) request contact to resolve the issue (52.5% of responses), 2) apologize for the problem (51.1%), 3) provide an explanation and/or information about the problem (48.9%), 4) defend the actions of the office against the accusation (28.3%), 5) thank the reviewer for the comments (26.0%), 6) question the validity or authenticity of the review (14.6%), 7) argue with the reviewer or review (8.7%), and 8) cite HIPAA regulations as the reason the office cannot address critical details of the complaint (3.2%) [Table 3].
Discussion
On average, orthodontists receive higher Google and Yelp ratings relative to many industries, including general dentistry.33,34 This finding matches the overall high patient satisfaction rates reported in orthodontic research conducted using questionnaires and surveys.12,35–37 In the current study, the average rating for orthodontists was 4.72 on Google and 4.42 on Yelp (Table 2A), which is better than average ratings for many industries that are estimated to be in range of 4.30–4.42 for Google, and 3.60–3.77 for Yelp.33,34,38 Average ratings for orthodontists are also higher than ratings found across dentistry, which surveys estimate to be 4.59 on Google.34 The current study found that 80.9% (998/1,234) of orthodontic offices have Google reviews, which is higher than the 74% of businesses across all industries with Google reviews.34
Ratings for orthodontists are higher on Google than Yelp. This difference could occur for numerous reasons. The Yelp rating system scores in increments of 0.5, and rounds to the nearest 0.5 interval. Yelp also directly averages all recommended reviews to obtain the final scores, and does not use other factors to arrive at the final average rating.46 In contrast, Google ratings are scored in increments of 1, and the average rating is rounded to the nearest tenth.47 Also, the final Google rating is not only calculated from user ratings, but also uses a “variety of other signals to ensure that the overall score best reflects the quality of the establishment”.47 Thus, methodological differences in how mean ratings are calculated on different websites may contribute to differences in average rating. Also, individuals who post to Yelp may be more critical, as the Yelp website is designed specifically for leaving reviews, whereas it is optional to write a review on Google after giving a star rating.
Based on online review metrics, orthodontics receives highly positive patient ratings relative to other fields. Indeed, 49.1% (366/746) of orthodontists with online reviews have no negative (1- or 2-star, including ratings without text) Google or Yelp reviews. Nonetheless, receiving negative reviews can be concerning to an orthodontic practice. Consequently, commercial firms market services to orthodontists to help manage their online reputation. Some orthodontists consider adding a non-disparagement clause to patient contracts to prevent being maligned on the internet.39 Negative online reviews can sometimes cause health care providers to file defamation lawsuits against the author of the negative review.40–42 The decision to pursue legal action should be considered carefully as there are protections for consumer free speech and some suggest that taking legal action may harm one’s reputation more than the presence of the negative review.43 A commonly recommended strategy is to overwhelm negative reviews by encouraging a large number of positive reviews.43 The current study found that >97% of Google reviews of orthodontists are positive (4- and 5-star) and only about 2% of reviews are negative (1- and 2-star). Yet, even with a huge predominance of positive reviews, it is easy to select the much smaller number of negative reviews to read, which could allow negative reviews to receive disproportionately more attention than positive ones. Another strategy to improve an office’s online reputation is to prevent the problems that cause patients to post negative reviews. Understanding the problems that lead to negative reviews may inform in-office training and suggest changes to office practices with the goal of improving patient satisfaction.
The first and second most frequently coded themes for patient dissatisfaction in this study were “quality of care/service” and “interpersonal” interactions, followed by “money/financial” issues as a close third. The first two themes support the findings of other qualitative research on orthodontic patient satisfaction.35 A systematic review reported the factors most commonly associated with satisfaction were perceived esthetic outcomes, psychological benefits, and quality of care as it relates to dentist-staff-patient interactions while the main causes of dissatisfaction were treatment duration, pain/discomfort, and use of retention appliances.14 Feldmann also found pain to be one of the main factors associated with dissatisfaction.35 Although these types of complaints were mentioned in the present study, they were not the most frequently mentioned in negative online reviews.
An important strength of the current study is that specific complaints could be analyzed in detail. The most frequently mentioned complaints were: 1) impolite/rude/mean, 2) bad outcome/complications/not finish, 3) errors/poor skill, ability, office organization, 4) money before patients/greedy, 5) no time for patient/factory environment, and 6) uncaring/patient not priority. Complaints tended to focus on poor interpersonal interactions, and a perceived lack of care and attention for the patient. Negative reviews often describe patients feeling less like an individual but more like a part moving through a high throughput assembly line factory, where the orthodontist and/or the office staff didn’t care about the individual and just wanted to deliver treatment quickly and profitably. The most common complaints under the money/financial theme were that: 1) offices were greedy or cared more about money than the well-being of the patient, 2) there were extra/hidden costs, and 3) treatment was too expensive or not worth the cost. It is interesting that patients complain about extra or hidden costs at about the same frequency as the actual price of treatment.
During the data analysis phase, brief written descriptions of complaints scored as “other” on the template were compiled and grouped into new categories based on similarity. As a consequence, the scoring template has been revised for future research (see online Appendix S1) to include: 1) lack of privacy during treatment, 2) sending patients to a different office location, 3) no itemized cost breakdown, and 4) poor office hours. An infrequent (16 out of the 956 reviews), but potentially important code, was added to accommodate reviews that mention actual or potential legal action or filing of a formal complaint with a third party. In addition, items that were either never or rarely scored were removed from the revised scoring template.
Consumers post online reviews for multiple reasons that include: providing potentially useful feedback to the business, informing other current or potential customers about one’s experience, expressing one’s emotions, or trying to elicit compensation or otherwise resolve the problem.44 Investigating the issues surrounding posting a managerial response to online reviews is an active topic of marketing research (e.g., in the hotel industry).44–46 Proserpio and Zervas45 believe it is a complex decision making process for a business to determine which reviews warrant a response, when a response should be posted, and what content should be included in a response. For the hotel industry, the research results have been mixed about whether posting a response will improve or worsen overall rating and it is not clear how changes in online ratings relate to changes in sales performance.44–46 Yelp has recommended posting a public response to a review and then communicating privately with the author of the review using Yelp’s Direct Message feature.47 To our knowledge, these issues have not been studied in orthodontics.
In the current study, posting a response to negative online reviews was a poorly effective strategy if the goal was for the author to change the review to a positive one. Indeed, no negative reviews on Google that received a response were updated. On Yelp, only 24.3% of the negative reviews that received a response were updated and the majority of those (76.5%) were not updated to a positive rating. However, our study was unable to assess whether some reviews might have been removed following an office response (e.g., by Direct Message), leaving open the possibility that office responses may cause some negative reviews to be removed.
For orthodontists with at least one negative online review, the average online rating for those who posted response(s) to negative online reviews (4.54 on Google; 4.15 on Yelp) was significantly better (Google: p=0.03; Yelp: p<0.01, see Table 3) than the mean rating for those who had never posted a response (4.35 on Google; 3.41 on Yelp). However, it is important to emphasize that the current study is observational and the association that offices posting replies to negative reviews also have better average online ratings does not demonstrate a causal relationship. Experimental research is needed to determine whether posting an appropriate response to negative reviews is an effective strategy to improve average online rating. Posting responses to reviews may have other effects as well. For example, a survey found that 44.6% of respondents indicate being more likely to visit a local business if it responds to negative reviews, whereas only 18.2% said they were not more likely, and 37.2% were neutral.33 Also, responding to reviews on Google increases traffic flow, which increases Google ratings indirectly due to the way Google determines its ratings.
Another important consideration is whether a potential patient’s decision to pursue treatment at an office is influenced by the content of how an office responds to a negative review (i.e., does posting an office response effectively counter the adverse effects of a negative review?). As indicated in Table 3, approximately two-thirds of offices choose not to reply to negative reviews while the responses posted by the remaining offices exhibit a range of response styles (e.g., wanting to resolve the complaint, apologizing, thanking, arguing). It is unclear what style of response, if any, is viewed most positively by potential patients and future research should investigate this question.
Of course, if an orthodontist posts a response to a patient review, it should adhere to HIPAA regulations. Although an individual may post opinions on a public domain about experiences as a patient, an office should not post information that would violate patient privacy. While scoring office responses for this study, the authors felt that some responses may have disclosed HIPAA protected information, which may reflect a lack of awareness on the part of the orthodontist or office staff. Orthodontists should understand that health care providers are held to a different standard than other businesses regarding how one may respond to online patient reviews (e.g., HIPAA laws). If an office chooses to respond to an online review, the AAO’s general counsel provides useful guidelines to consider and the response should comply with HIPAA regulations.43
This study has limitations. The scoring rules for coding reviews may lead to underestimates of who was responsible for certain types of complaints due to how the “not specified code” was used. Initially, the authors had difficulty reliably coding who was being blamed for a complaint when the review did not clearly attribute responsibility to an individual. Therefore, unless it was clear from the review who was considered responsible for the problem, the raters agreed to score who was responsible as being unspecified. This was done to prevent raters from making potentially biased and unreliable assumptions about who was considered responsible. Another potential limitation is the Yelp recommendation algorithm that determines which negative reviews are displayed on the webpage. Only negative reviews that were recommended were included in the study, since reviews placed into the not-recommended section are filtered out by Yelp with the intention of hiding reviews that have a high chance of being fake.48 The algorithm for filtering these reviews is not shared with the public. Thus, it is possible that not-recommended reviews contained types of complaints that were not assessed, and consequently not reported in this study. Nevertheless, the results presented reflect the available data that are visible on the website.
Conclusions
The percent of orthodontic offices with Google and Yelp reviews is higher than for most other professions and on average orthodontists receive better ratings than many other fields. Nevertheless, orthodontists do receive negative reviews. The main reasons for orthodontic patient dissatisfaction identified in negative online reviews expand upon frequently mentioned themes reported in previous orthodontic research and reveal several themes and specific concerns that have not been well described in previous research. This study also found that responding to negative online reviews had little effect on having that review improved. However, an association was found indicating that orthodontists that reply to negative online reviews also have better average online ratings, but it is unknown if this is a causal connection. By analyzing the content of negative online reviews, this study clarifies the causes of patient dissatisfaction and suggests areas that could be improved to increase patient satisfaction.
Supplementary Material
Methodological details of sample selection, data collection, content analysis of reviews, and the inter-rater-reliability of scoring reviews.
This Table lists the types of complaints that were mentioned in at least 1% of the negative reviews evaluated (i.e., .10 of the 956 reviews).
New Suggested Scoring Template.
HIGHLIGHTS.
Most orthodontists have online reviews; half of them have received a negative review
Negative reviews commonly cite the treatment phase and the exam-records-consult phase
The top 3 complaint categories are: Quality of Care, Interpersonal issues, and Money
The most common specific complaint refers to an impolite, rude or mean interaction
The majority of orthodontists do not post an online response to a negative review
Acknowledgments
This research was supported by the University of Washington Orthodontic Alumni Association.
References
- 1.Pascoe GC. Patient satisfaction in primary health care: A literature review and analysis. Eval Program Plann 1983;6:185–210. [DOI] [PubMed] [Google Scholar]
- 2.Rozenblum R, Bates DW. Patient-centered healthcare, social media and the internet: The perfect storm? BMJ Qual Saf 2013;22:183–86. [DOI] [PubMed] [Google Scholar]
- 3.Stelfox HT, Gandhi TK, Orav EJ, Gustafson ML. The relation of patient satisfaction with complaints against physicians and malpractice lawsuits. Am J Med 2005;118:1126–33. [DOI] [PubMed] [Google Scholar]
- 4.Jorgensen G Attracting orthodontic patients via the internet: A 20-year evolution. Am J Orthod Dentofacial Orthop 2015;148:939–42. [DOI] [PubMed] [Google Scholar]
- 5.Nelson KL, Shroff B, Best AM, Lindauer SJ. Orthodontic marketing through social media networks: The patient and practitioner’s perspective. Angle Orthod 2015;85:1035–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Ware JE, Hays RD. (1988). Methods for measuring patient satisfaction with specific medical encounters. Med Care 1988;26:393–402. [DOI] [PubMed] [Google Scholar]
- 7.Grol R Improving the quality of medical care: Building bridges among professional pride, payer profit, and patient satisfaction. JAMA 2001;286:2578–85. [DOI] [PubMed] [Google Scholar]
- 8.Korsch BM, Gozzi EK, Francis V. Gaps in doctor-patient communication: I. Doctor-patient interaction and patient satisfaction. Pediatrics 1968;42:855–71. [PubMed] [Google Scholar]
- 9.Tin-Oo MM, Saddki N, Hassan N. Factors influencing patient satisfaction with dental appearance and treatments they desire to improve aesthetics. BMC Oral Health 2011;11:6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Samorodnitzky-Naveh GR, Geiger SB, Levin L. Patients’ satisfaction with dental esthetics. Journal Am Dent Assoc 2007;138:805–8. [DOI] [PubMed] [Google Scholar]
- 11.Espeland LV, Stenvik A. Perception of personal dental appearance in young adults: relationship between occlusion, awareness, and satisfaction. Am J Orthod Dentofacial Orthop 1991;100:234–41. [DOI] [PubMed] [Google Scholar]
- 12.Bennett ME, Tulloch JC. Understanding orthodontic treatment satisfaction from the patients’ perspective: a qualitative approach. Clin Orthod Res 1999;2:53–61. [DOI] [PubMed] [Google Scholar]
- 13.Bennett ME, Camilla Tulloch JF, Vig KW, Phillips CL. Measuring orthodontic treatment satisfaction: questionnaire development and preliminary validation. J Public Health Dent 2001;61:155–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Pachêco-Pereira C, Pereira JR, Dick BD, Perez A, Flores-Mir C. Factors associated with patient and parent satisfaction after orthodontic treatment: a systematic review. Am J Orthod Dentofacial Orthop 2015;148:652–9. [DOI] [PubMed] [Google Scholar]
- 15.Maia NG, Normando D, Maia FA, Ferreira MÂF, do Socorro Costa Feitosa Alves M. Factors associated with long-term patient satisfaction. Angle Orthod 2010;80:1155–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Al-Omiri MK, Abu Alhaija ES. Factors affecting patient satisfaction after orthodontic treatment. Angle Orthod 2006;76:422–31. [DOI] [PubMed] [Google Scholar]
- 17.Anderson LE, Arruda A, Inglehart MR. Adolescent patients’ treatment motivation and satisfaction with orthodontic treatment: do possible selves matter?. Angle Orthod 2009;79:821–7. [DOI] [PubMed] [Google Scholar]
- 18.Birkeland K, Bøe OE, Wisth PJ. Relationship between occlusion and satisfaction with dental appearance in orthodontically treated and untreated groups. A longitudinal study. Eur J Orthod 2000;22:509–18. [DOI] [PubMed] [Google Scholar]
- 19.Tengilimoglu D, Sarp N, Yar CE, Bektas M, Hidir MN, Korkmaz E. The consumers’ social media use in choosing physicians and hospitals: the case study of the province of Izmir. Int J Health Plann Manage, 2017;32:19–35. [DOI] [PubMed] [Google Scholar]
- 20.Hedges Lisa. 2019. “How Patients Use Online Reviews.” Software Advice. Retrieved on April 18, 2019 from https://www.softwareadvice.com/resources/how-patients-use-online-reviews/.
- 21.Hanauer DA, Zheng K, Singer DC, Gebremariam A, Davis MM. Public awareness, perception, and use of online physician rating sites. JAMA 2014;311:734–5. [DOI] [PubMed] [Google Scholar]
- 22.Social media “likes” healthcare: From marketing to social business. Report from PwC Health Research Institute (April 2012) downloaded on April 18, 2019 from https://www.pwc.com/us/en/industries/health-industries/library/health-care-socialmedia.html. [Google Scholar]
- 23.Gao GG, McCullough JS, Agarwal R, Jha AK. A changing landscape of physician quality reporting: analysis of patients’ online ratings of their physicians over a 5-year period. J Med Internet Res 2012;14:e38. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Greaves F, Millett C. Consistently increasing numbers of online ratings of healthcare in England. J Med Internet Res 2012;14:e94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Huang E, Dunbar CL. Connecting to patients via social media: A hype or a reality?. Journal of Medical Marketing 2013;13:14–23. [Google Scholar]
- 26.Lagu T, Hannon NS, Rothberg MB, Lindenauer PK. Patients’ evaluations of health care providers in the era of social networking: An analysis of physician-rating websites. J General Intern Med 2010;25:942–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.López A, Detz A, Ratanawongsa N, & Sarkar U. What patients say about their doctors online: a qualitative content analysis. J General Intern Med 2012;27:685–92. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Rachel Henzell M, Margaret Knight A, Morgaine KC, S. Antoun J, Farella M. A qualitative analysis of orthodontic-related posts on Twitter. Angle Orthod 2013;84:203–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Bardach NS, Asteria-Peñaloza R, Boscardin WJ, Dudley RA. The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Qual Saf 2013;22:194–202. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Rastegar-Mojarad M, Ye Z, Wall D, Murali N, Lin S. Collecting and analyzing patient experiences of health care from social media. JMIR Res Protocols 2015;4:e78.. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Kilaru AS, Meisel ZF, Paciotti B, Ha YP, Smith RJ, Ranard BL, Merchant RM. What do patients say about emergency departments in online reviews? A qualitative study. BMJ Qual Saf 2016;25:14–24. [DOI] [PubMed] [Google Scholar]
- 32.Bondy WH, Zlot W. The standard error of the mean and the difference between means for finite populations. Am Stat 1976;30:96–7. [Google Scholar]
- 33.2018 ReviewTrackers Online Reviews Survey: Statistics and Trends. (n.d.). Retrieved on April 18, 2019 from https://www.reviewtrackers.com/online-reviews-survey/
- 34.Google Reviews Study: How Many Reviews Do Local Businesses Need? (posted on October 31, 2018) Retrieved on April 18, 2019 from https://www.brightlocal.com/learn/googlereviews-study/.
- 35.Feldmann I Satisfaction with orthodontic treatment outcome. Angle Orthod 2014;84:581–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Oliveira PG, Tavares RR, Freitas JC. Assessment of motivation, expectations and satisfaction of adult patients submitted to orthodontic treatment. Dental Press J Orthod, 2013;18:81–7. [DOI] [PubMed] [Google Scholar]
- 37.Mollov ND, Lindauer SJ, Best AM, Shroff B, Tufekci E. Patient attitudes toward retention and perceptions of treatment success. Angle Orthod 2010;80(4):468–73. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Bialik C (2018, Sep 14). Restaurant Ratings On Yelp Are Remarkably Consistent, No Matter Who’s Writing Them, When, And Where [Web blog post]. Retrieved February 24, 2019 from https://www.yelpblog.com/2018/09/restaurant-ratings-on-yelp-areremarkably-consistent-no-matter-whos-writing-them-when-and-where
- 39.Jerrold L (2014). Disparaging disparagement. Am J Orthod Dentofacial Orthop, 146(2), 264–265. [DOI] [PubMed] [Google Scholar]
- 40.O’Donnell J, Alltucker K (2018, July 18). Doctors, hospitals sue patients who post negative comments, reviews on social media. https://www.usatoday.com/story/news/politics/2018/07/18/doctors-hospitals-suepatients-posting-negative-online-comments/763981002/
- 41.Rozner L (2018, May 28). Manhattan Doctor Sues Patient for $1 million for Posting Negative Reviews Online. Retrieved from <https://newyork.cbslocal.com/2018/05/29/million-dollar-online-review-lawsuit/>
- 42.[Google]. Score ratings for local places. (n.d.). Retrieved on April 18, 2019 from https://support.google.com/business/answer/4801187?hl=en&ref_topic=4596755
- 43.Dillard Kevin, et al. “The Business of Orthodontics Podcast.” Episode 21, 20 June 2017, aao.podbean.com/e/the-business-of-orthodontics-podcast-episode-21. [Google Scholar]
- 44.Chevalier JA, Dover Y, & Mayzlin D Channels of Impact: User reviews when quality is dynamic and managers respond. Marketing Science, 2018;37:688–709. [Google Scholar]
- 45.Proserpio D, Zervas G. Online reputation management: Estimating the impact of management responses on consumer reviews. Marketing Science, 2017;36:645–65. [Google Scholar]
- 46.Wang Y, Chaudhry A. When and how managers’ responses to online reviews affect subsequent reviews. J Mark Res 2018;55:163–77. [Google Scholar]
- 47.Washcovick Emily. (May 21, 2918). DOs and DON’Ts of Responding to Reviews on Yelp. Retrieved on April 14, 2019 from https://yelpblog.com/2018/05/dos-donts-responding-to-reviews-yelp.
- 48.[Yelp]. (2013, Nov 13). Why Does Yelp Recommend Reviews? Retrieved on April 18, 2019 from https://www.youtube.com/watch?time_continue=95&v=PniMEnM89iY
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Methodological details of sample selection, data collection, content analysis of reviews, and the inter-rater-reliability of scoring reviews.
This Table lists the types of complaints that were mentioned in at least 1% of the negative reviews evaluated (i.e., .10 of the 956 reviews).
New Suggested Scoring Template.