On Patient Safety: Are Online Reviews a Reliable Assessment of Safety?

A few years ago, shortly after moving to a new city, I developed a toothache. New in town, I didn’t know any local dentists, but colleagues had given me some suggestions. Sadly, like popular surgeons, good dentists have wait lists. And unlike some of my patients, I do not have a high pain threshold; I was desperate for relief. I decided to perform an online search and got an appointment that morning. The dentist’s reviews seemed pretty good, though the occasional negative one was sprinkled in there as well.

My new dentist told me I needed a root canal. I readily agreed; I thought anything had to be better than the pain I was in at that moment. Wrong. That root canal was the most physically painful experience of my life. The 2-hour procedure probably took longer than expected because the dentist and his assistant kept flinching every time I convulsed in pain. When it was over, I remember lying on the patient chair, exhausted, dehydrated, thinking about how wrong those reviews were and how dangerous this guy was. I was planning to post a scathing review once I had the energy.

Online reviews can give us plenty of information when it comes to restaurants and shopping, but how useful and reliable are they when it comes to patient care and safety? We constantly measure all sorts of outcomes in healthcare, and it is good that we do so: Rates of surgical site infection, readmission, reoperation, deep venous thrombosis, length of stay, patient satisfaction scores, and patient-reported outcomes, to name a few. Numerous mechanisms and databases support us in making these measurements: National Surgical Quality Improvement Program benchmarks, institutional Morbidity and Mortality boards, Press Ganey, the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) patient satisfaction scores, surgical registries, patient reported outcomes scores, and many more. But beyond all the “official” metrics exist yet another assessment of a physician: The online patient review.

Online patient-review sites like Yelp, ZocDoc, Healthgrades, and others generally feature a grading system and a comments section. How much do these sites influence a patient’s decision to come see you? According to the 2017 Local Consumer Review Survey research poll [3], they matter a lot. For the majority of consumers, online reviews influence their decision-making. The survey indicated that 85% of consumers trusted online reviews as much as a personal recommendation [3]. Additionally, after restaurants and hotels, healthcare reviews were the third most commonly read by consumers in the business category. According to the poll [3]:

• 93% of consumers read local reviews to decide if a business is good or not,

• 73% of consumers stated that positive reviews make them trust a local business more,

• 68% of consumers said that positive reviews made them more likely to use a business,

• 54% of consumers consider the average star rating to be the most important factor, and

• 40% say that negative reviews make them not want to use a business.

But does this also apply to healthcare? Can sites like Yelp be used as a measure of patient satisfaction, and more importantly, patient safety? Is this effect scientifically supported? From a scientific and statistical perspective, these sample sizes are likely too small to be a valid representation of the larger population’s experience. And besides, consumers consider their healthcare with a much more critical eye than what kind of pizza they are going to order, right?

Indeed, most of the population has never posted a review of a doctor or hospital. According to a research letter in JAMA, only 5% of the population has ever left an online review of a physician and 3% for a hospital [6]. Despite that, some evidence suggests that these reviews correlate with patient satisfaction scores on HCAHPS. Bardach and colleagues [1] correlated positive Yelp Scores to high HCAHPS ratings for hospitals that had as few as five Yelp ratings. In 2016, Ranard and colleagues [8] observed a similar correlation in hospitals with as few as three Yelp ratings. These studies examined thousands of online reviews and it was surprising to see how they appeared to correlate to formal metrics of patient satisfaction.

That said, there is a big difference between patient satisfaction and patient safety. Though it may be tempting to associate high patient satisfaction with high-quality of care, the distinction is very important and in some cases, contradictory. To borrow from the Rolling Stones, there is good evidence that giving patients what they want—in the name of raising satisfaction scores—may not be giving them what they need [11]. One study even found that the most-satisfied patients received more prescriptions, more hospitalizations, and were more likely to die within 4 years [5]. Given that the vast majority of consumers make a judgment based on as few as one to six online reviews [4], can we really expect the average internet user to take due diligence to differentiate satisfaction from safety?

And how much can we really trust online data? First, the data may not be broadly representative of the population; as earlier noted, more than 95% of the population has never left a review for a physician or hospital [6]. Also, reviews may not be valid; some research suggests that 10% to 30% of online reviews may be fake [12, 13]. Hosting websites use their own discretion when deciding to post certain comments, and their methodologies for doing so are not explicitly defined or disclosed. I reached out to several of these review sites to learn more about their posting criteria and only two responded (ZocDocs referred me to a Frequently Asked Questions page; Healthgrades responded with general interest, but did not respond to my followup queries).

In today’s era of social media manipulation, misinformation is easily conveyed online. The mere existence of businesses whose primary purpose is to manage the online reputation of other businesses [2, 9, 10] underscores this point. If we are going to evaluate something as important as patient safety and quality of care, we need data measurement methods that are not subject to so many confounders. Until there is full transparency in how these data are collected and adequate control of potential confounders, online reviews should not be used as a source of data to determine quality and safety of care.

The problem with the online review, like much of today’s social media information, is that accurate or not, it can be disproportionately influential. We, as a society, are still learning how powerful online (mis)information can be whether it is in a review of a physician, dentist, or even political candidate. Whether we like it or not, patients use these reviews to make decisions. I’ve used them myself, to select a dentist. But given how potentially influential and inaccurate online data can be, we need to be more mindful of where those data come from. More importantly, these online review sites need to be fully transparent with their methodology and accuracy. As much as I may be irritated with and disagree with formal quality metrics [7] and their lack of sufficient risk adjustment, at least there is a documented process to the analysis- something we don’t see from these online review sites.

And what about that dentist and the 2-hour pain session? When the throbbing stopped, I decided not to post that scathing review. I left that office without the pain I came in with, and I have been fine since. It was a good reminder that sometimes things don’t always go as smoothly as we might like. Posting the review I had in mind at the time could unfairly bias the next patient. In a world where physicians’ choices should be largely evidence-based, it is disheartening that our patients’ priorities may actually in part be driven by something so subjective and idiosyncratic as an online review. Patients should be able to share their experiences with other prospective patients, and they should have access to quality metrics on patient safety about their physicians and medical institutions. Most importantly, patients (as well as doctors and medical institutions), in the interest of quality care, deserve that information to be accurate and transparent in their sources, whether they are from national quality institutions or online review sites.