Patient and Caregiver Perceptions of Nursing Home Physicians: Insight from Yelp Reviews from 2009-2018

Abstract

Objectives:

To describe the content of reviews submitted on Yelp that pertain to nursing home physicians.

Design:

Retrospective qualitative study.

Setting and Participants:

US nursing homes reviewed on Yelp in 2009 - 2018 with reviews that discussed nursing home physicians (N=375 nursing homes in 31 states).

Measurements:

Content analysis was performed to detect recurrent themes and divergent ideas about nursing home physicians perceived by reviewers.

Results:

Average rating among nursing homes with physician reviews was 2.0 (standard deviation 1.5, range 1-5). The major themes from content analysis of reviews that discussed physicians were perceived attitudes of clinicians toward patients and caregivers, physician communication with patients or caregivers, and perceived clinical expertise of the physicians.

Conclusion:

Unsolicited online reviews of nursing homes that comment on patient and caregiver perceptions of physician care represent a small but growing number of online comments about nursing home. These reviews contain important information about aspects of physician care valued by nursing home patients and caregivers. However, the sample of comments about physicians represents a small proportion of online reviews and is not representative of all nursing homes. As online reviews grow in number and become more representative, these data could be used by physicians to improve perceptions of their care by the NH patients and their families, and by NHs to inform expectations of their medical director and attending physicians.

Introduction

Nursing homes (NHs) increasingly recognize the important role of physicians in care quality and patient outcomes.^1,2 Over the past decade, the prevalence of physicians who practice exclusively in NHs (colloquially known as ‘SNFists’) increased by 34% to 48%.^3,4 However, little is known about patient and caregiver experience with physician care in NHs. Prior studies on this topic focused on the experiences of caregivers of patients at the end-of-life.^1,5 The goal of this study was to evaluate the reviews about NH physicians using data from a large social media platform (Yelp.com).

Referring providers, patients, and caregivers increasingly turn to online sources to obtain information about the quality of healthcare facilities.^6–9 For NHs, Medicare’s Nursing Home Compare (NHC) has been publishing star ratings based on clinical quality measures, inspections, and staffing information for over a decade.¹⁰ However, the site’s use by consumers has been limited by distrust of the information available on the website.¹¹ Furthermore, while other CMS Compare sites like Hospital Compare include patient-reported outcomes such as satisfaction or perceived experience with care, NHC does not.¹² Unsolicited consumer reviews of healthcare facilities have been published online by Yelp since 2005.^13–17 The volume of reviews about NHs on Yelp grew since 2015 when Yelp added publicly available information about health inspections to the reviews.¹⁸ Two advantages of platforms such as Yelp is that they are able to publish the reviews as soon as they are submitted, and the open-ended nature of the reviews. The disadvantages of unsolicited online reviews include lack of oversight and transparency, which limit a NH’s ability to correct inaccurate reviews or use reported information in quality improvement efforts. Although Yelp uses proprietary algorithms to identify fraudulent submissions, their success rate is unknown.¹⁸

Prior studies of online reviews of NHs identified several topics consistently mentioned in the reviews,^7,19 but did not focus on comments about physicians. As online reviews of NHs proliferate, it is important to learn what aspects of physician care are described positively or negatively by the reviewers. Thus, our objective in this study was to describe the content of online NH reviews pertaining to physicians.

Methods

Data Sources and Study Sample

Yelp is a free online platform that publishes free-text unedited comments and calculates a cumulative average 5-point rating of all submissions.¹⁸ We collected all reviews and ratings in the NH category posted between August 27, 2005 and January 17, 2019, consistent with the terms of use. To verify that the facility reviewed was a NH and obtain NH characteristics (e.g., size), we merged Yelp data to the NHC database. Yelp facility names were matched to Medicare facility identifiers via two rounds of matching: (1) an exact string match between the names and zip codes, and (2) a “fuzzy” match allowing facility names to vary by at most 3 characters as long as the facilities were located in the same zip code. All facilities matched in the second step were checked by hand and ambiguous matches were deleted. Lastly, to select the final sample of reviews that included information about physicians for qualitative analyses, we searched all reviews for relevant keywords (“doctor”, “physician”, “MD”). All identified reviews were checked manually during coding to confirm that they commented on NH physicians.

Coding

Reviews were imported into NVivo Software Version 12.1 (QSR International, Melbourne, Australia). Two authors (KR, HW) reviewed a 5% random sample of reviews independently to develop the initial codebook, followed by iterative revisions of the codebook to narrow down the components until consensus was reached (final codebook available as Supplementary Table S1). All reviews were coded by one of us (HW) and a 10% random sample of reviews was independently re-coded by a third co-author (KF). The inter-rater correlation coefficient was measured and ranged between 0.87 and 1.00 across all nodes indicating high inter-rater reliability and coding consistency.²⁰

Analysis

After all transcripts were coded, thematic content analysis was performed to detect recurrent themes about perceptions of NH physician care. All emerging themes were reviewed iteratively by the authors and discussed in a group until consensus on the main themes was reached. A detailed audit trail was maintained throughout the study, including codebook development, resolution of coding inconsistencies, and analyses.²¹

We performed three additional analyses. First, we compared the characteristics of NHs in our sample to the characteristics of all NHs in the NHC database between 2014-2018. Second, we compared the ratings of reviews that mentioned physicians vs. those that did not mention physicians using the two-sample t-test. Third, we reviewed a 5% random sample (n=450) of reviews that did not mention physicians to confirm that the initial sample selection strategy did not miss any relevant reviews about physicians. This process did not identify any reviews that were misclassified.

Statistical analyses were performed using Stata, version 13 (College Station, Texas) and NVivo. The study was exempt from review by the institutional review board of the University of Pennsylvania.

Results

Study Sample Characteristics

Of the 9,796 reviews about 1,618 NHs, physician care was reviewed in 564 reviews that were about 375 NHs in 31 states. The reviews posted in 2005-2014 rarely mentioned physicians (n=48). The number of reviews about physicians increased 5-fold between 2014 (n=29) and 2018 (n=159) (Figure 1). On average, Yelp reviews that mentioned NH physicians had lower ratings compared to reviews that did not mention NH physicians (2.0 out of 5.0 possible points, standard deviation [SD] 1.5 vs. 2.9, SD 1.8, p<0.001).

Figure.

Nursing Home Physician Reviews on Yelp 2009-2018

More than half (56.0%) of the NHs in the sample were located in the West, 18.7% in the South, 13.9% in the Northeast, and 11.5% in the Midwest (Table 1). And almost all of the NHs were located in urban areas (97.0% vs. 2.9% in rural areas). Few NHs were hospital-based (2.7%). Supplement Figure shows the distribution of online ratings. Yelp ratings followed a bimodal distribution with most NHs receiving the lowest rating (65.4%) and 15.8% receiving the highest rating. In contrast, NHC star ratings of these facilities were skewed toward more stars (higher rating), with 34.9% of the NHs rated the highest (5-stars) and 23.8% second highest (4-stars).

Table 1.

Characteristics of Nursing Homes in the Sample vs. All US Nursing Homes¹

	Study Sample (N=375)		All Nursing Homes (N=14,787)		p-value
Characteristic	N	%	N	%
Region
Northeast	52	13.9	2,662	18.0	0.45
South	70	18.7	5,397	36.5	0.002
Midwest	43	11.5	4,599	31.1	0.006
West	210	56.0	2,129	14.4	<0.001
Size
Small (<100 beds)	148	39.5	6,550	44.3	0.24
Medium (100-199 beds)	179	47.7	7,246	49.0	0.73
Large (200 or more beds)	48	12.8	991	6.7	0.11
Ownership
Non profit	84	22.4	5,161	34.9	0.02
For profit	291	77.6	9,626	65.1	<0.001
Location
Urban	364	97.0	10,868	73.5	<0.001
Rural	11	2.9	3,919	26.5	0.08
Any advanced practitioners (NP/PA)	199	53.1	7,526	50.9	0.54
Part of a chain	283	75.5	8,665	58.6	<0.001
Hospital-based facility	10	2.7	577	3.9	0.85

¹For comparison, the characteristics for all NHs in 2014-2018 were obtained from the Medicare Nursing Home Compare data files.

Main Themes

Three major themes that emerged from thematic content analysis of the reviews were (1) perceived attitudes of physicians toward the patients and caregivers, (2) physician communication with patients or caregivers, and (3) perceived clinical expertise of the physicians (Table 2). Nearly a third of reviews (29.3%) commented on physician’s attitudes perceived by the patient or caregiver toward themselves, the NH, and/or their job. Reviewers that reported positive experiences perceived that their physician at the NH was attentive, empathetic, and caring. Several reviewers mentioned that they felt their physician “really cared” and was sincerely concerned about them/their loved one.

Table 2.

Major Themes and Representative Quotes

Theme	Definition and topics covered	Example Quote	Frequency Reported, %
Attitudes toward the patients and caregivers	Perceived attentiveness, empathy, and concern for the patient’s well being. The way caregiver concerns are received. Perceived prioritization of patient care over other responsibilities. Perceived ulterior motives (e.g., financial gain).	“The doctor my mother had was not the best either just didn’t seem to want to be there or enjoyed her job.” “One plus the two doctors on staff are really great - both are very caring and informative.”	29.3
Communication with patients or caregivers	Frequency and thoroughness of communication, responsiveness to questions, clarity of explanations.	“[My mother’s] doctor always got back to me within a day when I had questions and I felt like they really listened to me.” “There are no doctors to be seen. I was told there was one doctor who stops in once a week so we were never quite clear on [my relative’s] treatment plan.”	66.0
Perceived clinical expertise	Descriptions of diagnostic or treatment decisions made, clinical issues addressed, medications prescribed, discharge instructions.	“Ailments that can only be handled by a doctor are taken care of in a manner that makes you feel confident in their abilities - which in turn relieves anxiety.” “We had to ask repeatedly over a period of several days to make changes to her care, while she was losing a significant amount of weight and looking sicker and sicker each day her doctor maintained things were fine.”	34.9

Physician communication with other staff, patients, or caregivers was the most common theme discussed in the reviews. Two-thirds (66.0%) of the reviews discussed physician communication. The topics covered by reviewers included the frequency of communication with patients and caregivers (ranging from “never” to “every day”), quality (e.g., clarify of explanations, tone, professionalism, protection of patient confidentiality), mode (e.g., in-person, via telephone, text, paper chart or notes, by passing messages/information to and from nurses or other staff) and duration of communication (e.g., “3 minutes”, “half hour”), and perceived responsiveness to questions or concerns. Some reviewers perceived that physicians were easily reachable to answer questions or discuss concerns; whereas others reported that reaching the physician was “impossible”. Slightly over a third (34.9%) of reviews commented on the reviewers’ perceptions of the NH physician’s clinical expertise. These comments included topics such as identification, diagnosis, or treatment of acute complaints, medication prescribing, and discharge planning. The recurrent themes and representative quotes in reviews of NHs with the lowest and the highest Yelp ratings are shown in Supplement Table S2.

Discussion

Reviews of NH physicians represent a small but growing subset of NH reviews published on Yelp. Although NHs represented in those reviews are located across 31 states, the reviewed facilities are disproportionally urban and located on the West coast. Yelp ratings associated with reviews that mentioned physicians were on average lower than the Yelp ratings of reviews that did not mention physicians and lower than the NHC star rating for the same NHs. Overall, NHs with reviews that mentioned physicians were not representative of the population of US NHs. Qualitative analysis of the reviews revealed three recurrent themes discussed by the reviewers including communication, perceived clinical expertise, and attitudes of the NH physicians. Information on these aspects of NH care that are valued by the patients and caregivers are not currently part of other public reporting efforts on NH quality. Efforts to improve these aspects of NH care may result to better patient and caregiver satisfaction with NH care. Incorporating patient- and caregiver-reported measures of physician communication (e.g., availability, frequency, and responsiveness), for instance, into formal public reporting on NH quality such as NHC may better inform NH selection process for patients and their caregivers. However, Yelp reviews about NH physicians represent a small fraction of all NH reviews and the number of reviews for each NH in the sample is small. In contrast, NHC does not publish quality data based on fewer than 20 assessments. Efforts to improve sampling and transparency of reviews published on Yelp would make this information more actionable for NHs and the public.

Our findings suggest that access to high-quality physician care is important to NH patients and their caregivers and that patients and caregivers expect higher frequency and quality of communication from their physician during their stay. The recent trends of increased adoption of NH-based physicians by NHs may represent one strategy to address these needs.^3,4,22,23 However, whether clinicians that specialize in NH practice are more readily available to patients and their families depends on a number of factors, such as the number of NHs they work in, travel time between facilities, patient volume and acuity. Future studies should evaluate the impact of physician practice arrangements in NHs on the patient and caregiver experience.

Our findings are consistent with prior studies of online reviews of physicians. A 2015 analysis of Yelp reviews of dermatology practices found that physician-specific themes included communication skills, perceived attitude or temperament, and clinical expertise.²⁴ Prior survey studies of patient experience with physician care in NHs at the end of life reported that participants perceived poor communication and care coordination between physicians and other medical staff as well as with patients and caregivers.^1,5 In follow-up interviews to those surveys, family members of patients who died in a NH reported difficulty reaching physicians,²⁵ poor quality of communication with physicians,²⁶ and perceived incomplete hand-offs between external physicians and NH staff.²⁷

This study has several limitations. First, the sample size of reviews that discussed NH physicians was small and the majority of NHs in the sample had only 1 or 2 reviews. Therefore, we were unable to interpret the ratings or reviews for individual NHs. Second, the NHs in the sample were not representative of all NHs in the U.S. However, by including all Yelp reviews posted during the study interval, we were able to summarize the content of NH physician-specific comments on Yelp. Third, we were unable to confirm that the reviewers had a first-hand experience with the facilities. Fourth, this qualitative study was not designed to account for differences in other aspects of NH practice, such as direct care staffing, that affect patient and caregiver experience with NH care.

Unsolicited online reviews of NHs contain information about NH physicians unavailable from other public reporting sources about NH quality, including perceptions of physician attitudes toward patients and caregivers, physician communication with patients or caregivers, and perceived level of clinical expertise of the physician. However, the sample of comments about physicians represents a relatively small proportion of online reviews and far from being representative of all U.S. NHs. As online reviews grow in number and become more representative, these data could be used by physicians to improve perceptions of their care by the NH patients and their families, and by NHs to inform expectations of their medical director and attending physicians.

Supplementary Material

supinfo

Table S1. Codebook

Table S2. Differences in Main Themes and Representative Quotes between Reviews of Nursing Homes with the Lowest vs. Highest Yelp Ratings

Figure. Nursing Home Reviews about Physicians by Yelp Rating vs. Overall Nursing Home Compare Rating