Online Customer Reviews of Assisted Living Communities: Association with Community, County and State Factors

Abstract

Objectives:

Online reviews provided by users of assisted living communities may offer a unique source of heretofore unexamined data. We explored online reviews as a possible source of information about these communities and examined the association between the reviews and aspects of state regulations, while controlling for assisted living, county, and state market-level factors.

Design:

Cross-sectional, observational study.

Setting and Participants:

Sample included 149,265 reviews for 8,828 communities.

Methods:

Primary (e.g., state regulations) and secondary (e.g., Medicare Beneficiary Summary Files) data were used. County-level factors were derived from the Area Health Resource Files, and state-level factors from the integrated Public Use Microdata series. Information on state regulations was obtained from a previously compiled regulatory dataset.

Average assisted living rating score, calculated as the mean of posted online reviews, was the outcome of interest, with a higher score indicating a more positive review. We used word cloud to visualize how often words appeared in 1-star and 5-star reviews. Logistic regression models were used to determine the association between online rating and a set of community, county, and state variables. Models were weighted by the number of reviews per assisted living bed.

Results:

Overall, 76% of communities had online reviews. We found lower odds of positive reviews in communities with greater proportions of Medicare/Medicaid residents (OR=0.986; p<0.001), while communities located in micropolitan areas (compared to urban), and those in states with more direct care worker hours (per week per bed) had greater odds of high rating (OR=1.722; p<0.001 and OR=1.018, p<0.05, respectively).

Conclusions and Implications:

Online reviews are increasingly common, including in long-term. These reviews are a promising source of information about important aspects of satisfaction, particularly in care settings which lack a public reporting infrastructure. We found several significant associations between online ratings and community-level factors, suggesting these reviews may be a valuable source of information to consumers and policy makers.

Brief summary:

Online reviews of assisted living communities may be a promising source of information about important aspects of consumer satisfaction. We found several significant associations between online ratings and community and state factors.

INTRODUCTION

Assisted living has become a critical component of the US residential long-term care, providing over 1 million beds in 31,000 communities.¹ As the number of these communities grew in the last two decades, so has the range of models now referred to as assisted living, and the services they provide.^2,3 With this growth and changes, concerns about standards of care and quality in assisted living have persisted.^4-6

Today, assisted living communities serve largely older, Medicare beneficiaries who are frail, experience significant functional and cognitive impairments, and for whom this is often their last home.^7-9 Yet, unlike nursing homes, which are required to disclose information on numerous aspects of care quality, there are no federal reporting mandates and only a handful of states require some reporting from assisted living communities. At least half of all states do not have sufficient administrative oversights of these communities, and generally provide very limited information about assisted living.⁵

In absence of such oversight, and the corresponding lack of data collection, consumers choosing to enter assisted living make their choices almost entirely based on location, price and/or the amenities, and largely through the “word of mouth” recommendations. Yet, a prior study has shown that assisted living residents reported quality as the most important factor in their choice of a community.¹⁰ A recent review of state websites of assisted living communities showed that even when information on satisfaction and quality was provided (69%), most was dated and difficult to find.¹¹ [see Appendix note 1] Because of this information gap, online reviews provided by assisted living users, including residents and their family members, may be an important source of information. Although online reviews for a variety of residential care providers have been available for years, there has been only a handful of studies on their reliability and their value to the consumers is not clear. For nursing home providers, a few studies of online ratings have demonstrated statistically significant associations between online ratings and quality measures such as deficiency citations¹² or hospital readmissions.¹³ National studies of online reviews specific to assisted living communities have not been available. A notable exception is a recent study that examined the relationship between online Google reviews and a patient-centered outcome measure of home time, i.e., time spent in assisted living as opposed to being in a nursing home or a hospital. Finding a significant positive association between online reviews and home time, the study showed that these reviews may be an important source signaling care quality in these communities.¹⁴

Although states certify and regulate many aspects of assisted living operations such as staffing, training, admission and retention of residents, and others, regulators do not know if these requirements make a difference in the quality and satisfaction with the care provided. To date, several studies have shown that assisted living state regulations influence provider safety culture,¹⁵ as well as the care that the residents receive, including hospitalizations, place of death and hospice use.^8,16 There have been no studies on the relationship between online reviews reflecting consumer satisfaction and state regulations.

Motivated by these gaps in the literature, we: 1) explored online reviews as a possible source of data on assisted living consumer satisfaction; and 2) examined the association between online reviews and state regulations on resident admission/retention and on staffing, controlling for community and county as well as state market-level factors.

METHODS

Data and Sample

First, following a previously published methodology,¹⁷ we identified 16,682 assisted living communities with Medicare beneficiary residents in 2018-2019. Using a community name and physical address we obtained place IDs through Google Maps. Place ID uniquely identifies a place on Google Maps. We then submitted the list of community IDs to Outscraper (Google maps data scraper) to obtain all reviews attached to each ID. A total of 196,599 reviews posted between July 27, 2007, and March 11, 2022, were collected, representing 12,705 communities.

We subset the study sample to include only the most recent reviews posted between April of 2018 and March of 2022. On average, in that sample, assisted living communities had 4 reviews (SD=5.6). To minimize the risk of bias resulting from communities with a small number of reviews, we excluded communities with fewer than 4 reviews from our analysis. Our main analytical sample contained 149,265 reviews for 8,828 assisted living communities (mean/stdev=16.9/16.8).

To obtain community-level characteristics, we used previously compiled directory of 2018-2019 assisted living,¹⁷ the Medicare Beneficiary Summary Files, and the rural-urban commuting area codes (https://depts.washington.edu/uwruca/ruca-uses.php). County-level market factors were derived from the Area Health Resource Files, and state-level market factors were obtained from the integrated Public Use Microdata series for 2018-19 (https://usa.ipums.org). Information on state assisted living regulations was obtained from a previously compiled state regulatory dataset.¹⁸

Variables

The outcome of interest was the average community rating score, calculated as the mean of all posted scores. Online scores range from 1 to 5 stars, with a higher score indicating a more positive review. Based on the distribution of all scores, we dichotomized scores at median with a score <=4.2 defined as low and >4.2 as high.

The main independent variables of interest included state regulations pertaining to assisted living staffing by direct care workers, licensed staff, and resident admission/retention policies.¹⁸ Direct care workers, who provide hands-on care and support to the residents, are comprised of personal care aides, nursing assistants, and home care aides.¹⁹ Staffing regulations were, following prior studies,^15,16 defined as specific – when minimum staff numbers were specified or required in proportion to residents - otherwise as non-specific. The domain of state regulations on admission/retention included 9 items (e.g., cannot safely manage medication, not able to provide appropriate services),⁸ coded on a continuous scale, with a higher number reflecting higher regulatory specificity (Table S1 – Appendix).

Community-level covariates included the number of certified beds (continuous), proportion of residents who were Medicare/Medicaid eligible, and location (urban, suburban/large town, or small town/rural). County-level covariates included the number of assisted living and nursing home beds per 1,000 people age 65+. At the state-level, for individuals who worked in residential care settings other than nursing homes, we included: total hours direct care workers worked per week divided by number of assisted living beds; the ratio of licensed staff (licensed practical and registered nurses) to all nursing staff, per week per bed; and the average hourly wages for direct care workers, and practical and registered nurses.

Statistical Analysis

Analyses were performed at the assisted living level. We conducted a preliminary validation of online reviews using state-level consumer surveys conducted in Ohio (see Appendix note 2); showing a positive and a statistically significant association between online reviews and consumer survey-based satisfaction scores. We used word cloud to visualize how often words appeared in 1-star and 5-star reviews.²⁰ Common words (e.g., “the”, “a”, “an”, “in”) were removed before creating the word cloud. The most frequent words are shown in bigger and bolder letters. Logistic regression models were used to determine the association between rating status (high-rated vs. low-rated) and a set of community, county, and state variables. To adjust for the variation in reviews across communities, models were weighted by the number of reviews per assisted living bed. For sensitivity analysis, we also estimated one model restricted to larger communities (with 25 or more beds) that had at least 4 online reviews (N=7,812), and another model for larger communities with at least 20 reviews (N=2,223).

The study was approved by the Institutional Review Board of the University of Rochester.

RESULTS

Overall, 24% of the communities had no online reviews [((12,705/16,682)-1)x100]. Figure S1 (Appendix) shows that for communities with at least 4 reviews, the distribution of ratings is skewed by those receiving ratings of 4 or higher. Sample characteristics by community rating are depicted in Table 1. Assisted living with lower rating were statistically significantly larger, with a higher proportion of Medicare/Medicaid residents, and more likely located in urban areas. They were also located in states with lower direct care workers’ pay and fewer work hours. Significant variations by state in online ratings were also evident (Figure S2).

Table 1:

Sample characteristics by assisted living rating level

	All ALs with at least 4 reviews
	Low rating (<=4.2)	High rating (>4.2)	p-value
Number of assisted living communities	N=4,542	N=4,286
Number of reviews	N=64,	N= 84,881
Average no. of reviews per community	14.2	19.8
AL-level Factors
Mean (SD) bed size	85.09 (52.64)	81.82 (49.98)	0.003
Proportion of Medicare/Medicaid eligibles	24.96 (29.69)	15.12 (23.78)	<0.001
Location			0.002
Urban	84.41%	81.33%
Suburban/large town	9.49%	11.22%
Small town/rural	5.57%	6.58%
Missing	0.53%	0.86%
County-level Factors
AL beds per 1000 people age 65+	31.65 (16.75)	29.91 (15.87)	<0.001
Nursing home beds per 1000 people age 65+	32.19 (15.46)	32.79 (16.65)	0.078
State-level Factors
DCWs: average hourly wage (2018-19)	15.18 (6.25)	15.52 (8.21)	0.028
Licensed nursing staff avg. hourly wage (2018-19)
Licensed Practical Nurses	23.31 (6.21)	23.62 (6.34)	0.021
Registered Nurses	35.62 (26.52)	35.82 (27.04)	0.73
DCWs: total hours worked per week per bed (2018-19)	11.16 (4.76)	11.75 (6.42)	<0.001
Licensed staffing work hours ratio (2018-19)	21.10 (6.68)	21.23 (7.03)	0.37
Admission & retention regulatory score (9 items)	4.63 (1.13)	4.53 (1.21)	<0.001
DCW staffing regulations			0.003
Not required or number not specified	18.96%	21.47%
Required and minimum specified or in proportion to residents	81.04%	78.53%
Licensed staffing regulations			0.012
Not required or number not specified	73.25%	70.86%
Required and minimum specified or in proportion to residents	26.75%	29.14%

Notes: Online reviews statistics: mean=16.9, median=12, std. deviation=16.8

DCWs = personal care aides, nursing assistants, personal and home care aides.

Word cloud visualization (Figure 1) depicted comments provided in online reviews and their frequency, respectively for reviews coded as being the worst (score=1) and the best (score=5). Regardless of the score, staffing seemed to be the most important issue, with the reviewers respectively criticizing or praising staff availability, caring, and communication (see Table S1, Appendix, for examples of comments associated with the reviews). Among the most negative reviews, references to assisted living cleanliness and the general environment were also common. The most positive reviews emphasized care and the attention given to the residents, while also commenting on the quality of the assisted living environment.

Figure 1:

World cloud visualization display of online comments for assisted living communities rated as 1-star and 5-star.

We found several community, county, and state-level factors that were associated with the online ratings (Table 2). In all three study samples (all assisted living, larger assisted living, and larger with at least 20 reviews), communities with greater shares of Medicare/Medicaid residents had lower odds of high rating (respectively OR=0.986, 0.987, 0.989; p<0.001), while those located in micropolitan areas, compared to urban, had greater odds of higher rating (OR=1.722, 1.799, 3.997; p<0.001). In the main sample (all), communities located in states with more direct care worker hours/week/bed had higher odds of high rating (OR=1.018; p<0.01), as did those in the sample with at least 20 reviews (OR=1.031; p<0.05), but this association was not statistically significant in the analysis with larger communities and at least 4 reviews.

Table 2:

Association of online rating score with assisted living, county, and state-level factors: Regression models weighted by number of reviews per bed

Variables	ALs with at least 4 reviews	Larger ALs (25+ beds) with at least 4 reviews	Larger ALs (25+beds) with at least 20 reviews
	Odds Ratio (high vs. low)	Odds Ratio (high vs. low)	Odds Ratios (high vs. low)
AL-level Factors
AL beds	1.001 (0.001)	1.000 (0.001)	0.999 (0.001)
Proportion of duals in AL	0.986***(0.002)	0.987***(0.002)	0.989***(0.003)
Suburban/large town (ref: urban)	1.722***(0.219)	1.799***(0.244)	3.997***(1.233)
Small town/rural (ref: urban)	1.501 (0.354)	1.423 (0.264)	2.904**(1.059)
County-level Factors
AL beds per 1000 people 65+	0.994 (0.003)	0.992*(0.003)	0.999 (0.005)
Nursing home beds per 1000 people 65+	0.998 (0.003)	0.996 (0.003)	0.991 (0.005)
State-level Factors
DCWs: average hourly wage (2018-19)	1.015 (0.022)	1.008 (0.022)	1.019 (0.037)
LPNs: average hourly wage (2018-19)	1.018 (0.011)	1.012 (0.011)	0.979 (0.019)
RNs: average hourly wage (2018-19)	1.001 (0.005)	0.999 (0.004)	1.007 (0.008)
DCWs: total hours worked per week per bed (2018-19)	1.018**(0.006)	1.014 (0.008)	1.031*(0.015)
Licensed staff work hours ratio (2018-19)	0.997 (0.007)	1.004 (0.007)	1.017 (0.012)
Admission & retention score	0.997 (0.036)	0.958 (0.036)	0.968 (0.061)
DCW staffing regulations (ref: not required or specified)
Specified or in proportion to residents	0.928 (0.089)	0.912 (0.089)	1.118 (1.186)
Licensed staffing regulations (ref: not required or specified)
Specified or in proportion to residents	0.958 (0.007)	0.997 (0.007)	0.985 (0.138)
Observations	8,591	7,812	2,223

Note: Robust standard errors in parentheses.

^***p<0.001

^**p<0.01

^*p<0.05

DCWs = personal care aides, nursing assistants, personal and home care aides.

DISCUSSION

This is, to our knowledge, the first study to examine the association between assisted living online reviews and community, county, and state-level factors that may impact residents’ and their families’ care experiences. Consistent with a prior study of assisted living consumer preferences,²¹ we found staff availability and help, food and meals, and the general environment to be the most commonly mentioned in online reviews. While associations with county and state level factors were not always consistent across our study samples, associations with community-level proportion of Medicare/Medicaid eligible residents and its geographic location were sizable and persistent throughout. Not surprisingly, communities with a high proportion of Medicare/Medicaid resident, which may signal resource availability, have worse online reviews. To the extent that these reviews may reflect care quality, such an association is consistent with prior studies showing Medicare/Medicaid residents and communities with higher penetration of these residents having worse outcomes.^8,22 While other studies have found significant associations, for example between staff assessment of patient safety and assisted living state regulations,¹⁵ we found no such relationship. It is possible that staff perceptions of care and patient safety are more sensitive to state regulatory variations, while consumer perceptions are somewhat more removed as management practices may be able to make widespread staff shortages somewhat less transparent by hiring temporary staff, relying on staff working overtime or taking additional shifts.²³

As assisted living communities continue to expand concerns about the viability of the model involving real estate, hospitality, and increasingly the healthcare sectors has raised concerns about care quality in this setting.⁴ In absence of concerted state or federal efforts to assess and report assisted living quality, online reviews constitute a possible, yet underexplored, source of data. Further work is needed to establish the validity and reliability of these measures of care quality and/or consumer satisfaction.

Limitations

We note several limitations. First, only about half of the assisted living communities had at least 4 reviews within the study period. It is not clear why some communities had no reviews at all, how accurately this number reflects consumers’ satisfaction or what biases may be inherent. Second, all reviews were obtained from Google and the results may not be generalizable to reviews from other platforms. Third, although we conducted a preliminary validation of online reviews in one state (Appendix note 2), larger, full-fledged validation studies are needed.

Conclusions and Implications

Online consumer reviews are increasingly common in most service industries. As such, it is perhaps not surprising that this source of information is becoming more prevalent in the long-term care sector. Although direct consumer reviews are a promising source of information about important aspects of consumer satisfaction or perhaps even care quality, particularly in assisted living communities that lack public quality reporting infrastructure, almost nothing is known about the utility of these reviews. We found several significant associations between online ratings and community-level factors, suggesting these reviews may be a valuable source of information to consumers and to policy makers.