Use of social media to assess the impact of equitable state policies on LGBTQ patient experiences: An exploratory study

Introduction

The Institute of Medicine recognizes that LGBTQ persons experience a disproportionate burden of disease and poorer health outcomes compared to the general population (1). Equitable policies at the state level that protect LGBTQ persons have been implemented to reduce discrimination towards this group. However, limited research has evaluated these equitable policies because of the difficulty of capturing LGBTQ patient experience. Previous studies have shown that LGBTQ persons report increased rates of discrimination across a wide variety of healthcare settings (2–4) which may prevent them from disclosing their LGBTQ status (5, 6). Traditional surveys such as the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey do not capture patient experience as it relates to sexual orientation and gender identity and few studies include LGBTQ-related questions (5). The goal of this research was to use a social media big dataset to evaluate the impact of equitable policies on patient experiences for LGBTQ persons.

Methods

A supervised machine learning classifier was built to identify tweets related to healthcare patient experience as documented in a previous study (7). Tweets related to patient experience were collected from February 2013 to February 2017. To assign LGBTQ status, we examined the Twitter user profile. A user who used any of the terms “lesbian,” “gay,” “bisexual,” “transgender”, “trans”, “queer”, “LGBT”, “LGBTQ”, “intersex”, “homosexual”, or “cis”, in their profile description was deemed an LGBTQ user. The user population that did not defined themselves as being LGBTQ on their Twitter profile was used as the comparison group in this study. To check for consistency in content between the LGBTQ and non-LGBTQ datasets, the top 1000 most frequently mentioned words in LGBTQ and non-LGBTQ user tweets were extracted and a proportion test was conducted to reveal any differences between the two groups.

Tweet sentiment was defined as the attitude of the patient towards their healthcare experience and obtained using Valence Aware Dictionary for Sentiment Reasoning (VADER), a widely accepted rule-based sentiment classifier (8, 9). A state-level measure of LGBTQ-related policies was obtained from the Movement Advancement Project (MAP), an independent non-profit that provides rigorous research on policy equity (10), and linked with the Twitter dataset based on location of tweet post. The LGBTQ “state policy tally,” from MAP measures state protection for LGBTQ persons, whereby each state’s tally counts the number of positive laws and policies that help drive equality for LGBTQ persons. Each positive (protective) law adds one point to the state tally and each negative (harmful) law, a law that restricts access to LGBTQ rights, services or programs, subtracts one point from the tally. A state’s possible overall tally ranges from −10 to 34. Linear regression was used to quantify the effect of the enactment of one positive LGBTQ policy on patient experience tweet sentiment. An interaction term was included to quantify the differential effect on LGBTQ versus non-LGBTQ persons. Data analysis was conducted using the scikit-learn (11) and statsmodels (12) packages in Python.

Model:

$$\begin{array}{l}tweetsentiment\text{~}LGBTQmembership\text{+}StatePolicyTally\text{+}LGBTQmembership\\ \text{*}StatePolicyTally\end{array}$$

Results

The total number of users in the patient experience dataset was 1,376,084 users. Out of these users, 13,689 (1.00%) self-identified as LGBTQ and 1,362,395 (99.00%) did not self-identify as LGBTQ. The number of LGBTQ users that had available geolocation data was 5,545 and the number of non-LGBTQ users was 445,919; only tweets with geolocation data were used for analysis. A proportion test of the top 1000 most common words found no significant differences in proportion between LGBTQ users and non-LGBTQ users. The top 10 most frequent words by LGBTQ and non-LGBTQ users are provided in Table 1. In order of most frequent to least frequent terms, the top 10 terms were: hospital, doctor, pain, surgery, care, sick, urgent, medicine, nurse, and today.

Table 1.

Keyword analysis of LGBTQ versus non-LGBTQ user tweets, United States, 2013–2017

Rank	Keyword	* Percent occurrence of keyword in top 1000 words
		LGBTQ users	Non-LGBTQ users
#1	hospital	1.75	1.88
#2	pain	1.49	1.54
#3	doctor	1.42	1.48
#4	surgery	1.10	1.22
#5	care	1.03	1.10
#6	urgent	0.79	0.89
#7	sick	0.64	0.83
#8	medicine	0.37	0.49
#9	today	0.29	0.31
#10	need	0.28	0.30

^*No significant differences were found between the frequency of the top words discussed in LGBTQ versus non-LGBTQ user tweets

Table 2 describes the results of the linear regression model. On average, LGBTQ users had a 0.0266-point lower patient experience sentiment compared to non-LGBTQ users. States with higher state policy tally had higher patient experience sentiment for both LGBTQ and non-LGBTQ groups. A one-point increase in state policy tally had a 0.0008-point increase in sentiment for non-LGBTQ users and a 0.0008 + 0.0009 = 0.0017 point increase in sentiment for LGBTQ users.

Table 2.

Results of linear regression model of tweet sentiment based on LGBTQ identification and state policy tally, United States, 2013–2017

	Coefficient	SE	T-statistic	p-value	95% Confidence Interval
Intercept	−0.0680	0.001	−70.506	<0.001	−0.070 to −0.066
LGBTQ user	−0.0266	0.009	14.962	0.002	−0.044 to −0.009
State Policy Tally	0.0008	<0.0001	−3.027	<0.001	0.001 to 0.001
State Policy Tally x LGBTQ user	0.0009	<0.0001	2.046	0.041	0.000 to 0.002

Discussion

Based on our findings, more supportive LGBTQ policies were associated with higher patient experience sentiment and this association between protective LGBTQ state policies was two times greater for LGBTQ versus non-LGBTQ users. Additionally, our results reveal that LGBTQ patients and non-LGBTQ patients use similar terminology to report on their healthcare experiences. This may indicate they have comparable experiences but different sentiment toward these experiences.

The MAP policy tally aggregates nearly 40 different LGBTQ-related laws and policies that cover healthcare, relationship and parental recognition, non-discrimination, religious exemption laws, LGBT youth, criminal justice, and identity documents, and involves both sexual orientation and gender identity (13). Our results may reflect the translation of these wide-ranging policies onto patient experiences through their impact on the overall legal environment and social climate. Nevertheless, different LGBTQ laws are likely to confer different degrees of harm or benefit and also can affect different elements of LGBTQ individuals’ health care. Analysis of particular policies to measure their differential impact was not conducted in this study, but is an important area for further research.

Limitations

Findings in this study are associative and causal claims on the effect of LGBTQ policies on patient experience cannot be made from these results. Although our study evaluated tweet sentiment regarding healthcare experience, we did not study tweet sentiment regarding LGBTQ status. Our dataset included tweets describing any patient experience by LGBTQ users; we did not require that the tweets comment on how users perceived their LGBTQ status to have impacted the quality of healthcare received. Thus, we were unable to conduct a sentiment analysis of LGBTQ status on healthcare experience, which may have differed between the LGBTQ and comparison group. Future studies should analyze the perceptions of LGBTQ status on patient experiences.

The keywords methodology used to identify LGBTQ users may have incorrectly placed some LGBTQ users in the non-LGBTQ group and vice versa. For instance, the term cis was used because of its frequent use within the LGBTQ community to indicate that their gender identity matches the sex, they were assigned to. (14) However, persons who do not identify as LGBTQ may also use this term may have been incorrectly classified into the LGBTQ group. While we were able to manually confirm that 80–90% of LGBTQ categorized users were indeed LGBTQ users, we were not able to place a number on the accuracy of non-LGBTQ categorization. Another data limitation is that only users with identifiable geolocation were used in the analysis of this study (<50% for LGBTQ users and the comparison group). This may reduce the generalizability of the findings.

Finally, our data is derived from the online social media site Twitter which may not be representative of the broader US patient population. However, most national or state surveys lack the appropriate questions pertaining to gender and sexual identity making it difficult to conduct large-scale research (15, 16). Furthermore, research shows that LGBTQ persons are more open to defining their LGBTQ status in their online social networks when compared to offline networks (17, 18). In fact, Twitter is more popular as a social media space for LGBTQ users compared to other social media networks like Facebook (17). Therefore, our study may have captured a group of LGBTQ patient participants normally excluded from traditional patient experiences studies.

Conclusion

Measuring patient experience can be difficult and research has indicated that surveys that capture a breath of the healthcare experiences that include communications and interactions with providers and the care team are more strongly correlated with health outcomes (4, 19). Therefore, measuring patient experiences through Twitter may enable us to capture a broader and more organic picture of patient experience compared to traditional surveys which can be limited by question specificity and social desirability bias – especially within vulnerable population groups.(7, 19) Findings in our study suggest that certain state level factors may influence LGBTQ patient experience, but further study in this area needs to be conducted to determine a causal effect. Public health officials, providers, and healthcare administrators should use Twitter as a supplemental dataset to monitor patient experiences among LGBTQ patients and to evaluate LGBTQ policies.