Abstract
Dementia profoundly impacts patients and their families, making it essential to understand the experiences and concerns offamily caregivers for enhanced support and care. This study introduces a novel approach to analyzing tweets from individuals whose family members suffer from dementia. We preprocessed our collected Twitter (now X) data using advanced natural language processing techniques and enhanced conventional topic model—Gibbs Sampling Dirichlet Multinomial Mixture Model (GSDMM)—with term-weighting strategies to improve topic clarity. This enhanced approach enabled the identification of key topics among dementia-affected families, offering semantically rich and contextually coherent topics, demonstrating that our method outperforms the state-of-the-art BERTopic model in clarity and consistency. Leveraging ChatGPT 4 alongside two human experts, we uncovered the multifaceted challenges faced by family caregivers. This work aims to provide healthcare professionals, researchers, and support organizations with a valuable tool to better understand and address the needs offamily caregivers.
Introduction
The demographic shift towards the globally aging population has precipitated a marked increase in the prevalence of dementia, a condition characterized by a progressive decline in cognitive function that significantly affects the lives of both individuals diagnosed with the condition and their families. Current estimates suggest that the number of people living with dementia worldwide is set to rise from 55 million in 2023 to 115 million by 2050, underlining the urgent need for innovative approaches to support and care [1, 2, 3]. In the United States, with nearly 6.7 million individuals currently diagnosed with dementia, the overwhelming majority rely on the invaluable support provided by family and other informal caregivers [4]. This caregiving cohort, numbering over 11 million, faces substantial psychological, physical, and economic challenges, magnifying the societal impact of dementia [4, 5, 6].
Research indicates that dementia caregivers are subject to significantly higher levels of stress, depression, and physical health issues than those caring for individuals without dementia, with one study identifying a 33.9% prevalence rate for depression among dementia caregivers—a figure notably higher than that for caregivers of individuals with schizophrenia [7, 8, 9]. Moreover, the compounded stress and emotional burden can elevate the risk of serious health conditions, including cardiovascular diseases [10]. Economically, the care provided by dementia caregivers was estimated to be worth about 339.5 billion dollars in 2022 [4], a figure that is expected to rise in tandem with the increasing prevalence of dementia, thereby exacerbating the economic strain on families and society.
Amid these challenges, the advent of social media platforms such as Twitter (now X) has provided a unique lens through which the experiences and concerns of dementia family caregivers can be explored and understood. Recognizing this opportunity, our study proposes a novel yet straightforward approach to analyze Twitter data generated by individuals with family members suffering from dementia. This research aims not only to shed light on the diverse issues faced by these individuals but also to contribute to the broader field of natural language processing and topic modeling.
To this end, as shown in Figure 1, we have collected and pre-processed data from Twitter using state-of-the-art natural language processing techniques, including spam removal, lemmatization, stopword removal, compound word segmentation, and spell checking. Our approach enhances conventional topic model—specifically, Gibbs Sampling Dirichlet Multinomial Mixture Model (GSDMM)—by incorporating two term-weighting strategies,”Log” and “BDC”. The “Log” strategy is designed to diminish the impact of common stopwords across the corpus, while “BDC” targets the reduction of weights for dementia topic-specific stopwords. These modifications aim to refine the topic models’ ability to identify key topics discussed by individuals impacted by dementia, resulting in a set of well-defined topics characterized by semantically related and contextually coherent words.
Figure 1:
Overview of this work. We first collected tweets data from Twitter (now X). Then we processed the data for quality control. We then used the proposed term-weighted topic models to generate important topics. Finally we employed two experts and ChatGPT4 to interpret the topics.
Further elevating our analysis, we employ ChatGPT 4, alongside two human experts, to provide insightful interpretations of the identified topics. Our experimental results show that these enhancements greatly improve the performance of the conventional topic models, surpassing even state-of-the-art models like BERTopic [11] in terms of coherence. Through this innovative methodology, we offer a valuable tool for healthcare professionals, researchers, and support organizations, enabling them to better understand and address the needs of those affected by dementia, thus contributing meaningfully to the domain of natural language processing and topic modeling.
Related Works
Topic Modeling Topic modeling is a statistical tool used to explore latent, or hidden variables and information in large datasets. It is popular and commonly applied to various types of data, including scRNA-seq data [12], EHR data [13], environmental data [14], and even video data [15]; however, text data is particularly well-suited for this tool. The Latent Dirichlet Allocation (LDA) [16] is one of the most classic and popular generative probabilistic topic models. This unsupervised model assumes that each document in a text corpus is represented as a random mixture over multiple latent topics [16]. In other words, for LDA, each word in a document is assumed to be generated randomly from one of the latent topics. The Dirichlet Multinomial Mixture (DMM) model was first proposed by Nigam et al. [17]. It differs from LDA due to its distinct assumption and generative process. DMM assumes that each document contains only one topic and that words in the document are sampled from the same topic. In the original work by Nigam et al., DMM is implemented based on expectation maximization (EM) [17]. Based on the concept of the original DMM, Yin and Wang modified LDA to combine Gibbs sampling [18] with DMM and proposed the GSDMM [19].
BERTopic [11] is a state-of-the-art topic modeling framework structurally different from the probabilistic models like LDA and GSDMM. It is composed of document embedding, document clustering, and topic representation [11]. In the document embedding, BERTopic applies the pre-trained Sentence-BERT (SBERT) to convert documents to dense vector representations. Next, in document clustering, the framework first reduces the dimension of the vector representations of the documents using UMAP [20] and then applies the HDBSCAN [21], a hierarchical, density-based, spatial clustering method, to generate semantic clusters or topics. Eventually, in topic representation, the semantic topics are represented as bags of words by ranking words in each semantic cluster based on class-based TF-IDF (c-TF-IDF) [11], a measure for representing the importance of a word to a document cluster. The performance of BERTopic can vary depending on the specific SBERT model used.
Term Weighing Term weighting is widely used in natural language processing to adjust the importance of terms in texts by assigning weights to them. In topic modeling, term weighting is applied to enhance model performance by emphasizing more relevant terms. Topic modeling studies [22, 23, 24] utilize term weighting to filter out unimportant stopwords and highlight informative words and semantic themes.
Wilson and Chew et al. [22] presented a term weighting study using LDA, discussing the term weighting strategy called “Log” term weight. Log term weight assumes that the words with higher occurrence frequency tend to be less semantically informative in the corpus [22]. For example, the common stopwords, like “he”, “am”, “the”, and “of”, are general words with high occurrence but carrying little useful information. The Log term weight therefore aims to reduce the importance of these frequent words by assigning small weights to them and vice versa. A limitation of the Log weight is that it assigns overly high weights to words with very low occurrence. To address this issue, Li et al. [24] further modified the Log weight by adding an upper bound, designed using number of unique words in the corpus, for the weight. Besides the Log weight, Wang et al. [25] developed a supervised, entropy-based term weight called balanced distributional concentration (BDC) weight, originally for text categorization. Yang et al. [23] later applied this term weight to LDA to cope with domain-specific stopwords, which are words appearing across most of the topics and concealing the semantic themes in the topics. For instance, for a text dataset collected using the keyword “Covid”, or in other words, in the “Covid” domain, its domain-specific stopwords, such as “Covid”, “Covid-19”, and “mask”, may appear in most of the topics and conceal the semantically informative words.
In this paper, we leverage both the Log term weight and BDC term weight strategies to significantly improve the performance of topic modeling on the tweets data posted by dementia caregivers that we collected.
Preliminaries
In this section, we will introduce the background information about the topic modeling model GSDMM we used in this study. Table 1 explains the relevant notations used in this paper.
Table 1:
Basic mathematical notations used in the paper
| Notation | Notation Explanation | Notation | Notation Explanation |
|---|---|---|---|
| N vocab | number of unique words in the vocabulary | z | latent topic |
| D | text corpus | set of latent topics | |
| T | total number of documents | d | dth document notation |
| K | total number of topics | each document | |
| W | total number of words in corpus | n | nth word notation in a document |
| N | number of words | w | a word |
| number of documents | k | kth topic | |
| α | hyperparameter α in DMM | ϕ | distribution over words for a topic |
| β | hyperparameter β in DMM | θ | distribution over latent topics |
| m | term weight | ¬d | exclusion of a token of dth document |
| ¬(d, n) | exclusion of a token at position n of dth document |
Gibbs Sampling Dirichlet Multinomial Mixture Model GSDMM contains hyperparameters K for the number of latent topics, α for the distribution over topics for the whole corpus (corpus-topic distribution), and β for the distribution over words for each topic (topic-word distribution/latent topic) [16]. GSDMM generates each document according to the following process [19]:
-
1.
Randomly choose K latent topics, where each latent topic has a topic-word distribution ϕk ~ Dirichlet(β).
-
2.
Randomly choose a distribution over latent topics for the text corpus D, θ ~ Dirichlet(α).
-
3.For the dth (d ∈ {1,…,T}) document with N words in the text corpus where = (wd,1, wd,2, wd,3, …, wd,N):
-
(a)Randomly choose a latent topic zd for the dth document: zd ~ Multinomial(θ).
-
(b)Randomly choose a word wd,n from the chosen latent topic: wd,n ~ Multinomial(ϕzd).
-
(a)
Since GSDMM assumes that each document is sampled from only one semantic topic, it theoretically performs better over short texts with potentially few topics than long texts with multiple themes.
Methods
In this section, we introduce the method used in this study. We propose the ensemble method of Log and BDC term weighting strategies into GSDMM to analyze the topics within the tweets data posted by dementia caregivers.
Topic Models To implement GSDMM, one needs to compute the latent distributions ϕ and θ shown in the generative procedure of GSDMM in the Preliminaries section to sample each word w in each document and the topic assignment z of each document using Gibbs sampling.
In GSDMM, the Gibbs sampling sequentially resamples each topic zd according to the conditional probability [19]:
| (1) |
where D represents the text corpus; α is the hyperparameter for ϕ; β is the hyperparameter for θ; Nvocab is the total number of unique words in the vocabulary of the corpus; z represents a generalized topic; zd represents the topic that generates words in dth document; is the set of all topics excluding the topic zd; is the number of documents generated by topic z without considering dth document; represents the number of occurrence of word w in dth document; Nd is the total number of words in dth document; is the number of occurrence of word w in topic z without considering dth document; N¬d,z is the total number of words in topic z without considering dth document. By repeating the Gibbs sampling process until convergence, we have:
| (2) |
where ϕz,w is the occurrence probability of word w in topic z; is the number of occurrence of word w in topic z; Nz is the total number of words in topic z; θz is the probability that a document is generated by topic z; is the number of documents generated by topic z; T is the total number of documents in the corpus; K is the total number of topics in the corpus. By combining each θz, we can obtain the estimation of θ, which is the distribution over topics for the text corpus. Similarly, by assembling ϕz,w for each word w together, we can obtain the approximation of ϕz, which is the distribution over words for each topic z. In real world, ϕz is the bag of words representation of each topic and can be used for word cloud visualization, word importance analysis, and coherence evaluation.
Term Weighting Strategy Log weight is designed to punish the general words with high occurrence but low information via using the axiom of information theory that an event a’s information content in bit is equal to = − log p(a). To punish the general words and avoid extremely large weight on words with very low occurrence, we applied the modified Log weight mlog for each word w with an upper bound is computed as follows:
| (3) |
where Nvocab is the number of unique words in the corpus; Nw is the total number of word w in corpus; W is the total number of words in corpus. The upper bound is defined as and avoids extremely large weight of word with very low occurrence by assigning a fixed weight of to it.
To punish the domain-specific stopwords appearing across topics, we applied the BDC weight mBDC for each word w that is computed as follows [25]:
| (4) |
where K is the total number of topics in corpus; p(w|k) is the occurrence frequency of each word w in a given topic k; BH(w) is the balanced entropy of word w across topics in the corpus, in which BH(w) ∈ [0, log(K)]. High BH(w) indicates that w is a domain-specific stopword with high occurrence across topics with low importance, and according to Equation 4, BDC weight punishes w by assigning a small weight. Since the BDC weight needs the prior knowledge of p(w|k) and p(w|i), prior topic modeling results are required to compute BDC weight [23]. Therefore, BDC weight is called a supervised weight.
In our study, each unique word in the corpus has a corresponding term weight. To ensure mlog(w) ∈ [0, 1] and mBDC(w) ∈ [0, 1] for each unique word w, we applied the min-max normalization in Equation 5 to the term weights:
| (5) |
where m is the general term weight; mmax is the maximum term weight in the corpus; mmin is the minimum term weight in the corpus; mnormalized is the min-max normalized general term weight. To obtain a term weight that can punish both the general words and domain-specific words simultaneously, we multiplied the Log weight and BDC weight together as shown in Equation 6 to obtain a term weight called mlog -BDC(w) weight. To ensure mlog -BDC(w) ∈ [0, 1], we first multiplied the normalized mlog(w) and mBDC(w) and then further normalized the resultant mlog -BDC(w) according to Equation 5:
| (6) |
Term-Weighted Topic Models As previously discussed, deploying the term weights to topic models like LDA and GSDMM requires altering the corresponding weight of each unique word in the text corpus. This altering on the generative process can be demonstrated directly in the Gibbs sampling equation of GSDMM:
| (7) |
where is the number of occurrence of word v in topic z excluding dth document; m(v) is the general term weight of word v which can be mlog(v), mBDC(v) or the proposed mlog -BDC(v). The altered estimation of ϕz,w is computed as follows:
| (8) |
In real-world implementation, the term-weighted models are achieved by changing the counting of each word w from 1 to m(w).
Materials and Experimental Design
Ethical Considerations The publicly available data used in this study were collected and analyzed in accordance with the Twitter Terms of Service. The Institutional Review Boards of the University of Pennsylvania and Cedars-Sinai Medical Center reviewed this study and deemed it exempt.
Data Collection In prior work [26], we trained a deep neural network classifier, based on a BERT model that was pre-trained on tweets [27], that achieved an F1-score of 0.962 (precision=0.946 and recall=0.979) for identifying Twitter users who reported having a family member with dementia. For the present study, we deployed the classifier on 362,979 tweets (excluding retweets) that were collected from the Twitter Streaming API between May 2021 and March 2023 and mentioned the same keywords as the tweets on which the classifier was evaluated. Among the 362,979 tweets, the classifier identified 231,870 tweets, posted by 124,062 users, that reported having a family member with dementia, which we used for topic modeling.
Data Preprocessing Although the dementia caregiver tweets were carefully collected, the tweets still contain spam tweets and noises. To mitigate the impact of these issues, we implemented a quality control pipeline composed of (1) spam removal, (2) lemmatization, (3) stopword removal, (4) compound word segmentation, and (5) spell check. In (1) spam removal step, the caregiver tweets were first cleaned by removing links, usernames, and non-word entities, lowercasing, and normalizing spacing. Then, the tweets were embedded into a 384 dimensional dense vector space. Then, these tweet embedding vectors were grouped based on usernames, and for each user, a cosine similarity matrix of tweets was computed. The cosine similarity matrix was then averaged to obtain a mean value, which measures how similar the tweets are. A high mean cosine similarity for a user indicates that the tweets by the user are either similar or identical, and vice versa. We set a threshold with value 0.6 to distinguish high and low values. For users with cosine similarity ≥ 0.6 and tweet count ≥ 10, we sampled only one tweet per user to reduce redundancy and noise. For the rest, their tweets were kept. (2) Lemmatization reduces and unifies the forms of each word in the corpus. We applied the WordNetLemmatizer in the NLTK package [28] to achieve the goal. (3) In stopword removal section, we applied the English stopword list from the NLTK packcage to exclude the general words. Then, we manually excluded dementia-related keywords and family-related keywords used for data collection from the corpus to remove domain-specific words. (4) In compound word segmentation, we utilized the wordsegment1* package to break up informal compound words such as “bestthing” and “newworld” into single words “best”, “thing”, “new”, and “world” to prevent incorrect and rare words. In the final step of the quality control process, (5) spell check, we applied the pyspellchecker2* package to correct the misspelled words to further avoid rare words and filter noise. We had 224,862 out of the orginal 231,870 tweets that passed the quality control process.
Baseline Models The dementia caregiver tweets are considered short texts due to the limitation of 280 characters. Owing to the short length, these tweets are semantically sparse and tend to contain fewer semantic topics. The quality control process has filtered out most of the noise in the tweets. However, many general words not included in the English stopword list and domain-specific stopwords not included in the keyword lists remained in the tweets, concealing the semantic themes. To evaluate the performance of our Log-BDC-GSDMM on this short and noisy data, we applied plain LDA, Log-LDA, BDC-LDA, Log-BDC-LDA, plain GSDMM, Log-GSDMM, and BDC-GSDMM as baseline models. Besides the probabilistic models, we also applied three BERTopic variants with different transformer-based embedding methods from Hugging Face, including all-MiniLM-L6-v2 [29], all-mpnet-base-v2 [29], and all-distilroberta-v1 [29], as baseline models to evaluate the performance.
Evaluation Metrics Topic coherence is a measure to evaluate how semantically coherent the words in the bag-of-words topic representation are. In other words, it measures the quality of the topic-word distribution ϕz, (bag of words in real world) of the obtained topics. In our study, we applied a coherence score called Cv [30]. The computation of Cv contains four steps. (1) In a given topic z, the top N most frequent words with the highest occurrence are picked. (2) Then, for the word w in the N top words of topic z, the normalized point-wise mutual information between w and other words in the text corpus is computed to create a vector representation of the word w. This vector encodes information about the concurrence of w and other words in the text. (3) In the topic z, the cosine similarity between and the vector representation of another word u in the top N words of z is computed to evaluate how similar the two words are. (4) After computing the cosine similarity of each pair of words in the top N words of z, a mean cosine similarity is computed from these similarities, and the mean value is taken as the coherence score Cv of the topic z.
Setting For the probabilistic models, we have set both hyperparameters α and β to 0.1. Additionally, we have set the number of iterations to 30 in order to improve computational efficiency. Then, we fed the 224,862 tweets to the probabilistic models and the BERTopic models (11 models in total) with total number of topics K = 5, 10, 15, 20 and random state = 10, 20, 30, 42, which count to 11 × 4 × 4 = 176 experiments, to obtain a comprehensive evaluation of the performance of topic models. We used the K bag-of-words topic representations (topic-word distributions where each is ϕk) from the topic models as the results for analysis of performance and interpretation. For each bag-of-words topic representation, we kept only the top 20 words with the highest occurrence and importance for further analysis. In the performance evaluation, for each model, we computed the coherence score Cv of every topic representation and obtained an average coherence score over topics. We used this average score of a model to represent the overall quality of topic modeling results and therefore performance of the model. Then, for each total number of topics K, we averaged the Cv scores of each model again across the random states to avoid coincidences caused by randomness. We compared the coherence scores to evaluate model performance. For topic interpretation, we ran Log-BDC-GSDMM with K values ranging from 1 to 20, and manually selected the K value that best represented the semantic themes by checking for merged or redundant topics. A merged topic contains a mixture of words from different semantic themes in the top 20 words, while a redundant topic has very similar top 20 words to another topic. After manual selection, we used both manual and automatic interpretation by GPT-4 on our topic results, and compared the consistency of the two approaches.
GPT-4 Prompt Engineering To leverage GPT-4 to interpret the topics, we set up the prompt by assigning GPT-4 a role as an expert in dementia caregiver care proficient in interpreting the hidden theme beneath bag-of-words topic representations. Then, we provided the information about data type, content, and collection method to GPT-4. We also mentioned that the order of words in the topic representation marks the importance rank of words to the topic and provided an example. Next, we informed GPT-4 that all words in a topic are about family caregivers of dementia patients. Finally, we fed the topic representation into GPT-4 for both a short label and a expanded interpretation.
Results and Discussion
The model performance comparison results are shown in Figure 2. According to Figure 2, Log-BDC-GSDMM has the highest Cv coherence score among the models for every total number of topics, which are 0.574, 0.559, 0.554, and 0.528 for K = 5, 10, 15, and 20. This indicates that in all our experiment scenarios, using a combination of term weighting strategies and the GSDMM model on short texts can efficiently uncover latent topics in the tweets. We can notice that, except the plain GSDMM, all term-weighted GSDMM models perform better than term-weighted LDA models for all K. This suggests that GSDMM is more suitable for short-text topic modeling than LDA because of its advantage in assuming each document contains only one theme. The exception, where plain GSDMM performs worse than plain LDA, may be caused by the existence of noise from general and domain-specific words in the tweets.
Figure 2:
Model Performance Evaluation on Dementia Caregiver Tweets. This evaluation compares the Log-BDC-GSDMM with 10 baseline models, including plain GSDMM, Log-GSDMM, BDC-GSDMM, plain LDA, Log-LDA, BDC-LDA, Log-BDC-LDA, BERTopic-all-MiniLM-L6-v2, BERTopic-all-mpnet-base-v2, and BERTopic-all-distilroberta-v1 with total number of topics K = 5, 10, 15, and 20. Each color in the comparison represents one of the topic models. Red represents GSDMM models; blue represents LDA models; green represents BERTopic models. Performance is evaluated using the averaged Cv coherence score of each model, discussed in the Evaluation Metrics subsection.
Regarding Figure 2, we can also notice that all term-weighted GSDMM models perform better than the plain GSDMM, which shows that term weights can effectively improve the performance of GSDMM and indicates that term weight can address the noisy impact caused by general words and domain specific words during topic modeling.
Except for BDC-LDA with K = 5, all other term-weighted LDA models perform better than plain LDA, which further indicates the efficiency of term weighting in removing noise and improving topic modeling quality. For the exception that BDC-LDA performs worse than plain LDA when K = 5, one possible reason is that the underlying assumption of LDA, which posits that documents are mixtures of topics, does not perform well when applied to short tweets, leading to inconsistent results. The same reason may also explain why Log-BDC-LDA does not outperform other LDA models like Log-BDC-GSDMM in each case of K.
All the BERTopic models show poor performance on the caregiver tweets. In the case of K = 15 and K = 20, the BERTopic models perform even worse than plain GSDMM and LDA. One explanation for the poor performance could be that the presence of general words and domain-specific words negatively impacts the document embeddings by the sentence transformers in BERTopic models and obscures the semantic information in tweet embeddings. The obscure embeddings then causes a cascade of poor performance of the HDBSCAN and C-TF-IDF sections in BERTopic, which leads to poor-quality topics and low performance of BERTopic models.
By manually examining the results of the Log-BDC-GSDMM models with different values of K, we selected the model with K=13 to avoid merged or redundant topics. With the 13 bag-of-words topic representations each containing top 20 words, we interpreted each representation using both manual method and GPT-4. For manual interpretation, two of our experts carefully evaluated and interpreted each topic to generate semantic topic labels. For GPT-4 interpretation, we used the prompt described in the prompt engineering section to generate both a concise topic label and an expanded explanation for each topic. Then, we utilized the manual labels to qualitatively check the quality of GPT-4 labels. The manual and GPT-4 labels are displayed in Table 2.
Table 2:
Semantic topic demonstration and interpretation of top 20 words (top 10 words shown here due to page limit).
| Topic ID | Top 10 words in topic | Expert Label | GPT-4 Label |
|---|---|---|---|
| 1 | pay, pension, account, tax, money, attorney, cost, bank, bill, council | Cost of Care | Financial Planning and Healthcare Access |
| 2 | donate, honor, awareness, raise, donation, fundraiser, association, research, charity, support | Charities/Fundraisers | Charitable Support and Awareness |
| 3 | symptom, behavior, research, brain, robin, aphasia, genetic, anger, loss, personality | Dementia Symptoms | Neurological and Behavioral Symptoms |
| 4 | biden, joe, president, trump, jill, stutter, vote, shuffle, gait, behavior | Suspicions of Joe Biden Exhibiting Dementia | Political Figures’ Health and Public Perceptions |
| 5 | sorry, loss, condolence, prayer, peace, hug, horrible, grieve, cruel, lose | Grieving/Condolences | Empathy and Condolences |
| 6 | cancer, covid, raise, pas, birthday, visit, move, home, christmas, ago | Death | Life Events and Challenges |
| 7 | covid, jab, flu, vaccine, blood, vax, booster, kidney, clot, certificate | Comorbidities/Causes of death | Health Risks and Interventions |
| 8 | cream, ice, cook, eat, recipe, chocolate, food, cake, coffee, play | Cooking/Eating | Culinary Enjoyment and Activities |
| 9 | sing, song, music, play, movie, beautiful, smile, piano, bennett, lyric | Entertainment | Emotional Engagement and Memory |
| 10 | na, remember, gon, phone, name, forget, wan, call, shit, visit | Emotional/Mental Health Impact on Caregivers | Memory and Communication Challenges |
| 11 | sleep, door, bed, night, walk, room, bathroom, clothe, police, house | Loss of Activities of Daily Living | Safety and Nighttime Challenges |
| 12 | support, job, carers, pay, social, help, home, staff, care, work | Healthcare Facilities | Caregiving Support and Challenges |
| 13 | mask, covid, hospital, vaccinate, lockdown, visitor, rule, visit, allow, ambulance | Covid-19 Pandemic | COVID-19 Pandemic Measures |
Table 2 also contains the first 10 words of the top 20 words in each topic representation for efficiency of presentation. Based on our observation, the GPT-4 labels are generally consistent with the expert labels but more comprehensive, which demonstrates the powerful semantic interpretation ability of the GPT-4 model. According to the expanded interpretations of topics by GPT-4, (1) topic 1 highlights the intricacies of managing costs, legal matters, insurance, and engaging with social and community resources to support the well-being of dementia patients; (2) topic 2 emphasizes community involvement, research support, and advocacy efforts, symbolized by activities like walks and donations, aiming for advancements in cure research and enhancing societal support for affected individuals and caregivers; (3) topic 3 focuses on the cognitive and behavioral changes due to dementia and highlights the importance of understanding genetic factors and neurological impacts; (4) topic 4 suggests political discourse surrounding President Biden, touching on personal attributes (like stuttering and gait), behavioral observations, and partisan perspectives, which reflects public and political scrutiny of leadership qualities, health speculation, and the polarized nature of political debate; (5) topic 5 expresses the sympathy and support for those grieving over dementia-related losses, emphasizing emotional solidarity and comfort; (6) topic 6 focuses on chronic illness impacts life milestones and daily living, highlighting the emotional and physical challenges of caregiving; (7) topic 7 emphasizes health crises and medical treatments impact dementia care, emphasizing the importance of vaccinations and managing comorbidities; (8) topic 8 is about the joyful moments in caregiving through cooking, eating, and sharing meals, highlighting the importance of pleasure and nourishment; (9) topic 9 focuses on how music and movies evoke memories and emotions, offering comfort and connection to dementia patients and their caregivers; (10) the theme of topic 10 involves communicating and sharing moments, navigating memory challenges, and the emotional ups and downs mark the dementia caregiving experience; (11) topic 11 involves nighttime wandering, safety concerns, and emergency situations, highlighting the complex care needs of dementia patients; (12) topic 12 mainly discusses caregiving’s emotional, financial, and social challenges, necessitating support, respite, and appreciation within healthcare and community systems; (13) topic 13 is mainly about pandemic protocols impacting hospital visits and care, emphasizing vaccination, testing, and personal protective measures. These interpretations provide us with a picture of the experiences and concerns the dementia caregivers experienced while caring for their family members with dementia.
Consistent with the findings from previous studies [4, 31], our interpretation revealed the presence of financial and social dilemmas faced by the family caregivers, as indicated by terms like “bill”, “donate”, “social”, and “job” in topics 1, 2, and 12. These challenges align with existing research [31, 32] on financial strain and social isolation experienced by family caregivers. In the meantime, we also identified the emotional distress among family caregivers indicated by “grieve”, “sorry”, “horrible”, and “cruel” due to losses of their family members and condolences from topic 5. This finding is consistent with the literature [7] on the psychological strain of dementia family caregivers. Moreover, topics 3, 6, 7, and 13 suggest the caregivers’ heightened concerns about the physical health of their family members with dementia and their own physical health during the pandemic, which aligns well with the results of previous literature [33]. These findings demonstrate the efficiency of our Twitter data and our topic modeling method using Log-BDC-GSDMM and GPT-4 in uncovering the experiences and concerns of dementia family caregiver population.
Conclusion
In this study, we proposed an ensemble of two term-weighting strategies to enhance the Dirichlet Multinomial Mixture Model for more precise topic identification. Our methodology, supported by the analytical capabilities of ChatGPT 4 and expert human interpretation, successfully uncovered the nuanced challenges faced by dementia caregivers, offering clearer and more relevant insights than the current state-of-the-art BERTopic models. This work not only advances the field of natural language processing and topic modeling but also provides healthcare professionals and support organizations with critical insights to better address the needs of caregivers affected by dementia, demonstrating a significant step forward in understanding and supporting dementia caregiving. A limitation of this work is the fixed hyperparameters, α and β, and number of iterations. The current model is not optimal, and the topic results can be further improved by hyperparameter tuning in the future. One possible future work is to incorporate interpretability [6, 34] and fairness [35, 36, 37] into our framework to come up with a more trustworthy topic modeling system.
Acknowledgments
This work was supported in part by the NIH grants R01 LM011176, P30 AG073105, U01 AG068057, U01 AG066833 and S10 OD023495.
Footnotes
Figures & Tables
References
- 1.Cheng ST. Dementia caregiver burden: a research update and critical analysis. Current psychiatry reports. 2017;19:1–8. doi: 10.1007/s11920-017-0818-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.World Health Organization. Available from: https://www.who.int/news-room/fact-sheets/detail/dementia.
- 3.Prince M, Wimo A, Guerchet M, Ali GC, Wu YT, Prina M. The global impact of dementia: an analysis of prevalence, incidence, cost and trends. Alzheimer’s disease international; 2015. World Alzheimer report 2015. [Google Scholar]
- 4.Association A. 2023 Alzheimer’s disease facts and figures. Alzheimer’s & Dementia. 2023;19(4):1598–695. doi: 10.1002/alz.13016. [DOI] [PubMed] [Google Scholar]
- 5.Kasper JD, Freedman VA, Spillman BC, Wolff JL. The disproportionate impact of dementia on family and unpaid caregiving to older adults. Health Affairs. 2015;34(10):1642–9. doi: 10.1377/hlthaff.2015.0536. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.He W, Hou B, Demiris G, Shen L. Interpretability study for long interview transcripts from behavior intervention sessions for family caregivers of dementia patients. AMIA Jt Summits Transl Sci Proc. 2024;2024:201–10. [PMC free article] [PubMed] [Google Scholar]
- 7.Pinquart M, S¨orensen S. Differences between caregivers and noncaregivers in psychological health and physical health: a meta-analysis. Psychology and aging. 2003;18(2):250. doi: 10.1037/0882-7974.18.2.250. [DOI] [PubMed] [Google Scholar]
- 8.Sallim AB, Sayampanathan AA, Cuttilan A, Ho RCM. Prevalence of mental health disorders among caregivers of patients with Alzheimer disease. Journal of the American Medical Directors Association. 2015;16(12):1034–41. doi: 10.1016/j.jamda.2015.09.007. [DOI] [PubMed] [Google Scholar]
- 9.Thunyadee C, Sitthimongkol Y, Sangon S, Chai-Aroon T, Hegadoren KM. Predictors of depressive symptoms and physical health in caregivers of individuals with schizophrenia. Nursing & Health Sciences. 2015;17(4):412–9. doi: 10.1111/nhs.12205. [DOI] [PubMed] [Google Scholar]
- 10.Patterson TL, Grant I. Interventions for caregiving in dementia: physical outcomes. Current Opinion in Psychiatry. 2003;16(6):629–33. [Google Scholar]
- 11.Grootendorst M. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. arXiv preprint arXiv:220305794. 2022.
- 12.Pancheva A, Wheadon H, Rogers S, Otto TD. Using topic modeling to detect cellular crosstalk in scRNA-seq. PLOS Computational Biology. 2022;18(4):e1009975. doi: 10.1371/journal.pcbi.1009975. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Meng Y, Speier W, Ong M, Arnold CW. HCET: hierarchical clinical embedding with topic modeling on electronic health records for predicting future depression. IEEE J Biomed Health Inform. 2020;25(4):1265–72. doi: 10.1109/JBHI.2020.3004072. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Girdhar Y, Giguere P, Dudek G. Experimental Robotics: The 13th Int. Symposium on Experimental Robotics. Springer; 2013. Autonomous adaptive underwater exploration using online topic modeling; pp. p. 789–802. [Google Scholar]
- 15.Hendel A, Weinshall D, Peleg S. Asian Conference on Computer Vision. Springer; 2010. Identifying surprising events in videos using bayesian topic models; pp. p. 448–59. [Google Scholar]
- 16.Blei DM, Ng AY, Jordan MI. Latent dirichlet allocation. Journal of machine Learning research. 2003 Jan;3:993–1022. [Google Scholar]
- 17.Nigam K, McCallum AK, Thrun S, Mitchell T. Text classification from labeled and unlabeled documents using EM. Machine learning. 2000;39:103–34. [Google Scholar]
- 18.Geman S, Geman D. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on pattern analysis and machine intelligence. 1984;(6):721–41. doi: 10.1109/tpami.1984.4767596. [DOI] [PubMed] [Google Scholar]
- 19.Yin J, Wang J. A dirichlet multinomial mixture model-based approach for short text clustering. Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 2014. pp. p. 233–42.
- 20.McInnes L, Healy J, Melville J. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:180203426. 2018.
- 21.McInnes L, Healy J, Astels S, et al. hdbscan: Hierarchical density based clustering. J Open Source Softw. 2017;2(11):205. [Google Scholar]
- 22.Wilson A, Chew PA. Term weighting schemes for latent dirichlet allocation. human language technologies: The 2010 annual conference of the North American Chapter of the Association for Computational Linguistics. 2010. pp. p. 465–73.
- 23.Yang K, Cai Y, Chen Z, Leung Hf, Lau R. Exploring topic discriminating power of words in latent dirichlet allocation. Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. 2016. pp. p. 2238–47.
- 24.Li X, Zhang A, Li C, Ouyang J, Cai Y. Exploring coherent topics by topic modeling with term weighting. Information Processing & Management. 2018;54(6):1345–58. [Google Scholar]
- 25.Wang T, Cai Y, Leung Hf, Cai Z, Min H. 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI) IEEE; 2015. Entropy-based term weighting schemes for text categorization in VSM; pp. p. 325–32. [Google Scholar]
- 26.Klein AZ, Magge A, O’Connor K, Gonzalez-Hernandez G. Automatically identifying Twitter users for interventions to support dementia family caregivers: annotated data set and benchmark classification models. JMIR Aging. 2022;5(3):e39547. doi: 10.2196/39547. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Nguyen DQ, Vu T, Tuan Nguyen A. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics; 2020. BERTweet: a pre-trained language model for English Tweets; pp. p. 9–14. [Google Scholar]
- 28.Bird S, Klein E, Loper E. “ O’Reilly Media, Inc.”; 2009. Natural language processing with Python: analyzing text with the natural language toolkit. [Google Scholar]
- 29.Reimers N, Gurevych I. Proc. of Conf. on Empirical Methods in Natural Language Processing. Association for Computational Linguistics; 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. [Google Scholar]
- 30.Syed S, Spruit M. 2017 IEEE International conference on data science and advanced analytics (DSAA) Ieee; 2017. Full-text or abstract? examining topic coherence scores using latent dirichlet allocation; pp. p. 165–74. [Google Scholar]
- 31.Brodaty H, Donkin M. Family caregivers of people with dementia. Dialogues in clinical neuroscience. 2009;11(2):217–28. doi: 10.31887/DCNS.2009.11.2/hbrodaty. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Tu J, Li H, Ye B, Liao J. The trajectory of family caregiving for older adults with dementia: difficulties and challenges. Age and ageing. 2022;51(12):afac254. doi: 10.1093/ageing/afac254. [DOI] [PubMed] [Google Scholar]
- 33.Sunmoo Y, Broadwell P, Alcantara C, Davis N, Haeyoung L, Bristol A, et al. Analyzing topics and sentiments from Twitter to gain insights to refine interventions for family caregivers of persons with Alzheimer’s disease and related dementias (ADRD) during COVID-19 pandemic. Studies in health technology and informatics. 2022;289:170. doi: 10.3233/SHTI210886. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Hou BJ, Zhou ZH. Learning with interpretable structure from gated RNN. IEEE transactions on neural networks and learning systems. 2020;31(7):2267–79. doi: 10.1109/TNNLS.2020.2967051. [DOI] [PubMed] [Google Scholar]
- 35.Donini M, Oneto L, Ben-David S, Shawe-Taylor JS, Pontil M. Empirical risk minimization under fairness constraints. Advances in neural information processing systems. 2018. p. 31.
- 36.Tarzanagh DA, Hou B, Tong B, Long Q, Shen L. Uncertainty in Artificial Intelligence. PMLR; 2023. Fairness-aware class imbalanced learning on multiple subgroups; pp. p. 2123–33. [PMC free article] [PubMed] [Google Scholar]
- 37.Hou B, Mondrag´on A, Tarzanagh DA, Zhou Z, Saykin AJ, Moore JH, et al. PFERM: A Fair Empirical Risk Minimization Approach with Prior Knowledge. AMIA Summits on Translational Science Proceedings. 2024;2024:211. [PMC free article] [PubMed] [Google Scholar]


