Abstract
INTRODUCTION
Behavioral and psychological symptoms in dementia (BPSD) are dynamic phenomena with a high amount of intraindividual variability. We applied a multilevel framework to identify subsyndromes (between‐person factors) that represent clinically relevant profiles of BPSD and identify symptom clusters (within‐person factors) that represent contextually driven daily symptom experiences.
METHODS
This study used an intensive longitudinal design in which 68 co‐residing family caregivers to persons living with dementia were recruited to proxy report on their care recipient's daily symptom experiences of 23 different BPSD for eight consecutive days (n = 443 diaries). A multilevel exploratory/confirmatory factor analysis was used to account for nested data and separate within‐person variances from between‐level factor estimates.
RESULTS
Exploratory factor analysis identified a 4‐between 3‐within factor structure based on fit statistics and clinical interpretability.
DISCUSSION
This study offers major methodological and conceptual advancements for management of BPSD within Alzheimer's disease and related dementias by introducing two related but distinct concepts of subsyndromes and symptom clusters.
Highlights
Because behavioral and psychological symptoms of dementia (BPSD) are dynamic temporal phenomenon, this introduces measurement error into aggregate group‐level estimates when trying to create subsyndromes. We propose a multilevel analysis to provide a more valid and reliable estimation by separating out variance due to within‐person daily fluctuations.
Using a multilevel exploratory factor analysis with intensive longitudinal data, we identified distinct and meaningful groups of BPSD. The four factors at the between‐person level represented subsyndromes that are based on how BPSD co‐occurred among persons with Alzheimer's disease (AD). These subsyndromes are clinically relevant because they share features of established clinical phenomena and may have similar neurobiological etiologies.
We also found three within‐person factors representing distinct symptom clusters. They are based on how BPSD clustered together on a given day for an individual with AD and related dementias. These clusters may have shared environmental triggers.
Keywords: behavioral symptoms, multilevel analyses, neuropsychiatric symptoms, subsyndromes, symptom cluster, symptom management
1. BACKGROUND
Most patients with Alzheimer's disease and related dementias (ADRD) experience behavioral and psychological symptoms of dementia (BPSD), though most do not experience all types of BPSD or experience them in a predictable pattern. 1 As reported by family caregivers, the most prevalent BPSDs include apathy, depression, and agitation, while the most distressing BPSDs include delusions, agitation, and irritability. 2 As BPSD represents a heterogeneous group of clinical symptoms, there has been an effort to identify subsyndromes based on co‐occurrence or clustering of symptoms. The hypothesis is that these subsyndromes may represent clinically and mechanistically distinct phenomena. 3 , 4 Thus, the identification of subsyndromes would inform clinical management and pharmaceutical development.
More than 60 studies have attempted to identify subsyndromes of BPSD, primarily through factor analysis of cross‐sectional data. 5 Factor analysis creates groupings of interrelated variables based on the hypothesis that they share underlying mechanistic properties. 6 Among these studies, there has been some consistency in terms of the number of subsyndromes (three to four) and overall types of subsyndromes (affective, hyperactivity, psychosis, euphoria). 7 , 8 However, multiple systematic reviews have demonstrated that no two studies have identified the same factor structure or item loadings underlying the subsyndromes and that subsyndromes do not appear stable over time. 5 , 9 , 10
The lack of reproducibility of subsyndromes may be related to measurement issues. BPSD is a dynamic temporal phenomenon with a high degree of intra‐individual variability, which is masked when relying on aggregate group‐level estimates. 11 , 12 , 13 A seminal study demonstrated that for a group of persons with dementia, the average occurrence of BPSD appeared unchanged over a 3‐month period. However, individual rates of change were significantly different from the group's trend. This means that on an individual level, BPSD is not stable over short periods of time. 13 Furthermore, environmental stressors (e.g., caregivers shouting aggressively) can significantly increase the number of different types of BPSD that a person with dementia experiences on a given day as well as the next day. 11 Therefore, there are strong within‐person processes underlying BPSD that need to be accounted for in group‐level estimates. Not accounting for the individual variability in BPSD introduces error into the estimation of subsyndromes. 14
Within a multilevel factor analysis framework, the factors produced at each level have a different interpretation with distinct implications for clinical research and practice. At the between‐person level, the factors represent how an individual's BPSD co‐occurs compared to other people with ADRD. This represents the traditional subsyndrome concept or a group of symptoms related through a shared neurobiological etiology, 5 such as gray matter volume atrophy. 15 In other words, subsyndromes represent a profile of symptoms that stands out among all the BPSD that a person with dementia may experience. At the within‐person level, the factors represent how an individual's daily BPSD co‐occurs relative to their own usual BPSD. This represents a symptom cluster, or a group of temporally related fluctuating symptoms that may share environmental triggers but do not necessarily share the same neurobiological etiology.
Subsyndromes would be a target for pharmaceutical development and inform medical management, especially as they may be predictive of different health outcomes. 5 Symptom clusters would be a target for environmental and caregiver‐based interventions and inform daily care strategies. This means that for a single BPSD, there may be multiple avenues for intervention.
A review of published subsyndrome analyses questioned whether “statistically derived symptom groups are meaningful in everyday terms.” 5 Accordingly, the purpose of this paper is to use an innovative approach of intensive longitudinal data collected via daily diary reports of symptoms and a multilevel analytic framework to disaggregate the between‐person and within‐person effects driving BPSD. We argue that this approach can (1) identify subsyndromes (between‐person factors) that represent clinically relevant profiles of BPSD, and (2) identify symptom clusters (within‐person factors) that represent contextually driven daily symptom experiences.
2. METHODS
2.1. Study design and procedures
This study uses an intensive longitudinal design and caregiver‐reported data from a nightly diary survey over 8 days. The diary captures the occurrence, context, and timing of 23 BPSD. RedCap, a secure, web‐based survey platform, was used for data collection. After passing eligibility and providing consent, participants received a link to the baseline survey via e‐mail and text. After completion of the baseline survey, participants received daily e‐mails each evening at 8 pm with a link to the nightly diary. They were asked to complete it within a 4‐hour window (8 pm to midnight).
2.2. Study sample
Participants included persons aged ≥ 18 years, co‐residing with a person with mild cognitive impairment or dementia who spends at least 12 hours a day providing care. The AD8 was used as a proxy measure to determine mild cognitive impairment or dementia. 16 The AD8 is a valid and reliable tool for caregiver proxy report that has strong concurrent validity with the Clinical Dementia Rating scale, good inter‐rater reliability and stability, and excellent discrimination between persons with and without cognitive impairment regardless of etiology (area under the curve 0.92, 95% confidence interval 0.88–0.95). 16 Participants were excluded if they did not have reliable access to the internet or the care recipient had a score < 3 on the AD8. The most common reason for ineligibility was that the caregiver did not live with the person with dementia.
Regional social and news media (the southeastern United States), in addition to snowball sampling, were used for recruitment. Advertising materials directed interested persons to the website, where they were provided a description of the study and a link to an initial screening form. From there, a rigorous three‐step process was used to assess the authenticity of participants (i.e., that they are not fraudulent persons or spam‐bots). The initial online screening form was used to exclude potential participants based on invalid answers, including their location. This step also prevented spam‐bots from accessing the data collection platform, RedCap. Participants who passed this step were then sent a link to an eligibility survey in RedCap. Those who passed the eligibility survey were sent an e‐mail with a link to the consent form. After clicking “yes” to indicate their consent in RedCap, participants were notified via e‐mail of a second‐step verification survey. These answers were checked for agreement with the answers on the initial eligibility survey and verified the authenticity of participants by requiring endorsements of two images with the correct answer and a coherent response to the question, “Tell us in a few words why you are interested in participating in this study.” A multistep protocol for recruiting and enrolling participants is recommended as a best practice in online research methods. 17 , 18
RESEARCH IN CONTEXT
Systematic review: Many studies have identified potential subsyndromes based on groupings of behavioral symptoms. However, these findings have not been reproducible. This may be due to measurement error that does not account for the dependencies in the data.
Interpretation: We propose a new multilevel framework for conceptualizing groupings of behavioral symptoms. Subsyndromes represent how a person's symptoms occur in relation to others with Alzheimer's disease and related dementias. Symptom clusters represent how a person's daily symptoms co‐occur relative to their own usual symptoms. Findings identified four subsyndromes and three symptom clusters.
Future directions: Understanding the dual but related concepts of subsyndromes and symptom clusters creates a new paradigm for symptom management. Future work should investigate whether subsyndromes have a shared neurobiological basis and whether symptom clusters are environmentally and contextually driven.
2.3. Measures
For consistency with previous research on subsyndromes, the Neuropsychiatric Inventory (NPI) was the primary measure used to identify different domains of BPSD. 19 It was supplemented with other scales for BPSD commonly used in daily diary research and items identified as integral through previous pilot work. These items were generated from data collected for a pilot study about BPSD and caregiving outcomes. 20 In the pilot study, caregiver participants were asked to respond to diary surveys twice a day for 21 days and had the option to write in BPSD on the daily diaries that were not captured completely by the NPI. Two geriatric nurse scientists reviewed the open‐ended answers to identify any conceptually unique symptoms that were mentioned consistently across the days sampled and among participants sampled. Because these were open‐ended responses, participants provided detailed anecdotes about the symptoms, which provided context to evaluate conceptual distinctions between their written‐in answers and existing NPI items.
Based on this review, three distinct symptoms were identified that had a high degree of content validity based on clinical experiences. First, compulsive behaviors were identified, particularly behaviors related to ordering and completeness. While the NPI includes repetitive behaviors, compulsions are clinically distinct. Because autism is a neurological condition with many behavioral symptoms that are often proxy reported by caregivers, we included an item on compulsions from the Repetitive Behaviors Scale—Revised rather than generating a new item. 21 Second, vocalizations were also common, and, while not measured on the NPI, have been identified as a frequent BPSD among residents in nursing homes. 22 Thus, a researcher‐generated item was included. Third, toileting issues were also frequently mentioned. Specifically, this does not refer to general incontinence; rather, these are situations in which the person with dementia mistakenly relieves themself somewhere besides a toilet (e.g., a trashcan). All items are found in Table S1 in supporting information.
2.3.1. Neuropsychiatric Inventory
A self‐report scale for dementia family caregivers measuring BPSD 19 included screening items measuring the domains of hallucinations, delusions, depression, anxiety, elation, apathy, disinhibition (split into socially inappropriate disinhibition and sexual disinhibition), irritability/lability, and eating difficulties (10 items total).
2.3.2. Daily record of behavior
A self‐report scale for dementia family caregivers measuring BPSD on a daily basis 2 included items measuring the domains of agitation–care–resistance (6 items), agitation–restlessness (1 item), verbal aggression (1 item), physical aggression (1 item), and memory problems (1 item).
2.3.3. Repetitive behaviors scale revised—Compulsive behaviors subscale
This scale measures parental self‐report of autistic children's compulsive behaviors. Though it has not been used in dementia research, our pilot work provides high content validity for the domain of aberrant motor behavior–compulsions. Subitems were aggregated to create a 1‐item measure representing ordering, completeness, checking, hoarding, and tapping. 21
2.3.4. Researcher‐generated items
To measure the domain of aberrant motor behavior–non‐verbal vocalizations, we created an item based on the Typology of Disruptive Vocalizations. 22 Based on pilot work, we also included an item on toileting difficulties (e.g., using things other than the toilet to relieve themselves).
2.4. Analysis
The purpose of this study was to identify subsyndromes (between level) and symptom clusters (within level) by clustering 23 BPSD indicator items collected with daily diaries over eight sequential days. Traditional data reduction methods, such as exploratory factor analysis, were not appropriate due to multiple daily diaries from each participant. Therefore, a multilevel exploratory factor analysis (MLEFA) approach was used. MLEFA partitions the variance–covariance matrix at the within‐person and the between‐person level components and then computes factor extractions at each level. 23 Using MLEFA, one can explore the dimensionality of an instrument when the data have a multilevel structure at each specific level. Because this dataset is longitudinal in nature, the within level represents the day‐to‐day fluctuations within a person while the between level represents interpersonal differences. A multilevel model is justified by an intraclass correlation coefficient (ICC), the ratio of the between‐level variance to the total variance of an item. A rule of thumb is to consider a multilevel approach when the ICC is > 0.05. 24 More recent research suggests using an ICC > 0.20. 25 The ICCs of BPSD items in this study ranged from 0.16 to 0.53 with a mean of 0.27. Therefore, the MLEFA approach was justified.
Multiple steps were involved in determining the optimal number of factors at each level. First, we examined the models that varied the numbers of within‐level factors while between‐level factors were held constant. Once the number of within‐level factors was determined, we examined models with varied numbers of between‐level factors. Both levels included a maximum of five factors. Five was selected as the maximum number of factors because factor solutions six and above had single indicator factors that were not theoretically or clinically meaningful. This may have indicated there was a factor only because the model was forced to produce one. Models selected at within and between levels were merged to create the final model.
MLEFA model parameters were estimated using maximum likelihood estimation with an expectation maximization algorithm. The Geomin option in Mplus as an oblique rotation was used. Oblique rotation was selected because, theoretically, all the factors were assumed to correlate. We chose candidate models using comparative fit index (CFI), 26 Tucker–Lewis fit index (TLI), 27 Akaike information criteria (AIC), 28 Bayesian information criteria (BIC), 29 and sample size adjusted BIC (aBIC). 30 In addition, root mean square error of approximation (RMSEA), 31 standardized root mean square residual within (SRMR‐within), 32 and SRMR‐between were used to judge the fit of the final model to the data. The data analysis was performed on Mplus 8.8. 33
3. RESULTS
3.1. Study participants and descriptive statistics
The sample of caregiver participants (N = 68) was, on average, 69% female, 42% White, and 54% Black, with an average age of 42 years (Table 1). Care recipients with dementia were 54% female, with an average age of 74 years. In total, 68 participants completed 443 daily diaries over the follow‐up period. Participants did not fill out some of their diaries (18.43% were missing). As a result, the average number of diaries per individual was 6.51 (standard deviation 1.84) out of eight expected. The dataset was screened for missed or skipped questions in the diaries. We found no item‐level missing data. Table 2 summarizes the frequencies of the BPSD items along with the ICCs. There were no differences in the frequency of total BPSDs by race or ethnicity. ICCs indicated a substantial amount of item variance at both within‐ and between‐person levels. Table S1 shows one item on care resistance. While it was originally measured as five separate items, their sum was used to improve the MLEFA model fit as we found that these items measured a related concept and performed similarly when included as individual items.
TABLE 1.
Sample demographic characteristics.
| Caregiver characteristics | %N | M (SD) |
|---|---|---|
| Age | 42.07 (15.58) | |
| Female | 69% (48) | |
| Race | ||
| White | 42% (29) | |
| Black | 54% (37) | |
| Hispanic/Latino | 29% (20) | |
| Relationship type | ||
| Spousal | 14% (10) | |
| Adult child/parent | 62% (43) | |
| Adult grandchild/grandparent | 19% (13) | |
| Care recipient characteristics | ||
| Age | 74.46 (8.81) | |
| Female | 54% (37) | |
| AD8 | 7.70 (0.71) | |
| Household characteristics | ||
| Number of persons living in house | 3.68 (1.57) | |
| Average number hours dyad spend together each day | 19.7 (4.29) |
Abbreviation: SD, standard deviation.
TABLE 2.
Items, frequencies, ICC, and final model loadings.
| Within‐F2 | Between‐F1 | Between‐F2 | Between‐F3 | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Behavioral Item | Days present N/% | ICC |
Within‐F1 Behavioral |
Psychological–mood |
Within‐F3 Psychosis‐like |
Aggressive–agitation | Depression–affective | Manic–agitation |
Between‐F4 Hallucinations |
| Eating difficulties | 53 (12%) | 0.188 | 0.20 * | −0.04 | 0.07 | −0.08 | −0.12 | 0.13 | 0.1 |
| Uncooperative | 97 (22%) | 0.269 | 0.28 * | −0.02 | 0.02 | 0.16 | −0.09 | 0.12 | 0.36 * |
| Toileting mishaps | 45 (10%) | 0.287 | 0.22 * | −0.1 | 0.12 | −0.01 | 0.07 | −0.09 | −0.2 |
| Verbal aggression | 73 (16%) | 0.317 | 0.28 * | 0.16 | 0.01 | 1.14 * | 0.15 | 0.03 | 0 |
| Physical aggression | 24 (5%) | 0.166 | 0.52 * | −0.05 | −0.04 | 0.64 * | 0.21 | 0.03 | 0.03 |
| Memory‐related symptoms | 167 (38%) | 0.379 | 0.09 | 0.03 | 0.16 | 0.04 | 0.2 | 0.33 * | −0.01 |
| Delusions | 54 (12%) | 0.369 | 0.22 * | −0.06 | 0.28 * | 0.09 | 0.87 * | 0.02 | 0.34 |
| Hallucinations | 44 (10%) | 0.247 | 0.17 | 0.13 | 0.33 * | 0.02 | 0.63 * | −0.01 | 0.91 * |
| Depression | 63 (14%) | 0.203 | −0.01 | 0.76 * | 0 | 0.05 | 1.01 * | −0.15 | −0.08 |
| Anxiety | 47 (11%) | 0.163 | 0.13 | 0.38 * | −0.02 | −0.25 | 0.44 * | 0.39 | −0.2 |
| Euphoria | 30 (7%) | 0.23 | 0.17 * | 0.08 | −0.1 | −0.37 * | 0.05 | 0.41 * | 0.19 |
| Apathy | 48 (11%) | 0.216 | −0.02 | 0.11 | 0.42 * | −0.04 | 0.75 * | 0.07 | 0.06 |
| Impulsivity | 35 (8%) | 0.336 | 0.20 * | 0.16 | 0.01 | 0.05 | −0.11 | 1.05 * | −0.03 |
| Sexual disinhibition | 16 (4%) | 0.289 | 0.34 * | 0.02 | −0.1 | 0.05 | 0.06 | 0.83 * | −0.52 * |
| Mood lability | 51 (12%) | 0.217 | 0.19 * | 0.23 * | 0.03 | 0.18 | −0.03 | 0.84 * | 0.2 |
| Compulsions | 67 (15%) | 0.349 | 0.13 | 0.02 | 0.13 | −0.09 | 0.27 | 0.67 * | 0.29 |
| Wandering | 77 (17%) | 0.21 | 0 | 0.15 | 0.22 * | 0.36 * | −0.17 | 0.26 | −0.09 |
| Vocalizations | 57 (13%) | 0.271 | 0.41 * | 0.01 | −0.25 | −0.31 * | 0.64 * | 0.16 | −0.02 |
| Care resistance ** | 429 (19%) | 0.534 | 0.20 * | −0.13 | 0.07 | 0.46 * | 0.04 | 0.09 | 0 |
Abbreviation: ICC, intraclass correlation.
p ≤ 0.05.
Original items were summed because they measure very similar content.
3.2. Model selection
The decision on the final number of factors for within‐ and between‐level models was guided by Ji et al. 25 and the clinical interpretability of item loadings. TLI, changes in CFI, and BIC were the primary statistics used to determine the number of factors at the within level. For between‐level models, changes in CFI and AIC were used. Table 2 displays the factor solutions for the models. It illustrates that as the number of factors increased, the fit of the models to the data improved. For the within‐level only, one‐ and two‐factor solutions had low TLI, whereas models with more than three factors failed to converge. Therefore, a three‐factor solution was selected. For the between‐level‐only models, low AIC and high CFI favored four‐ and five‐factor solutions. A four‐factor solution was selected for the final model because the fifth factor had no significant item loadings.
3.3. Final model
The final model had four between‐level and three within‐level factors, with final model loadings shown in Table 2. The model fit was acceptable to good based on CFI and TLI and very good based on RMSEA (CFI = 0.89, TLI = 0.84, AIC = 4194.45, BIC = 4935.39, aBIC = 4360.98, RMSEA = 0.025, SRMR‐within = 0.04, SRMR‐between = 0.094). Inter‐factor correlations (Table 3) were small to moderate for both within‐level (0.31–0.44) and between‐level (0.23–0.89) models, indicating that factors measured unique constructs across levels. The inter‐factor correlations of the model are summarized in Table 4, and correlations between the factors and items are in Table S1.
TABLE 3.
Within‐ and between‐level model selection statistics.
| Number of factors within model * | Model fit statistics | |||||||
|---|---|---|---|---|---|---|---|---|
| Within level | Between level | CFI | TLI | AIC | BIC | aBIC | SRMR.W | SRMR.B |
| 1 | 0 | 0.81 | 0.58 | 3931.722 | 4165.055 | 3984.163 | 0.06 | 0 |
| 2 | 0 | 0.90 | 0.74 | 3896.854 | 4203.871 | 3965.855 | 0.05 | 0 |
| 3 | 0 | 0.94 | 0.81 | 3891.77 | 4268.378 | 3976.412 | 0.04 | 0 |
| 0 | 1 | 0.70 | 0.32 | 3999.902 | 4233.235 | 4052.343 | 0 | 0.16 |
| 0 | 2 | 0.80 | 0.50 | 3954.874 | 4261.892 | 4023.876 | 0 | 0.14 |
| 0 | 3 | 0.90 | 0.70 | 3914.395 | 4291.003 | 3999.037 | 0 | 0.11 |
| 0 | 4 | 0.95 | 0.84 | 3897.557 | 4339.662 | 3996.919 | 0 | 0.10 |
| 0 | 5 | 0.97 | 0.87 | 3904.093 | 4407.602 | 4017.256 | 0 | 0.08 |
Abbreviations: aBIC, sample size–adjusted Bayesian information criterion; AIC, Akaike information criterion; BIC, Bayesian information criterion; CFI, comparative fit index; SRMR.B, standardized root mean square residual between; SRMR.W, standardized root mean square residual within; TLI, Tucker–Lewis index.
0 indicates that the level was saturated (held constant).
TABLE 4.
Inter‐factor correlations for each level.
| Between‐F1 | Between‐F2 | Between‐F3 | Between‐F4 | |
|---|---|---|---|---|
| Within‐F1 | 1 | 0.12 | 0.27 | 0.11 |
| Within‐F2 | 0.05 | 1 | 0.40 * | –0.11 |
| Within‐F3 | 0.16 | –0.08 | 1 | 0.11 |
Note: The correlations below the diagonal are the within‐level inter‐factor correlations, the above the diagonal correlations are the between‐level inter‐factor correlations.
Statistically significant correlation estimate.
The resulting factors at the between‐person level represented four subsyndromes that were comprised of specific BPSD items: aggressive–agitation (wandering, care resistance, verbal aggression, physical aggression), depression–affective (vocalizations, apathy, delusions, hallucinations, anxiety, depression), manic–agitation (memory problems, euphoria, lability, impulsivity, compulsions, sexual disinhibition), and hallucinations (hallucinations, uncooperativeness). If items were cross‐loaded and one of the loadings was negative, then the item was counted on the factor with the positive loading. Eating and toileting difficulties did not perform well at the between‐person level.
The factors at the within‐person level represented three symptom clusters: behavioral symptoms (eating difficulties, toileting difficulties, uncooperativeness, verbal/physical aggression, impulsivity, sexual disinhibition, care resistance, euphoria, vocalizations, delusions, lability), psychological–mood symptoms (depression, anxiety, lability), and psychosis‐like symptoms (delusions, hallucinations, apathy, wandering). Memory problems and compulsive behaviors did not load on any within‐person factor.
4. DISCUSSION
Using a multilevel exploratory factor analysis with intensive longitudinal data, we identified distinct and meaningful groups of BPSD. The four factors at the between‐person level represented subsyndromes that are based on how BPSD co‐occurred among persons with AD. These subsyndromes are clinically relevant because they share features of established clinical phenomena and may have similar neurobiological etiologies.
The depression–affective subsyndrome is consistent with geriatric depression. If untreated, it includes a higher prevalence of comorbid psychosis and anxiety than what is found in younger populations. 34 This subsyndrome could represent two things. First, there is a well‐established link between late‐life depression and incident dementia, including Alzheimer's disease and vascular dementia. 35 Possible etiological mechanisms that link depression and dementia suggest cumulative contributions of vascular changes, inflammatory processes, and increased amyloid production. 36 Second, depression can be a consequence of a dementia diagnosis. Either way, this subsyndrome points to the importance of assessment and treatment for geriatric depression among persons with dementia.
There is clinical and face validity for hallucinations to appear as their own subsyndrome. In our analysis they were grouped with uncooperativeness. First, hallucinations are a hallmark feature in Lewy body dementia compared to other dementias. 37 Thus, it may stand out as its own subsyndrome among all ADRDs. Second, compared to other BPSD, hallucinations are related to distinct neuroanatomical changes indicating a unique pathogenesis. 38 Last, because health‐care professionals lack knowledge and confidence regarding the management of psychosis, it is likely that family caregivers lack the skills to manage hallucinations at home. 39 Therefore, it is reasonable to expect that they would group with general uncooperativeness. At the between level, hallucinations were the only BPSD to cross‐load, which may indicate they are a byproduct of unmanaged depression in the depression–affective subsyndrome and their own unique subsyndrome.
Aggressive–agitation reflects a common neuropsychiatric phenomenon. Wandering, care resistance, and verbal and physical aggression are often identified together. 40 While some have argued that care resistance is its own clinical phenomenon, 41 our findings support that these individual BPSD fall within the International Psychogeriatric Association provisional consensus definition of agitation. 42 Importantly, we did find some agitation symptoms grouped separately to form a manic–agitation subsyndrome representing a symptom profile of mania in persons with ADRD. Mania is often associated with behavioral agitation in older adults, 43 with our findings indicating that manic–agitation is distinct from aggressive–agitation. Neurological changes contribute to secondary mania, which is more common among older adults compared to younger. 44 Moreover, manic episodes are more common in dementia than in other chronic diseases. 45 As there is virtually no guidance for the management of manic–agitation in ADRD, these findings support further investigation of mania as a distinct form of agitation; its neurobiological basis in ADRD is a high‐priority area for future research.
We found three within‐person factors representing distinct symptom clusters. They are based on how BPSD clustered together on a given day for an individual with ADRD. Interestingly, the factors generally split into behavioral versus psychological symptoms. The behavioral symptom cluster included BPSD with known shared environmental triggers. For example, vocalizations, physical aggression, wandering, impulsivity, and disinhibition are related to physical environment (e.g., cleanliness, lighting) and social environment (e.g., attitudes of caregivers, availability of activities). 46 These clusters might be amendable to strategies that moderate the context of daily caregiving. The psychological symptoms split into two factors: mood symptoms (depression, anxiety, lability) and psychosis‐like symptoms (delusions, hallucinations, apathy, wandering). Fluctuating affect can also be triggered by various levels of environmental under‐ or overstimulation. Multisensory experiences and virtual reality are promising interventions for emotional stabilization in ADRD. 47 , 48 , 49 Psychosis‐like symptoms may reflect daily fluctuations in BPSD as it has been suggested that BPSD follows patterns of escalation and de‐escalation over the course of the day. 12 Superimposed hypo‐ and hyperactive delirium may also contribute to the clustering of psychosis‐like symptoms. 50
There is support for the proposed subsyndromes and symptom clusters found in previous factor analytic studies. The subsyndromes we identified are comparable to the commonly found groupings identified by a systematic review including affective symptoms, psychosis, hyperactivity, and euphoria (sometimes including disinhibition). 5 A review of subsyndrome analyses that used the NPI found that an affective subsyndrome was identified in 20% to 33% of groupings, and a psychosis subsyndrome was present in 13% to 20% of groupings. 9 At the within level, our “behavioral symptom” cluster is similar to the “HIDA domain” (hyperactivity–impulsivity–irritability–disinhibition–aggression–agitation). This is an alternative grouping that has guided clinical management guidelines. 51 Importantly, work to date has focused on subsyndromes, 5 and this is the first study to explore symptom clusters. As such, our findings clarify some of the inconsistencies in previous work on this topic. The systematic review found that certain BPSD had no consistencies in how they loaded including apathy, eating disturbances, disinhibition, and aberrant motor behavior (including compulsions). 5 Through a multilevel framework we found that eating difficulties did not load at the between level while compulsions did not load at the within level. Moreover, apathy clustered with different BPSD at the between level versus within level. Because previous attempts to create subsyndromes used a single‐level approach, this would account for the difficulty in replicating findings for these BPSD.
4.1. Strengths and limitations
The findings may apply to a general, community‐based population of ADRD because our sample included all types of dementia, regardless of the official diagnosis. Different subsyndromes and symptom clusters may be found within different types of dementia with different clinical features, which is an area for future research. However, from a pragmatic standpoint, a parsimonious model of all ADRD may be more desirable to guide symptom management research and clinical practice given the frequency of comorbid diagnoses and their additive effects on BPSD burden. 38 An additional limitation is that, like most studies, data relied on proxy reporting from caregivers. The inclusion of persons with mild cognitive impairment and earlier stages of dementia is an area for future research.
One of the biggest methodological challenges in this study was the lack of established research approaches to data reduction with longitudinal datasets. Obtaining model reliability estimates was problematic, as there are no established methods when the number of factors varies at each level. The short follow‐up period of 8 days limited calculations on the factor extraction at the within‐person level. Additionally, there is a lack of guidance on determining sample sizes, so we relied on general rules of multilevel structural equation models. 52 Our within‐level sample size exceeds recommendations, and there are no recommendations to judge our sample size at the between level. However, because our between‐level solution had favorable fit statistics and significant factor loadings, we believe our sample size is sufficient. Replication of these findings in larger samples, including confirmatory factor analyses, is an area for future research. Nevertheless, our intensive longitudinal data coupled with a multilevel analytic approach strengthened the rigor of this study. It enabled a more valid measurement of outcomes by decomposing variance structures.
4.2. Future directions
A major critique of the subsyndrome concept is that prior work has not been able to demonstrate the stability of the subsyndromes over time. 53 Thus, future work using multilevel frameworks is needed to assess subsyndrome stability as well as to examine how persons with ADRD transition in and out of subsyndromes as their disease progresses over time. Future work is needed to identify individual susceptibility to the subsyndromes and understand the neurobiological underpinnings driving them, which could inform new pharmacological targets. The gut microbiome has been linked to a number of behavioral and psychological symptoms (e.g., depression, anxiety, compulsions) in other mental health disorders and should be further examined in BPSD in relation to subsyndromes. 54 Future work should also investigate whether the individual subsyndromes have a “sentinel symptom”—a primary symptom driving the mechanistic process, which influences the occurrence of other symptoms within the group. This nuanced analysis can be accomplished through complex data modeling approaches, such as Bayesian networks. 6 , 55 Future studies should include examinations of biological contributions of race, ethnicity, and sex. Finally, it is likely that persons experiencing different subsyndromes may have different health outcomes. Identification of those differences can inform long‐term care planning.
To validate the concept of symptom clusters, it is important to identify whether they are predicted by different environmental and contextual triggers. Future work should also examine temporal relationships among the clusters to understand how they influence one another, 12 using approaches such as ecological momentary assessment to allow for temporal ordering. For example, it may be that exacerbated depression and anxiety symptoms amplify the overall sensitivity to environmental stimuli, thus increasing behavioral symptoms. Therefore, to reduce overall behavioral symptoms, we may need a targeted intervention for the psychological–mood symptom cluster. This is a marked difference from the prevailing clinical paradigms for preparing dementia caregivers. Many non‐pharmacologic interventions have been developed to support caregivers in the day‐to‐day management of BPSDs. Yet, most produce small effect sizes with unknown duration, 56 possibly because these studies measure BPSD as single items or composite scores. Our results uncover the opportunity to re‐conceptualize the complex and dynamic nature of BPSD through nuanced behavioral profiles that can further advance symptom science in ADRD. 11 , 12 , 57
In conclusion, this study offers major methodological and conceptual advancements for the management of BPSD within ADRD by introducing two related but distinct concepts of subsyndromes and symptom clusters. This framework offers a bridge between the so‐called “medical” and “environmental/nursing” models and emphasizes the importance of a multiple pathways approach for the management of BPSD. Further development of subsyndromes and symptom clusters will elucidate novel, targeted interventions that ultimately will allow us to offer more tools to persons with AD and their family caregivers to manage their chronic illness.
CONFLICT OF INTEREST STATEMENT
The authors have no conflicts of interest to declare. Author disclosures are available in the supporting information.
CONSENT STATEMENT
Consent was provided by all human subjected in accordance with the institutional review board approved protocol.
Supporting information
Supporting Information
Supporting Information
ACKNOWLEDGMENTS
The authors have nothing to report. Funding was provided by the Alzheimer's Association Research Grant Program (2019‐AARG‐644673).
Pickering CEZ, Winstead V, Yildiz M, Wang D, Yefimova M, Pickering AM. Subsyndromes and symptom clusters: Multilevel factor analysis of behavioral and psychological symptoms of dementia with intensive longitudinal data. Alzheimer's Dement. 2024;20:6699–6708. 10.1002/alz.14075
[Correction added on 17 August 2024, after first online publication: The affliations for the first two authors were corrected.]
REFERENCES
- 1. Hope T, Keene J, Fairburn CG, Jacoby R, McShane R. Natural history of behavioural changes and psychiatric symptoms in Alzheimer's disease. Br J Psychiatry. 2018;174(1):39‐44. doi: 10.1192/bjp.174.1.39 [DOI] [PubMed] [Google Scholar]
- 2. Fauth EB, Gibbons A. Which behavioral and psychological symptoms of dementia are the most problematic? Variability by prevalence, intensity, distress ratings, and associations with caregiver depressive symptoms. Int J Geriatr Psychiatry. 2014;29(3):263‐271. doi: 10.1002/gps.4002 [DOI] [PubMed] [Google Scholar]
- 3. Robert P, Verhey FR, Byrne EJ, et al. Grouping for behavioral and psychological symptoms in dementia: clinical and biological aspects. Consensus paper of the European Alzheimer disease consortium. Eur Psychiatry. 2005;20(7):490‐496. doi: 10.1016/j.eurpsy.2004.09.031 [DOI] [PubMed] [Google Scholar]
- 4. Geda YE, Schneider LS, Gitlin LN, et al. Neuropsychiatric symptoms in Alzheimer's disease: past progress and anticipation of the future. Alzheimers Dement. 2013;9(5):602‐608. doi: 10.1016/j.jalz.2012.12.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. van der Linde RM, Dening T, Matthews FE, Brayne C. Grouping of behavioural and psychological symptoms of dementia. Int J Geriatr Psychiatry. 2014;29(6):562‐568. doi: 10.1002/gps.4037 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Harris CS, Dodd M, Kober KM, et al. Advances in conceptual and methodological issues in symptom cluster research: a 20‐year perspective. ANS Adv Nurs Sci. 2022;45(4):309‐322. doi: 10.1097/ans.0000000000000423 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Aalten P, de Vugt ME, Lousberg R, et al. Behavioral problems in dementia: a factor analysis of the neuropsychiatric inventory. Dement Geriatr Cogn Disord. 2003;15(2):99‐105. doi: 10.1159/000067972 [DOI] [PubMed] [Google Scholar]
- 8. Aalten P, Verhey FR, Boziki M, et al. Neuropsychiatric syndromes in dementia. Results from the European Alzheimer Disease Consortium: part I. Dement Geriatr Cogn Disord. 2007;24(6):457‐463. doi: 10.1159/000110738 [DOI] [PubMed] [Google Scholar]
- 9. Canevelli M, Adali N, Voisin T, et al. Behavioral and psychological subsyndromes in Alzheimer's disease using the neuropsychiatric inventory. Int J Geriatr Psychiatry. 2013;28(8):795‐803. doi: 10.1002/gps.3904 [DOI] [PubMed] [Google Scholar]
- 10. Connors MH, Seeher KM, Crawford J, Ames D, Woodward M, Brodaty H. The stability of neuropsychiatric subsyndromes in Alzheimer's disease. Alzheimers Dement. 2018;14(7):880‐888. doi: 10.1016/j.jalz.2018.02.006 [DOI] [PubMed] [Google Scholar]
- 11. Pickering CEZ, Yefimova M, Wang D, Maxwell Christopher D, Jablonski R. Dynamic structural equation modelling evaluating the progressively lowered stress threshold as an explanation for behavioural symptoms of dementia. J Adv Nurs. 2022;78(8):2448‐2459. doi: 10.1111/jan.15173 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Woods D, Buckwalter K. Taking another look: thoughts on behavioral symptoms in dementia and their measurement. Healthcare. 2018;6(4);126. doi: 10.3390/healthcare6040126 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Fauth EB, Zarit SH, Femia EE, Hofer SM, Stephens MAP. Behavioral and psychological symptoms of dementia and caregivers’ stress appraisals: intra‐individual stability and change over short‐term observations. Aging Ment Health. 2006;10(6):563‐573. doi: 10.1080/13607860600638107 [DOI] [PubMed] [Google Scholar]
- 14. Curran PJ, Bauer DJ. The disaggregation of within‐person and between‐person effects in longitudinal models of change. Annu Rev Psychol. 2011;62:583‐619. doi: 10.1146/annurev.psych.093008.100356 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Chen Y, Dang M, Zhang Z. Brain mechanisms underlying neuropsychiatric symptoms in Alzheimer's disease: a systematic review of symptom‐general and –specific lesion patterns. Mol Neurodegener. 2021;16(1). doi: 10.1186/s13024-021-00456-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Galvin JE, Roe CM, Xiong C, Morris JC. Validity and reliability of the AD8 informant interview in dementia. Neurology. 2006;67(11):1942‐1948. doi: 10.1212/01.wnl.0000247042.15547.eb [DOI] [PubMed] [Google Scholar]
- 17. Tarzia L, Valpied J, Koziol‐McLain J, Glass N, Hegarty K. Methodological and ethical challenges in a web‐based randomized controlled trial of a domestic violence intervention. J Med Internet Res. 2017;19(3). doi: 10.2196/jmir.7039 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Teitcher JEF, Bockting WO, Bauermeister JA, Hoefer CJ, Miner MH, Klitzman RL. Detecting, preventing, and responding to “fraudsters” in internet research: ethics and tradeoffs. J Law Med Ethics. 2015;43(1):116‐133. doi: 10.1111/jlme.12200 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Cummings JL, Mega M, Gray K, Rosenberg‐Thompson S, Carusi DA, Gornbein J. The neuropsychiatric inventory: comprehensive assessment of psychopathology in dementia. Neurology. 1994;44(12):2308‐2308. doi: 10.1212/wnl.44.12.2308 [DOI] [PubMed] [Google Scholar]
- 20. Pickering CEZ, Yefimova M, Maxwell C, Puga F, Sullivan T. Daily context for abusive and neglectful behavior in family caregiving for dementia. Gerontologist. 2020;60(3):483‐493. doi: 10.1093/geront/gnz110 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Lam KSL, Aman MG. The repetitive behavior scale‐revised: independent validation in individuals with autism spectrum disorders. J Autism Dev Disord. 2006;37(5):855‐866. doi: 10.1007/s10803-006-0213-z [DOI] [PubMed] [Google Scholar]
- 22. Cohen‐Mansfield J, Werner P. Typology of disruptive vocalizations in older persons suffering from dementia. Int J Geriatr Psychiatry. 1997;12(11):1079‐1091. doi: 10.1002/(sici)1099-1166(199711)12:113C1079::aid-gps6893E3.0.co;2-p [DOI] [PubMed] [Google Scholar]
- 23. MuthÉN BO. Multilevel covariance structure analysis. Sociol Methods Res. 2016;22(3):376‐398. doi: 10.1177/0049124194022003006 [DOI] [Google Scholar]
- 24. van de Vijver FJR, Poortinga YH. Structural equivalence in multilevel research. J Cross Cult Psychol. 2016;33(2):141‐156. doi: 10.1177/0022022102033002002 [DOI] [Google Scholar]
- 25. Ji Y, Luo W, Lai MHC, Yoon M, Chen L‐S, Kwok O‐M. Model‐selection‐based approaches to identifying the optimal number of factors in multilevel exploratory factor analysis. Struct Equ Modeling. 2021;28(5):763‐777. doi: 10.1080/10705511.2021.1916394 [DOI] [Google Scholar]
- 26. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107(2):238‐246. doi: 10.1037/0033-2909.107.2.238 [DOI] [PubMed] [Google Scholar]
- 27. Tucker LR, Lewis C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika. 1973;38(1):1‐10. doi: 10.1007/bf02291170 [DOI] [Google Scholar]
- 28. Akaike H. Factor analysis and AIC. Psychometrika. 1987;52(3):317‐332. doi: 10.1007/bf02294359 [DOI] [Google Scholar]
- 29. Schwarz G. Estimating the dimension of a model. Ann Stat. 1978:461‐464. [Google Scholar]
- 30. Sclove SL. Application of model‐selection criteria to some problems in multivariate analysis. Psychometrika. 1987;52(3):333‐343. doi: 10.1007/bf02294360 [DOI] [Google Scholar]
- 31. Steiger JH. Structural model evaluation and modification: an interval estimation approach. Multivar Behav Res. 1990;25(2):173‐180. doi: 10.1207/s15327906mbr2502_4 [DOI] [PubMed] [Google Scholar]
- 32. Jöreskog KG, Sörbom D. LISREL 7: A Guide to the Program and Applications. SPSS; 1989. [Google Scholar]
- 33. Muthen LK, Muthen BO. Mplus User's Guide. 8th ed. Muthen & Muthen; 2017. [Google Scholar]
- 34. Tampi RR, Young J, Hoq R, Resnick K, Tampi DJ. Psychotic disorders in late life: a narrative review. Ther Adv Psychopharmacol. 2019;9. doi: 10.1177/2045125319882798 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35. Diniz BS, Butters MA, Albert SM, Dew MA, Reynolds CF. Late‐life depression and risk of vascular dementia and Alzheimer's disease: systematic review and meta‐analysis of community‐based cohort studies. Br J Psychiatry. 2018;202(5):329‐335. doi: 10.1192/bjp.bp.112.118307 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Butters MA, Young JB, Lopez O, et al. Pathways linking late‐life depression to persistent cognitive impairment and dementia. Dialogues Clin Neurosci. 2008;10(3):345‐357. doi: 10.31887/DCNS.2008.10.3/mabutters [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Devanand DP, Lee S, Huey ED, Goldberg TE. Associations between neuropsychiatric symptoms and neuropathological diagnoses of Alzheimer disease and related dementias. JAMA Psychiatry. 2022;79(4):359‐367. doi: 10.1001/jamapsychiatry.2021.4363 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Gibson LL, Grinberg LT, Ffytche D, et al. Neuropathological correlates of neuropsychiatric symptoms in dementia. Alzheimers Dement. 2023;19(4):1372‐1382. doi: 10.1002/alz.12765 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Chen J, El‐Den S, Pham L, O'Reilly CL, Collins JC. Healthcare professionals’ knowledge, confidence and attitudes in relation to psychosis care: a systematic review. Int J Soc Psychiatry. 2023;69(8):1856‐1868. doi: 10.1177/00207640231194490 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40. Cohen‐Mansfield J. Agitated behavior in persons with dementia: the relationship between type of behavior, its frequency, and its disruptiveness. J Psychiatr Res. 2008;43(1):64‐69. doi: 10.1016/j.jpsychires.2008.02.003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Jablonski‐Jaudon RA, Winstead V, Jones‐Townsend C, Azuero A, Mahoney E, Kolanowski AM. Revising the resistiveness to care scale. J Nurs Meas. 2016;24(2):72‐82. doi: 10.1891/1061-3749.24.2.E72 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42. Cummings J, Mintzer J, Brodaty H, et al. Agitation in cognitive disorders: International Psychogeriatric Association provisional consensus clinical and research definition. Int Psychogeriatr. 2015;27(1):7‐17. doi: 10.1017/S1041610214001963 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Cipriani G, Danti S, Carlesi C, Lucetti C, Nuti A, Di Fiorino M. Shedding light on manic syndromes and dementia. OBM Geriat. 2019;3(3):1‐1. doi: 10.21926/obm.geriatr.1903070 [DOI] [Google Scholar]
- 44. Brooks JO, Hoblyn JC. Secondary mania in older adults. Am J Psychiatry. 2005;162(11):2033‐2038. doi: 10.1176/appi.ajp.162.11.2033 [DOI] [PubMed] [Google Scholar]
- 45. Nilsson FM. Enduring increased risk of developing depression and mania in patients with dementia. J Neurol Neurosurg Psychiatry. 2002;73(1):40‐44. doi: 10.1136/jnnp.73.1.40 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46. Sloane PD, Mitchell CM, Preisser JS, Phillips C, Commander C, Burker E. Environmental correlates of resident agitation in Alzheimer's disease special care units. J Am Geriatr Soc. 1998;46(7):862‐869. doi: 10.1111/j.1532-5415.1998.tb02720.x [DOI] [PubMed] [Google Scholar]
- 47. Pinto JO, Dores AR, Geraldo A, Peixoto B, Barbosa F. Sensory stimulation programs in dementia: a systematic review of methods and effectiveness. Expert Rev Neurother. 2020;20(12):1229‐1247. doi: 10.1080/14737175.2020.1825942 [DOI] [PubMed] [Google Scholar]
- 48. Brimelow RE, Dawe B, Dissanayaka N. Preliminary research: virtual reality in residential aged care to reduce apathy and improve mood. Cyberpsychol Behav Soc Netw. 2020;23(3):165‐170. doi: 10.1089/cyber.2019.0286 [DOI] [PubMed] [Google Scholar]
- 49. Matsangidou M, Solomou T, Frangoudes F, et al. Affective out‐world experience via virtual reality for older adults living with mild cognitive impairments or mild dementia. Int J Environ Res Public Health. 2023;20(4). doi: 10.3390/ijerph20042919 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Hosker C, Ward D. Hypoactive delirium. BMJ. 2017;357:j2047. doi: 10.1136/bmj.j2047 [DOI] [PubMed] [Google Scholar]
- 51. Keszycki RM, Fisher DW, Dong H. The hyperactivity–impulsivity–irritiability–disinhibition–aggression–agitation domain in Alzheimer's disease: current management and future directions. Front Pharmacol. 2019;10. doi: 10.3389/fphar.2019.01109 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Holtmann J, Koch T, Lochner K, Eid M. A comparison of ML, WLSMV, and Bayesian methods for multilevel structural equation models in small samples: a simulation study. Multivar Behav Res. 2016;51(5):661‐680. doi: 10.1080/00273171.2016.1208074 [DOI] [PubMed] [Google Scholar]
- 53. Connors MH, Seeher KM, Crawford J, Ames D, Woodward M, Brodaty H. The stability of neuropsychiatric subsyndromes in Alzheimer's disease. Alzheimers Dement. 2018;14(7):880‐888. doi: 10.1016/j.jalz.2018.02.006 [DOI] [PubMed] [Google Scholar]
- 54. Sampson Timothy R, Mazmanian Sarkis K. Control of brain development, function, and behavior by the microbiome. Cell Host Microbe. 2015;17(5):565‐576. doi: 10.1016/j.chom.2015.04.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55. Miaskowski C, Barsevick A, Berger A, et al. Advancing symptom science through symptom cluster research: expert panel proceedings and recommendations. J Natl Cancer Inst. 2017;109(4). doi: 10.1093/jnci/djw253 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56. Walter E, Pinquart M, Heyn PC. How effective are dementia caregiver interventions? An updated comprehensive meta‐analysis. Gerontologist. 2020;60(8):e609‐e619. doi: 10.1093/geront/gnz118 [DOI] [PubMed] [Google Scholar]
- 57. Beck C, Richards K, Lambert C, et al. Factors associated with problematic vocalizations in nursing home residents with dementia. Gerontologist. 2011;51(3):389‐405. doi: 10.1093/geront/gnq129 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Supporting Information
Supporting Information
