Measuring Symptoms in Community-dwelling Older Adults: The Psychometric Properties of a Brief Symptom Screen

Abstract

Background

With aging, the probability of experiencing multiple chronic conditions is increased, along with symptoms associated with these conditions. Symptoms form a central component of illness burden and distress. To date, most symptom measures have focused on a particular disease population.

Objective

We sought to develop and evaluate a simple symptom screen using data obtained from a representative sample of community-dwelling older adults.

Methods

Psychometric analyses were conducted on 10 self-reported dichotomous symptom indicators collected during in-person interviews from a sample of 1000 community-dwelling older adults. Symptoms included shortness of breath, feeling tired or fatigued, problems with balance or dizziness, perceived weakness in legs, constipation, daily pain, stiffness, poor appetite, anxiety, and anhedonia.

Results

Over one-third of the sample (37.4%) had 5 or more concurrent symptoms. Stiffness and feeling tired were the most common symptoms. Confirmatory factor analyses were performed on the 10 symptoms for single factor and bifactor (physical and affective) models of symptom reporting. Goodness of fit indices indicated better fit for the bifactor model (χ²_df=10=89.6, p<0.001) but the practical significance of the improvement in fit was negligible. Differential item functioning (DIF) analyses showed some differences of relatively high magnitude in location parameters by race; however, because the DIF was in different directions, the impact on the overall measure was most likely lessened.

Conclusion

Among community-dwelling older adults, a large proportion experienced multiple co-occurring symptoms. This Brief Symptom Screen can be used to quickly measure overall symptom load in older adult populations, including those with multiple chronic conditions.

Introduction

Advances in modern medicine have contributed to a greater proportion of adults surviving acute illness and living with multiple conditions. The experience of multiple conditions is compounded by symptoms associated with these conditions. These symptoms contribute to illness burden in ways that are not predictable on the basis of the diagnosed disorder(s) alone.

Symptom number and characteristics have been studied in cancer populations and conditions such as AIDS and chronic organ failure (1–4); little research has examined symptoms among community-dwelling older adults. Many symptom measures were developed initially for cancer patients (5–7) and evaluate symptoms over short time intervals. Often these measures focus on a particular disease without capturing the role of non-index conditions on symptoms (8–9). Non disease-specific symptom inventories are needed (10) for older adults with multiple conditions.

In this study we sought to understand symptom experience in a population based sample of older adults by identifying symptoms common in this population, performing confirmatory factor analyses on the symptom indicators, testing biases in symptom endorsement, and exploring the associations between an inventory of symptoms and other indicators of function and self-rated health. We targeted symptoms in both physical and affective domains; we hypothesized that acceptable fit and convergent validity with related measures would be observed for a single factor underlying the 10- symptoms. We also examined symptom-level biases in the scale. Methods

Participants

The UAB Study of Aging (SOA) is a population based longitudinal study of 1000 community dwelling older adults residing in Alabama. Participants were recruited from a stratified random sample of Medicare beneficiaries aged 65 years and older living in 5 counties in central Alabama. Recruitment of participants was stratified to achieve a balanced sample by sex, race (African American and White), and urban – rural residence.

Procedures

A full description of the methods has been published elsewhere (11–12). Baseline in-home interviews conducted between 1999 and 2001 included questions regarding sociodemographic characteristics, medical status and history, mental health status, activities of daily living, mobility (Life Space Assessment), and symptoms. The UAB Institutional Review Board approved the study protocol.

Measures

Symptoms

We conceptualized symptoms as self-reported negative perceptions that reflect a person’s subjective experience. We identified symptoms that were queried in the SOA cohort and distinguished between physical and affective symptoms. Physical symptoms refer to unpleasant sensations in the body, whereas affective symptoms refer to unpleasant feelings or emotions (13). From 15 symptoms measured at baseline, we chose 10 symptoms for further analyses because they were reported by at least 10% of the study population and did not by themselves reflect a specific condition or diagnosis (e.g., chronic diarrhea and feeling faint had prevalence rates under 10%). We did not include “events” like incontinence episodes or falls. Physical symptoms included shortness of breath, feeling tired, dizziness, weakness, daily pain, stiffness, and constipation. Six of these symptoms were asked with yes/no questions. For pain, we used the response of “daily pain” as a way of capturing more chronic rather than intermittent pain. The 3 affective symptoms (poor appetite, anxiety, and anhedonia) were measured on ordinal rating scales, and dichotomous categorizations were later applied to these responses (Table 1). We examined only dichotomous item responses so that our analyses would reflect an overall indication of symptom presence and cumulative burden that would be easy to score in applied clinical and research settings.

Table 1.

Brief Symptom Screen

Symptom	Question and Answer	Score
Pain	How frequently over the past 4 weeks have you experienced pain? Not at all Up to 4–6 times per week Daily	1, if daily
Shortness of breath	Do you have problems with shortness of breath? Yes No	1, if yes
Fatigue	Do you have problems with feeling tired or fatigued? Yes No	1, if yes
Dizziness	Do you have problems with balance or dizziness? Yes No	1, if yes
Weakness	Do you have problems with weakness in legs? Yes No	1, if yes
Stiffness	Do you have problems with stiffness? Yes No	1, if yes
Constipation	Do you have any constipation? Yes No	1, if yes
Poor appetite	Would you say your appetite is usually: Very good Good Fair Poor	1, if poor
Anhedonia	During the past 4 weeks, how often have you had little interest or pleasure in doing things? Always Very often Sometimes Almost never Never	1, if sometimes, very often or always
Anxiety	During the past 4 weeks, how often have you been bothered by your nerves? Always Very often Sometimes Almost never Never	1, if sometimes, very often or always

Variables Used to Assess the Convergent Validity between an Overall Symptom Score and Related Outcomes included activities of daily living (ADLs), Life Space Assessment (LSA), self-rated health and comorbidity. ADLs were measured as a sum of self-care activities for which persons reported having difficulty performing independently (bathing or showering, dressing or undressing self, using the toilet, eating, walking, getting outside, going up and down stairs). Scores ranged from 0 to 7 with higher scores reflecting lower function. The UAB SOA LSA measures mobility and participation in society and is based on the distance through which a person reports moving over the month prior to assessment. LSA scores range from 0 to 120; lower scores represent lower mobility (14). In addition to baseline functional measures, 4-year follow-up ADL scores and life-space scores were used in the analyses.

Self-rated health was assessed by asking “In general, would you say your health is excellent, very good, good, fair, or poor?” (15). We calculated an unweighted comorbidity count, assigning one point for each diagnosis in the Charlson Comorbidity Index (16).

Statistical Analyses

Our conceptual model was informed by the perspective that these symptoms are indicators of an underlying attribute of illness burden, represented by both conditions captured traditionally by comorbidity assessment and potentially by conditions that exist but may not be so easily captured because of lack of recognition by clinicians or by the older adult themselves (due to dysthymia, cognitive impairment, or a sense that these symptoms are part of normal aging). All ten symptoms were subjected to parallel analysis with a scree plot to identify the minimum number of factors underlying the set of symptoms (17). We performed confirmatory factor analysis (CFA) on the symptom indicators (18) to compare a single factor model to a bifactor model based on a hypothetical distinction between physical and affective types of symptoms (19). CFA examines the interrelationships among a set of indicator variables by considering those indicator variables to be effects of a smaller number of underlying latent factors (20). The DIFFTEST option in Mplus (21) was used in conjunction with the weighted least squares estimator to examine the statistical significance of any improvements in fit from the single factor to a bifactor model (22). The comparative fit index (CFI) and the root mean square error of approximation (RMSEA) were used to examine absolute model fit while also taking model complexity into account. A CFI greater than 0.95 and an RMSEA less than 0.05 were considered indicative of excellent fit (23).

We further examined whether the factor parameters of the single factor model differed significantly by sex, race, age rural versus urban residence, and comorbidity to gain further insights into possible group differences in the severity of the symptom indicators using the IRTLRDIF analysis package (24). These analyses provided tests of differential item functioning (DIF) by first estimating a latent variable model in which parameters (symptom discriminations and locations) are fixed to be equal across the grouping variable (sex, race, age, urban versus rural, comorbidity) and then comparing this model using likelihood ratio tests with subsequent models in which parameters for a given indicator are free to vary by the grouping variable (24–25). For significant DIF by location, subsequent multiple indicator multiple cause (MIMIC) analyses were conducted to determine the practical significance of any threshold differences. Using a criterion suggested by Cole et al. (26), symptoms with an odds ratio of endorsement greater than 2.0 for one group compared to the other were considered to show practically meaningful DIF.

For convergent validity, we assessed unadjusted and covariate-adjusted relationships between our overall Brief Symptom Screen (BSS) and demographic variables, comorbidity, baseline and 4-year ADL score, SPPB, baseline and 4-year Life Space Assessment, and self-rated health. Covariate-adjusted relationships were examined using multiple linear regression analyses and estimated associations after controlling for demographic variables and comorbidity.

Results

Table 2 displays descriptive information for individual measures used in these analyses. More than one-third of the participants reported 5 or more of the 10 symptoms. The 10 symptoms in the BSS demonstrated good variability (Mean = 3.70, SD = 2.66, Median = 3, Min = 0, Max = 10), and Cronbach’s alpha measure of internal consistency was 0.76, suggesting acceptable reliability for this brief assessment of symptom experience.

Table 2.

Participant Characteristics

Characteristic	N = 1000
DEMOGRAPHIC CHARACTERISTICS
Age (SD)	75.31 (6.72)
Gender (% Female)	49.9%
Race (% African American)	50.0%
Urban / Rural (% Rural)	51.4%
Prevalence of Individual Symptoms
Shortness of breath	35.1%
Feeling tired or fatigued	47.9%
Problems with balance or dizziness	35.0%
Weakness	38.9%
Daily pain	38.6%
Stiffness	49.0%
Constipation	36.2%
Poor appetite	17.2%
Anxiety	36.0%
Anhedonia	36.5%
Mean Total Number of Symptoms – Sym10 (SD)	3.70 (2.66)
0 Symptoms	12.9%
1–2 Symptoms	25.3%
3–4 Symptoms	24.4%
5–6 Symptoms	20.0%
7–8 Symptoms	12.1%
9–10 Symptoms	5.3%
Mean Number of ADL Difficulties (SD)∫	1.30 (1.85)
Mean Short Physical Performance Battery Score (SD)∫	6.84 (3.25)
Mean Life Space (SD)∫	64.11 (24.93)

^∫The theoretical and actual ranges for these scores were the same and are as follows: ADL Difficulties: min = 0, max = 7; Short Physical Performance Battery Score: min = 0, max = 12; Life Space Score, min = 0, max = 120.

Parallel analysis with a scree plot suggested that a single factor was sufficient to explain the variability among the ten symptoms. The largest eigenvalue was 4.41, indicating that 44.1% of the variability in the items can be accounted for by the first component. The second eigenvalue was 1.14, and the remaining 8 eigenvalues were all less than 1.0.The CFA results are summarized in Figure 1, where we report factor loadings and thresholds for each symptom on a two-parameter probit metric traditionally used in item response theory. In the single factor model, each symptom freely loaded on the only factor, and the variance of that factor was set to 1.0. This single factor model provided moderately good fit to the observed data (CFI = 0.95, RMSEA = 0.063) and supported the single symptom experience dimension. The factor variances of the bifactor model (Figure 1, panel B) were also set at 1.0, and the fit was excellent for this model (CFI = 0.98, RMSEA = 0.045). A likelihood ratio test comparing the single factor model to the bifactor solution supported distinguishing between physical and affective symptoms (χ²_df=10=89.6, p<0.001). However, factor loadings for only one symptom (anhedonia) were larger for the domain-specific factor than for the general factor. Together with results of the parallel analysis, the findings suggest a strong unidimensionality underlying the reporting of these 10 dichotomous symptoms.

Figure 1. General Symptom Burden Confirmatory Factor Analysis Models (N=1000).

Item factor loadings and threshold parameters are parameterized in a two-parameter probit metric.

From the single-factor CFA, we plotted the magnitude of the relationship between each symptom and the underlying factor against the location of the symptom along the level of the symptom experience trait in Figure 2. All symptoms were moderately to highly correlated with the underlying factor, as indicated by symptom discrimination parameters. The “fatigue” symptom was most related to the rest of the symptoms, while constipation was the least related. The symptom locations along the X-axis suggest that most symptoms had similar probabilities of endorsement and thus provided the most information in the central part of the scale. Stiffness and fatigue were the most commonly endorsed symptoms and thus provided more information in the less severe range of the symptom inventory, while poor appetite was the least prevalent symptom and provided information in the more severe range of the symptom experience scale.

Figure 2. Scatterplot of symptom discrimination by symptom location on the symptom burden scale (N=1000).

Symptom discrimination and location (threshold) parameters are parameterized in a two-parameter probit metric. This graphic plots the magnitude of the relationship between each symptom and the underlying factor from a single-factor CFA (symptom discrimination) against the location of the symptom along the level of the symptom burden trait.

Examinations of DIF revealed few differences on item parameters as a function of sex, race, age, urban/rural residence, or comorbidity. For symptom factor loadings or discrimination parameters, only one statistically significant difference (p < .05) was observed across 50 possible examinations (10 symptoms times 5 grouping factors). A significantly smaller factor loading was found for stiffness for participants 75 or more years of age compared to younger participants. For symptom thresholds or location parameters, statistically significant DIF was observed in 18 out of 50 possible instances, but in most instances the difference was small and practically negligible. In only 4 of 50 possible instances did the location DIF exceed an odds ratio of 2.0 from a MIMIC model such that one group had an odds of symptom reporting that was more than twice as high as the other group after holding constant the level of the underlying latent trait. Three of these 4 instances involved significant DIF by race. Controlling for overall symptom experience, Whites were more likely than African Americans to report feeling tired or fatigued (OR = 4.12, 95% confidence interval (CI) = 2.47, 6.88) and daily pain (OR = 2.10, 95% CI = 1.51, 2.92), whereas African Americans were more likely than Whites to report anhedonia (OR = 2.40, 95% CI = 1.75, 3.31). In addition, men had higher endorsement rates than women for shortness of breath (OR = 2.48, 95% CI = 1.79, 3.41) after controlling for overall symptom experience.

Table 3 displays the correlations and standardized regression coefficients for convergent validity between the BSS and self-rated health, ADL difficulty (baseline and at 4 years post-baseline), and Life-Space (baseline and at 4 years). Generally, correlations between all variables and the factor were somewhat higher than, but comparable to, corresponding correlations with respective sum scores. The adjusted estimates, controlled for sex, race, age, urban/rural and the sum of comorbidities, suggested high correlations between ADL difficulty and BSS score(r=0.50). Similarly, correlations with BSS score demonstrated acceptable convergent validity with ADL difficulty (r=0.34) and life space (r=-0.34) measured 4 years later. These findings indicate similar convergent correlations for the single latent factor and our cumulative measure of 10 symptoms, and that both measures were significantly associated with measures of self-rated health, ADL difficulty, and mobility.

Table 3.

Standardized Associations with Symptom Burden (as measured by the Brief Symptom Screen) before and after Adjusting for Covariates.

Variables	Latent Factor (S.E.)		Brief Symptom Screen Score (S.E)
	Unadjusted^	Adjusted~	Unadjusted^	Adjusted~
Age	0.16 (0.04)**	--	0.16 (0.03)**	--
Gender	0.12 (0.04)**	--	0.11 (0.03)**	--
Race	0.09 (0.04)*	--	0.10 (0.03)*	--
Rural	0.20 (0.04)**	--	0.17 (0.03)**	--
Multimorbidity	0.33 (0.03)**	--	0.29 (0.03)**	--
ADL	0.66 (0.02)**	0.59 (0.03)**	0.57 (0.02)**	0.50 (0.03)**
Life Space	−0.44 (0.03)**	−0.39 (0.04)**	−0.39 (0.03)**	−0.34 (0.03)**
Self-Rated Health	0.58 (0.03)**	0.51 (0.03)**	0.52 (0.02)**	0.45 (0.03)**
ADL at 4 years	0.41 (0.04)**	0.34 (0.64)**	0.37 (0.03)**	0.30 (0.04)**
Life Space at 4 years	−0.44 (0.04)**	−0.39 (0.04)**	−0.40 (0.03)**	−0.34 (0.04)**

S.E. Standard error

^**<0.001,

^*<0.05

^{^}Pearson correlation coefficients.

^~Standardized regression coefficients from a multiple linear regression controlling for age, gender, race, rural, and multimorbidity.

ADL: Activities of Daily Living;

Discussion

The results of these analyses support the potential usefulness of the BSS as a tool to quickly measure overall symptom experience in older adults with chronic conditions. The scree plot from the parallel analysis and the modest improvement in fit from the single factor to the bifactor CFA model support our goal to obtain a simple unidimensional measure of overall symptom reports. Tests of possible DIF generally indicated invariance across major demographic groups in factor loadings or discriminations. More frequent differences in thresholds were noted, especially when comparing African Americans to Whites, but the direction of those differences were in opposing directions, which minimizes any bias that might be observed in the overall BSS score. Because we used data from a diverse population of community dwelling older adults with a wide array of chronic conditions, these initial findings are likely to be more generalizable than studies of symptom measures obtained in older adults with discrete chronic conditions.

Remarkably, prevalence estimates of shortness of breath, fatigue, and pain were very similar to those described in Walke’s smaller study of 226 COPD, CHF and cancer patients, even though our study encompassed a larger cohort of older adults with a wider range of conditions (27). Because the BSS was not derived from patients with primarily one specific condition, we captured high prevalence symptoms that are often not assessed such as stiffness, problems with balance or dizziness, weakness, and constipation. It therefore could be used as a self-report tool or could be practically be filled out with the assistance of a proxy or clinician with the expectation that positive symptoms would warrant more in-depth assessment.

Higher symptom scores were associated with decreased function and mobility and lower levels of self-rated health, underscoring its convergent validity as a measure of overall symptom experience and symptom burden (28).

Several limitations of the BSS warrant mention. First, no qualitative analyses or cognitive interviews were conducted as part of the psychometric evaluation of these questions. Second, the questions in the BSS do not measure the distress a person may have about their symptoms. Finally, the DIF observed when comparing racial groups warrants further study. Future qualitative work is required to provide insight regarding how fatigue, pain and anhedonia are conceptualized by different socio-demographic groups. Despite these limitations, the BSS provides a rapid overview of symptoms commonly experienced by older adults and characterizes symptom experience regardless of the person’s underlying condition(s). The BSS could be linked to additional severity and distress queries for any symptom that screened positively.

With the rise of multimorbidity, there is an increasing need for symptom assessment tools that are not disease specific. Use of condition-specific tools for patients with multiple conditions is cumbersome. We describe a brief symptom inventory that can be used for older adults that is not condition specific. Future studies will be needed to understand its utility for identifying and alleviating symptoms that bear negatively on older adults’ quality of life.