Test-retest reliability of a psychological well-being scale in hospitalized older adults

Abstract

Background and aims

To examine the reliability of a 6-domain psychological well-being instrument in older patients admitted to an acute care hospital unit.

Methods

A prospective reliability study was conducted using a convenience sample of 40 hospitalized patients aged 65 or older. The main measure was a 6-domain psychological well-being instrument including self-acceptance, positive relations with others, autonomy, environmental mastery, purpose of life, and personal growth.

Results

The mean age was 76.3 years [standard deviation (SD)=6.1], 72.5% were white and 57.5% were men. The mean length of stay was 4.9 days (SD=3.1). Test-retest (admission and discharge) intraclass correlation (ICC) values for the six domains were self-acceptance (0.79), positive relations with others (0.72), autonomy (0.79), environmental mastery (0.66), purpose in life (0.79), and personal growth (0.78).

Conclusions

The 6-domain psychological well-being instrument demonstrated good reliability among a sample of hospitalized older patients. This instrument may be useful in a clinical setting to predict outcomes related to patient health and recovery.

INTRODUCTION

An accumulating body of evidence shows the beneficial effects of psychological well-being on physical and functional health (1-12). In large community-based samples of older adults, high levels of positive emotion have been shown to independently predict lower incidence of cardiovascular disease and disability in basic activities of daily living (1-3). For example, Ostir et al. (4) found a direct relationship between positive affect scores at baseline and mobility, functional status, and survival 2 years later in a population-based sample of 2282 Mexican Americans 65 to 99 years of age. Another investigation demonstrated positive affect scores showed a strong inverse association with stroke incidence in a sample of 2478 older whites and blacks (5). Ryff et al. (6) reported a significant association between psychological well-being and biomarkers such as HDL cholesterol, cortisol and hemoglobin, Steptoe et al. (7) also found significant associations between positive emotion and lower cortisol levels and heart rate, and reduced fibrinogen stress responsivity in 228 London-based civil servants aged 35-55 years old. Still other evidence suggests that psychological well-being may promote adaptive coping abilities and build resilience to stressful life events including hospitalization (8-11).

Health care accreditation agencies, such as the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), have recognized the importance of assessing psychological well-being and proposed that the measurement of health outcomes and quality indicators include patient-centered measures such as satisfaction, quality of life and patient well-being (13). Most instruments used to assess psychological well-being, however, were originally developed for use in community populations and their reliability and sensitivity in hospitalized and/or less healthy older adults is unknown. The goal of the current study was to test the reliability of a psychological well-being instrument (14, 15) in older adults hospitalized for acute illness.

The first step in establishing the clinical usefulness of any instrument is to determine the reliability and consistency of the data collection process. If the reliability of the psychological well-being instrument can be demonstrated, the instrument may be useful in clinical trials to examine factors related to resiliency and recovery in hospitalized and frail older adults – a population that is increasing significantly (16, 17).

METHODS

Study population

Data included a sample of 40 patients admitted to the Acute Care for Elders (ACE) unit at the University of Texas Medical Branch (UTMB) in January 2006. The ACE unit, at UTMB, was the first hospital unit in Texas designed exclusively to care for acutely ill older adults. In 2005, the ACE unit was expanded from 20 to 52 beds to better serve the area’s growing older population.

Eligible patients included those aged 65 or older admitted with an acute medical event (n=74). Patients with an admitting diagnosis of cardiovascular, pulmonary, or kidney/urinary tract infection comprised 85% of the admissions to this unit (n=63). Eligible patients also had to be cognitively appropriate by standard nurse assessments about orientation to person, place and time. Screened patients (n=60) were approached to participate in the study by interviewers trained in clinical research techniques. The response rate was 67%. Patients who refused to participate (n=20) did not significantly differ in age, gender or ethnicity from those who participated. Informed consent was obtained from each patient prior to the start of the interview and the research was conducted within the ethical guidelines established by the University of Texas Medical Branch Institutional Review Board.

In-hospital interviews occurred at two time points: within 24 h of admission and within 24 h of discharge. The admission interview occurred immediately after the patient signed the consent form. The discharge interview occurred on average 5 days later. Admission and discharge interviews each required 30-40 min to complete and chart abstraction required an additional 20 min. Interviews were temporarily halted if the patient became fatigued or required routine nursing care. The majority of all interviews (95%) were completed without interruption.

Measures

The psychological well-being instrument (14) includes 6 domains each containing 9 items, and has been widely used in community-based surveys. It summarizes common mental health, clinical and life span developmental theories into six core domains including: 1) self-acceptance; 2) purpose in life; 3) environmental mastery; 4) personal growth; 5) positive relations with others; and 6) autonomy (14, 15) (Table 1). The psychometric properties were originally tested on a sample of 321 healthy men and women (14). Findings included high internal consistency for the 6 domains (Cronbach’s alpha’s from 0.86-0.93), and good test-retest reliability with Pearson product moment coefficients over a six week period ranging from 0.81-0.88.

Table 1. Domains and description of psychological well-being instrument.

Domain	Description
Self-acceptance	Self-acceptance is a key feature of positive men- tal health. Individuals with high self-acceptance tend to have a positive attitude toward them- selves and look forward toward new challenges
Purpose in life	Purpose in life is critical to well-being. Individuals who score high on purpose in life feel that the present and future have meaning
Environmental mastery	Environmental mastery indicates the ability to choose or create suitable environments. Individuals who score high on environmental mastery are able to create or manage complex environments
Personal growth	Personal growth is the ability to continue to de- velop one’s potential over the life course. Indi- viduals who score high on personal growth are of- ten open to new experiences and challenges
Positive relations with others	Positive relations with others are associated with enhanced physical health. Individuals who re- port positive relations with others often have trusting relationships and strong social networks
Autonomy	Autonomy refers to self-determination or inde- pendence. Individuals with high levels of auton- omy often follow their own convictions and stan- dards

Patients in the current study were asked to rate each item on a scale of 1 to 6, with 1 indicating strong disagreement and 6 indicating strong agreement. Responses were totaled for each domain (potential range of 9-54) and higher scores indicated greater psychological well-being. Table 1 describes the 6 domains including self-acceptance, purpose in life, environmental mastery, personal growth, positive relations with others and autonomy.

Covariates

Covariates included sociodemographic characteristics (age, gender, ethnicity, marital status, and education) and length of stay. Age was used as a continuous variable with a range of 65 to 88. Marital status was categorized as married or unmarried (separated, widowed, divorced, or never married). Length of stay was used as a continuous variable, calculated in days.

Statistical Analysis

Sociodemographic characteristics were examined using descriptive and univariate statistics for continuous variables. Test-retest reliability was analyzed using the intraclass correlation (ICC) approach. The ICC expresses measurement error and agreement as the relationship between true and observed variance. The coefficients were derived from an analysis of variance model and assumed time (first vs second interview) as a fixed effect. Model assumptions for the ICC analysis were tested and met. All analyses were performed using SAS statistical software Version 9.0 (SAS Institute, Inc., 2000).

RESULTS

Table 2 shows the sociodemographic characteristics and average length of stay for the sample population (n=40). The mean age was 76.3 (SD=6.1), 57.5% were men and most were married (75.0%). The average level of education was 11.3 years (SD=4.0) and the majority of patients interviewed were white (72.5%). The average length of stay was 4.9 days (SD=3.1).

Table 2. Demographic characteristics and length of hospital stay for older patients admitted to the Acute Care for Elders (ACE) unit (n=40).

Characteristic	n	%	Mean	SD (range)
Total	40
Gender
Men	23	57.5
Women	17	42.5
Race
White	29	72.5
Black	4	10.0
Hispanic	6	15.0
Other	1	2.5
Married
Yes	30	75.0
No	10	25.0
Age (years)			76.3	6.1 (65-88)
Education			11.3	4.0 (2-21)
Length of stay (days)			4.9	3.1 (2-20)

Figure 1 shows correlation coefficient scores between admission and discharge interviews for patients across the 6 psychological well-being domains. Correlations were moderately to strongly associated and ranged from a low of 0.67 (environmental mastery) to a high of 0,82 (autonomy).

Fig. 1.

Individual correlations for domains of psychological well-being between admission and discharge (n = 40).

ICC values for the psychological well-being instrument were analyzed next. Findings indicated good consistency (test-retest reliability), with ICC values ranging from 0.66 to 0.79 across the six domains. Five of the 6 psychological well-being domains had ICC values greater than 0.70 including: self-acceptance (0.79, p<0.01), autonomy (0.79, p<0.01), purpose in life (0.79, p<0.01), personal growth (0.78, p<0.01) and positive relations with others (0.72, p<0.01). Environmental mastery had the lowest ICC value (0.66, p<0.01). Table 3 shows mean psychological well-being scores at admission and discharge for each domain. Mean domain scores did not significantly differ between admission and discharge interview, although mean scores did decrease across each of the 6 domains by about 1 point.

Table 3. Mean psychological well-being scores from admission and discharge from the Acute Care for Elders (ACE) unit (n=40).

Domains of psychological well-being	Admission Mean (SD)	Discharge Mean (SD)	p
Self-acceptance	44.9 (7.3)	43.3 (7.3)	0.33
Positive relations with others	44.5 (7.2)	43.7 (7.4)	0.62
Autonomy	43.6 (6.2)	41.8 (6.6)	0.21
Environmental mastery	44.9 (5.9)	44.1 (5.7)	0.56
Purpose in life	39.3 (8.3)	37.4 (9.3)	0.34
Personal growth	39.5 (7.6)	37.8 (7.4)	0.30

Table 4 shows the intercorrelations among the 6 domains of psychological well-being at admission. Intercorrelations ranged from −0.02 to 0.62. In general, low to moderate correlations were found between domain pairings, although higher correlations were observed between self-acceptance and autonomy (0.58) and between self-acceptance and environmental mastery (0.62).

Table 4. Intercorrelations of psychological well-being domains at admission to the Acute Care for Elders (ACE) unit (n=40).

r	SA	PR	AU	EM	PL	PG
SA	-
PR	0.39	-
AU	0.58	0.26	-
EM	0.62	0.38	0.48	-
PL	0.44	0.09	0.22	0.37	-
PG	0.24	−0.02	0.13	0.11	0.38	-

SA: self-acceptance; PR: positive relations with others; AU: autonomy; EM: environmental mastery; PL: Purpose of life; PG: personal growth.

We examined whether ICC correlations varied across social and demographic factors (gender, marriage, race, age, and education). None of these associations were statistically significant for any of the psychological well-being scales (data not shown).

DISCUSSION

The majority of psychological well-being instruments in use today were developed in relatively healthy community-based populations. In this study, the goal was to present reliability estimates for an established psychological well-being instrument in a sample of older patients hospitalized with an acute illness. Our results can be summarized as follows. The 6-domain psychological well-being instrument that included self-acceptance, purpose in life, environmental mastery, personal growth, positive relations with others and autonomy demonstrated good reliability among older patients with ICC values ranging from 0.66 to 0.79. Intercorrelations ranged from a low of −0.02 to a high of 0.62. Mean scores for each of the 6 domains did not significantly differ between admission and discharge interview. On average psychological well-being domain scores declined by about one point over the assessment period.

Overall, our findings indicated that a 6-domain psychological well-being instrument can produce consistent results in a sample population of older hospitalized adults, although our reliability estimates were marginally lower than what has been previously reported in community dwelling adults (14). From the original study that included 321 healthy men and women (14), findings showed that the internal consistency for the 6 domains ranged from 0.86-0.93 and test-retest reliability coefficients ranged from 0.81-0.88 over a 6-week period. Differences in reliability estimates between the current study and others is likely due to several factors including differences in how the instruments were administered (i.e., in person vs telephone interview), where the test was administered (hospital vs community) and differences in study populations (ill older patients vs healthy middle to older-aged adults).

Our results also suggested some degree of overlap between domains of self-acceptance and autonomy (0.58), and self-acceptance and environmental mastery (0.62). Other studies have also shown overlap between specific domains of the psychological well-being instrument. Ryff (14), for example, reported a strong correlation between self-acceptance and environmental mastery (0.76). Strong correlations also have been reported between self-acceptance and purpose in life (0.72) and between purpose in life and personal growth (0.72) (14). High intercorrelations reported in the current study and others suggest the possibility of fewer than 6 psychological well-being domains, and further suggest the need to explore the factor structure of this instrument in a hospitalized older patient population.

Nonetheless, results obtained from this study are important and indicate that the psychological well-being of acutely ill older adults can be reliably collected in a hospital setting. In using reliable instruments designed to assess the positive in a patient’s life, clinicians and health care providers may be in a better position to identify pathways that lead to recovery. A recent National Research Council (18) report has recommended the study of positive psychosocial factors in promoting health and well-being stating that findings are likely to contribute meaningful information into why some individuals do well and others do poorly in the face of a health challenge. Accreditation agencies have also recognized the importance of considering how the growing population of older patients view their health, and have recommended that the subjective evaluations of these patients be included as part of medical care received (19).

The use of a reliable psychological well-being instrument in the clinical setting may also prove useful in focusing attention on ways that could help narrow health disparities in underserved minority populations. A recent study by Sands et al. (20) that included 2364 patients aged 70 or older hospitalized with an acute illness found non-Hispanic blacks to report significantly less improvement in instrumental activities of daily living (IADLs) than non-Hispanic whites at a 1- and 3-month follow-up interview. Differences in 1ADL status could not be explained by demographics, in-hospital care, severity or type of admitting illness, or the initial level of functioning or cognitive impairment. The study suggested that differential rates of functional recovery may be due to other factors and to better understand the causes of ethnic disparities required broadening the focus from factors surrounding the acute illness and hospitalization (20). One recommendation of an IOM 2003 report “Unequal Treatment: Confronting Racial and Ethnic Disparities in Healthcare,” was that future research consider the patient’s attitudes and expectations in the clinical encounter (21, 22).

The current study has limitations. One limitation is the convenience sample of 40 older hospitalized adults. The demographics and personal characteristics of our sample are consistent with other groups of hospitalized older adults (17), but the sample was not randomly selected and the results cannot be generalized to other older adult populations.

While test-retest reliability is a standard psychometric approach to determine the consistency and repeatability of patient characteristics and behaviors, intraindividual reliability testing is subject to several limitations (23). Persons may remember and repeat previous responses and there may be a learning or practice effect present whenever measures are collected repeatedly. In the current study we found no statistically significant correlation between the interval between test and re-test and the ICC values for any of the six scales. We recognize, however, that establishing the reliability of an instrument is an on-going process and it will be important to replicate these findings using different samples of hospitalized older adults.

Another limitation is reliance on self-reports to collect information on personal characteristics and behaviors including psychological well-being. As with all self-report instruments, a bias may result if individuals respond to items in ways that are socially desirable. However, research examining patient perceptions has found that they are among the best predictors of use of medical services and are strongly correlated with function and mortality, even after clinical factors have been controlled (24). Psychological well-being is based on personal perception and there is growing recognition that person-centered measures of psychological attributes such as patient satisfaction and well-being are valuable and valid indicators of health status (13, 24).

In summary, our findings indicated that items from the psychological well-being instrument (14) can provide consistent information for hospitalized older adults and support its use in an acute care setting. Future studies using this instrument will contribute to our understanding the linkages between psychological well-being and factors that help patients cope and adapt to health challenges.