Measuring disability in older adults: The International Classification System of Functioning, Disability and Health (ICF) framework

Abstract

Background

Despite the importance of disability to geriatric medicine, no large scale study has validated the activity and participation domains of the International Classification System of Functioning, Disability, and Health (ICF) in older adults. The current project was designed to conduct such as analysis, and then to examine the psychometric properties of a measure that is based on this conceptual structure.

Methods

This was an archival analysis of older adults (n = 1388) who had participated in studies within our Claude D Pepper Older Americans Independence Center. Assessments included demographics and chronic disease status, a 23-item Pepper Assessment Tool for Disability (PAT-D) and 6-min walk performance.

Results

Analysis of the PAT-D produced a three-factor structure that was consistent across several datasets: activities of daily living disability, mobility disability and instrumental activities of daily living disability. The first two factors are activities in the ICF framework, whereas the final factor falls into the participation domain. All factors had acceptable internal consistency reliability (>0.70) and test–retest (>0.70) reliability coefficients. Fast walkers self-reported better function on the PAT-D scales than slow walkers: effect sizes ranged from moderate to large (0.41–0.95); individuals with cardiovascular disease had poorer scores on all scales than those free of cardiovascular disease. In an 18-month randomized clinical trial, individuals who received a lifestyle intervention for weight loss had greater improvements in their mobility disability scores than those in a control condition.

Conclusion

The ICF is a useful model for conceptualizing disability in aging research, and the PAT-D has acceptable psychometric properties as a measure for use in clinical research.

Introduction

The earliest and most prominent conceptual framework on disability was described by sociologist Saad Nagi.¹ It was subsequently given formal attention in the medical community through the World Health Organization’s (WHO) International Classification System of Impairments, Disability and Handicaps (ICIDH-1).² Recently, the ICIDH-1 was revised through the International Classification System of Functioning, Disability, and Health (ICF).³ The ICF model states that disabilities include a range of behaviors that can be partitioned into those that involve either discrete tasks/actions; namely, “activities” or “participation” in life situations.

In the late 1980s, we constructed a brief 23-item measure of difficulty with functioning – now called the Pepper Assessment Tool for Disability (PAT-D) – that targeted both discrete tasks and several social/role behaviors for use in the Wake Forest University Claude D Pepper Older Americans Independence Center.⁴ Because we now have PAT-D data on over 1000 older adults with a variety of chronic health conditions, we have an excellent opportunity to examine the structural integrity of the activity/participation distinction espoused by the ICF. In addition, our existing datasets enable us to evaluate the psychometric properties of the PAT-D in relation to this new dimensional structure. Jette et al.,⁵ using items from the late-life function and disability instruments, have provided initial support for the ICF framework;^6,7 however, as noted by these authors, their study relied on a relatively small convenience sample of older adults (n = 150), and the participation items had a different response scale than the activity items.

Methods

Measures

Pepper assessment tool for disability

The PAT-D self-administered questionnaire consists of 23 items that include a range of activities that assess mobility, activities of daily living (ADL) and instrumental activities of daily living (IADL). Responses are made on a five-point Likert scale ranging from 1 (“usually did with no difficulty”) to 5 (“unable to do”) or a box can be checked that reads “usually did not do for other reasons”.⁴

Performance measure

Participants completed a 6-min walk test as an objective measure of physical function. This measure has good validity and reliability for use with older adult populations⁸ and has been used in our own Pepper Center to assess mobility disability.⁹ Performance is measured as the total distance covered in meters.

Data

Data are from four studies of older adults that included the PAT-D: (i) the Observational Arthritis Study in Seniors (OASIS, n = 480);¹⁰ (ii) the Arthritis, Diet and Activity Promotion Trial (ADAPT, n = 318);¹¹ (iii) the Reconditioning Exercise and Chronic Obstructive Pulmonary Disease Trial (REACT, n = 291);¹² and (iv) the Trial of Angiotensin Converting Enzyme Inhibition and Novel Cardiovascular Risk Factors (TRAIN, n = 299).¹³ The protocols for these studies were approved by the Institutional Review Board and all participants completed informed consents. OASIS was a 3-year observational study designed to determine the mechanisms underlying the progression of disability in older men and women who had chronic knee pain. ADAPT was a clinical trial comparing the efficacy of long-term exercise and dietary weight loss, alone or in combination, on improvements in physical function and physical symptoms in older, overweight and obese adults with knee osteoarthritis. REACT was designed to determine the effect of long-term exercise training on the physical functioning of older adults with chronic obstructive pulmonary disease. Finally, TRAIN was a double-blind cross-over randomized placebo-controlled trial of older adults who were at high risk for cardiovascular disease (CVD) that assessed the effects of 6 months of treatment with either fosinopril or placebo on selected biomarkers of inflammation.

Psychometric procedures

Exploratory factor analysis

The empiric factor structure of the PAT-D items was analyzed using exploratory factor analysis (EFA). Our emphasis was on the delineation of meaningful underlying factors as revealed by the data and the interpretation of these factors. Scree plots and other methods outlined below were used to assess the dimensional structure of the PAT-D as follows:

1. Standard EFA was conducted on each dataset with comparisons made across the different datasets.

2. Based on the results obtained in 1, we combined datasets and performed another EFA. We then compared the results of the combined analysis with each individual dataset to assess robustness and stability of the factor structure.

3. In the final step, we refined the final factor model to enhance the interpretability of the scales by removing items that were problematic.

Internal consistency reliability

Measures of internal consistency for each domain of the PAT-D as determined from EFA were assessed at baseline. An alpha of 0.7 or higher was considered indicative of measure reliability.¹⁴ The internal consistency of the overall summary score was measured by the Pearson correlations of each domain score with the summary score.

Test–retest reliability

Test–retest reliability was measured by comparing scores across two different time points among respondents in the control group of the ADAPT study who had no important changes in functioning in the period between assessments. No change was operationalized as a difference in walking speed of less than 2 standard deviations (SD, 0.40 m/s). Strong correlations and intraclass correlation coefficients (ICC, >0.7) were used as support for the stability and reliability of the PAT-D.^14,15

Construct validity and sensitivity to change

Construct validity was evaluated using three different approaches. First, we conducted a median split on the 6-min walk test data and examined the summary and domain scores for the PAT-D for these two groups. The hypothesis was that slow walkers would have poorer scores on the PAT-D summary and domain scales than fast walkers. Second, using the ADAPT data, we correlated the PAT-D total score with the WOMAC index of functional status¹⁶ to assess convergent validity. Third, we compared PAT-D scores for individuals with and without three chronic diseases that are known to influence self-reported physical functioning: CVD, diabetes and cancer. Based on existing work in the area of self-reported physical health,¹⁷ we hypothesized that the most severe effects would be observed for CVD and diabetes.

Finally, to evaluate sensitivity to change, we examined PAT-D scores in the ADAPT study. Recall that this was an 18-month randomized clinical trial of older, obese adults who had knee osteoarthritis and who were assigned to one of four different treatment groups: (i) health education control; (ii) diet only; (iii) exercise only; and (iv) diet plus exercise. In a previous publication, we demonstrated that participants in the diet plus exercise treatment condition made significant improvement in 6-min walk time as compared to the control group.¹¹ Thus, we predicted that participants assigned to the diet plus exercise treatment group would experience the greatest 18-month change in self-reported physical function, particularly in the mobility domain because this was the target of the ADAPT intervention. The diet plus exercise treatment arm in ADAPT was a standard cognitive behavioral program for treating obesity. It involved: (i) an intensive phase of one individual and three group meetings each month for 4-months; (ii) a transition phase involving contact every other week for 2 months that consisted of three group meetings and one individual meeting; and (iii) a maintenance phase that involved group meetings and phone contacts alternated every 2 weeks. Exercise was prescribed three times each week and consisted of 30 min of aerobic work and 15 min of resistance training. After 18-months, the average weight loss from the intervention was 5.7% of bodyweight.¹¹

Results

Participant characteristics

Characteristics of the participants in the four studies are shown in Table 1. The mean age for all studies was 65 years or older. There were similar numbers of men (n = 725) and women (n = 657) across studies. Most participants were white and had at least a high school education.

Table 1.

Participant characteristics (mean standard deviation and n percent) for studies examined

Variable	Study
	ADAPT (n = 318)	REACT (n = 316)	TRAIN (n = 299)	OASIS (n = 480)
Age	68.52 (6.21)	67.60 (8.46)	65.94 (7.41)	71.82 (5.00)
Gender
Male	88 (27.85)	173 (54.75)	168 (56.95)	235 (48.96)
Female	228 (72.15)	143 (45.25)	127 (43.05)	245 (51.04)
Race
White	240 (75.95)	276 (87.34)	220 (74.58)	396 (82.50)
African American	70 (22.15)	39 (12.34)	69 (23.39)	63 (13.13)
Hispanic	1 (0.32)	1 (0.32)	1 (0.34)	0 (0.00)
Other	5 (1.59)	0	5 (1.69)	21 (4.38)
Education
<High school diploma	35 (11.51)	38 (12.03)	31 (10.44)	96 (20.04)
High school graduate	56 (18.42)	62 (19.62)	77 (25.93)	105 (21.92)
Some college	113 (37.17)	124 (39.24)	88 (29.63)	135 (28.18)
College graduate	41 (13.49)	39 (12.34)	55 (18.52)	60 (12.52)
Graduate/professional	59 (19.41)	53 (16.77)	46 (15.49)	83 (17.33)

ADAPT, the Arthritis, Diet and Activity Promotion Trial; OASIS, the Osteoporosis and Sodium Intake Study; REACT, the Reconditioning Exercise and Chronic Obstructive Pulmonary Disease Trial; TRAIN, the Trial of Angiotensin Converting Enzyme Inhibition and Novel Cardiovascular Risk Factors.

Factor structure

The factor structure and scree plots across the studies were highly similar. The three largest eigenvalues were greater than 1 and were well separated from the remaining eigenvalues, supporting a three-factor structure. Consequently, we fitted a three-factor solution with varimax rotation to each dataset and examined the factor structure for each. After appropriately reordering the factors across the studies, all four studies again had similar factor loading patterns. Thus, we combined the datasets across the studies; Table 2 shows the three-factor solution for this merged dataset (n = 1379).

Table 2.

Factor structure of Pepper Assessment Tool for Disability (PAT-D, n = 1379): item loadings for varimax rotation

Item	Factor 1 Basic ADL	Factor 2 Mobility	Factor 3 IADL
Getting in and out of the car	0.701	0.290	0.269
Using the toilet including getting on and   off the toilet	0.663	0.185	0.236
Moving in and out of a chair	0.654	0.350	0.187
Moving in and out of a bed	0.633	0.294	0.280
Dressing yourself	0.514	0.147	0.497
Bathing or showering	0.484	0.312	0.433
Gripping with your hands	0.410	0.226	0.217
Feeding yourself†	0.222	0.043	0530
Raising your arms above your head†	0.360	0.263	0.365
Walking several blocks	0.175	0.767	0.192
Walking one block	0.139	0.702	0.328
Climbing several flights of stairs	0.356	0.700	0.073
Lifting heavy objects	0.244	0.645	0.134
Climbing one flight of stairs	0.377	0.640	0.113
Lifting/carrying something as heavy as   10 lbs	0.280	0.625	0.281
Taking care of a family member	0.380	0.449	0.417
Visiting with relatives or friends	0.217	0.303	0.628
Participating in community activities	0.184	0.451	0.550
Doing light housework	0.153	0.456	0.498
Managing your money	0.149	0.121	0.497
Using the telephone	0.216	0.028	0.474
Doing errands†	0.336	0.494	0.479
Preparing your own meals†	0.141	0.309	0.631

^†Denotes that these items were removed from the final subscales. Instructions for completion of the measure read as follows: “How much difficulty, if any do you have with each of these activities? Think about the past month. How hard was it to do the activity because of your health?”. ADL, activities of daily living; IADL, instrumental activities of daily living.

Upon reviewing the items within each domain, we decided to remove four items from the list. The item “doing errands” was excluded from the IADL subscale because its factor loadings were evenly distributed over the three domains, which might be a consequence of the generic wording of the item. Another item on this same scale, “preparing your own meals”, was also removed because of concerns regarding gender bias. In addition, two items were removed from the ADL scale due to severe floor effects: “feeding yourself” and “raising your arms above your head”. As a result of these exclusions, the final version of the PAT-D contained 19 items that had a factor structure supporting the activity and participation domains of the ICF model of disability; that is, the basic ADL and mobility scales belong to the activity domain, whereas the IADL scale represents the participation domain. All three scales fall under the rubric of “disability” in that the PAT-D summary score has acceptable psychometric properties that are described below.

Internal consistency and test–retest reliability

The Cronbach’s alphas for internal consistency for the three domains and overall summary score were all above 0.7: α = 0.77 for IADL, α = 0.76 for ADL, α = 0.87 for mobility and α = 0.82 for the summary score.

Means, standard deviations and ranges for the three domains and summary scores are presented in Table 3. The possible ranges for the domains were as follows: 7–35 for basic ADL, 6–30 for IADL, 6–30 for mobility and 19–95 for the summary score. Scaled scores, which transform raw scores back to the original 1–5 Likert scale format, are also shown in Table 3. Only 4% of subjects were excluded in the test–retest analysis because their walking speed significantly changed. All of the Pearson correlations for test–retest reliability except the basic ADL domain (ρ = 0.65) were above 0.7, whereas all of the ICC were in the range 0.6–0.7.

Table 3.

Description of domain and summary PAT-D scores (n = 1379) and test–retest data

Domain	Mean (SD)	Scaled mean† (SD)	Min	Max	r†	ICC†
Basic ADL	11.15 (3.98)	1.57 (0.57)	7.0	29.29	0.81	0.91
Mobility	13.51 (5.29)	2.25 (0.88)	6.0	30.00	0.77	0.89
IADL	8.37 (2.91)	1.39 (0.49)	6.0	27.86	0.68	0.85
Summary	33.03 (10.73)	1.73 (0.56)	19.0	82.00	0.85	0.93

^†Scale mean is on a scale of 1 (“usually did with no difficulty”) to 5 (“unable to do”).

^†Based on ADAPT control group with n = 75. ICC, intraclass correlation coefficients; SD, standard deviation.

Construct validity and sensitivity to change

Comparisons of summary and domain scores on the PAT-D for slow (mean [SD] speed = 1.00 [0.17] m/s) versus fast (1.35 [0.12] m/s) walkers can be found in Table 4. In every case, fast walkers self-reported better function on the PAT-D than slow walkers. The effect sizes for the summary and domain scores ranged from moderate to large (0.41–0.95). In addition, evidence for the convergent validity of the PAT-D is evident from significant correlations with the WOMAC at the baseline, 6- and 18-month follow-up visits in ADAPT: Pearson correlations for each visit were 0.65, 0.69 and 0.73, respectively (P < 0.0001).

Table 4.

Comparison of PAT-D scores for slow versus fast walkers in ADAPT

Domain	Slow walkers (n = 170)		Fast walkers (n = 144)		Effect size	P-value
	Mean (SD)	Scaled mean (SD)	Mean (SD)	Scaled mean (SD)
Basic ADL	13.37 (4.33)	1.91 (0.62)	11.93 (3.54)	1.70 (0.51)	0.41	0.001
Mobility	16.07 (5.62)	2.67 (0.94)	12.47 (3.77)	2.08 (0.62)	0.95	<0.0001
IADL	9.79 (3.52)	1.63 (0.59)	8.16 (2.33)	1.36 (0.39)	0.67	<0.0001
Summary	39.23 (12.04)	2.06 (0.63)	32.56 (8.36)	1.71 (0.44)	0.80	<0.0001

We also conducted analyses between the PAT-D scores of older adults with and without three chronic conditions: CVD (n = 309; with : without disease ratio, 171:138), diabetes (n = 300; with : without disease ratio, 270:30) and cancer (n = 302; with : without disease ratio, 255/47). Those with CVD reported greater disability on all PAT-D scales as compared to those without the disease (P < 0.001). Effect sizes for the CVD comparisons were consistent across scales and ranged 0.51–0.56. Effect sizes were similar in magnitude and in the expected direction for diabetes; however, the IADL scale did not reach conventional levels of statistical significance (P = 0.06) with an effect size of 0.37. For cancer, the effect sizes for the PAT-D scales were smaller with both the mobility and IALD scales failing to reach statistical significance: 0.38 for basic ADL (P = 0.02), 0.29 for mobility (P = 0.07), 0.17 for IADL (P = 0.28) and 0.33 for the summary score (P = 0.04).

Finally, the descriptive data and Student’s t-test results on the 18-month differences between the control and diet plus exercise group for each of the PAT-D scales can be found in Table 5. As expected, changes in mobility appear to be driving changes in the summary score; however, it is interesting to note that both basic ADL and IADL scores trend in a direction that favors the active treatment group.

Table 5.

Control versus diet plus exercise group differences in PAT-D scores in ADAPT

Domain	Control (n = 70)		Intervention (n = 63)		Effect size	P-value
	Mean (SD)	Scaled mean (SD)	Mean (SD)	Scaled mean (SD)
Basic ADL	12.19 (4.47)	1.74 (0.64)	10.90 (4.19)	1.56 (0.60)	0.29	0.10
Mobility	14.01 (5.65)	2.34 (0.94)	12.02 (5.19)	2.00 (0.87)	0.35	0.04
IADL	9.11 (3.47)	1.52 (0.58)	8.04 (2.85)	1.35 (0.49)	0.31	0.07
Summary	35.40 (12.27)	1.81 (0.65)	31.05 (11.40)	1.63 (0.60)	0.36	0.04

Discussion

The factor analyses support the activity and participation domains proposed by the ICF framework. Two scales from the PAT-D fall into the realm of the ICF classification of an activity and were labeled basic ADL and mobility. The third scale, IADL, fell into the participation domain in that it captured several important activities that are embedded in social roles. Parenthetically, these three scales are consistent with the conceptual structure of disability as articulated by Katz¹⁸ and reinforced in previous validation work on the ICF framework by Jette.⁵

Two comments on the overall factor structure of the PAT-D seem warranted. First, the original version of this measure published in 1995 had 23 items.⁴ In the current version, four items were deleted for one of three reasons: floor effects, item ambiguity and potential gender bias. Floor effects can be problematic in that they place constraints on the generalizability of a measure. In this regard, there is a movement toward the use of computerized adaptive testing (CAT) in the development of new measures that assess functioning.^6,7 This technology permits the development of a pool of items that vary widely in level of difficulty. Through the use of mathematical algorithms, participants are quickly directed to questions of item difficulty that match their capacities. This enables a single instrument to be used with participants who have varied levels of ability and can be employed across multiple settings, for example, in the context of independent community-based populations or with those who reside in assisted-living facilities.

Second, five of the seven items for the ADL domain had strong item loadings that were unique to this scale. Two items – “dressing yourself” and “bathing/showering” – cross-loaded on the IADL domain. This would appear to be due to the relatively low physical demands of the tasks associated with these items. Similarly, three items out of six identified as the IADL domain – “taking care of a family member”, “participating in community activities” and “doing light housework” – cross-loaded on the mobility domain. We do not view these cross-loadings as problematic because the decision to place any individual item within a particular domain is based both on empiric and conceptual rationale. Moreover, the scree plots and Cronbach alpha reliabilities from the EFA support the three-factor structure of the PAT-D.

The secondary aim involved evaluating the construct validity and sensitivity to change of the PAT-D. Consistent with the study hypotheses, we found that participants labeled as fast walkers in the ADAPT study reported significantly lower disability in the area of basic ADL, mobility and IADL than slow walkers. Convergent validity was supported by the moderate correlation between the PAT-D and the WOMAC measure in ADAPT. Additionally, as compared to individuals with no chronic disease, participants with a history of CVD and diabetes self-reported greater levels of disability than those with cancer. This pattern across the three diseases is consistent with published data on the SF-36 Physical Health Composite Index.¹⁷ Finally, the mobility and summary scores of the PAT-D were favorably influenced by an 18-month weight loss intervention in ADAPT.¹¹ These data illustrate that the PAT-D is sensitive to change even in a long-term lifestyle intervention that had modest levels of adherence.

In summary, data from this study support the “activity” and “participation” domains proposed by the ICF for disability research in older adults and captures some of the higher levels of competence described by Shibata et al.¹⁹ The PAT-D has acceptable psychometric qualities and enables researchers to evaluate the effects of interventions on three specific domains of disability: basic ADL, mobility and IADL. With this new factor structure in place, we plan to reexamine existing Pepper datasets to ascertain whether the domain scores of the PAT-D offer new insights into the relationship between lifestyle interventions and disability that may have been masked in our prior analyses by reliance on a single summary score.