Consumer and Professional Ratings of the Importance of Functional Status Components

Abstract

As the population ages and chronic disease becomes the more dominant form of illness, measures of functional loss and disability assume greater importance in the assessment of both quality of life and the cost-effectiveness of care. The authors studied the responses of consumers and health care professionals regarding the impact on dependency of various levels of disability. Striking differences in perception were noted, raising concerns about the ability of those providing care to assume that the recipients share their values about what is important. This study makes clear the need for more research on functional outcome measurements that incorporate the values of consumers.

Introduction

Health care is becoming more patient-centered, or at least that goal is being more widely articulated. Patients' preferences and rights to play a central role in health care decisions are being more widely discussed. One area in which patient values are extremely important deals with the relative emphasis placed on the alternative outcomes of care. Whether judging how successful treatment has been or in weighing the risks and benefits of alternative treatment strategies, some means of summarizing the overall effects of care is necessary. As chronic disease becomes the dominant form of illness, functional status, which may encompass a wide variety of domains (including physical, emotional, and social functioning), assumes an ever-larger role as a means of describing outcomes.

Interest is growing in examining the more global effects of disease and treatment on overall functioning, and therefore attention has been focused on activities of daily living (ADLs) and instrumental activities of daily living (IADLs) and the need for a summary measure. In general, there is wide appreciation that such a measure should cover a broad spectrum, ranging from the basic activities needed to ensure independent function (ADLs) to more complex activities (IADLs) (Katz et al., 1963; Lawton and Brody, 1969). These two levels of function have been shown to form a general hierarchy, with IADLs being lost first (Spector et al., 1987). The basic ADLs have similarly been shown to exhibit Guttman properties of ordinality (Katz and Stroud, 1989).

The most commonly used summary measures of these functions have relied on simple sums of areas of dependency. For example, the inability to carry out 3 of 5 specified ADLs was proposed as the basis for eligibility for long-term care (LTC) benefits under the unsuccessful Clinton health proposal (Stone and Murtaugh, 1990). However, a simple summing, although administratively facile, loses substantial information. Such a step relies on the assumption that all included activities are of equal weight, or that they can only be lost in a fixed ordinality, such that the sum represents a cumulative report.

Indeed, neither is the case. Even a casual examination of one's own experience and beliefs would reveal that different ADLs make quite distinct contributions to overall disability. For example, some activities occur several times a day (e.g., using the toilet or eating), whereas others occur less often (e.g., transferring or dressing). Some IADL activities occur less often than daily (e.g., managing money or shopping).

The traditional approach to simply summing dependencies presents at least two types of problems: As already noted, it assumes that each type of functional loss is equivalent, and it requires that some arbitrary cut point be established to distinguish dependent from independent. For example, one might decide that needing a little assistance to perform an activity constitutes dependency, or, alternatively, the threshold for dependency might be set as needing a great deal or complete assistance. These various thresholds would result in quite different patterns of inclusion and exclusion for the counts of dependencies. Not only may different levels of dependency be treated as equivalent, but also, useful information can be lost, especially if the threshold is set high.

To create a summary ADL measure that would combine different levels of dependency within each ADL and IADL item, we conducted a survey of 27 professionals, representing the variety of disciplines involved in LTC. The professional respondents included 2 physicians (a psychiatrist and a geriatrician), 10 LTC administrators, 3 nurses, 2 social workers, a physical therapist, and a psychologist. We asked each respondent to rate the relative importance of both ADL and IADL domains and the levels of function within each domain, in terms of their contribution to overall disability (Finch, Kane, and Philp, 1995). This ratio scale was used as the dependent variable in studies of the outcomes of post-hospital care (Kane et al., 1996a, 1996b).

We chose to poll professionals on the basis that they were the ones who would be held accountable. However, a reasonable criticism could be raised about the importance of ascertaining the values from older consumers as well.

To address this concern, we interviewed frail older persons, using the same approach originally used with the LTC professionals. Our original goal was to recruit a sample of persons living in nursing homes, but too few of these individuals were able to perform the necessary cognitive tasks. Instead, we used a sample of older persons living in sheltered housing settings attached to nursing homes. These people were living on their own but receiving congregate meals and assistance and were judged to be at risk for LTC. Although members of this group were not incapacitated to the point of requiring institutionalization, they were no longer living independently in the community. Also, several of the ADL and IADL items used, such as house cleaning, do not apply to those living in a total institution such as a nursing home.

Methods

Residents in two sheltered housing programs attached to larger LTC campuses in Minneapolis were invited to participate in this study. Written notices were posted in several places in each setting, and staff helped to recruit volunteers. Older persons opting to participate met with project staff in each housing program's common area at a designated time. At each meeting the purpose of the study was explained and instructions given. The respondents completed the forms independently during the session with a project staff member available to answer any questions that arose. Nonetheless, the task proved too complicated for some participants. A total of 61 at-risk elderly participated in the research (30 in one sheltered housing site and 31 in the other). Out of the 61 forms, 10 had to be discarded because of incomplete data, yielding a total of 51 completed forms. To preserve anonymity no demographic data on the participants were collected.

The form used in the study was composed of two sections, corresponding to two rating tasks. The first section of the form contained a list of 13 ADL and IADL items (Table 1). As a reference point, the inability to bathe oneself was assigned an arbitrary score of 500. Participants were then instructed to rate the remaining 12 items in terms of their severity relative to the base established in the bathing item. If they believed that debilitation resulting from a given ADL or IADL presented was more severe than the bathing item, they were instructed to give it a higher score. Alternatively, if it was not as debilitating, then a lower score would be given. No restrictions were placed on the scores they could use. This first section provides a comparative weighting for all of the ADLs and IADLs (similar to Guttman scaling). The only limitation placed on the values that would be assigned was that nothing would be less than zero.

Table 1. Professional and Consumer Ratings for ADL and IADL Domains.

Functional Domain	Professional (N = 27)		Consumer (N = 51)
	Mean	Rank	Mean	Rank
Cleaning the House or Doing the Laundry	201	1	419	1
Shopping	224	2	503	3
Walking Outside	335	3	515	4
Using the Telephone	339	4	563	6
Preparing Meals	462	5	543	5
Taking Medications	481	6	669	8
Bathing	500	7	500	2
Dressing	537	8	613	7
Walking or Managing Wheelchair	800	9	720	9.5
Transferring	847	10	720	9.5
Using the Toilet	866	11	798	12
Feeding Self	1,005	12	732	11

NOTES: ADL is activities of daily living. IADL is instrumental activities of daily living.

SOURCE: Kane, R.L., Rockwood, T., Finch, M., and Philp, I., 1997.

The second section of the form asked the participant to assign a value between 0 and 100 to various dependency levels associated with each of the 13 ADL and IADL items. For each ADL and IADL item, three to four dependency levels were presented, depending on the item. The extreme categories were preassigned values; for complete independence, e.g., needing no assistance to bathe oneself, a value of zero was provided; alternatively, for complete dependence, e.g., unable to bathe oneself at all, a value of 100 was provided. Between these extremes, one or two levels of assistance (e.g., needing a little help to bathe oneself, needing a lot of help to bathe oneself) were presented. Participants were instructed to assign a value between 0 and 100 to these two levels of assistance. This portion of the questionnaire provided a relative weight for the dependence level for each ADL and IADL. Measuring dependence in this manner takes into account that “needing a little help” for one activity is not the same as “needing a little help” for another activity, thus the measure of dependence is specific to each ADL and IADL.

To compute the scoring to be used in the analysis, the value assigned in the first section was multiplied by the dependency value assigned in the second. This produces three or four scores for each ADL and IADL (one for each level of performance). In the state of complete independence for each ADL and IADL, one of these scores was always zero and hence was not used in the analysis. The remaining 34 scores can be ordered to generate a ranking for each of the ADLs and IADLs that incorporates an assessment of the importance of each item relative to the others (section one), as well as the item-specific weighting associated with dependence (section two).

Analysis

The analysis of the distribution of the 34 ADL variables among the groups drew upon three separate statistical techniques. A non-parametric test, Wilcoxon sign rank test (Hollander and Wolfe, 1973), and two parametric tests: Bonferroni Means test and ordinary least-squares regression were used to evaluate the differences between groups.

To compare the relative value of items developed through this process with another widely used weighted scale, the Barthel Index (Mahoney and Barthel, 1965), we used seven of the items from the Barthel Index that approximated those used in the current scale. The maximal value for these seven items was obtained and the relative value of the intermediate levels was calculated by dividing each by the total. Because the Barthel assigns greater weights to independence, and our approach assigns higher scores to greater dependence, we had to reverse the Barthel scoring approach to facilitate comparison.

Results

Table 2 compares the geometric means of the professional and consumer ratings. They are quite different. In 10 instances the consumer ratings are significantly different from the professional ratings. In four instances the opposite applies. In general the consumer ratings are higher for the IADL items, and the professional ratings higher for the more disabled levels of the ADLs. This pattern can be better seen when the two sets of ratings are displayed graphically as in Figure 1.

Table 2. Professional and Consumer Ratings for ADL and IADL Items.

Disability Item	Professional (N = 27)		Consumer (N = 51)
	Mean	Rank	Mean	Rank
Needs Help Cleaning the House	8,284	1	24,607	2
Needs Some Help With Shopping	11,503	2	29,985	5
Walks Outside	11,894	3	34,748	8
Needs a Little Help With Using the Telephone	15,075	4	28,210	4
Unable to Clean the House	20,196	5	41,960	11
Needs a Little Help With Bathing	21,442	6	21,323	1
Needs a Little Help With Dressing	21,828	7	25,372	3
Unable to Shop	22,407	8	50,392	19
Needs a Lot of Help With Using the Telephone	25,881	9	42,931	12
Needs Some Help With Preparing Meals	26,194	10	32,411	6
Needs Some Help With Medications	31,671	11	43,480	13
Unable to Walk Outside	33,518	12	51,568	20
Unable to Use the Telephone	33,981	13	56,372	23
Needs a Little Help With Transferring	45,986	14	35,392	16
Needs Help to Walk	38,517	15	35,686	9
Unable to Walk But Can Use Wheelchair	38,776	16	48,598	17
Needs a Lot of Help With Bathing	41,629	17	37,421	10
Needs a Lot of Help With Dressing	43,747	18	45,602	15
Unable to Prepare Meals	46,296	19	54,313	21
Unable to Manage Medications	48,144	20	66,960	27
Needs a Little Help With Feeding Self	48,305	21	34,632	7
Unable to Bathe Self	50,000	22	50,000	18
Needs a Little Help With Using the Toilet	50,423	23	43,843	14
Unable to Dress	53,781	24	61,372	25
Not Bedridden But Unable to Walk	71,459	25	62,431	26
Needs a Lot of help With Transferring	73,211	26	57,465	24
Needs a Lot of Help With Using the Toilet	78,050	27	67,137	28
Bedridden	80,018	28	72,058	29.5
Unable to Transfer	84,703	29	72,058	29.5
Unable to Use the Toilet	86,653	30	79,803	32
Needs a Lot of Help With Feeding Self	89,364	31	54,965	22
Unable to Feed Self	100,537	32	73,235	31

NOTES: ADL is activities of daily living. IADL is instrumental activities of daily living.

SOURCE: Kane, R.L., Rockwood, T., Finch, M., and Philp, I., 1997.

Figure 1. Mean Rating of Individual Activities of Daily Living (ADL) Items by Professionals and Consumers.

The level of agreement can also be explored in terms of the relative ranking given each rating by the two groups. Figure 2 shows the correlation between the rankings for professionals and consumers. A higher ranking implies more disability. The mean rank order assigned by one group was plotted against the mean rank of the other group assigned to the corresponding item. The overall slope (i.e., the correlation coefficient calculated) is quite high (0.816).

Figure 2. Correlation Between Consumers' and Professional Rank Ordering.

To get a better idea of where the differences in the two groups' ratings lay, we compared the ratings for the importance of complete disability in each major domain. The results are shown in Table 1. The level of agreement with regard to rank order is quite high, although once again the values assigned by consumers tend to be higher for IADLs, whereas the experts assign higher scores for the ADLs.

One way to reduce the apparent variation between these groups might be to norm the two sets of scores, reducing each to a comparable scale by dividing each set of ratings by the total score assigned by that group. However, such a step would not likely affect the distributions, because the total scores for the two groups are well within the same order of magnitude. The sum of the consumers' ratings was 1,546,923 and the experts was 1,442,879.

To estimate the effects of using one or the other set of weightings, we applied the weights to three hypothetical patients. One was chosen to represent a mildly disabled person with only IADL needs; the second was a severely physically disabled person; and the third was someone with functional losses associated with dementia. As shown in Table 3, the consumer-based score is much higher than that using the experts' weightings for the mildly disabled case (consistent with the earlier observed bias). The scores are also identical for the severely disabled person, despite the experts' higher weightings for several ADLs. The consumers' weightings also led to higher scores for the dementia case, again likely reflecting the IADL dependencies.

Table 3. Effects of Professional and Consumer Scores on 3 Hypothetical Cases.

Patient	Using Experts' Ratings	Using Consumers' Ratings
Patient 11	77,652	130,483
Patient 22	462,867	464,705
Patient 33	290,615	344,775

¹Needs help cleaning the house; needs some help shopping, preparing meals, and with medications.

²Unable to transfer or use the toilet; needs a lot of help feeding, bathing, and dressing; unable to walk but can use wheelchair; unable to prepare meals or clean the house; needs some help shopping.

³Unable to clean the house, manage medications, use the telephone, or prepare meals; needs a little help feeding self, dressing, bathing, and using the toilet.

SOURCE: Kane, R.L., Rockwood, T., Finch, M, and Philp, I., 1997.

Another way to compare the effects of using consumers' or experts' values is to compare their magnitude with those from established scales. Perhaps the best known disability scale is the Barthel Index, which was created to measure the effectiveness of rehabilitation (Granger, Albrecht, and Hamilton, 1979). The Barthel Index uses a total of 100 points across 10 dimensions. Just the opposite of the approach we have used, it assigns maximal points to states of independence in each domain. Thus, its point values distinguish levels of independence but render total dependency in each dimension equivalent (i.e., by assigning all values of zero). (By contrast, we would argue that it makes more clinical sense to treat independence in all domains as equivalent and discriminate among causes of dependence.) Only 7 of the 10 Barthel items correspond to items in the current scale. (Grooming, ascending or descending stairs, and bowel control are not included in our domains.) To create a more comparable scale, we reversed the Barthel weights, assigning the maximum value used by Mahoney and Barthel (1965) for each item to the most dependent state.

Table 4 presents a comparison of the item values generated from consumer and expert ratings and the corresponding value from the modified Barthel score. Both are created such that complete dependence in all domains yields a value of 75 (i.e., the value of the original 7 Barthel items). Using either set of values produces quite different relative emphasis from those based on the Barthel weights for complete dependence. Compared with the Barthel values, feeding, using the toilet, bathing, and incontinence receive more importance, whereas transferring and walking receive less. This difference may reflect the rehabilitation context for which the Barthel Index was originally developed, as opposed to the more general geriatric context of the new scale. The new scale also permits more gradations for the intermediate dependency state. In general the Barthel state for needing help seems to correspond closer to “needing a little help” in the new scale. The same distinctions between professional and consumer values noted earlier apply here as well, although they are muted by the extensive rounding.

Table 4. Comparison of Consumer and Professional Ratings to Modified Barthel Rating.

Item	Barthel Value1	Consumer Mean	Professional Mean
Feeding
Unable	10	12	14
Needs Help	5	—	—
A Lot	—	9	12
A Little	—	5	7
Transferring
Unable	15	11	11
A Lot of Help	10	9	10
A Little Help	5	6	6
Using the toilet
Unable	10	13	12
Needs Help	5	—	—
A Lot	—	11	11
A Little	—	7	7
Bathing
Unable	5	8	7
Needs Help	—	—	—
A Lot	—	8	5
A Little	—	3	3
Walking on level surface
Unable	15	—	—
Bedridden	—	11	11
Not Bedridden/Unable to Walk	—	10	10
Needs Help	5	6	5
Uses a Wheelchair	10	8	5
Dressing
Unable	10	10	7
Needs Help	5	—	—
A Lot	—	7	6
A Little	—	4	3
Controlling Bladder
Frequent Accidents	10	12	14
Occasional Accidents	5	10	7

¹Actual Barthel value has been reversed to correspond to scale used in this study.

SOURCE: Kane, R.L., Rockwood, T., Finch, M., and Philp, I., 1997.

Discussion

Some limitations to this study should be acknowledged. The samples of respondents for both the professional and consumer groups were chosen on the basis of convenience and cannot be said to accurately represent any larger population. Ideally, one might have wished to tap the values of persons already actively involved in receiving LTC, but the rating task proved too complex for most of the nursing home residents we first approached. Instead, we were left to work with older persons who were somewhat more intact but nonetheless were living in adjacent facilities in situations that afforded at least some components of assistance with daily living. To address concerns about representativeness, one may wish to replicate this study on a more randomly selected sample.

It should also be noted that the samples of professionals and consumers were totally independent. The professionals did not provide care for these specific consumers. Hence, their points of reference were separate. This is not a study of agreement about the rating of dependency for specific individuals. Rather, it addresses the emphasis that different constituencies place on the different elements and the levels of dependence within each.

The numbers generated by this approach can be daunting. Scores that span a range from 0 to 1.5 million are too large to be clinically useful. For practical purposes of communication, it may prove easier to take advantage of the ratio scale properties to convert the scores developed into a more familiar range of 0 to 100. Because the emphasis was on dependence, higher scores were assigned to reflect greater dependence. This approach corresponds to that used by Katz et al. (1963), but, as noted in the text, another widely used scoring system, developed by Mahoney and Barthel (Mahoney and Barthel, 1965) does just the opposite, assigning higher values to greater independence.

The results from these admittedly convenient samples suggest that older consumers and aging experts place quite different importance on various aspects of disability. Consumers viewed dependency in IADLs as more of a loss than did the professionals; the professionals were more concerned about inability to perform basic ADLs.

This discrepancy raises several issues. At a minimum, these findings raise serious red flags about the ability of those providing care to assume that the recipients share their values about what is important. Providers who feel comfortable about exercising judgments on behalf of their clients may be motivated to give some additional thought before undertaking this task.

The major importance of using some approach to value weighting and being sensitive to whose values are incorporated lies in developing measures that reflect the importance of outcomes. Outcomes should reflect the importance of the individual components to those who must live with the consequences. Although ADL scores may also be used as criteria for allocation of LTC, value weighting is less relevant in that pursuit. The relative importance of individual ADL and IADL items in determining the need for LTC is best estimated by examining the statistical relationship of each to maintaining an independent existence or receiving regular assistance. Although the concepts seem closely related, the elements that are believed to be most important to a person may not necessarily be those that lead to a need for institutional care.

In an era of consumer-centered care, it behooves health professionals to understand more about the preferences of their clients. If patients and providers of care hold different views about what types of losses are most important, it seems unlikely that both parties will agree on the best course of treatment, if that treatment has any specific effects. If one assumes that LTC produces only diffuse results with little specificity, then the differences in preferences may matter little. Indeed, if such an assumption is correct, little measurement is needed, because one type of effect will be much like another. The only determining feature will be the nature of the order of functional loss, which must presumably occur independently of disease or treatment. However, if one adopts a more focused model, in which treatment is intended to address particular problems, then the major benefits will be measured by the relevant outcomes, but consideration needs to be given to the effects on overall function as well. Some method for summarizing this state should prove useful. If the summary is sensitive to the source of the values, as indicated here, this effect should be carefully considered in developing the summary procedure.

From a policy perspective, an outcome system that incorporates the values of the appropriate constituency will be more likely to detect relevant changes. The question then arises of whose values should be primary. The finding of substantial differences in professional and consumer ratings suggests at the least that a great deal more discussion is needed.

Our society supports greater consumer sovereignty. The results of this study suggest that professionals cannot adequately represent the values of consumers. The question then is how to best integrate consumer values. At one level, some argue that consumers should make the ultimate choices about what is done to or for them. In most instances consumers have that right. However, making such a decision correctly involves a number of complicated steps and a great deal of information. In essence, the relative effectiveness of alternative therapies must be weighed. For each one, the values assigned to each risk and benefit must be assessed. In practice, few decisions are made this precisely or deliberately. Even those who might wish to pursue a structured decision process will find themselves missing many critical elements of information about the efficacy of alternative approaches.

Nonetheless, a more structured approach to decisionmaking is indicated. One step in that process is helping consumers to identify what they truly hope to accomplish as a result of the care they seek. To estimate these goals, consumers have to first articulate what they seek and then assign some type of importance weights to the various competing ends. The underlying premise of this effort is that one important role for consumers lies in establishing the relative priority assigned to various outcome states. It is useful to consider how such a goal can be achieved.

If health care professionals are to become more active advocates of better decisionmaking for consumers and facilitators, they will need some sort of structured approach to the process. The techniques described here represent one step in that direction. Although one may want or need to go through the entire rating exercise each time an important decision is made, some method of deliberately considering alternatives and estimating their relative importance is warranted. Pressures of time and money will make it unlikely that deliberate decisionmaking is employed often. But such omissions may prove unwise, both monetarily as well as professionally. Consumers will have to accept more responsibility for the results of decisions in which they participate actively. Whether expressed in terms of increasing satisfaction or avoiding malpractice litigation, it behooves providers to consider investing the time to truly probe consumer expectations.

The results reported here emphasize mean differences between samples of professionals and consumers. The differences in the means are useful in pointing to discrepancies that should be addressed when considering the extent to which professional judgment can be used in lieu of specifically eliciting consumer preferences. The ultimate role for consumer preferences would be to allow each individual to identify the aspects of outcomes that he or she wants to maximize in a given situation. Rather than assigning some uniform set of weights based on the average values of a dominant party or some type of compromise, it might be preferable to allow each patient to establish his or her own values about the relative importance of different outcomes. Such an approach would make it difficult to compare successes among different patients unless one were prepared to assess outcomes specifically in terms of the extent to which they met patient preferences.

An issue worthy of future research would be the extent to which consumers' values vary among themselves. The present sample is too small and too limited to test the effects of varying levels of consumer disability on perceptions of importance. Other work that examines patients' utilities suggests that these can differ widely between those who have a given problem and those who seek to avoid contracting it (Torrance, 1987). Similarly, those already quite dependent may view the elements contributing to dependency differently from those who are less impaired.