Functional status after critical illness: agreement between patient and proxy assessments

Abstract

Background: assessment of baseline functional status of older patients during and after intensive care unit (ICU) admission is often hampered by challenges related to the critical illness such as cognitive dysfunction, neuropsychological morbidity and pain. To explore the reliability of assessments by carefully chosen proxies, we designed a discriminating selection of proxies and evaluated agreement between patient and proxy responses by assessing activities of daily living (ADLs) at 1 month post-ICU discharge.

Methods: patients ≥60 years old admitted to the medical ICU were enrolled in a prospective parent cohort studying delirium. Proxies were carefully screened at ICU admission to choose the best available respondent. Follow-up interviews, including instruments for ADLs, were conducted 1 month after ICU discharge. We examined 179 paired patient–proxy follow-up interviews. Kappa statistics assessed inter-observer agreement, and McNemar's exact test assessed response differences.

Results: patients averaged 73.3 ± 8.1 years old with 29% having evidence of cognitive impairment. Proxies were most commonly spouses (38%) or children (39%). Overall, there was substantial (κ ≥ 0.6) to excellent agreement (κ ≥ 0.8) between patients and proxies on assessment of all but one basic and one instrumental ADL.

Conclusion: proxies carefully chosen at ICU admission show high levels of inter-observer agreement with older patients when assessing current functional status at 1 month post-ICU discharge. This motivates further study of proxy assessments that could be used earlier in critical illness to assess premorbid functional status.

Introduction

Functional impairment has been associated with increased mortality in older patients requiring intensive care unit (ICU) admission [1–3], and it has been associated with other morbidities including increased costs, in-hospital events, and longer ICU and hospital lengths of stay [3, 4]. ICU survivors may have ongoing functional impairment beyond baseline [5, 6]. As a measure of disability that can include ability to perform activities of daily living (ADLs), functional impairment is commonly assessed by patient interview. Twenty per cent or more of older persons may be unable or unwilling to participate in health surveys [7], and at least half of hospitalised older patients cannot be directly interviewed because of limitations that include cognitive or sensory deficits [8, 9]. Such challenges may still be barriers to assessing functional impairment during recovery from critical illness due to well-described sequelae in ICU survivors such as chronic pain, weakness and neuropsychological morbidities [5, 6, 10, 11]. Thus, a reliable proxy instrument for assessment of functional status following critical illness would be useful. As such, the primary aim of this study is to determine the utility of using well-screened proxies to assess ADLs for older ICU patients surviving to 1 month post-ICU discharge.

Methods

This study is a secondary analysis of a prospective medical ICU cohort of adults ≥60 years old. The parent study has been described in detail previously [12–14]. Proxies and/or patients provided informed consent according to procedures approved by the Institutional Review Board of the Yale University School of Medicine.

Data collection

At the time of ICU admission, patient proxies were screened for having a long-term relationship with the patient and for seeing the patient frequently to assure that they could answer questions about patient status. Details of the screening procedures have been described elsewhere [12, 13] and are detailed in Figure 1.

Figure 1.

Proxy screening protocol.

Patients surviving to 1 month post-ICU discharge and their proxies had follow-up interviews wherein current functional status was assessed. Interviews were conducted independently with each respondent blinded. If patients were still hospitalised or living in a nursing facility 1 month post-ICU discharge, questions about IADLs were not attempted for lack of relevancy; questions about BADLs were attempted, but answers were unknown for some respondents because of the patient's dependent status. Details about interview content are available in the Supplementary data available in Age and Ageing online. Reasons for partial interviews are shown in Supplementary data, Figure S1 available in Age and Ageing online.

Outcome

The primary outcome was an agreement between patients and proxies in assessment of ADLs 1 month post-ICU discharge.

Statistical analysis

Descriptive statistics were ascertained as appropriate. Simple percent agreement between patients and proxies was calculated. Kappa statistics (κ) were used to assess inter-observer agreement and as an index of agreement exceeding chance. McNemar's exact tests were used to assess systematic over- or underrating of functional status by proxies compared with patients' self-assessments [15]. Intra-class correlation coefficients (ICCs) assessed agreement between composite scores of BADLs and IADLs to provide a global view of agreement [15]. Additional details are in the Supplementary data available in Age and Ageing online.

Analyses were performed using SAS version 9.3 (SAS Institute, Inc., Cary, NC, USA).

Results

Of the 309 patients enrolled in the parent cohort at ICU admission, 214 were alive 1 month after ICU discharge and thus were eligible for the current study. Of these, 205 (96%) completed 1 month post-discharge interviews to assess functional status. Reasons for non-completion are included in Supplementary data, Figure S1 available in Age and Ageing online. Proxy interviews were completed for 179 (87%) of the 205 patients with follow-up.

There were 158 paired responses for BADLs and 110 paired responses for IADLs. The most common reason for ADL data not obtained was patients still being hospitalised or in a nursing facility (Supplementary data, Figure S1 available in Age and Ageing online). Patient characteristics at ICU admission are shown in Supplementary data, Table S1 available in Age and Ageing online. Proxies were most commonly spouses (38%) or children (39%). Additional details are in the Supplementary data available in Age and Ageing online.

Percent agreement was substantial (>80%) for all ADLs; inter-observer agreement was highly significant for all BADLs and 6 of 7 IADLs (Table 1). There was a noticeable discrepancy between percent agreement and κ for using the telephone, the only ADL for which κ was not significant. The responses for this IADL were very unbalanced with 104 (94.5%) patient–proxy pairs responding that no assistance was needed.

Table 1.

Agreement between patient and proxy regarding patient's activities of daily living

Activities of daily living (ADLs)	% Agreement	κ Statistic (95% CI)
Basic ADLsa
Bathing	91	0.82 (0.73, 0.91)
Grooming	80	0.54 (0.40, 0.68)
Transferring from bed to chair	89	0.76 (0.65, 0.87)
Walking across small room	91	0.79 (0.69, 0.89)
Dressing	87	0.73 (0.61, 0.84)
Eating	89	0.64 (0.48, 0.79)
Toileting	90	0.76 (0.65, 0.87)
Instrumental ADLsb
Using telephone	95	0.27 (−0.17, 0.71)
Getting to places out of walking distance	95	0.86 (0.75, 0.97)
Shopping for groceries	92	0.68 (0.48, 0.87)
Preparing own meals	87	0.64 (0.47, 0.81)
Doing housework	94	0.68 (0.46, 0.90)
Taking own medications	85	0.70 (0.56, 0.83)
Managing own money	84	0.69 (0.55, 0.82)

CI, confidence interval.

^aAnalysis reflects 158 available paired patient–proxy responses.

^bAnalysis reflects 110 available paired patient–proxy responses.

There were no significant response differences between patients and proxies for BADLs using McNemar's exact test. For IADLs, proxies overrated disability relative to patients for taking medications (P = 0.0042) and managing money (P = 0.049). Although no other significant response differences were seen, we note that this study was not designed to detect response differences.

ICCs for ADL composite scores between patients and proxies were very strong for both BADLs and IADLs: ICC = 0.89 (95% CI 0.85, 0.92), P < 0.0001 and 0.83 (0.76, 0.88), P < 0.0001, respectively.

Discussion

This study shows that carefully screened proxies are reliable for assessing older adults' ADLs 1 month post-ICU discharge. Response difference was minimal, even with almost a third of patients having cognitive impairment. We recommend this proxy screening method as a way to identify reliable surrogates.

This study has multiple strengths. This is a well-characterised prospective cohort with very high enrolment of eligible patients, minimal loss to follow-up of ICU survivors and excellent proxy participation. Proxies were meticulously screened, and identification of proxies did not rely solely on the patient.

Several limitations warrant mention. Because most patients could not be interviewed at admission, paired interviews only took place 1 month post-ICU discharge. Data for IADLs and BADLs were not obtainable from many patients still hospitalised or in a nursing facility. There was no objective functional assessment to confirm validity of responses. Detailed proxy information was not collected.

This study differs from prior literature in its meticulous and standardised screening for the best available proxy. This may be the reason our results differ from prior studies that have tended to show less robust agreement [16–18] or underrating of functional status by proxies referent to patients [9, 15].

In general, proxies are willing to answer questions about functional status, and proxies living with patients show higher agreement than those who do not [15]. Among proxies not living with patients, agreement rises with time spent together [19]. The more observable the function being assessed the better the patient–proxy agreement is [9]. These epidemiologic observations informed the development of the proxy screener used in our study. For research on older adults dependent on proxy responses, systematic screening may improve the reliability of proxy responses.

The EuroQol health-related quality of life instrument (EQ-5D) [20] has been used in at least three studies of critically ill patients [16–18]. In this instrument, functional impairment is graded more globally as a component of quality of life using mobility, self-care and usual activities. In Dinglas et al. [16], survivors of acute lung injury and their self-identified proxies were interviewed near hospital discharge regarding patients' pre-hospital quality of life. However, agreement concerning functional status was poor. Badia et al. [17] examined patients transferred from ICU to a step-down unit, and they used proxies recruited at ICU admission. Patients identified their proxies as someone who lived with or had known them for ≥3 years. They found ‘moderate to good agreement’ between patients and proxies for functional status. Finally, Diaz-Prieto et al. [18] assessed a large ICU cohort with a mixture of trauma, surgical and medical illness. Similar to Badia et al., proxies were screened to have lived with or known the patient for ≥3 years. On ICU admission, proxies completed the EQ-5D to reflect baseline status, and patients completed the EQ-5D when their health status allowed. Moderate agreement was seen for mobility, self-care and usual activities (κ > 0.5). Whereas improved agreement in the latter two studies may reflect better proxy screening, it might plausibly have been inflated by the more global functional assessment in the EQ-5D. This is somewhat analogous to summary measures that tend to have better agreement, because errors associated with each item are dampened [9]. Recall bias among patients and proxies rating premorbid function during acute illness might also alter results. This study shows excellent agreement for most ADLs, and we hypothesise that standardised proxy screening was a major determinant.

Conclusion

Well-chosen proxies are reliable for assessing most components of functional status of older adults who survive to 1 month post-ICU discharge. Careful choosing of proxies likely predicts their reliability as informants of functional status. The screening tool presented in this study could be adapted to research that leverages proxy responses for functional assessments, including studies of proxy assessments at ICU admission.

Key points.

• This study examines proxy assessments of functional impairment in older patients surviving 1 month after ICU discharge.

• We demonstrate that standardised screening successfully identifies reliable proxies as informants of functional impairment.

• This screening tool could be leveraged to identify the best available proxy informants of ADLs in research or clinical practice.

Conflicts of interest

None declared.

Funding

This work was supported by the National Institute on Aging at the National Institutes of Health [5K23 AG023023 to M.A.P., and P30 AG21342 to Yale Claude D. Pepper Older Americans Independence Center] and the American Heart Association [10FTF3440007 to A.M.A]. These sponsors played no role in the design, execution, analysis, interpretation of data or writing of this study.

Supplementary data

Supplementary data mentioned in the text are available to subscribers in Age and Ageing online.