Baseline Quality of Life before Intensive Care: A Comparison of Patient versus Proxy Responses

Abstract

Objective

To compare acute lung injury (ALI) patients’ self-reported, retrospective baseline quality of life (QOL) before their intensive care hospitalization with population norms and retrospective proxy estimates.

Design

Prospective cohort study using the Short Form 36 (SF-36) QOL survey.

Setting

13 intensive care units at 4 teaching hospitals in Baltimore, MD, USA.

Patients

136 ALI survivors and their designated proxies.

Interventions

Both patients and proxies were asked to estimate patient baseline QOL before hospital admission using the SF-36 survey.

Measurements and Main Results

Compared to population norms, QOL scores were lower in ALI patients across all 8 domains, but the difference was significantly greater than the minimum clinically important difference in only 2 of 8 domains (Physical Role and General Health). The mean paired difference between patient versus proxy responses revealed no clinically important difference. However, kappa statistics demonstrated only fair to moderate agreement for all domains. Bland-Altman analysis revealed that for all domains, proxies tended to overestimate QOL when patient scores were low and underestimate QOL when patient scores were high.

Conclusion

Retrospective assessment of QOL prior to hospitalization revealed that ALI patients were consistently lower than population norms, but the magnitude of this difference may not be clinically important. Proxy assessments had only fair to moderate agreement with patient assessments. Across all 8 SF-36 QOL domains, proxy responses represented an attenuation of patient QOL estimates.

Introduction

Quality of life (QOL) is an important outcome measure in evaluating survivors of critical illness.(1–6) ICU survivors have impaired QOL versus age- and sex-matched population norms, and these impairments can persist for months to years.(1;2;7–11) Among critically ill patients, those with acute lung injury/acute respiratory distress syndrome (ALI) may be especially likely to have poor QOL, given their high severity of illness, prolonged ICU stay, and frequent physical and mental health morbidities.(1;2;12)

Baseline QOL prior to hospitalization for critical illness is an important consideration when analyzing the subsequent effects of critical illness and ICU treatment on long-term QOL. In addition, baseline QOL is useful in predicting mortality and assisting decision-making regarding the use of life-sustaining treatments in the ICU.(13) Because critical illness is frequently sudden and unexpected, baseline QOL generally cannot be prospectively measured in most ICU patients. In these circumstances, proxies may estimate patient QOL, or ICU survivors may be asked to retrospectively assess their QOL. Both of these QOL assessment methods are subject to bias.

It is unclear whether proxy estimates of patient QOL are accurate. In one study of ALI patients, the average difference between paired patient and proxy responses was insignificant in 5 of 8 domains of the Short Form-36 (SF-36) QOL survey, but patient-proxy agreement was only fair (kappa statistic = 0.30 – 0.40).(14) However, other studies have concluded that proxy estimates are an acceptable substitute for patients’ self-reported QOL in several circumstances, including during chronic illness(15), prior to hospital admission (16–19), or after ICU discharge.(20)

Given these conflicting results, our goal was to further evaluate patient versus proxy assessments of baseline QOL in ALI survivors. Our study has two specific objectives: (1) to compare baseline QOL of ALI survivors to age- and sex-matched population norms, and (2) to evaluate the agreement of proxy versus patient estimates of baseline QOL using the Short Form General Health Survey (SF-36).

Materials and Methods

Study Design and Participants

Data for this analysis was obtained from the Improving Care of Acute Lung Injury Patients (ICAP) study, an ongoing prospective cohort study.(21) The ICAP study population was consecutively enrolled from 13 intensive care units at four teaching hospitals in Baltimore, USA. Eligible patients were ≥ 18 years old and mechanically ventilated with ALI as defined by the American-European Consensus Conference criteria.(22) Relevant exclusion criteria evaluated at the time of ALI diagnosis included preexistence of: 1) comorbid disease with a life expectancy of <6 months; 2) any pre-existing communication or language barrier; 3) known pre-existing cognitive impairment; and 4) no fixed address. The study was approved by the institutional review boards (IRB) of the Johns Hopkins University and all participating institutions.

Before hospital discharge, consenting ALI survivors provided the name and contact information for their closest proxies. For this study, both patients and proxies were asked to estimate the patient’s baseline QOL, defined as QOL just before the onset of the illness that resulted in the hospitalization with ALI. This QOL assessment was completed by both patient and proxy as soon as possible after patients regained capacity and provided consent for the ICAP study.(23), for patients and via phone interview for proxies who were generally not available in hospital after patient consent. Proxies were explicitly instructed to respond using the patient’s perspective of QOL as previously described.(24;25)

The SF-36 QOL instrument has 36 questions evaluating eight separate domains: Physical Functioning, Physical Role, Bodily Pain, General Health, Vitality, Social Functioning, Emotional Role, and Mental Health. The responses for each domain are scored and transformed to a 0–100 scale, with higher scores reflecting better QOL. In our study, when a domain could not be scored for either a patient or proxy (e.g. incomplete survey response), that specific domain was omitted from analysis. The SF-36 has been validated in survivors of critical illness(1;2;26) and frequently used to retrospectively assess baseline QOL in ICU survivors.(16;18;27;28) Similar to prior studies, the wording for each SF-36 item was revised slightly to evaluate baseline QOL.(29)

Statistical Analysis

For each SF-36 domain, the following statistical analyses were performed: (1) mean difference between the patient score and the associated age- and sex-matched U.S. population norm, and (2) mean difference between each patient-proxy pair. For the patient-population and patient-proxy differences, unpaired and paired t-tests were used, respectively, to test for a statistically significant difference from zero and from the minimal clinically important difference (MCID) for each domain, as estimated for patients with chronic pulmonary disease.(30;31)

Agreement between patient and proxy responses was measured using the Cohen’s kappa statistic (unweighted and weighted). The kappa statistic can range from −1 (complete disagreement), to 0 (no agreement), to +1(perfect agreement). For these statistics, we treated each SF-36 domain score as an ordinal variable.(32;33) For the weighted kappa, weights were assigned using a standard method for linear weighting proposed by Cicchetti and Allison.(34) This approach assigns a weight of 1 for perfect agreement and 0 for the largest possible disagreement, with weights of all other misclassifications determined linearly. Two sets of kappa and weighted kappa statistics were calculated. For the first set, perfect agreement was defined as a difference between the patient and proxy score of less than or equal to the state change (i.e. the smallest possible change in score based on a 1 unit change in response for 1 question within the domain) for the particular SF-36 domain. For the second set, perfect agreement was defined as a difference between the patient and proxy score of less than the MCID for a given domain. Based on the kappa statistic, patient-proxy agreement was qualitatively described according to recommendations from Landis and Koch: poor (κ < 0), slight (κ 0 – 0.2), fair (κ 0.21 – 0.4), moderate (κ 0.41–0.60), substantial (κ 0.61–0.8), or almost perfect (κ ≥ 0.81).(35)

In addition, Bland-Altman (B-A) plots were used to explore the relationship between the differences in the patient and proxy responses as a function of the patient response.(36) A traditional B-A plot would display the average of the patient and proxy responses along the horizontal axis. However, for this analysis, it was assumed that the patient response is measured without error, so that the patient response is most reflective of the true underlying quality of life and most appropriate for the x-axis. Linear regression models, evaluating both linear and quadratic relationships, were used to estimate the mean difference in patient and proxy responses as a function of the patient response.

For all analyses, p <0.05 was considered statistically significant. All data were stored and analyzed using STATA version 10.0 (College Station, TX), except kappa statistics which were calculated using R software (University of Auckland, New Zealand).

Results

Within the ICAP study, 193 participants were potentially eligible for this patient-proxy QOL analysis. Of these potential participants, 47 were not eligible for the following reasons: no consent (primarily due to patient lack of capacity secondary to cognitive impairment, n = 30), no proxy available (n=10), or death or hospice care prior to completion of surveys (n = 7). Ten otherwise eligible participants were excluded because the patient and/or proxy declined to complete the survey (n = 10). Hence, the SF-36 data were analyzed for 136 patient-proxy pairs. Of 1088 possible patient-proxy domain scores (8 domains for each of the 136 patient-proxy pairs), only 18 (1.7%) could not be scored and reported due to missing data. Table 1 describes baseline characteristics of the patients included in this study.

Table 1.

Description of study participants

Baseline characteristic	N=136
Age, median (IQR) years	49 (40–60)
Male, no. (%)	72 (53)
Race, no. (%)
White	87 (64)
African-American	47 (35)
Other	2 (1)
Preexisting pulmonary disease1, no (%)	31 (23)
Charlson Comorbidity score, (median, IQR)	1.0 (0.0–3.0)
APACHE II score, median (IQR)	22 (17–27)
ICU Admission Diagnosis, no. (%)
Respiratory, including pneumonia	78 (57)
Gastrointestinal	17 (13)
Infectious disease	9 (7)
Trauma	6 (4)
Cardiovascular	4 (3)
Other	22 (16)
ICU Type, no. (%)
Medical	107 (79)
Surgical	19 (14)
Trauma	10 (7)

Abbreviations: ALI – acute lung injury; APACHE II – Acute Physiology and Chronic Health Evaluation II; IQR – inter-quartile range

¹Includes the documented diagnosis of any obstructive or restrictive lung disease, asthma, dyspnea with moderate or less than moderate activity associated with lung pathology, known use of home oxygen, chronic hypoxia or hypercapnea, severe pulmonary hypertension, and history of lung transplant.

The mean paired difference for the patient-population comparison demonstrated significantly greater population norms for all SF-36 domains except for Vitality where this difference did not reach statistical significance (p=0.12) (Table 2). For 2 of the 8 domains (Physical Role and General Health) the patient-population difference was significantly greater than the corresponding MCID.

Table 2.

Patient versus population norms for baseline quality of life

Domain	Na	Mean Population normb	Mean Patient estimate	Mean paired difference [95% CI] (population - patient)	P-value for difference >0c	MCID (30)	P-value for difference ≥ MCIDd
Physical Function	136	82.1	69.6	12.5 [7.0, 18.0]	<0.001	10.0	0.190
Physical Role	135	81.9	64.0	17.9 [11.9, 23.9]	<0.001	12.5	0.038
Bodily Pain	133	70.1	62.8	7.3 [1.4, 13.2]	0.008	10.0	0.820
General Health	133	70.3	56.4	13.9 [9.2, 18.7]	<0.001	10.0	0.049
Vitality	134	58.9	56.4	2.6 [−1.6, 6.7]	0.120	12.5	>0.99
Social Function	134	84.2	68.5	15.7 [10.2, 21.3]	<0.001	12.5	0.130
Emotional Role	132	87.6	75.6	12.0 [6.3, 17.8]	<0.001	8.3	0.099
Mental Health	133	75.5	68.8	6.7 [2.6, 10.9]	0.001	10.0	0.940

Abbreviations: CI – confidence interval; MCID – minimal clinically important difference

^aWhen a domain could not be scored for either a patient or proxy, that specific domain was omitted from analysis

^bAge- and sex-matched U.S. population norm (41)

^cP-value for testing if the mean paired difference is greater than zero

^dP-value for testing if the mean paired difference is greater than or equal to the MCID (30)

The mean paired difference for the patient-proxy comparison demonstrated significantly greater patient values for all domains except Emotional Role where this difference did not reach statistical significance (p=0.14) (Table 3). General Health was the only domain where the patient-proxy difference was greater in magnitude than the corresponding MCID, but this difference was not significant (p = 0.29). All but one patient-proxy pair had at least one domain where the difference in scores was greater than the MCID (median = 5, IQR = 3). When differences in survey timing for patients and proxies were analyzed, patient-proxy pairs with one or more domains with a score difference ≥ 60 did not differ significantly from patient-proxy pairs with all domain score differences < 60 (Wilcoxon rank sum test, p=0.14).

Table 3.

Patient versus proxy comparison for baseline quality of life

SF-36 domain	Na	Mean Patient estimate	Mean Proxy estimate	Mean paired difference [95% CI] (patient - proxy)	P-value for difference >0b	MCID (30)	P-value for Difference ≥ MCIDc
Physical Function	136	69.6	63.4	6.2 [0.42, 12.0]	0.036	10.0	0.903
Physical Role	135	64.0	56.7	7.3 [1.6, 13.0]	0.012	12.5	0.964
Bodily Pain	133	62.8	56.1	6.7 [0.9, 12.4]	0.023	10.0	0.875
General Health	133	56.4	45.1	11.2 [6.7, 15.8]	<0.001	10.0	0.294
Vitality	134	56.4	49.8	6.6 [2.0, 11.2]	0.005	12.5	0.994
Social Function	134	68.5	61.2	7.3 [1.2, 13.4]	0.020	12.5	0.954
Emotional Role	132	75.6	71.0	4.6 [−1.6, 10.8]	0.140	8.3	0.880
Mental Health	133	68.8	63.3	5.5 [1.5, 9.4]	0.007	10.0	0.987

Abbreviations: CI – confidence interval; MCID – minimal clinically important difference

^aWhen a domain could not be scored for either a patient or proxy, that specific domain was omitted from analysis

^bP-value for testing if the mean paired difference is greater than zero

^cP-value for testing if the mean paired difference is greater than or equal to the MCID (30)

The weighted kappa statistic, with perfect agreement defined as both a difference of less than or equal to a “state change” or a difference less than the MCID, revealed similar results with patient-proxy agreement being “fair” for 7 of the 8 domains (range: 0.32 – 0.43) (Table 4). The unweighted kappa, as expected, demonstrated a lower level of agreement with generally “slight” agreement between the patient-proxy pairs.

Table 4.

Agreement between patient and proxy baseline quality of life scores

SF-36 domain	State Changea	Kappab [95% CI]	Weighted Kappab,d [95% CI]	MCID (30)	MCID Kappac [95% CI]	MCID Weighted Kappac, d [95% CI]
Physical Function	5.0	0.16 [0.11, 0.21]	0.36 [0.24, 0.47]	10.0	0.21 [0.14, 0.29]	0.36 [0.25, 0.48]
Role Physical	6.25	0.13 [0.08, 0.18]	0.38 [0.27, 0.50]	12.5	0.17 [0.09, 0.24]	0.39 [0.27, 0.51]
Bodily Pain	10.0	0.08 [0.02, 0.14]	0.34 [0.22, 0.46]	10.0	0.08 [0.02, 0.14]	0.34 [0.22, 0.46]
General Health	5.0	0.02 [−0.01, 0.05]	0.35 [0.25, 0.45]	10.0	0.10 [0.03, 0.17]	0.37 [0.26, 0.47]
Vitality	6.25	0.07 [0.03, 0.11]	0.32 [0.23, 0.42]	12.5	0.17 [0.09, 0.25]	0.35 [0.24, 0.45]
Social Function	12.5	0.18 [0.12, 0.25]	0.34 [0.23, 0.46]	12.5	0.18 [0.12, 0.25]	0.34 [0.23, 0.46]
Role Emotional	8.3	0.16 [0.09, 0.24]	0.40 [0.28, 0.52]	8.3	0.16 [0.09, 0.23]	0.40 [0.28, 0.52]
Mental Health	5.0	0.06 [0.02, 0.10]	0.41 [0.31, 0.51]	10.0	0.17 [0.09, 0.24]	0.43 [0.33, 0.54]

Abbreviations: CI – confidence interval; COPD – chronic obstructive pulmonary disease; MCID – minimal clinically important difference; QOL – quality of life; SF-36 – Short-Form 36

^aA state change is defined as the smallest possible change in score based on a 1 unit change in response for 1 question within a domain

^bPerfect agreement is defined as having an absolute difference between the patient and proxy score of less than or equal to the state change

^cPerfect agreement is defined as having an absolute difference between the patient and proxy score of less than the MCID

^dFor the weighted kappa, weights were assigned using linear weighting which assigns a weight of 1 for perfect agreement and 0 for the largest possible disagreement, with other weights of misclassifications determined linearly.(34)

For all 8 SF-36 domains, the B-A analysis revealed that for both relatively low and high patient scores, the proxy estimate tended to attenuate toward a more moderate score. The General Health domain demonstrates this pattern of attenuation, especially with higher patient scores (Figure 1).

Figure 1. Bland Altman plot for the SF-36 General Health domain.

This figure displays the relationship of the patient SF-36 scores to the difference between the patient and proxy scores (i.e. patient minus proxy). The area between the dashed horizontal lines represents a difference of 0 ± the minimum clinically important difference (MCID) for patients with chronic pulmonary disease.(30) Each circle represents one patient-proxy pair. The fitted line represents the linear regression model used to estimate the mean difference in patient and proxy responses as a function of the patient response. The dashed lines surrounding the fitted line represents 95% confidence intervals.

Discussion

This prospective cohort study of 136 ALI patients compared patients’ retrospective baseline SF-36 QOL estimates with age- and sex-matched population norms and retrospective proxy estimates. The ALI patient cohort demonstrated lower baseline QOL scores than population norms for all 8 SF-36 domains, but this difference was both clinically important and statistically significant for only the Physical Role and General Health domains. Proxy estimates had only fair agreement with patients, with proxy results consistently demonstrating an attenuation of patient scores for all domains.

A small number of studies have compared baseline QOL in ICU survivors with population norms. In a systematic review of QOL in ARDS patients,(2) three studies specifically evaluated baseline QOL using the SF-36 survey.(28;29;37) All three studies similarly demonstrated that when compared to population norms, ICU survivors had lower mean baseline SF-36 scores for all domains. However, one additional recent study of 46 ALI patients demonstrated that for all domains except General Health, patients had SF-36 QOL scores that were the same or better than population norms.(14) Two studies of general ICU patient populations used proxy responses to estimate baseline QOL, with one study finding a trend toward lower SF-36 scores vs. population norms for all domains,(27) and the other finding a similar trend in 6 of 8 domains.(8) In our study, when compared to population norms, patients had lower average SF-36 scores for all domains. However, only the Physical Role and General Health domains had differences that were both statistically significant and greater than the estimated MCID. The prior studies did not compare the magnitude of the patient-population difference to an MCID estimate.

Several factors may contribute to the diversity in these results, including varying sources for the baseline QOL estimate (i.e. patients, proxies, or mixed patient and proxy), differences in the presence of comorbidities between study cohorts, and insufficient normative data for patients >65 years old in some countries. Compared to the most recent ALI study cohort which reported baseline QOL scores that were the same or better than population norms, our cohort had a similar median patient age and APACHE II severity of illness. However, the burden of patients’ baseline comorbidity was likely greater in our study since the majority of patients were recruited from hospitals serving inner city patients and the prevalence of documented preexisting pulmonary disease was two-fold greater (23% vs. 11%).(10;14)

Existing studies also demonstrate variable results for agreement between patient and proxy estimates of baseline QOL in ICU survivors. A recent study of ALI patients reported findings similar to our study with patient-proxy agreement that was only fair despite a mean paired difference of less than the MCID in 7 of 8 SF-36 domains.(14) However, studies of other ICU populations have reported fair to excellent agreement using the SF-36 and other QOL instruments.(16;18;19;38) Furthermore, one review article of 23 studies of non-ICU patients with chronic illness concluded that proxy estimates are reasonably accurate and that substantial discrepancies are rare.

Few studies have been able to identify factors underlying these varied results regarding patient-proxy agreement. Of the five studies previously discussed, only two used the SF-36 (14;18). Hence, the difference in the results may be due to different QOL instruments. There are no studies directly comparing the level of patient-proxy agreement using different QOL instruments. The retrospective nature of baseline QOL assessments introduces potential recall bias from the patient’s ICU stay. For example, in one study, patient and proxy QOL estimates differed significantly in six domains at hospital discharge, but agreement improved at 6 months, suggesting that temporal proximity to the acute hospital illness may have an effect on agreement.(20) However, existing research demonstrates that severity of illness, type of admission, patient education level, and the nature of the patient-proxy relationship do not affect patient-proxy agreement for QOL prior to ICU admission.(14;16;18;20;38)

Proxy attenuation of patient QOL estimates, demonstrated by the B-A analyses in our study, is a novel finding within ICU QOL research. This relationship was present for all 8 SF-36 domains and illustrates the limitations of the mean paired difference statistic in comparing patient-proxy estimates. Despite this finding being present across all SF-36 domains, the magnitude of differences within patient-proxy pairs markedly varied by domain such that pairs did not consistently have particularly large or small differences across all domains. More investigation is needed to understand this relationship and its potential implications on proxy decision-making for critically ill patients.

Our study has several potential limitations. First, the use of an MCID based on chronic pulmonary diseases may not be appropriate for ALI patients. Moreover, the MCID was determined via an expert consensus panel since no estimate based on patient report is available for ALI or general ICU patients. Second, our study does not include the necessary data to permit reporting on the nature of the relationship of the proxy to the patient or the precise timing differences between completion of the SF-36 survey within the patient-proxy pairs. However, the closest available proxy (designated by the patient) was used in this study, and prior research has demonstrated that the nature of this relationship does not significantly affect patient-proxy agreement.(16;20;38) In addition, analysis of the available data within the study suggests that the magnitude of patient-proxy differences was not associated with the estimated timing differences. Third, although the number of capable patients and/or proxies declining to participate in the QOL survey was low (5%), similar to other related studies,(16;38) approximately 15% of patients could not complete the baseline SF-36 survey due to cognitive impairment, as commonly observed during the early post-ICU in-patient period.(39) Despite this, our final sample size makes an important contribution since it is 2- to 3-fold larger than prior studies of patient-proxy SF-36 QOL comparisons.(14;18) Finally, in our study, the SF-36 survey was administered in-person for patients and via phone for proxies, a difference which may introduce a response-mode bias.(40) We attempted to minimize this bias by administering the survey via an interviewer (rather than self-administered) for both patients and proxies. Moreover, we felt that selecting a single mode of administration (i.e. changing to in-person for proxies or to phone after hospital discharge for patients) would have resulted in a greater bias due to non-response.

Conclusions

Our retrospective assessment of QOL prior to hospitalization revealed that ALI patients were consistently lower than population norms, but the magnitude of this difference may not be clinically important. Furthermore, proxy assessments had only fair to moderate agreement with patient assessments, largely due to a proxy attenuation of patient responses in all 8 SF-36 domains.