Validation of a New Clinical Tool for Post-Intensive Care Syndrome

INTRODUCTION

In 2010, the Society of Critical Care Medicine organized a task force to raise awareness of the long-term cognitive, psychological, and physical impairments in survivors of critical illness. Impairments in these three domains are collectively known as post-intensive care syndrome (PICS).¹ PICS affects 50–70% of intensive care unit (ICU) survivors, and its effects can persist for 5–15 years after ICU hospitalization.² One major barrier in the assessment of PICS is the lack of a single, validated clinical tool to rapidly assess impairments in all three domains of PICS.¹

The Healthy Aging Brain Care Monitor Self Report version (HABC-M SR) is a 27-item questionnaire that evaluates cognitive, functional, and psychological domains³ (Figure 1). Patients demarcate the frequency of target symptoms (cognitive, psychological, and functional) in the past 2 weeks. The HABC-M SR can be administered face to face, over the phone, or via the Internet. It has been validated in older patients with normal cognition, mild cognitive impairment, early-stage dementia, and late-life depression. The aim of this study was to validate face to face administration of the HABC-M SR as a rapid assessment tool for PICS.

Figure 1.

HABC-Monitor. Self-Report Version.

MATERIAL AND METHODS

Subjects and Setting

A total of 261 patients were recruited from July 2011 to May 2017 at the Critical Care Recovery Center (CCRC) at Eskenazi Hospital, one of the first ICU adult survivor clinics in the United States.⁴ Eskenazi Hospital serves a racially diverse, underserved population in the Indianapolis metropolitan area. Inclusion criteria were that patients needed to be 18 years or older, admitted to the Eskenazi ICU, on mechanical ventilation or delirious for >48 hours, recommended for follow-up by a critical care physician, and had a Mini-Mental State Examination (MMSE) greater than 17. Exclusion criteria were enrollment in hospice or palliative care services. Patients who did not have an HABC-M SR (n = 86) or neuropsychological testing (n = 33) were also excluded from the study. The final sample included a subgroup of 142 patients who completed both the HABC-M SR and the standardized assessments. Standardized assessments were done at the initial visit, and the HABC-M SR was completed within a week or less during the same visit or a subsequent follow-up visit (with a mean gap of 7.2 days ± 10.0). Standardized assessments included those to examine cognition (either CERAD or RBANS), psychological symptoms (PHQ-9 or Geriatric Depression Scale; PTSS-10; and GAD-7), and functional levels (PSMS and IADL self-report). We chose to utilize the CERAD and GDS-15 during the initial assessments since these tests had been validated with the HABC-M SR, but as the CCRC referral base began to include younger ICU survivors, we decided to switch the cognitive and depression assessments to the RBANS and PHQ-9, respectively. The PSMS and IADL self-ratings can be used across all age ranges. Institutional Review Board approval was obtained to conduct retrospective analysis of de-identified clinical data.

Healthy Aging Brain Care-Monitor Self Report (HABC-M SR)

The HABC-M SR was developed by an interdisciplinary panel of dementia experts, and was validated in patients with a MMSE score greater than 17.³ The HABC-M SR (Supplementary Data, Table 1) is a 27 item self-administered tool to evaluate the cognitive, functional, and psychological symptoms. The cognitive subscale was composed of 6 questions about memory, orientation, and judgment. The functional subscale was composed of 11 questions about instrumental activities of daily living (IADLs) and activities of daily living (ADLs). The psychological subscale was composed of 10 questions about depression, psychotic, and anxiety symptoms. Each question is rated on the patient’s perceived frequency of the symptom over the past two weeks: 0 = Not at all (0–1 day), 1 = Several Days (2–6 days), 2 = More than half the days (7–11 days), 3 = Almost daily (12–14 days). The maximum scores for cognitive, functional, and psychological subscales are 18, 33, and 30 respectively. The maximum total score is 81. Higher numbers for the three subscales and the total score correlate with higher severity of symptoms.³

Standardized Assessments of Cognition

All patients completed the MMSE, a 30 point questionnaire which assesses for cognitive impairment.⁵ They also completed the Trail Making Test (TMT), which consists of 2 parts to measure processing speed (TMT-A) and executive functioning (TMT-B).⁶ The amount of time to complete each part is the score. In TMT-A, the patient sequentially connects 25 encircled numbers distributed on a sheet of paper. In TMT-B, the patient connects alternating between numbers and letters in ascending order (e.g., 1, A, 2, B, 3, C, etc.). Patients then completed either the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) or the Consortium to Establish a Registry for Alzheimer’s Disease Neuropsychological Battery (CERAD-NB). The RBANS is a neuropsychological screening tool validated in a wide range of neuropsychiatric disorders.^7,8 It yields five index scores (attention, language, visuospatial/constructional abilities, immediate memory, and delayed memory) and a Total Scale score (40–160 points). The CERAD-NB is a standardized test battery designed to detect cognitive deficits in Alzheimer’s dementia.⁹ It consists of 8 subtests (verbal fluency, Boston naming, mini-mental state exam, word list learning, constructional praxis, word list recall, word list recognition, and constructional praxis recall) which measure general cognition, semantic fluency, graphomotor construction ability, confrontation naming, and verbal learning and memory.

Standardized Assessments of Psychological Symptoms

All patients completed either the Geriatric Depression Scale-30 (GDS-30) or the Patient Health Questionnaire-9 (PHQ-9) to assess depressive symptoms. The GDS-30 is a 30-item, self-report, yes/no assessment to measure depression in the elderly.¹⁰ The PHQ-9 is a 9-question self-administered scale where patients rate frequency of depressive symptoms on a 0–3 scale (0 = not at all, 1 = several days, 2 = more than half the days, 3 = nearly every day) over the past two weeks.¹¹ A subset of patients also completed the Post-Traumatic Symptom Scale (PTSS-10), a 10 item self-report post-traumatic stress disorder (PTSD) screen with a total severity score ranging 10–70. The items are based on the Diagnosis and Statistical Manual of Mental Disorders (DSM)-III criteria for PTSD. The PTSS-10 has since been validated in patients with acute respiratory distress disorder after ICU treatment using the Structured Clinical Interview for DSM-IV.¹²

Standardized Assessments of Physical Functioning

Patients or informal caregivers completed the Physical Self-Maintenance Scale (PSMS), a 6 item questionnaire that assesses the patient’s ability to complete the activities of daily living (ADLs).¹³ The patient’s level of functioning for independent activities of daily living (IADLs) was also assessed with a modified version of the Lawton IADL scale; participants were asked to rate on a three point Likert scale (1 = completely independent to 3 = completely dependent) on seven IADLs (their ability to telephone, traveling, shopping, preparing meals, housework, medication management, and finances).

Data Collection

At the initial visit, the critical care physician completed a history and interview with both the patient and informal caregiver (if one was available) and performed a full physical examination including vitals and a neurologic examination. A healthcare professional or psychometrist administered the HABC-M SR and the standardized assessments of cognition, psychological symptoms, and functional symptoms as described above. Medical history and medication lists were collected from patients, informal caregivers, and electronic medical records.

Statistical Analysis

Internal consistency of the scale items was assessed using Cronbach’s alpha. Convergent-divergent validity was assessed performing two separate analyses. First, the relationship between the HABC-M SR scales and standardized cognitive, psychological, and functional scales was examined. It was expected that the individual scales from the self-report would correlate highest with the external scales that belonged to the same domain (e.g. the HABC-M SR psychological scale would correlate with the GDS-30, PHQ-9, PTSS-10). Then generalized linear models were used to test whether this association remained significant after adjusting for age, gender, and education. Results for the HABC-M SR from the CCRC cohort were then compared to the sample used in the original validation study for the HABC-M SR. Patients for the original validation study for the HABC-M SR were recruited if they had 1) at least one visit to primary care during the period from January 1, 2008 to April 1, 2011, 2) were age ≥65 years, and 3) had either a diagnosis of cognitive impairment or had received at least one prescription of a cholinesterase inhibitor or memantine or had any ICD-9 code indicating depression or had received at least one prescription of a selective serotonin reuptake. Fisher’s exact test was used to test for differences in percentage of patients at the floor of each scale, while the Mann-Whitney test was used to test for differences in scale scores across the two populations. All statistical analyses were performed using the SAS 9.4 version (Cary, NC).

RESULTS

Patient characteristics

Characteristics of the study patients are shown in Table 1. The average age was 52.3 years (standard deviation (SD) 13.0), and less than half (48%) were female. The cohort reflected the diversity of the Indianapolis metropolitan area. Nearly half were African American (46%), and the average education was 11.8 years (SD 2.3). The most common comorbidities were hypertension (70%), alcohol use disorder (67%), chronic obstructive pulmonary disease (COPD) or other lung disease (46%), and depression (46%). In terms of hospitalization characteristics, the mean length of ICU stay was 9 days and the mean length of hospital stay was 17.1 days. 46% of the patients had an episode of ICU delirium. Most patients (94%) required ventilator support during their ICU stay.

Table 1.

Characteristics of Patients with Post-Intensive Care Syndrome (PICS).

Demographics
Mean age in years (SD)	52.3 (13.0)
Mean years of education (SD)	11.8 (2.3)
Female, N (%)	68 (47.9)
Race
African-American N (%)	64 (45.7)
Other N (%)	9 (6.4)
White N (%)	67 (47.9)
Comorbidities
Alcohol use disorder (current or previous), N (%)	58 (40.8)
Tobacco use disorder (current or previous), N (%)	112 (78.9)
History of depression, N (%)	64 (46.0)
CNS disorder, N (%)	36 (26.1)
Cardiac disease, % N (%)	48 (34.8)
Hypertension, % N (%)	97 (70.3)
Diabetes mellitus, N (%)	38 (27.5)
COPD and other lung disease, % N (%)	64 (46.4)
Cancer, N (%)	20 (14.5)
Hospital characteristics*
Mean Length of hospitalization days (SD)	17.1 (15.5)
Length of ICU days (SD)	12.2 (13.1)
Delirium during entire hospitalization, N (%)	63 (45.7)
Respiratory failure, N (%)	129 (93.5)
Initial CCRC visit information
Time between initial visit in CCRC and discharge from the hospital (days)	89.3 (54.2)

N = 142 for patients with PICS.

Continuous variables were expressed as average (SD). Dichotomous variables were expressed as % (N).

^ǂHistory of depression was defined as a diagnosis of depression based on informant report or chart diagnosis of depression.

^*Hospital stay with sentinel ICU stay resulting in CCRC referral

Central nervous disease (CNS). Chronic obstructive pulmonary disease (COPD). Critical Care Recovery Center (CCRC). Intensive care unit (ICU). Post-intensive care syndrome (PICS). Standard deviation (SD).

Reliability and scale score features of HABC-M SR

Table 2 shows the internal-consistency reliability and score distributions of the HABC-M SR. The internal consistency of the HABC-M SR scales was good to excellent (0.83–0.92). All the subscale and total scores were positively distributed, but still covered a wide range of possible answers. The interscale correlation between all of the subscales was moderate (0.61–0.70) (Supplementary Table 1), but indicated that the subscales were distinct.

Table 2.

HABC-M SR score features: internal-consistency reliability and score distributions

HABC-M SR Scales	# of Items	Cronbach’ s Alpha	# of possible levels	Range	Mean	Median	SD	% Floor	% Ceiling
Cognitive	6	0.83	18	0–15	3.7	2	4.1	33.3	0.0
Functional	11	0.83	33	0–24.75	6.3	3.3	6.8	25.0	0.0
Psychological	10	0.84	30	0–22	6.4	5.0	6.0	20.1	0.0
Total	27	0.92	81	0–57.75	16.3	12.5	14.5	12.3	0.0

Internal consistency of the scale items was assessed using Cronbach’s alpha. Healthy Aging Brain Center Monitor Self-Report (HABC-M SR). Standard deviation (SD).

Construct validity of HABC-M SR

Table 3 shows the construct validity of the subscale and total scores in HABC-M SR. The psychological subscale had the strongest correlations with the standardized measures of psychological symptoms, PHQ-9 (Spearman correlation coefficient 0.73) (n = 67), GDS-30 (0.74) (n = 56), and PTSS-10 (0.68) (n = 59). The cognitive subscale strongly correlated with only the delayed memory measure of the CERAD-NB (−0.51) (n = 56), but did not correlate with any of the measures on the RBANS (n = 76). The functional subscale correlated with the PSMS (−0.26). All these relationships remained significant after adjusting for age, gender, and education.

Table 3.

Construct validity of HABC-M SR in CCRC patients

External Scales	Cognitive Score	Functional Score	Behavioral / Psychological Score	Total Score
Cognitive Measures
MMSE	−0.08	−0.11	−0.06	−0.09
RBANS (n = 76)
Total	−0.20	−0.24*	−0.12	−0.22
Immediate Recall	−0.20	−0.20	−0.11	−0.19
Visuospatial	−0.21	−0.20	−0.18	−0.25*
Language	0.06	0.04	0.09	0.07
Attention	−0.16	−0.21	−0.08	−0.16
Delayed Memory	−0.20	−0.22	−0.06	−0.19
Trail A	−0.14	−0.15	−0.08	−0.14
Trail B	−0.10	−0.17	0.14	−0.07
CERAD-NB (n = 56)
Fluency	−0.38*	−0.19	−0.05	−0.19
Naming	−0.22	−0.16	−0.04	−0.18
Praxis	−0.06	−0.18	−0.002	−0.12
Delayed Memory	−0.51**	−0.42*	−0.18	−0.40*
AMNART	−0.18	−0.11	−0.23	−0.18
Delayed Praxis	−0.25	−0.27*	−0.11	−0.25
Tokens	−0.35*	−0.26	−0.18	−0.33*
Trails A	−0.02	−0.19	0.02	−0.09
Trails B	−0.07	−0.05	0.01	−0.04
Psychological Measures
PHQ-9 (n = 67)	0.59**	0.38*	0.73**	0.58**
GDS-30 (n = 56)	0.65**	0.51**	0.74**	0.70**
PTSS-10 (n = 59)	0.45**	0.34*	0.68**	0.54**
Functional Measures
PSMS (n = 116)	−0.19*	−0.26*	−0.16	−0.22*
Number of independent IADLs (n=109)	−0.17	−0.26*	−0.20*	−0.23*

Values represent Spearman correlation coefficients. Consortium to Establish a Registry for Alzheimer’s Disease Neuropsychological Battery (CERAD-NB). Care Recovery Center (CCRC). Patient Health Questionnaire-9 (PHQ-9). Geriatric Depression Scale-30 (GDS-30). Instrumental Activities of Daily Living (IADL). Mini-Mental State Examination (MMSE). Physical Self-Maintenance Scale (PSMS). Post-Traumatic Symptom Scale (PTSS-10). Repeatable Battery for the Assessment of Neuropsychological Status (RBANS).

^*P < 0.05.

^**P < 0.001.

Comparison of HABC-M SR between CCRC and primary care populations

Table 4 compares the subscale and total HABC-M SR scores of CCRC patients to primary care patients (n = 291; mean age 72.7 years ± 6.7). The CCRC patients had significantly worse scores for all subscales and total scale on HABC-M SR. The average total HABC-M SR score for CCRC patients was nearly double the total score for patients seen in primary care. These relationships remained significant after adjusting for age and gender. Patients seen in primary care were more likely to report no cognition, psychological, and functional symptoms compared to the patients seen in the CCRC (Supplementary Table 2).

Table 4.

Comparison of HABC-M scores in CCRC and primary care populations.

	CCRC Population (n =142)		Primary Care Population (n = 291)
	Mean (SD)	Median (IQR)	Mean (SD)	Median (IQR)	P-value
Cognitive	3.7 (4.1)	2 (0–6)	1.9 (2.9)	0 (0–3)	<0.001
Psychological	6.4 (6.0)	5 (1–11)	3.2 (4.2)	2 (0–5)	<0.001
Functional	6.3 (6.8)	3.3 (0.6–11)	3.2 (4.5)	2 (0–5)	<0.001
Total	16.3 (14.5)	12.5 (3.2–27.6)	8.3 (10.3)	4 (1–12)	<0.001

Mann-Whitney test was used to test for differences in scale scores across the two populations.

Critical Care Recovery Center (CCRC). Healthy Aging Brain Center Monitor (HABC-M). Interquartile range (IQR). Standard deviation (SD).

DISCUSSION

While epidemiologic studies suggest a fairly high prevalence of PICS in the post-ICU population, this syndrome remains underrecognized.^1,2 One major barrier to the recognition of PICS is that it affects multiple domains (physical, psychological, cognition). In today’s era of subspecialized care, the full spectrum of the symptomology may not be captured in the post-hospital setting. Therefore the need for a clinical tool that rapidly assesses all these domains is much needed. Our findings suggest that the HABC-M SR may be such a clinical tool. The HABC-M SR psychological and functional subscales were found to be reliable tools to measure the severity of symptomology in PICS. Although the HABC-M cognitive scale demonstrated low correlations with the cognitive performance measures, the highest correlation was with the CERAD Delayed Memory, which is the area most related conceptually to the HABC-M SR cognitive scale. This suggests that the cognitive subscale in PICS may have limited validity.

Despite this limitation of HABC-M SR, patients seen in the CCRC still reported higher severity of cognitive, psychological, and functional symptoms compared to the patients seen in primary care. Although our population is much younger than the original targeted population for the HABC-M SR, a number of studies have suggested that despite their chronological age, younger ICU survivor patients may be suffering from the insults of aging similar to those seen in older patients, including significant cognitive deficits. Therefore, clinicians may consider having ICU survivors who report cognitive symptoms to undergo more detailed neuropsychological testing. However, normal cognitive subscale scores should not deter clinicians from doing further evaluation if they have concerns about patients’ cognition based on their history and examination.

Most importantly, this study lays the groundwork for future development of self-report cognitive scales in PICS, similar to those being developed and studied for AD. While neuropsychological assessment (an interview with a reliable informant and full testing battery) remain the gold standard for a cognitive disorders workup, there are significant logistical and resource barriers to administering neuropsychological assessments on a wide scale. The HABC-M SR cognitive subscale was modeled after the brief self-report and informant report tools used for mild cognitive impairment and early Alzheimer’s disease, such as the Cognitive Change Index¹⁴ and the Measurement of Everyday Cognition¹⁵, and additional work will be needed to refine and validate a self-report or informant-based screening tool for cognitive symptoms in PICS. Having reliable cognitive screening tools can then help clinicians decide who to appropriately refer for further assessment.

Future studies will need to examine whether the cognitive subscale of the HABC-M caregiver version is a reliable tool which can accurately capture the severity of cognitive symptoms in PICS. The gold standard for cognitive assessment consists of a clinical interview and detailed neuropsychological battery. Despite the lack of correlation between detailed cognitive testing and cognitive subscale score, there is still some relative value since patients in the CCRC did report more cognitive symptoms than patients in primary care.

The major strength of our study is that we have demonstrated that the HABC-M-SR, an easy to use, standardized clinical tool, has potential as a screening tool to rapidly assess the wide range of symptoms seen in PICS. Many other studies have utilized a wide range of tools to measure psychological symptoms, cognitive performance, and physical functioning.^16–20 These tools can be time-intensive, may require additional training for health care professionals to administer, and are often used in the research setting involving subspecialty care. It requires little to no training for health care professionals to administer, can be completed within 5 minutes, and can be administered in a wide range of health care settings (primary care and subspecialty outpatient care). The HABC-M SR can also be repeated for longitudinal follow-up of these symptoms. While the number of ICU survivor clinics is rapidly growing, access to this subspecialty care remains quite limited. Developing a tool to be able to rapidly screen ICU survivors for these symptoms in other settings (most notably in primary care) can increase the likelihood that ICU survivors are referred to the appropriate subspecialty care that they need.

There are some limitations to our study. First, the HABC-M SR can only be administered in patients with normal cognition, mild cognitive impairment, or early stage dementia. Future studies will need to examine whether the caregiver version of the HABC-M will be a valid measure of symptoms of PICS in patients with moderate to severe dementia. Patients who screen negative on HABC-M SR are less likely to have symptoms suggestive of PICS, but clinicians should always interpret HABC-M SR, as with all screens, in the context of the patients’ history and physical examination. Second, the HABC-M SR was administered in a subspecialty ICU survivor clinic. ICU survivor clinics are more common in Europe, but still fairly rare in the US, which means many ICU survivors rely on primary care physicians for hospital discharge follow-up. Although additional studies are need to validate this screen in larger populations and on longitudinal follow-up, clinicians may find the HABC-M SR helpful as a screen for symptoms in ICU survivors and follow-up with additional history and examination as indicated. Although the HABC-M SR has been used in older patients in primary care who had concerns for mild cognitive impairment, early stage dementia, or late-life depression, it has not been used in primary care to screen younger ICU survivors who may have undetected cognitive, mental health, or functional symptoms that could suggest a diagnosis of PICS. Future studies will need to examine whether primary care practitioners can accurately administer the HABC-M SR for ICU survivors, and then appropriately diagnose and refer for the management of PICS based on the results of the HABC-M SR. The HABC-M SR is a clinical tool that can be administered through multiple modalities (face to face, over the phone, and via the internet). Future studies will need to validate whether alternative modalities of administration of the HABC-MR SR via telephone or internet show similar results. Other limitations include that the functional subscale does not directly correlate with the physical impairments in PICS and selection bias in terms of patients who participate in this study.

CONCLUSION

As the number of ICU survivors increases, PICS is now becoming a major public health issue. Post discharge care for these survivors is fragmented.²¹ Despite the growth of ICU survivor clinics, the majority of ICU survivors will also continue to receive their care in primary care.^21,22 Therefore, healthcare professionals in all disciplines and specialties need clinical assessment tools for PICS that can be used in a wide range of outpatient settings. The use of such tools allow healthcare professionals to recognize patients are experiencing the symptoms of PICS and then referring to critical care or other relevant subspecialty expertise for the management of PICS. These tools need to be short and easy for healthcare professionals with little to no expertise in PICS to use. The HABC-M SR is one such tool, and future studies will need to examine potential barriers to the adoption of HABC-M SR in the outpatient settings for the diagnosis of PICS.