Abstract
Objective
To evaluate the internal consistency, validity, responsiveness, and minimal important difference of the Functional Status Score for the Intensive Care Unit (FSS-ICU), a physical function measure designed for the intensive care unit (ICU).
Design
Clinimetric analysis.
Settings
Five international data sets from the United States, Australia, and Brazil.
Patients
819 ICU patients.
Intervention
None.
Measurements and Main Results
Clinimetric analyses were initially conducted separately for each data source and time point to examine generalizability of findings, with pooled analyses performed thereafter to increase power of analyses. The FSS-ICU demonstrated good to excellent internal consistency. There was good convergent and discriminant validity, with significant and positive correlations (r = 0.30 to 0.95) between FSS-ICU and other physical function measures, and generally weaker correlations with non-physical measures (|r| = 0.01 to 0.70). Known group validity was demonstrated by significantly higher FSS-ICU scores among patients without ICU-acquired weakness (Medical Research Council sumscore ≥48 versus <48) and with hospital discharge to home (versus healthcare facility). FSS-ICU at ICU discharge predicted post-ICU hospital length of stay and discharge location. Responsiveness was supported via increased FSS-ICU scores with improvements in muscle strength. Distribution-based methods indicated a minimal important difference of 2.0 to 5.0.
Conclusions
The FSS-ICU has good internal consistency and is a valid and responsive measure of physical function for ICU patients. The estimated minimal important difference can be used in sample size calculations and in interpreting studies comparing the physical function of groups of ICU patients.
Keywords: Reproducibility of Results, Intensive Care, Cross-Sectional Studies, United States, Australia, Brazil
Introduction
Critically ill patients frequently experience long-lasting impairments in physical functioning after discharge from the intensive care unit (ICU).(1-5) There is a growing body of research aimed at evaluating ICU-based interventions that may reduce these impairments and growing interest in measures of physical function for critically ill adults.(6-8)
The Functional Status Score for the Intensive Care Unit (FSS-ICU) is a physical function measure specifically designed for the ICU that has not had comprehensive evaluation of its clinimetric performance.(9;10) The FSS-ICU includes 5 functional tasks (rolling, transfer from spine to sit, sitting at the edge of bed, transfer from sit to stand, and walking). Each task is evaluated using an 8-point ordinal scale ranging from 0 (not able to perform) to 7 (complete independence; see Web Table 1 for example scale; instrument and scoring details available at www.ImproveLTO.com). The total FSS-ICU score ranges from 0 to 35, with higher scores indicating better physical functioning.
Our objective was to evaluate the internal consistency, construct and predictive validity, responsiveness, and minimum important difference (MID) of the FSS-ICU in ICU patients across different in-patient assessment time points and across international ICU settings.
Methods
This analysis was conducted in accordance with the Consensus-based standards for the selection of health measurement instruments (COSMIN) guideline for evaluating the measurement properties of instruments.(11)
Study Design
We performed a clinimetric evaluation of the FSS-ICU using data from 5 international data sets: 2 from USA,(9;12) 1 from Australia,(13;14) and 2 from Brazil. All data sets were approved by the appropriate ethics review boards and, where required, informed consent was obtained.
The USA-Kho data set (n=34) was a randomized pilot trial of neuromuscular electrical stimulation (NMES) that enrolled patients requiring mechanical ventilation for ≤4 days in 3 medical and surgical ICUs in an academic medical center in Baltimore, MD, between 2008 and 2013.(15;16) The randomized intervention of NMES versus a sham control group did not have a significant effect on the FSS-ICU score, so intervention and control groups were pooled for this analysis.
The USA-Needham data set (n=59) was a quality improvement (QI) project that enrolled patients requiring mechanical ventilation for ≥4 days in a single medical ICU at an academic medical center in Baltimore, MD, during 2007.(9;12) This project used a structured QI framework to improve functional mobility via physical and occupational therapy. The QI versus pre-QI periods did not have a significant difference in the FSS-ICU score, so both periods were pooled for this analysis.
The Australia data set (n=66) included consecutive enrolled patients requiring mechanical ventilation for >48 hours in 2 mixed medical-surgical ICUs and received routine care in Melbourne, Australia between 2012 and 2014.(13)
The Brazil-da Silva data set (n=99) included consecutive patients admitted in a single mixed (trauma, neurosurgical, cardiovascular) ICU and received routine physical therapy (no intervention) in at a public hospital in Brasilia, Brazil in 2014, using a Portuguese version of FSS-ICU developed with independent forward and backward language translation. The FSS-ICU data was collected as part of the routine care of physical therapy evaluation.
The Brazil-Neto data set (n=561) included consecutive patients ≥60 years old admitted in 4 ICUs (3 medical-surgical, 1 surgical) and received routine physical therapy (no intervention) at a private hospital in Brasilia, Brazil between 2013 and 2014, using a Portuguese version of FSS-ICU translated by the Brazilian investigators. The FSS-ICU data was collected as part of the routine care of physical therapy evaluation.
Study Measures
The FSS-ICU was evaluated prior to hospitalization (via proxy, evaluating the 2-month period prior to hospitalization), and at ICU awakening, ICU discharge and hospital discharge for both USA studies; at ICU awakening, ICU discharge and hospital discharge for the Australian study; at ICU admission and ICU discharge for both Brazilian studies.
Well-established measures of physical function, available within the data sets, were used to assess convergent and known group validity of the FSS-ICU. These measures were the Lawton Instrumental Activity of Daily Living (IADL) score(17) (range: 0 to 8, with higher scores indicating better status), the Katz Activities of Daily Living (ADL) score(18) (range: 0 to 6, with higher scores indicating better status), manual muscle testing (MMT, using the Medical Research Council (MRC) sumscore, range: 0 to 60, with higher scores indicating greater strength, and <48 indicating ICU-acquired weakness (ICUAW)),(19;20) and hand grip strength (in kilograms, and as percent predicted using normative data(21;22)), ICU mobility scale (IMS; range: 0 to 10, with higher score indicating better mobility),(23) ICU and hospital length of stay (LOS), and hospital discharge location (home vs. healthcare facility).
To assess discriminant validity, measures that were available and expected to have little to no relationship with FSS-ICU were used. These included body mass index (BMI), continence status (from ADL scale), hemodialysis status and home oxygen use at hospital discharge, steroid and insulin use on the hospital ward and at hospital discharge.
We used two outcome measures to assess predictive validity of FSS-ICU, similar to prior research:(13;24-26) post-ICU hospital LOS (i.e., number of days between ICU and hospital discharge), and hospital discharge location (home vs. healthcare facility).
To assess FSS-ICU’s responsiveness, changes in FSS-ICU scores across two time points (ICU awakening/admission to ICU discharge, ICU discharge to hospital discharge, and ICU awakening to hospital discharge) were evaluated and were compared to changes across the same two time points for the MMT and ADLs.
Statistical Analysis
Analyses initially were conducted separately for each data set and assessment time point to evaluate generalizability of these individual findings by time point, patient sample, and study setting, then pooled analyses across studies were performed, whenever feasible and appropriate (i.e. when there were similar results among individual data sets), to increase statistical power. All analyses were performed using Stata 13.1 (StataCorp, College Station, TX).
Floor and Ceiling Effects
Floor and ceiling effects were evaluated by examining the percentage of assessments with the minimum and maximum FSS-ICU scores, respectively.
Internal Consistency
Pearson correlations were used to identify pairwise correlations between the five FSS-ICU items, and Cronbach’s alpha was used to examine the internal consistency of the FSS-ICU total score.(27)
Concurrent Construct Validity
We used Pearson correlations (for continuous measures) and biserial correlations (for binary measures) to examine convergent and discriminant validity. To evaluate convergent validity, we hypothesized that the measures evaluated would be at least moderately correlated (|r| >0.40) with the FSS-ICU. To evaluate discriminant validity, we hypothesized that measures evaluated would have negligible to weak correlations (|r| <0.30). We hypothesized significant negative correlations between FSS-ICU and ICU and hospital LOS. For known group validity, we conducted two-sample t-tests for group differences in FSS-ICU by ICUAW status (MMT ≥48 versus <48) and hospital discharge location (home vs. healthcare facility). We hypothesized that patients without (vs. with) ICUAW or discharged to home (vs. healthcare facility) would have significantly higher FSS-ICU scores.
Predictive Validity
As done in prior research,(13;24;25) we used two sample t-tests, and linear and logistic regression models to test the association of FSS-ICU at ICU discharge with post-ICU hospital LOS and hospital discharge location. In addition, the area under a Receiver Operating Characteristic (ROC) curve (i.e. C statistic) was calculated for FSS-ICU with discharge location. We hypothesized that patients with higher FSS-ICU scores at ICU discharge would have a shorter post-ICU hospital LOS and be discharged to home (vs. healthcare facility).
Responsiveness
Responsiveness was examined in three ways. First, we tracked FSS-ICU scores across the expected recovery trajectory. Differences in mean FSS-ICU scores between consecutive time points were tested using paired t-tests. Second, we calculated the effect size for changes over time (mean difference in FSS-ICU scores between two time points divided by the standard deviation (SD) at first time point).(28) Third, we evaluated change over time in the FSS-ICU relative to patients’ change in MMT and ADL scores, with changes categorized as “significant improvement” if MMT and ADL scores at the later assessment was ≥1 SD higher than the earlier assessments. A comparison group was comprised of patients whose scores increased <1 SD or declined over the period.(29)
Estimating MID
We used the following distribution-based methods to estimate MID:(30;31) standard error of measurement (SEM), minimal detectable change 90 (MDC-90), 0.2 SD, and 0.5 SD.(32)
RESULTS
Patient Characteristics
Across the 5 studies, the mean (SD) age of patients ranged from 54 (15) to 75 (9) years, and the mean (SD) Acute Physiology And Chronic Health Evaluation (APACHE) II score ranged from 12 (7) to 26 (7) (Table 1). Both Brazilian studies had older patients and lower APACHE II scores. There was a wide range of ICU admission diagnoses across the studies, with respiratory failure being the most common primary diagnosis (42% in the combined data set).
Table 1.
Pre-hospitalization patient characteristics
| Patient characteristics | USA – Kho (n=34) | USA – Needham (n=59) | Australia (n=66) | Brazil – da Silva (n=99) | Brazil – Neto (n=561)a | Combined (n ≤819) |
|---|---|---|---|---|---|---|
| Age (years), mean (SD) | 55 (16) | 54 (15) | 58 (17) | 66 (10) | 75 (9) | 70 (13) |
| Male, n (%) | 17 (50) | 19 (32) | 40 (61) | 35 (35) | 276 (49) | 387 (47) |
| BMI (kg/m2), mean (SD) | 27 (7) | 29 (11) | 28 (7) | 28 (8) | ||
| ADL scoreb, mean (SD) | 6 (1) | 5 (2) | 5 (1) | |||
| IADL scorec, mean (SD) | 6 (3) | 4 (3) | 5 (3) | |||
| FSS-ICUd, mean (SD) | 34 (4) | 31 (9) | 32 (8) | |||
| APACHE II severity of illnesse, mean (SD) | 25 (7) | 26 (7) | 21 (7) | 14 (7) | 12 (7) | 14 (8) |
| ICU admission diagnosisf, n (%) | ||||||
| Respiratory (incl. pneumonia) | 25 (76) | 39 (66) | 14 (21) | 30 (30) | 108 (42) | |
| Gastrointestinal | 3 (9) | 5 (8) | 12 (18) | 8 (8) | 28 (11) | |
| Sepsis, non-pulmonary | 0 (0) | 3 (5) | 13 (20) | 18 (18) | 34 (13) | |
| Cardiovascular | 2 (6) | 4 (7) | 18 (27) | 10 (10) | 34 (13) | |
| Trauma | 0 (0) | 0 (0) | 5 (8) | 20 (20) | 25 (10) | |
| Neurological | 0 (0) | 4 (7) | 0 (0) | 13 (13) | 17 (7) | |
| Other | 3 (9) | 4 (7) | 4 (6) | 0 (0) | 11 (4) | |
| Hospital LOS, mean (SD) | 35 (21) | 31 (20) | 28 (15) | 19 (4) | 33 (20) |
Abbreviations: BMI: body mass index; ADL: activities of daily living; IADL: instrumental activities of daily living; FSS-ICU: functional status score for the intensive care unit; APACHE: acute physiology and chronic health evaluation; Incl: including; LOS: length of stay.
The Brazil – Neto study doesn’t have ICU admission diagnosis data that fit into the above categories.
ADL score has a range of 0-6 with higher score indicating better functional status
IADL score has a range of 0-8 with higher score indicating better functional status;
FSS-ICU score has a range of 0-35 with higher score indicating better functional status;
APACHE II score has a range of 0-71, with higher score indicating greater severity of illness within first 24 hours of ICU admission.
Percentages may not sum to 100 (%) because of rounding.
Floor and Ceiling Effect
Minimal floor effect was observed (0.5%, 0.3%, and 0% at ICU admission/awakening, ICU discharge and hospital discharge, respectively). Some ceiling effect was observed later during recovery (0.7% at ICU admission/awakening, and then 11% and 21% at ICU and hospital discharge, respectively).
Internal Consistency
Good to excellent internal consistency was observed.(33) The correlation coefficients for pairwise correlation between each FSS-ICU items all positive and significant (p <0.05) in all data sets and at all time points. Across time points, Cronbach’s alpha for each study ranged from 0.90 to 0.94 (USA-Kho), 0.94 to 0.95 (USA-Needham), 0.91 to 0.93 (Australia), 0.78 to 0.91 (Brazil-da Silva), and 0.78 to 0.93 (Brazil-Neto).
Concurrent Construct Validity
Consistently across studies and time points (Table 2), we observed significant and positive correlations between FSS-ICU and other physical measures, and negative association with ICU and hospital LOS. These findings support concurrent validity. Known group validity was supported by significantly higher FSS-ICU scores among survivors without ICUAW (MMT ≥48 vs. <48) and among those discharged to home (vs. healthcare facility) (Web Table 2).
Table 2.
Construct validity: Cross-sectional relationship of FSS-ICU with other outcome measures across publications and assessment time points
| Construct and Convergent Validity (Pearson correlation)
|
Discriminant Validity (Biserial correlationa)
|
||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Time Point by Publication | N | IADL | ADL | MMT | Hand grip % predictted | Hand grip strength (kg) | ICU Mobility scaleb | ICU LOS | Hospital LOS | BMI (kg/m2) | Contin- ence item from ADL | Hemo- dialysis status | Need for home oxygen | Steroid usec | Insulin usec |
| Pre-hospitalization | |||||||||||||||
| USA - Kho | 32 | 0.48* | 0.73* | -0.04 | 0.09 | ||||||||||
| USA - Needham | 46-50 | 0.57* | 0.80* | <0.01 | 0.01 | ||||||||||
| Combined | 78-82 | 0.55* | 0.80* | -0.03 | 0.03 | ||||||||||
| ICU awakening/admissiond | |||||||||||||||
| USA - Kho | 20-29 | 0.48* | 0.81* | 0.44* | 0.41 | 0.46* | -0.01 | 0.50* | |||||||
| USA - Needham | 43-52 | 0.72* | 0.16 | ||||||||||||
| Australia | 19-66 | 0.62* | 0.30 | 0.30 | -0.17 | ||||||||||
| Brazil – da Silva | 99 | 0.38* | |||||||||||||
| Brazil – Neto | 561 | 0.32* | |||||||||||||
| Combined | 20-802 | 0.39* | 0.44* | 0.40* | 0.37* | 0.46* | -0.01 | 0.50* | |||||||
| ICU discharge | |||||||||||||||
| USA - Kho | 12-27 | 0.70* | 0.70* | 0.43* | 0.50* | 0.62* | 0.03 | 0.05 | 0.70* | 0.06 | -0.20 | 0.33 | |||
| USA - Needham | 39-47 | 0.76* | -0.18 | 0.31* | -0.26 | 0.30 | 0.05 | ||||||||
| Australiae | 20-66 | 0.68* | 0.62* | 0.70* | 0.69* | -0.24 | -0.21 | ||||||||
| Brazil – da Silva | 99 | 0.95* | -0.20* | ||||||||||||
| Brazil – Neto | 561 | 0.64* | -0.27* | ||||||||||||
| Combined | 27-800 | 0.70* | 0.60* | 0.50* | 0.59* | 0.86* | -0.25* | 0.05 | 0.70* | -0.20 | 0.19 | 0.11 | |||
| Hospital discharge | |||||||||||||||
| USA - Kho | 12-28 | 0.86* | 0.80* | 0.46* | 0.51* | -0.17 | -0.12 | 0.78* | 0.30 | -0.12 | -0.15 | 0.33 | |||
| USA - Needham | 15-44 | 0.81* | 0.80* | -0.34* | 0.11 | 0.29 | 0.14 | 0.22 | -0.12 | -0.24 | |||||
| Australia data | 8-19 | 0.39 | 0.51* | 0.61* | -0.46* | -0.37 | |||||||||
| Combined | 31-91 | 0.80* | 0.80* | 0.43* | 0.49* | -0.26* | -0.05 | 0.42* | 0.38* | 0.05 | -0.16 | -0.06 | |||
Abbreviations: FSS-ICU: functional status score for the intensive care unit; IADL: instrumental activities of daily living; ADL: activities of daily living; MMT: manual muscle testing; LOS: length of stay; BMI: body mass index.
p <0.05.
Biserial correlations evaluate a correlation when one variable is dichotomous, and were used to evaluate correlation with continence, hemodialysis, home oxygen, steroids and insulin use.
This Hodgson ordinal ICU mobility scale evaluates patients’ highest mobility level, during physical therapy assessment in the ICU, and ranges from 0 (lying in bed) to 10 (walking independently without a gait aid).
Represents any use of this medication on hospital ward (for the ICU discharge time point) and upon hospital discharge (for hospital discharge time point).
ICU awakening was defined using the De Jonghe criteria in USA-Kho and Australia studies; in the USA-Needham, study, it was defined based on Richmond Agitation Sedation Scale and ability to follow instructions to perform the assessment.
A total of 20 patients had hand grip data and 41 had ICU mobility scale data.
Consistent with our hypotheses, most associations were not statistically significant between FSS-ICU and BMI, hemodialysis, need for home oxygen, and steroid and insulin use. These findings support discriminant validity (Table 2).
Predictive Validity
We found evidence of predictive validity for duration of post-ICU hospital LOS in the USA-Needham study and in combined results across all studies, with significantly higher FSS-ICU scores at ICU discharge for survivors with below versus above the median post-ICU hospital LOS (Table 3). Linear regression analysis suggested that for a 1-unit increase in FSS-ICU score, post-ICU hospital LOS decreased by 0.27 days (p<0.01) in the combined results (Table 3). Prediction of discharge location was consistently significant across studies: survivors discharged to home were associated with a higher FSS-ICU at ICU discharge (23 vs. 16 in combined results, p<0.01). Logistic regression indicated that for 1 unit increase in FSS-ICU score, the odds of discharge to home increased by 11% (p<0.01) in combined results. The C-statistic for discharge location was 0.75 in the combined analysis, indicating that FSS-ICU can adequately predict discharge location.
Table 3.
Predictive validity of FSS-ICU at ICU discharge for duration of post-ICU hospital stay and discharge location.
| Mean FSS-ICU Score, below versus above median post- ICU hospital LOS | Post-ICU hospital LOS (continuous) | Discharge location | Discharge location (Home vs. Healthcare facility) | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
||||||||||
| Time point by publicationa | Below medianb | Above medianb | P-valuec | Linear regression coefficient | P-value | Home | Healthcare facilityd | P-valuec | Odds ratio (95% CI) | P-value | AUC |
| ICU discharge | |||||||||||
| USA–Kho | 23 | 19 | 0.19 | -0.23 | 0.19 | 25 | 16 | <0.01 | 1.23 (1.05, 1.45) | 0.01 | 0.83 |
| USA–Needham | 22 | 13 | <0.01 | -0.37 | <0.01 | 23 | 15 | <0.01 | 1.11 (1.02, 1.20) | 0.01 | 0.73 |
| Australia | 20 | 18 | 0.23 | -0.24 | 0.08 | 22 | 16 | <0.01 | 1.09 (1.02, 1.17) | <0.01 | 0.72 |
| Combined | 21 | 17 | <0.01 | -0.27 | <0.01 | 23 | 16 | <0.01 | 1.11 (1.06, 1.17) | <0.01 | 0.75 |
Abbreviations: FSS-ICU: functional status score for the intensive care unit; LOS: length of stay; CI: confidence interval; AUC: area under the receiver operating characteristics curve (i.e. C-statistic)
Sample size by post-ICU hospital LOS (below median, above median): USA-Kho (12, 14), USA-Needham (23, 21), Australia (32, 34). Sample size by discharge location (home, healthcare facility): USA-Kho (13, 15), USA-Needham (15, 29), Australia (37, 26).
Median LOS (in days) in different publications: USA-Kho: 10; USA-Needham: 10; Australia: 14; Combined: 11.
P-value calculated using two-sample t-test.
Healthcare facilities include nursing home, other hospital’s ICU or ward, or long-term ventilation facility.
Responsiveness
Mean FSS-ICU scores at each time point are shown in Web Figure 1. Consistent with the expected functional trajectory, the FSS-ICU score decreased from the baseline value prior to hospitalization to ICU admission/awakening, then increased at ICU and hospital discharge. Changes between each consecutive time points were statistically significant (p<0.01). In combined analysis, the median (inter-quartile range) FSS-ICU score was 35 (33-35) prior to hospitalization, 5 (5-10) at ICU admission/awakening, 20 (10-30) at ICU discharge, and 29 (20-34) at hospital discharge.
Although not always statistically significant, increased FSS-ICU scores were generally observed with improvements in muscle strength (Table 4), supporting responsiveness. The effect size was 2.02 from ICU awakening/admission to ICU discharge, suggesting good responsiveness. Only the USA-Kho study, with data on 24 26 patients, could be used to evaluate the FSS-ICU’s responsiveness to changes in ADL scores. This study showed a larger increase in FSS-ICU among survivors with >1 SD increase in ADL scores compared to those with negative or no change in ADL scores, although this difference was significant only when comparing ICU discharge to hospital discharge (Table 4).
Table 4.
Responsiveness to change of FSS-ICU score versus MMT and ADL scores
| Change in FSS-ICU
|
Change in FSS-ICU
|
Effect size for FSS-ICU | |||
|---|---|---|---|---|---|
| Time point by publicationa | MMT ScoreNegative or no changeb | MMT Score Significant positive changeb | ADL Score Negative or no changeb | ADL Score Significant positive changeb | |
| ICU awakening/admission to ICU discharge | |||||
| USA – Kho | 5.8 | 9.8 | 7.1 | 7.5 | 0.84 |
| USA – Needham | 5.8* | 12.0* | 0.99 | ||
| Australia | 3.9* | 11.9* | 0.84 | ||
| Brazil – Netoc | 13.5* | 16.5* | 2.62 | ||
| Combined | 10.0* | 17.5* | 7.1 | 7.5 | 2.02 |
| ICU discharge to hospital discharge | |||||
| USA – Kho | 8.1 | 16.3 | 5.2* | 13.8* | 0.93 |
| USA – Needham | 5.3 | 8.5 | 0.69 | ||
| Australia | 13.0 | 7.0d | 1.45 | ||
| Combined | 7.1 | 11.6 | 5.2* | 13.8* | 0.92 |
| ICU awakening to hospital discharge | |||||
| USA – Kho | 10.5* | 17.8* | 10.7 | 15.9 | 1.71 |
| USA – Needham | 8.4* | 15.3* | 1.78 | ||
| Australia | 21.4 | 19.7 | 2.51 | ||
| Combined | 11.1* | 15.6* | 10.7 | 15.9 | 1.85 |
Abbreviations: FSS-ICU: functional status score for the intensive care unit; MMT: manual muscle testing; LOS: length of stay.
P<0.05 by two-sample t-test for comparison of “negative or no change” versus “significant positive change” categories.
- MMT (no change, positive change):
- ICU awakening to ICU discharge: USA-Kho (13, 8), USA-Needham (30, 7), Australia (43, 23), Brazil – Neto (465, 73).
- ICU discharge to hospital discharge: USA-Kho (22, 3), USA-Needham (31, 6), Australia (8, 1).
- ICU awakening to hospital discharge: USA-Kho (15, 12), USA-Needham (22, 15), Australia (6, 3).
- ADL (no change, positive change):
- ICU awakening to ICU discharge: USA-Kho (22, 2).
- ICU discharge to hospital discharge: USA-Kho (15, 11).
The “no change” category represents all patients who did not increase by >1 standard deviation (SD) at last available FSS-ICU time points; except for Brazil – Neto study, no patient had a decrease in MMT or ADL scores of >1 SD. The SD used for different studies: MMT: USA-Kho (7), USA-Needham (10), Australia (6), Brazil – Neto (14), Combined (8); ADL: USA-Kho (3).
In Brazil – Neto study there were patients with significant decrease (>1 SD) of MMT score from ICU admission to ICU discharge, and the change in FSS-ICU in MMT score significant negative change group was 2.9 (n=22). This is significantly different from the change in FSS-ICU in MMT score no change or positive change groups.
Only one patient was found in this group.
MID
In the combined results, MID estimates based on the standard error of measurement and 0.2 SD were relatively consistent with 1.2-1.3 for ICU admission/awakening, 2.1-2.4 for ICU discharge, and 1.7-1.9 for hospital discharge (Table 5). Estimates based on MDC90 and 0.50 SD also were consistent, but larger, at 3.0-3.1, 5.3-5.4, and 4.3-4.5 for the same time points, respectively. Hence, the MID is estimated to be in the range of 2.0-5.0.
Table 5.
MID for FSS-ICU: distribution-based estimates
| Study (sample size) | USA – Kho (N=27-29) | USA – Needham (N=44-52) | Australia (N=19-66)a | Brazil – da Silva (N=99) | Brazil – Neto (N=561) | Combined (N=91-807) |
|---|---|---|---|---|---|---|
| Standard Error of Measurement | ||||||
| ICU awakening/admission | 1.8 | 1.7 | 1.8 | 1.0 | 1.1 | 1.3 (807) |
| ICU discharge | 1.9 | 2.1 | 2.0 | 2.0 | 2.4 | 2.4 (800) |
| Hospital discharge | 1.6 | 2.1 | 0.9 | 1.9 (91) | ||
| Minimal Detectable Change90 | ||||||
| ICU awakening/admission | 4.1 | 3.9 | 4.1 | 2.4 | 2.7 | 3.1 (807) |
| ICU discharge | 4.3 | 4.9 | 4.7 | 4.8 | 5.7 | 5.4 (800) |
| Hospital discharge | 3.7 | 4.9 | 2.2 | 4.5 (91) | ||
| 0.5 SD (moderate Cohen effect size) | ||||||
| ICU awakening/admission | 3.9 | 3.7 | 4.0 | 2.3 | 2.6 | 3.0 (807) |
| ICU discharge | 4.2 | 4.7 | 4.6 | 4.6 | 5.4 | 5.3 (800) |
| Hospital discharge | 3.6 | 4.7 | 2.1 | 4.3 (91) | ||
| 0.2 SD (small Cohen effect size) | ||||||
| ICU awakening/admission | 1.6 | 1.5 | 1.6 | 0.9 | 1.0 | 1.2 (807) |
| ICU discharge | 1.7 | 1.9 | 1.8 | 1.8 | 2.2 | 2.1 (800) |
| Hospital discharge | 1.4 | 1.9 | 0.8 | 1.7 (91) | ||
Abbreviations: MID: minimal important difference; FSS-ICU: functional status score for the intensive care unit; SD: standard deviation.
Only 19 patients at hospital discharge.
Discussion
Using data from 5 studies across 3 continents, we evaluated internal consistency, validity, responsiveness, and MID of FSS-ICU, an outcome measure assessing physical function in critically ill patients.(7;10;13) We found consistent and strong evidence of internal consistency and concurrent construct validity with expected findings for convergent, discriminant and known group validity tests. The similarity of these clinimetric analyses across individual studies demonstrates generalizability of results and supports pooling of data and analyses across studies, as done in prior research.(34-36)
The findings of convergent validity between the FSS-ICU and MMT agree with a prior smaller analysis.(13) Prior studies of the FSS-ICU also provided preliminary evidence of predictive validity and responsiveness,(10;13) which were expanded in our current analyses with larger sample size and more variables. Predictive validity was supported with FSS-ICU scores at ICU discharge significantly predicting post-ICU hospital LOS and hospital discharge location. An increase in FSS-ICU score was observed with improvement in muscle strength and ADLs, and FSS-ICU scores tracked the recovery trajectory of survivors from ICU awakening/admission to hospital discharge with a large effect size, supporting responsiveness. The MID for the FSS-ICU, based on multiple distribution-based methods, is estimated within a range of 2.0-5.0. These results were similar across various time points and the 5 data sets, supporting generalizability.
The results of this evaluation should be compared to similar evaluations of other published ICU-specific physical function measures, including: the Physical Function in Intensive care Test scored (PFIT-s),(13;24;37) Chelsea Critical Care Physical Assessment tool (CPAx),(38-40) Perme mobility scale,(41;42) Acute Care Index of Function (ACIF) score,(43) Surgical intensive care unit Optimal Mobilization score (SOMS),(25;26;44) and the IMS.(23) With respect to floor and ceiling effects, for the FSS-ICU, we detected a minimal floor effect (≤0.5%), but some ceiling effects at hospital discharge (≤21%), which may limit the instrument’s ability to detect improvement.(45) However, these findings compare favorably to other ICU-specific physical measures (Web Table 3). The CPAx has the lowest ceiling effects at ICU discharge;(39;40) however, it is important to note that CPAx differs from other ICU-specific measures (Web Table 3) as it involves evaluation of both physical function (whole body activities and grip strength) and respiratory (ventilation, oxygenation, and secretion clearance) measures.
For evaluation of validity, the PFIT-s, IMS, and CPAx also displayed concurrent construct validity with MMT (Web Table 3). Similar to FSS-ICU, PFIT-s also showed construct validity with hand grip strength and IMS, and there is a strong positive correlation between FSS-ICU and PFIT-s (rho=0.85-0.87, p<0.005) at ICU awakening and ICU discharge.(13) Our analyses also demonstrated appropriate divergent validity of FSS-ICU.
For predictive validity, a higher FSS-ICU, along with higher PFIT-s, IMS, SOMS, and ACIF scores, predict shorter hospital LOS and/or discharge location to home. The PFIT-s, IMS, and CPAx also demonstrated moderate to large responsiveness to change via effect size analyses. Although a prior study of the FSS-ICU demonstrated small responsiveness to change (effect size 0.46),(13;24;37) our current analysis demonstrated a large effect size (2.02) for FSS-ICU from ICU awakening/admission to ICU discharge, suggesting good responsiveness.
There is growing interest in identifying a core set of outcome measures which can be utilized across the continuum of recovery to measure response to interventions and monitor functional improvement. The FSS-ICU is a robust tool, which can be utilized to evaluate physical function in both the ICU setting and in the acute hospital setting for ICU survivors. The ability of FSS-ICU to be used in longer-term follow-up beyond acute hospitalization may be impacted by a ceiling effect. It is also important to consider clinical utility: the FSS-ICU takes 10 to 30 minutes to complete (depending on patient’s functional status), requires no additional equipment, and can be undertaken by the therapist at the bedside with standardized instructions readily available and thus can be easily integrated into routine critical care practice.
The strengths of our study includes performing a range of clinimetric analyses using 5 international data sets with relatively large combined sample size (N=819). Given that many of our findings were consistent across these data sets with different study designs, patient populations, and time points, help support generalizability of our findings. However, there are potential limitations. First, we only assessed internal consistency of the FSS-ICU and did not evaluate inter-rater and test-retest reliability, which should be examined in future research. Second, because of the heterogeneity in study design and data collection among studies, some measurements were not available in all studies and at all assessment time points, limiting our sample size for some analyses particularly for analyses of validity and responsiveness, which may have contributed to non-significant findings. Third, the Brazil-Neto study evaluated FSS-ICU in Portuguese without undertaking independent forward and backward translation process; however, its results were similar to analyses from the other datasets. Further cross-cultural validation is needed. Fourth, we could not calculate the MIDs using an anchor-based method as recommended (30;31) because of the lack of MIDs for MMT and other available physical measures that would be needed as anchors. However, the standard error of measurement (SEM) has been recommended among distribution-based MID methods (31) and estimates based on the SEM converged with those from 0.2 SD.(31;46) Future studies should compare anchor-based MIDs with distribution-based MIDs.
Conclusion
The FSS-ICU is an internally consistent, valid and responsive measure of physical function in the ICU and acute hospital ward setting. The estimated range for the MID of 2.0-5.0 will facilitate sample size calculations and interpretation of future group comparison studies in ICU patients.
Supplementary Material
Abbreviations: FSS-ICU: Functional Status Score for the Intensive Care Unit.
aAll paired t-test p-values are <0.01 for comparisons of the FSS-ICU score (range: 0 to 35) between each consecutive time point (from pre-hospitalization to ICU awakening, from ICU awakening/admission to ICU discharge, and from ICU discharge to hospital discharge) for each of the 5 data sets.
b Sample size at first available FSS-ICU time point for each study: USA-Kho (34), USA-Needham (59), Australia (66), Brazil – da Silva (99), Brazil – Neto (561).
Acknowledgments
We thank all of the patients who participated in this study.
Funding/Support: Funding from NIH (R24HL111895). Michelle Kho was funded by a Canadian Institutes of Health Research Fellowship Award and Bisby Prize; she currently holds a Canada Research Chair in Critical Care Rehabilitation and Knowledge Translation.
Footnotes
Author Contributions: MH had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. All authors have read and approved the final manuscript. DMN, KSC and MH developed the study concept and design. MH conducted statistical analysis and all authors interpreted the data. MH, KSC, SMP, and DMN drafted the manuscript and all authors have provided critical revisions for important intellectual content. This study was supervised by DMN.
The authors declare that they have no other relevant financial interests.
Reference List
- 1.Desai SV, Law TJ, Needham DM. Long-term complications of critical care. Crit Care Med. 2011 Feb;39(2):371–9. doi: 10.1097/CCM.0b013e3181fd66e5. [DOI] [PubMed] [Google Scholar]
- 2.Herridge MS, Tansey CM, Matte A, et al. Functional disability 5 years after acute respiratory distress syndrome. N Engl J Med. 2011 Apr 7;364(14):1293–304. doi: 10.1056/NEJMoa1011802. [DOI] [PubMed] [Google Scholar]
- 3.Fan E, Dowdy DW, Colantuoni E, et al. Physical complications in acute lung injury survivors: a two-year longitudinal prospective study. Crit Care Med. 2014 Apr;42(4):849–59. doi: 10.1097/CCM.0000000000000040. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Hough CL. Improving physical function during and after critical care. Curr Opin Crit Care. 2013 Oct;19(5):488–95. doi: 10.1097/MCC.0b013e328364d7ef. [DOI] [PubMed] [Google Scholar]
- 5.Fletcher SN, Kennedy DD, Ghosh IR, et al. Persistent neuromuscular and neurophysiologic abnormalities in long-term survivors of prolonged critical illness. Crit Care Med. 2003 Apr;31(4):1012–6. doi: 10.1097/01.CCM.0000053651.38421.D9. [DOI] [PubMed] [Google Scholar]
- 6.Tipping CJ, Young PJ, Romero L, et al. A systematic review of measurements of physical function in critically ill adults. Crit Care Resusc. 2012 Dec;14(4):302–11. [PubMed] [Google Scholar]
- 7.Parry SM, Granger CL, Berney S, et al. Assessment of impairment and activity limitations in the critically ill: a systematic review of measurement instruments and their clinimetric properties. Intensive Care Med. 2015 May;41(5):744–62. doi: 10.1007/s00134-015-3672-x. [DOI] [PubMed] [Google Scholar]
- 8.Connolly B. Describing and measuring recovery and rehabilitation after critical illness. Curr Opin Crit Care. 2015 Oct;21(5):445–52. doi: 10.1097/MCC.0000000000000233. [DOI] [PubMed] [Google Scholar]
- 9.Zanni JM, Korupolu R, Fan E, et al. Rehabilitation therapy and outcomes in acute respiratory failure: an observational pilot project. J Crit Care. 2010 Jun;25(2):254–62. doi: 10.1016/j.jcrc.2009.10.010. [DOI] [PubMed] [Google Scholar]
- 10.Thrush A, Rozek M, Dekerlegand JL. The clinical utility of the functional status score for the intensive care unit (FSS-ICU) at a long-term acute care hospital: a prospective cohort study. Phys Ther. 2012 Dec;92(12):1536–45. doi: 10.2522/ptj.20110412. [DOI] [PubMed] [Google Scholar]
- 11.Terwee CB, Mokkink LB, Knol DL, et al. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012 May;21(4):651–7. doi: 10.1007/s11136-011-9960-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Needham DM, Korupolu R, Zanni JM, et al. Early physical medicine and rehabilitation for patients with acute respiratory failure: a quality improvement project. Arch Phys Med Rehabil. 2010 Apr;91(4):536–42. doi: 10.1016/j.apmr.2010.01.002. [DOI] [PubMed] [Google Scholar]
- 13.Parry SM, Denehy L, Beach LJ, et al. Functional outcomes in ICU - what should we be using? - an observational study. Crit Care. 2015 Mar 29;19(1):127. doi: 10.1186/s13054-015-0829-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Parry SM, Berney S, Warrillow S, et al. Functional electrical stimulation with cycling in the critically ill: a pilot case-matched control study. J Crit Care. 2014 Aug;29(4):695–7. doi: 10.1016/j.jcrc.2014.03.017. [DOI] [PubMed] [Google Scholar]
- 15.Kho ME, Truong AD, Zanni JM, et al. Neuromuscular electrical stimulation in mechanically ventilated patients: a randomized, sham-controlled pilot trial with blinded outcome assessment. J Crit Care. 2015 Feb;30(1):32–9. doi: 10.1016/j.jcrc.2014.09.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Kho ME, Truong AD, Brower RG, et al. Neuromuscular electrical stimulation for intensive care unit-acquired weakness: protocol and methodological implications for a randomized, sham-controlled, phase II trial. Phys Ther. 2012 Dec;92(12):1564–79. doi: 10.2522/ptj.20110437. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969;9(3):179–86. [PubMed] [Google Scholar]
- 18.Katz S, FORD AB, MOSKOWITZ RW, et al. Studies of illness in the aged. The index of ADL: a standardized measure of biological and psychosocial function. JAMA. 1963 Oct 21;185:914–9. doi: 10.1001/jama.1963.03060120024016. [DOI] [PubMed] [Google Scholar]
- 19.Fan E, Ciesla ND, Truong AD, et al. Inter-rater reliability of manual muscle strength testing in ICU survivors and simulated patients. Intensive Care Med. 2010 Jun;36(6):1038–43. doi: 10.1007/s00134-010-1796-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.De Jonghe B, Sharshar T, Lefaucheur JP, et al. Paresis acquired in the intensive care unit: a prospective multicenter study. JAMA. 2002 Dec 11;288(22):2859–67. doi: 10.1001/jama.288.22.2859. [DOI] [PubMed] [Google Scholar]
- 21.Massy-Westropp NM, Gill TK, Taylor AW, et al. Hand Grip Strength: age and gender stratified normative data in a population-based study. BMC Res Notes. 2011;4:127. doi: 10.1186/1756-0500-4-127. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Bohannon RW. Reference values for extremity muscle strength obtained by hand-held dynamometry from adults aged 20 to 79 years. Arch Phys Med Rehabil. 1997 Jan;78(1):26–32. doi: 10.1016/s0003-9993(97)90005-8. [DOI] [PubMed] [Google Scholar]
- 23.Hodgson C, Needham D, Haines K, et al. Feasibility and inter-rater reliability of the ICU Mobility Scale. Heart Lung. 2014 Jan;43(1):19–24. doi: 10.1016/j.hrtlng.2013.11.003. [DOI] [PubMed] [Google Scholar]
- 24.Denehy L, de Morton NA, Skinner EH, et al. A physical function test for use in the intensive care unit: validity, responsiveness, and predictive utility of the physical function ICU test (scored) Phys Ther. 2013 Dec;93(12):1636–45. doi: 10.2522/ptj.20120310. [DOI] [PubMed] [Google Scholar]
- 25.Piva S, Dora G, Minelli C, et al. The Surgical Optimal Mobility Score predicts mortality and length of stay in an Italian population of medical, surgical, and neurologic intensive care unit patients. J Crit Care. 2015 Aug 5; doi: 10.1016/j.jcrc.2015.08.002. [DOI] [PubMed] [Google Scholar]
- 26.Schaller SJ, Stauble CG, Suemasa M, et al. The German validation study of the surgical intensive care unit optimal mobilitys score. J Crit Care. 2016 Apr;(32):201–6. doi: 10.1016/j.jcrc.2015.12.020. [DOI] [PubMed] [Google Scholar]
- 27.Knapp TR. Coefficient alpha: conceptualizations and anomalies. Res Nurs Health. 1991 Dec;14(6):457–60. doi: 10.1002/nur.4770140610. [DOI] [PubMed] [Google Scholar]
- 28.Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care. 1989 Mar;27(3 Suppl):S178–S189. doi: 10.1097/00005650-198903001-00015. [DOI] [PubMed] [Google Scholar]
- 29.Hough CL, Lieu BK, Caldwell ES. Manual muscle strength testing of critically ill patients: feasibility and interobserver agreement. Crit Care. 2011;15(1):R43. doi: 10.1186/cc10005. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Yost KJ, Eton DT. Combining distribution- and anchor-based approaches to determine minimally important differences: the FACIT experience. Eval Health Prof. 2005 Jun;28(2):172–91. doi: 10.1177/0163278705275340. [DOI] [PubMed] [Google Scholar]
- 31.Revicki D, Hays RD, Cella D, et al. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol. 2008 Feb;61(2):102–9. doi: 10.1016/j.jclinepi.2007.03.012. [DOI] [PubMed] [Google Scholar]
- 32.Cohen J. Statistical power analysis for the behavioral sciences. 2. Hillsdale, NJ: Lawrence Earlbaum Associates; 1988. [Google Scholar]
- 33.Bland JM, Altman DG. Cronbach’s alpha. BMJ. 1997 Feb 22;314(7080):572. doi: 10.1136/bmj.314.7080.572. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34.Chan KS, Friedman L, Dinglas VD, et al. Evaluating physical outcomes in ARDS survivors: validity, responsiveness & minimal important different of 4-meter gait speed test. Crit Care Med. 2016 doi: 10.1097/CCM.0000000000001760. In press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Chan K, Pfoh E, Denehy L, et al. Construct validity and minimal important difference of 6-minute walk distance in survivors of acute respiratory failure. Chest. 2015;147(5):1316–26. doi: 10.1378/chest.14-1808. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Pfoh E, Chan K, Dinglas V, et al. Cognitive screening among acute respiratory failure survivors: a cross-sectional evaluation of the mini mental state examination. Critical Care. 2015;5(19):220. doi: 10.1186/s13054-015-0934-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Nordon-Craft A, Schenkman M, Edbrooke L, et al. The physical function intensive care test: implementation in survivors of critical illness. Phys Ther. 2014 Oct;94(10):1499–507. doi: 10.2522/ptj.20130451. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Corner EJ, Wood H, Englebretsen C, et al. The Chelsea critical care physical assessment tool (CPAx): validation of an innovative new tool to measure physical morbidity in the general adult critical care population; an observational proof-of-concept pilot study. Physiotherapy. 2013 Mar;99(1):33–41. doi: 10.1016/j.physio.2012.01.003. [DOI] [PubMed] [Google Scholar]
- 39.Corner EJ, Hichens LV, Attrill KM, et al. The responsiveness of the Chelsea Critical Care Physical Assessment tool in measuring functional recovery in the burns critical care population: an observational study. Burns. 2015 Mar;41(2):241–7. doi: 10.1016/j.burns.2014.12.002. [DOI] [PubMed] [Google Scholar]
- 40.Corner EJ, Soni N, Handy JM, et al. Construct validity of the Chelsea critical care physical assessment tool: an observational study of recovery from critical illness. Crit Care. 2014;18(2):R55. doi: 10.1186/cc13801. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Perme C, Nawa RK, Winkelman C, et al. A tool to assess mobility status in critically ill patients: the Perme Intensive Care Unit Mobility Score. Methodist Debakey Cardiovasc J. 2014 Jan;10(1):41–9. doi: 10.14797/mdcj-10-1-41. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Nawa RK, Lettvin C, Winkelman C, et al. Initial interrater reliability for a novel measure of patient mobility in a cardiovascular intensive care unit. J Crit Care. 2014 Jun;29(3):475. doi: 10.1016/j.jcrc.2014.01.019. [DOI] [PubMed] [Google Scholar]
- 43.Bissett B, Green M, Marzanp V, et al. Reliability and utility of the Acute Care Index of Function in intensive care patients: An observational study. Heart & Lung. 2016;45(1):10–4. doi: 10.1016/j.hrtlng.2015.09.008. [DOI] [PubMed] [Google Scholar]
- 44.Kasotakis G, Schmidt U, Perry D, et al. The surgical intensive care unit optimal mobility score predicts mortality and length of stay. Crit Care Med. 2012 Apr;40(4):1122–8. doi: 10.1097/CCM.0b013e3182376e6d. [DOI] [PubMed] [Google Scholar]
- 45.de Vet H, Terwee CB, Mokkink LB, Knol DL. Measurement in Medicine - A Practical Guide. Cambridge: 2011. [Google Scholar]
- 46.Wyrwich KW, Tierney WM, Babu AN, et al. A comparison of clinically important differences in health-related quality of life for patients with chronic lung disease, asthma, or heart disease. Health Serv Res. 2005 Apr;40(2):577–91. doi: 10.1111/j.1475-6773.2005.00373.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Abbreviations: FSS-ICU: Functional Status Score for the Intensive Care Unit.
aAll paired t-test p-values are <0.01 for comparisons of the FSS-ICU score (range: 0 to 35) between each consecutive time point (from pre-hospitalization to ICU awakening, from ICU awakening/admission to ICU discharge, and from ICU discharge to hospital discharge) for each of the 5 data sets.
b Sample size at first available FSS-ICU time point for each study: USA-Kho (34), USA-Needham (59), Australia (66), Brazil – da Silva (99), Brazil – Neto (561).