Physical Activity of ICU Survivors during Acute Admission: Agreement of the activPAL with Observation

Abstract

Purpose: We estimated the agreement of a thigh-worn accelerometer, the activPAL, used to measure activity and sedentary parameters, with observed mobility assessments of intensive care unit (ICU) survivors. Method: We prospectively compared activPAL measurements with direct observation during assessments at discharge from the ICU or acute hospital in eight participants with a median age of 56 (1st–3rd quartile 48–65) years and an Acute Physiology and Chronic Health Evaluation II score of 23 (1st–3rd quartile 17–24). Frequency of sit-to-stand transitions; time spent standing, stepping, upright (standing and stepping), and sedentary (lying/sitting); and total steps were described; analysis was performed using Bland–Altman plots and calculating the absolute percent error. Results: All sit-to-stand transitions were accurately detected. The mean difference on the Bland–Altman plots suggested an overestimation of standing time with the activPAL of 31 (95% CI: −9, 71) seconds and underestimation of stepping time by 25 (95% CI: −47, −3) seconds. The largest median absolute percent errors were for standing time (21.9%) and stepping time (18.7%); time spent upright (1.7%) or sedentary (0.3%) was more accurately estimated. The activPAL underestimated total steps per session, achieving the largest percent error (70.8%). Conclusion: Because it underestimated step count, the activPAL likely incorrectly recorded stepping time as standing time, so that time spent upright was the measure of activity with the smallest error. Sedentary behaviour, including frequency of transitions, was validly assessed.

Wearable activity monitors can be used on hospitalized patients to objectively quantify daily activity levels and sedentary behaviour. However, data on the validity of activity monitoring in the acute hospital setting are limited to older medical or stroke inpatients^1–3 and are specific to the monitoring protocol, such as the device used, wear location, and recording or analysis method. Studies in acute populations have most commonly cited the device's validation among community-dwelling participants,^4–6 but it may not be generalizable to those who are acutely unwell because of the influence of the hospital environment and acute illness on physical activity patterns and quality of movement. Survivors of a critical illness present with a unique pattern of muscle dysfunction that exacerbates immobility-induced changes; this affects their short- and long-term activity levels, something that can be quantified by accelerometer monitoring.^7–9

In an intensive care unit (ICU), behavioural mapping suggests that patients rarely ambulate outside their rehabilitation sessions;¹⁰ as a result, therapy sessions are the main contributor to their cumulative daily activity. Because health care staff can overestimate active time and underestimate inactive time, objective activity and inactivity assessments such as accelerometry can provide a more accurate assessment of activity.¹¹ Accelerometry is advantageous because it is less intrusive and labour intensive than behavioural mapping or video observation, and it has the potential to be used for routine monitoring or in larger studies of the impact of the amount of activity on outcomes. Existing studies of accelerometry with ICU patients are limited because the accuracy of monitoring protocols has either not been tested^9,12 or has not been validated for out-of-bed activities.¹³

As part of a larger prospective study of activity monitoring in people admitted to an ICU (NCT 02881801), we sought to use concurrent measurement, using the activPAL accelerometer and direct observation, to test the hypothesis that there would be agreement between these two methods in classifying sit-to-stand transitions; time spent standing, stepping, upright (standing plus stepping), and sedentary (sitting or lying); and total number of steps during an unstructured mobility assessment of survivors of a critical illness on acute hospital wards.

Methods

The relevant institutional ethics review committees approved the procedures, and we conducted them in a single tertiary ICU at a hospital in Bedford Park, South Australia, Australia. Patients aged 18 years or older who required mechanical ventilation for 5 or more days were eligible for inclusion. Patients who were unable to ambulate independently 10 or more metres before admission, unable to provide informed consent, or medically deemed as needing palliative management were excluded. A total of 15 participants provided written informed consent after waking up from mechanical ventilation for potential assessment at the a priori defined milestones of discharge from the ICU or discharge from acute hospital.

Eligible participants from the larger study had been consecutively sampled, but to overcome the limitations of previous work,¹³ we purposely sought to assess a subset of patients who were capable of transitioning from sit to stand with maximal assistance from two people (no mechanical lifters) and were able to attempt stepping with or without a walking frame⁵ at discharge from the ICU. Of the 15 potential participants, 12 met this minimum mobility criterion between October 2015 and April 2016. However, assessments could only be completed with 8 participants. The choice of milestone (discharge from the ICU or acute hospital) was based on the availability of the treating physiotherapist, the participant, and the research observer for a scheduled mobility assessment. Each patient was assessed to his or her best functional capacity, while wearing the accelerometer to quantify his or her daily physical activity and sedentary behaviour.

Participants were fitted with a waterproofed activPAL3 (PAL Technologies, Glasgow, United Kingdom) on their non-dominant, anterior mid-thigh with strips of adhesive tape (Hypafix, BSN Medical, Hamburg, Germany). A thigh-worn accelerometer was chosen, as in previous studies,^2,5,6 because attaching a monitor at this site facilitates its wearability: It is less likely to be compromised by dressings, attachments, or invasive lines in acutely unwell patients. Minor variations in the mid-thigh position do not appear to affect the accuracy of outcome measurement.¹⁴ Unlike ankle-worn devices, the thigh-worn device can differentiate between posture and sit-to-stand transitions,¹⁵ although the former can be more accurate in determining step count at slower gait speeds.⁵

The activPAL recognizes posture (sitting or lying and upright), stepping, and transitions between postures using accelerometer-derived information about thigh position and accelerations.¹⁶ Further details of the activPAL algorithm are unknown because they are proprietary.¹⁶ The default setting for minimum upright period recording was reduced from 10 to 2 seconds, and the minimum sitting period remained at 10 seconds,¹⁷ based on the expectation that severely deconditioned patients might only stand for short periods. Each device was first waterproofed with a latex finger cot and dressing (OPSITE, Smith & Nephew Medical Ltd., Hull, United Kingdom). Data were transferred to a personal computer for processing with activPAL, version 7.1.18 (PAL Technologies, Glasgow, United Kingdom), and the raw data were then exported to Microsoft Excel.

Observational data were recorded and transcribed by a single physiotherapist with 10 years of clinical experience in an acute tertiary hospital setting and with the early rehabilitation of ICU patients. The physiotherapist used a Sony ICD-P28 digital voice recorder (Teaneck, NJ) to report changes in patients' body posture in real time using definitions for standing and stepping established a priori to determine the time spent upright and, for sitting, to determine the time spent sedentary (see the Appendix). Standing was determined when a patient was in their most upright posture, and a step was determined when the patient's foot lifted completely off the ground. Sitting was determined when the patient reached a seated position either from lying or from when the patient's bottom contacted the seated surface after a period of standing.

In real time, and with these definitions in mind, the observer spoke the words sit, stand, and step (for anticipated small numbers of steps, such as in transfers) into the voice recorder but numerically counted each series of steps when the patient was specifically marching on the spot or walking. A standardized form was used to transcribe observations to the nearest second using timing from the playback file (see the Appendix); activities were then coded consistently with the activPAL for each session before viewing the accelerometer results.

Data on the time interval and cumulative step count for each change in activity code (0=sedentary, 1 = standing, 2=stepping) were summarized from the activPAL event file to the nearest second for the corresponding period of observation and entered into IBM SPSS Statistics, version 21.0 (IBM Corporation, New York, NY). This was done by the same physiotherapist once accelerometer data had been obtained from all the study participants. The frequency of sit-to-stand transitions; time spent standing, stepping, upright (standing plus stepping), and sedentary (sitting or lying); and total number of steps over the session were noted. Descriptive data were reported as median (1st–3rd quartile), unless otherwise stated; full summary data for time and step count outcomes from both methods are presented in Table S1 online.

To be consistent with other studies that have compared the activPAL with observation (without video),^5,6 we used the Bland–Altman¹⁸ method to assess the difference between device and visualized measurements, plotted against the average of these measures; horizontal lines at the mean difference, ±1.96 SDs (limits of agreement), were also plotted. The precision of the mean differences and limits of agreement were determined by calculating 95% CIs, as described by Bland and Altman.¹⁸ Table S1 also includes summary data for the difference between the methods (activPAL—observation) and the absolute percent error ([activPAL—observation] / observation) × 100, calculated per participant. For the total number of steps, method agreement was also represented by a scatterplot, showing lines of perfect agreement and best fit (with Pearson's r and 95% CI), due to a systematic difference between the methods (see Figure S1 online).

Results

Eight participants completed the study; six were women aged median 56 (1st–3rd quartile 48–65) years, with an admission Acute Physiology and Chronic Health Evaluation II score of 23 (1st–3rd quartile 17–24) and ICU length of stay of 16 (1st–3rd quartile 12–38) days. The assessments occurred 18 (1st–3rd quartile 15–47) days after ICU admission; five participants were assessed at discharge from the ICU, and three participants were assessed at discharge from the hospital. The duration of observed sessions was 24 minutes, 57 seconds (1st–3rd quartile 12 minutes, 44 seconds–37 minutes, 14 seconds); there were 2 (1st–3rd quartile 1–3) sit-to-stand transitions observed per session (16 across all sessions) that were identified on 100% of occasions by the activPAL. Across all sessions, there were 11 stepping bouts (retrospectively defined from observed step counts as ≥10 consecutive steps); 1 (1st–3rd quartile 1–2) bout per participant and 58 (1st–3rd quartile 42–95) steps per bout were achieved as both marching on the spot (55% of bouts) and walking (45% of bouts).

Participants were observed to spend most of their time sedentary: a median of 20 minutes, 41 seconds (1st–3rd quartile 10 minutes, 39 seconds–31 minutes 59 seconds; see Table S1 online). In terms of active time, participants were observed to spend a median of 3 minutes, 1 second (1st–3rd quartile 1 minute, 52 seconds–5 minutes, 30 seconds), upright, consisting of more time standing—1 minute, 41 seconds (1st–3rd quartile 1 minute, 7 seconds–4 minutes, 18 seconds)—than time stepping—1 minute, 16 seconds (1st–3rd quartile 34 seconds–2 minutes, 55 seconds; see Table S1 online).

Figure 1 shows the mean (95% CI) differences between the activPAL and observation measurements. Time spent standing was overestimated by 31 (95% CI: −9, 71) seconds (Figure 1a), and time spent stepping was underestimated by 25 (95% CI: −47, −3) seconds (Figure 1b); the smallest mean difference between measures was for time spent upright (overestimated by 7 [95% CI: −33, 48] s; Figure 1c) and time spent sedentary (underestimated by 10 [95% CI: −59, 39] s; Figure 1d). The limits of agreement were ±1 minute, 34 seconds, for time spent standing; ±51 seconds for time spent stepping; ±1 minute, 36 seconds, for time spent upright; and ±1 minute, 54 seconds, for time spent sedentary. The largest absolute percentage errors were in time spent standing and stepping—that is, a median (interquartile range) of 21.9% (101.1%) and 18.7% (73.1%), respectively—compared with the more accurate measures of time spent upright (1.7% [23.5%]) and sedentary (0.3% [5.9%]; see Table S1).

Figure 1.

Bland-Altman plots of differences between objectively measured and observed time spent, in seconds: (a) standing, (b) stepping, (c) upright, and (d) sedentary. Grey shading represents the 95% CI for the mean difference; error brackets at the right of each plot represent the 95% CIs for the limits of agreement.

As many as a median of 85 (1st–3rd quartile 27–158) total steps observed per session were consistently underestimated by objective monitoring; these are all the points, shown on the scatterplot of total steps detected by activPAL monitoring versus observation, that fall below the line of perfect agreement (see Figure S1). For the line of best fit, r=0.997 (95% CI: 0.823, 0.994). The absolute percentage of error for total steps was a median (interquartile range) of 70.1% (28.6%).

Discussion

Like the study by Taraldsen and colleagues,² the activPAL correctly identified the number of sit-to-stand transitions. The point of equality (zero) for the mean difference between the two methods was within the 95% CIs on the Bland–Altman plots for all time-based variables except time spent stepping; this suggests acceptable agreement. More specifically, time spent upright and time spent sedentary had the smallest absolute percentage of error. Still, the activPAL appeared to overestimate time spent standing, possibly as a result of incorrectly registering time spent stepping as time spent standing and underestimating step count. The latter result is consistent with other studies of activPAL step detection in participants with slower walking speeds.^2,5

The clinical relevance of a difference in methods remains to be determined. However, given that the CIs for mean differences were within ±41 seconds and the limits of agreement were within ±1 minute, 36 seconds, for the time-based activity outcomes—equivalent to between 4% and 12% of the times spent active or walking reported in previous studies of ICU patients (16.8 minutes at or within 5 days in the ICU¹² to 13.7 minutes before discharge⁹)—we think that the activPAL provides a sufficiently valid estimate of time spent upright in people recovering from an ICU admission. As a positional device (rather than one that uses activity cut-points), the activPAL is capable of recording sedentary time as well as the transition between sedentary postures and quiet standing. This is important because quiet standing may indeed be therapeutic for deconditioned patients, and changes in response to treatment may first be seen in patterns of sedentary behaviour,¹⁵ such as breaks from sitting to standing. The measurement of transitions in the present study was 100% consistent between the activPAL and observation.

The systematic difference between the two methods in step detection in this sample of ICU survivors appeared to be larger than that seen in other populations who may mobilize with a slow gait speed or walking aid.⁵ This lower method agreement may have been due to the smaller number of occasions or shorter duration of stepping observed per participant. However, unlike a previous study, which compared an accelerometer with direct observation of ICU patients¹³ but observed therapeutic activity only in a bed or chair, all our participants attempted out-of-bed mobilization (standing or stepping). Although our observed mobility sessions were longer than those noted with behavioural mapping of ICU patients, it is not clear whether the time spent undertaking activities associated with ambulation within sessions was comparable.¹⁰ The median proportion of observed time spent stepping in the present study equated to 6.3% (1st–3rd quartile, 2.2%–9.5%). Alternatively, the low rate of step detection could have occurred in relation to how the steps were accumulated across different parts of an unstructured assessment (in transfer, marching on the spot, or walking) or to the differences in the stepping motor pattern of deconditioned patients after critical illness reflected in the accelerations detected by the activPAL. The accurate measurement of step count is likely to be clinically important in that walking during hospital admission for older adults is emerging as a measure of greater predictive ability for 30-day readmission than other measures of activities of daily living.¹⁹

This study had several limitations. First, we used the technique of direct observation^1,13 to compare the two methods rather than video recording,^2,4 and although video recordings enable interrater and intrarater reliability assessment of activity classifications and can limit errors in notation, it is challenging to position the camera appropriately in acute hospital and open ward settings. Also, we did not formally assess the reliability of the direct observation process, although we sought to mitigate sources of error by using set definitions for activities and a single, experienced investigator. Using direct observation of position and activity according to pre-defined categories is an established method,¹⁰ with high inter-observer agreement when assessed against video scenarios.^1,6,20 Unlike previous validation studies with acutely ill patients,^1,2 we chose to use a single monitor. Although this simplified the procedure, it limited the possibility of differentiating sitting from lying.² The limits of agreement and CIs for the differences between the methods on the Bland–Altman plots would be smaller if we had used a larger sample, thus enabling us to interpret the results with more confidence. Although small samples of specific clinical groups have been used in previous validation studies of accelerometry,^2,4,7 sample sizes in studies of acutely hospitalized ICU survivors that have used accelerometer monitoring reflect the challenges of research with critically ill patients and the need for further evaluations of accelerometry in the acute setting.

Conclusion

This study compared the agreement of the activPAL with direct observation in measuring activity and sedentary variables during mobility assessments of ICU survivors as they recovered on acute hospital wards; this method enhances the transferability of results to real clinical situations. The comparability of the activPAL with other accelerometer models used with ICU patients remains unknown.^9,12 Nonetheless, the single thigh-worn activPAL most consistently detected sit-to-stand transitions and time spent upright and sedentary, although the measurement of total steps was underestimated and should be interpreted with caution.

Key Messages

What is already known on this topic

Accelerometry can provide a valid measure of physical activity and sedentary time in stable healthy and clinical populations, but it has not been validated in the assessment of survivors of a critical illness in the acute hospital setting.

What this study adds

The agreement between the thigh-worn activPAL and observation in detecting sit-to-stand transitions in unstructured mobility assessments of intensive care unit survivors was excellent (100%). The agreement between these methods for time-based variables was best for time spent sedentary and upright (standing and stepping) and had the smallest percentage of error. However, the device underestimated step count, and the clinical relevance of this result remains to be determined.

Appendix

Observational Data Transcription Form

Validation Data

Time point: ICU discharge/hospital discharge

Session number: __________

Physiotherapist initials: __________

Activity Monitoring

ActivPAL

Serial number of device: ____________________

Time applied (24-hour time): ____________________

Date applied: ______ / ______ / ______

Applied to leg: left / right

Time set to start recording: __________

Observation

Date: __________

Start time: ____________

End time: _________

Direct observation/audio transcript

Instructions

Sequentially note the time (minutes:seconds) of each change in body position in a new row of the table below.

Position/Activity Definitions

Sit: The time when the patient has reached a seated position, with the trunk upright (≥45°) and legs lowered, on the edge of the bed from lying (not sitting up in bed, can only be sitting on edge of bed); or, following weight-bearing transfer, the time when the patient's bottom contacts the seated surface (supported in any type of chair or unsupported on the edge of the bed) after a period of standing.

Stand: The time when the patient is in their most upright position in standing.

Stepping: The time when the foot of the patient completely lifts off the ground (i.e., steps in transfer, marching on the spot, or walking).

Other movements: Can be described at discretion of observer (e.g., whether sitting in chair, wheelchair, etc.).

Time commenced observation	Body position				Other notes (e.g., no. of steps [step counts]) on transcription
	Laying	Sitting	Standing	Stepping