Characteristics of a protocol to collect objective physical activity/sedentary behaviour data in a large study: Seniors USP (understanding sedentary patterns)

Abstract

The Seniors USP study measured sedentary behaviour (activPAL3, 9 day wear) in older adults. The measurement protocol had three key characteristics: enabling 24-hour wear (monitor location, waterproofing); minimising data loss (reducing monitor failure, staff training, communication); and quality assurance (removal by researcher, confidence about wear). Two monitors were not returned; 91% (n=700) of returned monitors had 7 valid days of data. Sources of data loss included monitor failure (n=11), exclusion after quality assurance (n=5), early removal for skin irritation (n=8) or procedural errors (n=10). Objective measurement of physical activity and sedentary behaviour in large studies requires decisional trade-offs between data quantity (collecting representative data) and utility (derived outcomes that reflect actual behaviour).

Introduction

Physical activity (PA) and sedentary behaviour (SB) are important modifiable risk factors related to a range of health conditions, including mortality, cardiovascular and metabolic disease, and cancer.^1,2 Objective measures, using body-worn sensors, provide a detailed and accurate assessment of the amount of PA and SB undertaken by an individual in their daily life. In large-scale studies (e.g. n>400)³ use of self-report measures of both PA and SB are frequently justified for logistic rather than measurement considerations.^4,5 However, self-report measures typically overestimate PA (e.g. by 20 to 40 minutes per day)⁶ and underestimate SB (e.g. by two to four hours per day), ⁵ and may be measuring different constructs of physical behaviours compared with objective monitors.⁷

Using objective measurement of PA and SB in large-scale studies incurs practical and pragmatic challenges, different from the use of self-report, and often requires informed decisional trade-off between collecting large volumes of data and the utility and relevance of the outcomes that can be derived from such data. Costs are incurred both in terms of equipment (monitors and attachment consumables) and in terms of deployment and retrieval (staff costs, travel reimbursement, and postage).⁸ Data loss occurs through lack of compliance (people not wearing the monitor) and uncertainty about data utility (assurance that collected data reflects actual behaviour). Data loss can result in a smaller sample size than anticipated and potential selection bias both in terms of the demographics of those who do comply with wear protocols, and in terms of which days are measured.⁸

The first large scale study to use objective monitoring was the National Health and Nutritional Examination Survey (NHANES, 2003-2004). Using a hip worn ActiGraph monitor a 68% compliance rate was achieved from returned monitors, with data loss from either monitors not calibrated on return (5%) or not worn for a minimum of 4 valid (10 hour) days (27%);⁹ this did not include data lost from monitors that were not returned. To reduce data loss, recent large scale studies have attempted to increase compliance by opting for a wrist worn monitor. This was successful in increasing compliance from returned monitors (UK biobank, 93% providing 3 days of valid data;¹⁰ 70-80% 6 days valid wear NHANES 2011-2012).⁷ However, important concerns have been raised about the face validity of wrist worn monitors and their ability to provide accurate and interpretable measures of PA and SB, in particular time spent in postural sitting.^11,12,13 Thigh worn monitors (such as activPAL) which are able to clearly distinguish postural sitting,¹³ have been previously used in some large studies but not in population cohort studies (e.g. Walking away from diabetes, n=530 providing 67% with 7 days valid wear; AusDiab n=782 providing 79% with 7 days valid wear).¹⁴ The debate is, of course, whether any potential loss of data quality from monitor wear location is justified in order to provide a larger and potentially more representative sample of free-living PA and SB, and whether compliance should be the main aspect of methodology considered worthy of investment.

Specific protocols for successful objective data collection, including a level of detail which would allow replication by other studies, and covering the entire measurement chain, are rarely published in peer-reviewed articles.^3,14 The purpose of this brief report is to share the principles and details of the objective data collection protocol of PA and SB from one study (Seniors USP: understanding sedentary patterns). The protocol relies not only on increasing adherence but also on ensuring wear and data quality.

Methods

Briefly, the Seniors USP study¹⁵ collected objective PA and SB (primary outcome measure) data for at least 9 days (for 7-day analysis) using the activPAL3 monitor (PAL Technologies Ltd, Glasgow, UK), from older adults in three existing cohorts from within longitudinal studies (Lothian Birth Cohort 1936,¹⁶ West of Scotland Twenty-07 Study).¹⁷ The protocol and standard operating procedures (available at http://edshare.gcu.ac.uk/view/keywords/seniors%20usp%20sops.html) implemented a coherent package, which aimed to maximise both the volume and utility of the data collected. The key characteristics of the protocol were: enabling 24-hour wear; minimising data loss; and quality assurance. These key characteristics, along with details of the methods used to achieve them, are provided in Table 1.

Table 1.

Key Characteristics of the methodology and design of the Seniors USP study, which contribute to objective activity data quality and compliance.

Key Characteristic	Component of methodology
Enabling 24-hour wear	monitor selection monitor and wear location combination was selected which allows the monitor to be worn 24 hours a day for at least one week (activPAL3 [PAL technologies, Glasgow, UK] on the front of the thigh)  waterproof monitor heat sealed (P200-C heat sealer [Packer, Essex, UK] ) within Layflat plastic tubing (75mm wide x 150m long x 250 gauge [Packer, Essex, UK]) to eliminate reasons why the monitor may be removed (e.g. bathing, swimming)  Opsite flexifix [Smith&Nephew, London, UK] waterproof dressing placed over the sealed monitor  reduce chances of skin irritation appropriate materials used: hypoallergenic adhesive pad (PALstickie [PAL technologies, Glasgow, UK]), medical grade waterproof dressing (Opsite flexifix); food-safe plastic tubing (Packer layflat tubing)  enhance comfort hypoallergenic adhesive pad (PALstickie) provided padding between the skin and the monitor/waterproof tube, and reduced the likelihood of skin irritation  edges of the waterproof pouch trimmed with some to spare, to avoid hard corners which might dig in the skin  schedule appointments to avoid removal scheduling appointments when the participant knew they were having medical treatment (e.g. involving hospitalisation), or were scheduled to fly (to avoid the need to remove for airport security) was actively avoided
Minimising Data Loss	test all monitors before starting data collection all monitors tested (we had 4 researchers wearing 13 monitors per leg) for multiple days before using them for data collection  monitors that were not functioning correctly identified and sought assistance from manufacturer  set minimum battery level for programming 4.1v (value obtained through experience) was used as a minimum battery level for programming the monitor, to avoid data loss through monitor stopping recording  use wide programming times programmed to start recoding immediately, to allow confirmation that the monitor was collecting data (flashing green light)  programmed to record for 14 days (minimum required for full data collection was 9 days) to allow for delays in starting to wear the monitor  trained staff applied the monitor ensured the monitors were applied appropriately  ensured waterproof dressing was applied properly, minimising potential for water ingress (and consequent data loss through removal or monitor stopping)  used a checklist on application, including re-checking monitor was recording data (flashing green light)  common misunderstandings/procedural shortcuts were pre-identified and addressed in training (e.g. highlighting tips and errors).  communication between participants and research staff a central contact point was provided for participants to discuss concerns with a study researcher, which reduced inappropriate monitor removal  communication between central experts and fieldworkers key diagnostic data logged by staff applying/removing the monitor, for example battery level at programming and downloading, whether green light flashing at application.  key diagnostic data recorded centrally (on a secure cloud server) allowing easy review by all staff  data recorded to increase compliance and allow identification of systematic errors/deviation from protocol and monitor malfunction  member of staff with experience of data collection using the monitor assigned to triage technical issues with using the monitor  early and continuous quality assurance checks allowed identification of individual and systematic deviations from protocol, immediate feedback to staff, and engagement in process  reducing opportunities to lose monitor monitor removed by researcher, which reduced reliance on participants for data retrieval, for example remembering to bring monitor to appointment, losing the monitor while not worn, accidentally washing monitor placed in pocket.
Quality Assurance	increasing confidence monitor was worn monitor removed by researcher, allowing confirmation monitor still worn after end of analysis period  a message was provided to participants that monitor should not require re-attachment during data collection. Additional material to allow reattachment was not provided. Participants were not asked to prospectively record if monitor was not worn  assurance that monitor had not been reattached by participant was provided by using attachment materials that are not commonly available to participants  In the case that the monitor was removed by participant prior to research appointment, we then asked retrospectively for date and time of removal. This was close to date of removal to allow for reasonable recall, and was then checked with data record. Data processing was from midnight-midnight and not from specific time of removal, so day/date of removal was sufficient information  data inspection routinely performed by a single researcher close to time collected; difficult cases resolved by discussion with a second researcher  hierarchical review process was used (weekly graphical display, daily graphical display, raw acceleration data), to speed up routine cases but maintain in-depth review when required.  conducted with confidence that monitor was on the leg during data collection (i.e. looking for issues in battery/monitor failure, or thresholds not appropriate, e.g. known not to collect shuffling gait at slow speeds)

Enabling 24-hour wear

Enabling a 24-hour monitor wear protocol minimised data loss due to participant compliance with reporting and/or identification of wear times; identifying and dealing with non-wear time is a source of data loss and debate in studies without a continuous wear protocol.^10,14 However, for studies using SB as an outcome measure, the trade-off is a requirement to identify sleep to allow removal of sleep time during data processing; we used paper diaries to record sleep/wake times. Monitor selection is crucial, as the location that the monitor is worn on the body must not only be comfortable and suitable for continuous wear, but also provide robust information about the behaviour of interest. The activPAL3 provides a recognised gold standard measure of postural SB,^13,18 and is worn on the front of the thigh and is suitable for long-term wear including overnight when using attachment materials to reduce skin irritation. Based on reported reasons for lack of compliance in previous studies, further improvements in compliance can be made by taking care to make the monitor attachment comfortable to wear, effective waterproofing, and careful scheduling of research appointments to avoid times the participant might be more likely to remove the monitor (e.g. flights).

Minimising Data Loss

Data loss was minimised by adopting a protocol that reduced the likelihood and effects of monitor failure. At the start of the project, after receipt from the manufacturers and prior to being deployed in the field, each monitor was tested once to ensure it worked (individual calibration of activPAL monitors on each use is not required). Monitors were only programmed if they had a pre-defined minimum battery level. Wide programming limits (days recorded) including an immediate start, were selected to allow for minor variations in protocol and confirmation that the monitor was recording when attached. Eliminating extraneous data collected outside the study wear period is trivial in post-processing. Trained researchers attached the monitor, ensuring correct placement and reducing data loss through poor attachment. Although not strictly necessary, the monitor was also removed by trained staff; this reduced opportunities for loss through participant error and/or forgetfulness. Detailed standard operating procedures and staff training were developed to ensure consistent and effective implementation of the protocol. Communication was important. Participants were provided with a central study contact which allowed discussion of concerns and avoided unnecessary monitor removal. Additionally, reciprocal communication between fieldworkers and central research staff allowed the identification of deviations from protocol and procedure at monitor return, which could then be addressed through immediate feedback and/or additional training of fieldworkers.

Quality Assurance

The protocol was designed to provide confidence that the monitor was worn for the entire measurement period; only datasets with continuous wear for all included days were analysed and therefore no data imputation was conducted. Attachment of the monitor with single-use attachment materials and removal of the monitor by a researcher allowed a high level of certainty of continuous monitor wear. Although it was possible that a monitor that was still worn on removal by the researcher had been removed and re-attached by the participant, reattachment with the single-use attachment materials is both difficult and noticeable. In addition, spare attachment materials were not provided to the participants, and use of attachment materials that were not commonly available to participants meant that any participant reattachment would be identifiable by the researcher removing the monitor. In cases where the monitor was removed early, participant report of date and time of removal was recorded retrospectively at the research appointment. This was considered acceptable as we required recall of a single removal event to the precision of the day on which it occurred. In contrast to many other studies,¹⁴ we did not provide spare attachment materials or ask participants to record removal times prospectively. These measures were specifically adopted to encourage the expectation that monitors should not be removed. Although this will have prevented legitimate reattachment of the monitor if it was removed, it was balanced against increased certainty of wear/compliance. On-going quality assurance, as monitors were returned, was conducted by a single experienced researcher, with complicated cases resolved through discussion with a second researcher. Quality assurance of downloaded data was conducted with certainty that the monitor had been worn, reducing the need to make assumptions about participant behaviour (e.g. extended periods of sitting could be ascribed to the participant sitting, as the monitor was known to be worn). However, inconsistency with reported wear time and unusual data patterns were investigated in a hierarchical manner (week view, 24-hour view, and raw acceleration), and eliminated if a technical source for the discrepancy was identified.

Results

Forty-four percent of older adults approached to take part in the study agreed to wear a monitor. Only two of the monitors issued to participants (n=773) were not returned; in both cases, the monitor was removed early by the participant and subsequently lost. In this study, we achieved 700 datasets (91% of the 771 returned monitors) included in analysis, with a very stringent inclusion criteria of 24-hour data and 7 days of continuous wear; relaxing our inclusion criteria to 4 days of wear would have resulted in 97% of returned data included. Most data loss was attributed to early monitor removal (n=48); no reason for removal was recorded in 16 cases. Ten participants removed the monitor for unavoidable reasons, including skin irritation (n=8) and serious life events not related to wearing the monitor (e.g. bereavement, n=2). Twelve monitors were removed early due to procedural failures, including failure of attachment materials (n=8), water ingress under the dressing (n=2) and appointment scheduling errors (n=2). Ten participants removed the monitor early for their own convenience, for a variety of reasons, such as attending a night out, taking a last-minute holiday, or playing with a grandchild. Other reasons for data loss were: monitor failure (n=11; n=3 serious, e.g. data corruption; n=8 stopped early, i.e. low battery); removed during quality assurance (n=5, e.g. visible acceleration change in raw data did not trigger change in monitor categorisation); and missing/incomplete sleep diary (only relevant to SB outcome measures, n=7).

Discussion

In the Seniors USP study 91% of datasets from returned monitors with full 7 days data were included in analysis, achieving similar or higher proportion of data included from returned monitors whilst simultaneously including more days of data compared to national surveys using wrist-worn monitors (e.g. 93% including 3 days of data, UK biobank;¹⁰ 60-80% including 6 days of data, NHANES)⁷. Rates of agreement to wear the monitor (44%) in the current study were similar to uptake of the wrist-worn monitor from UK biobank (44%).¹⁰ This also compares favourably to other large studies that used the activPAL monitor (e.g. 67% of n=530 including 7 days of data, Walking away from diabetes;¹⁴ 79% of n=782 including 7 days of data, AusDiab;¹⁴ 95% of n=1506 including 5 days of data, ActiFE-Ulm)¹⁹. Participants were all recruited from established longitudinal cohorts, and although this was the first occasion that cohort participants had been asked to wear an activity monitor, this may have made them more compliant with study procedures. Nevertheless, an extremely low number of monitors were not returned (2 out of 773), facilitated by encouraging the expectation that the monitor should not be removed, and asking participants to wear the monitor until a second research appointment. This also removed the burden of remembering to wear the monitor from the participant. A number of decisional trade-offs are apparent in our protocol. Specifying and encouraging 24-hour wear allowed continuous wear and certainty that the data reflected behaviour; just 5 datasets were rejected during quality assurance. However, this was off-set by the need in this study to remove sleep from analysis, as the primary outcome measure was SB. We selected use of a paper diary, and lost 7 sets of data through incomplete diaries. Automatic algorithms to detect time in bed overnight²⁰ and distinguish lying from sitting²¹ are being developed, and may allow inclusion of this data in future studies. Additionally, PA data from the monitors could have been included in analysis. Provision of additional materials to allow participants to reattach monitors during data collection is common practice in many studies.¹⁴ We did not provide additional attachment materials to participants, and lost 10 datasets through poor initial attachment of the monitor or degradation during use (falling off or water ingress); this should be balanced against encouraging an implicit expectation of continuous wear in participants, and the loss of only 5 datasets during data assurance. The effectiveness (balance of sources of data loss) of not providing spare attachment materials may vary by study and population, depending on data collection duration and patterns of attachment degradation during use. Monitor attachment by a researcher (as opposed to by the participant) may have contributed to secure attachment, including familiarity with the materials. However, some data loss was unavoidable as monitors were removed for medical reasons (skin irritation) or because of serious life events. This will represent a source of data loss in any study. Data sets lost through early monitor removal for participant convenience are not unavoidable, but represent participant choice about compliance. Potentially, this could be addressed through communication of expectations, however, eight out of the ten data sets lost provided 6 days of data, which would have been included in other studies. Aspects of the objective measurement of PA/SB, for example monitor removal for convenience, degradation of attachment materials during wear, and remembering to adhere to study protocols, are likely to be affected by the population being studied. The generalisability of the components of the Seniors USP study protocol to other populations should be explored in future studies.

In studies wishing to assess the PA and SB of their participants, there is a clear need for the objective measurement of both PA and SB. However collecting objective measures of posture and movement in very large studies (e.g. UK Biobank, n~100,000),¹⁰ is difficult and requires adequate investment. It is important to clarify in which procedural aspects to invest. The protocol described here (available from http://edshare.gcu.ac.uk/view/keywords/seniors%20usp%20sops.html), was successful in a study of 773 participants, and has been adopted for use by larger studies (e.g. British Cohort Study 70, cohort n=17,000),²² demonstrating the potential for scaling up, although performance at that scale has not yet been evaluated. Although we report on individual items, it is important to understand that it is their combination that makes the protocol successful. In developing the protocol we took a holistic approach integrating the whole measurement and analysis chain, and taking some elements without understanding the co-dependency of the items might not be as effective. The protocol components which incurred the highest costs were the staff costs to allow monitor attachment and removal by a researcher at separate appointments. It is acknowledged that these might be the most difficult and costly components to increase in scale for larger studies. However, some large national surveys have face-to-face research appointments to collect other data (e.g. UK Biobank prior to the activity monitoring component)¹⁰, and it is feasible that monitor attachment could be integrated into such appointments. Additionally, staff costs of research appointments should be offset against the costs involved in purchasing additional monitors to cover monitor losses/non-return, which can be substantial. In the Seniors USP study, the purchase of a single additional monitor would have covered the costs of 20 research visits. This investment in the Seniors USP study, particularly the second appointment for monitor removal by a researcher, resulted in an extremely small number of monitors not being returned, which may represent the ideal scenario for reducing selection bias from consented participants. Alternative strategies, such as incentives paid to the participant on monitor return, may partially compensate for monitors not returned through lack of participant engagement, however they are less able to compensate for monitors not returned because they are lost or damaged after removal by the participant or during transit in the postal service.

In summary, there is growing research demonstrating that the objective measurement of physical activity and sedentary behaviour in large studies is feasible with a range of different monitors. Decisional trade-offs are made in protocols between data quantity (collecting representative data) and utility (derived outcomes that reflect actual behaviour). Paying increased attention to reporting the explicit methodological details of monitor use, across a wide range of studies, will allow future researchers to make appropriate and informed methodological decisions.