Skip to main content
PLOS ONE logoLink to PLOS ONE
. 2020 Sep 11;15(9):e0238690. doi: 10.1371/journal.pone.0238690

Cognition and motor function: The gait and cognition pooled index

Jacqueline K Kueper 1, Daniel J Lizotte 1,2,3,4, Manuel Montero-Odasso 1,5,*, Mark Speechley 1,4; for the Alzheimer’s Disease Neuroimaging Initiative
Editor: Antony Bayer6
PMCID: PMC7485843  PMID: 32915845

Abstract

Background

There is a need for outcome measures with improved responsiveness to changes in pre-dementia populations. Both cognitive and motor function play important roles in neurodegeneration; motor function decline is detectable at early stages of cognitive decline. This proof of principle study used a Pooled Index approach to evaluate improved responsiveness of the predominant outcome measure (ADAS-Cog: Alzheimer’s Disease Assessment Scale-Cognitive Subscale) when assessment of motor function is added.

Methods

Candidate Pooled Index variables were selected based on theoretical importance and pairwise correlation coefficients. Kruskal-Wallis and Mann-Whitney U tests assessed baseline discrimination. Standardized response means assessed responsiveness to longitudinal change.

Results

Final selected variables for the Pooled Index include gait velocity, dual-task cost of gait velocity, and an ADAS-Cog-Proxy (statistical approximation of the ADAS-Cog using similar cognitive tests). The Pooled Index and ADAS-Cog-Proxy scores had similar ability to discriminate between pre-dementia syndromes. The Pooled Index demonstrated trends of similar or greater responsiveness to longitudinal decline than ADAS-Cog-Proxy scores.

Conclusion

Adding motor function assessments to the ADAS-Cog may improve responsiveness in pre-dementia populations.

Introduction

The Alzheimer’s Disease (AD) Assessment Scale–Cognitive Subscale (ADAS-Cog) was developed in 1984 to assess cognitive dysfunction in AD [1]. The ADAS-Cog subsequently became a widely adopted outcome measure for assessing efficacy in clinical trials of antidementia treatments and is still used today. Multiple studies have reported relationships between cognitive and motor function in pre-dementia syndromes [27]. There is a need for outcome measures that reflect these advancements and are more responsive for present research settings, while maintaining compatibility with historical measurement techniques.

Pre-dementia syndromes such as mild cognitive impairment (MCI) involve decreased cognitive functioning of memory, language, and judgement, with decrements in between normal or expected age-associated cognitive decline and serious cognitive and functional deficits seen with dementia [8,9]. There are concerns about the responsiveness of the ADAS-Cog at pre-dementia stages of disease [1013]. Responsiveness is a form of validity defined as the ability to detect change [1417]. Change can be contextualized using three aspects: group versus individual level of measurement, between-person versus within-person comparison, and the type of change one is interested in detecting [14]. The responsiveness of any outcome measure is population and context specific; although the ADAS-Cog performs well for studies of dementia it does not meet the needs of studies earlier in the natural history of disease progression [14,16].

Responsiveness can be affected by measurement properties such as floor and ceiling effects of individual items. Accordingly, several modifications have improved ADAS-Cog responsiveness, including alternative scoring, removing tasks, and adding assessments of delayed recall, executive function, and activities for daily living [1012,1823]. An important property when modifying an outcome measure is backward compatibility as this allows the direct comparison of novel study results with previous literature based on the original measure. An advantage of maintaining backwards compatibility with the ADAS-Cog is the ability to compare results with the vast amount of previously conducted literature that uses this ‘de-facto’ gold standard measure.

Since the time of ADAS-Cog development, research has found motor function decline plays an important role in dementia and pre-dementia syndromes [24,25]. Motor function tests help assess aspects of severity or stage of dementia not captured by purely cognitive tests [24,25]. Performance on tests such as quantitative gait assessment has been associated with cognitive status, changes in cognition over time, and incident dementia [1,2634]. Furthermore, combined cognitive and gait impairments are more strongly associated with risk of cognitive decline and conversion to dementia than either component alone [31,35]. Dual-task gait performance (walking while simultaneously performing a cognitive task) has been associated with cognitive ability, different pre-dementia syndromes, and incident dementia [33,36,37]. The magnitude of change in gait during dual-tasking can be expressed as a dual-task gait cost (DTC), which adjusts for an individual’s baseline gait characteristics [38]. Importantly, the ability to maintain gait control while using cognitive resources underlies the ability to safely perform daily activities required for independent living [39]. Our literature review of modifications made to the ADAS-Cog since its development did not find any revisions whereby motor function or DTC assessments were added to the ADAS-Cog [13]. The overall aim of our study was to assess whether motor function assessments can be a helpful addition to cognitive outcome measures for detecting pre-dementia syndrome progression.

We hypothesized that adding assessments of motor function to the ADAS-Cog would improve responsiveness among older adults with pre-dementia syndromes. Our objectives were 1) to develop an outcome measure using a pooled index approach that includes quantitative gait and DTC assessments and is backwards-compatible to the ADAS-Cog and, 2) compare the responsiveness of the ADAS-Cog and the Pooled Index to group-level between-person differences in stage of pre-dementia disease progression at one point in time (baseline discrimination), and 3) compare the responsiveness of the ADAS-Cog, the Pooled Index, and different combinations of items to group-level within-person measured change over time in a pre-dementia sample (longitudinal decline).

Methods

We searched for a database containing ADAS-Cog scores, quantitative gait assessments, and prospectively measured conversion to dementia across different cognitive subgroups at baseline. Because we did not locate a database with all required items, we accessed two partially overlapping datasets that together had the required variables and developed a statistical model that approximated the ADAS-Cog, the ‘ADAS-Cog-Proxy’.

Study populations

The Gait and Brain Study

The Gait and Brain Study is an ongoing prospective cohort (clinicaltrial.gov identifier: NCT03020381) designed to determine whether quantitative gait deficits can predict cognitive and mobility decline, falls, and progression to dementia among community-dwelling older adults. Study details have been described elsewhere [31,33,40]. The study was approved by the University of Western Ontario Health Sciences Research Ethics Board (approval number: 17200), and written informed consent was obtained from participants at the time of enrollment. Participant recruitment began in 2007 from geriatric and memory clinics at hospitals affiliated with the University of Western Ontario in London, Ontario. Inclusion criteria were 65 to 85 years old, able to walk 10 meters without assistance, and absence of dementia. Exclusion criteria were lack of English proficiency, Parkinsonism or other neurological disorder affecting motor function, musculoskeletal disorders or joint replacements that affect gait performance (clinician-assessed), use of psychotropics that can influence motor performance, and major depression. At baseline, eligible participants were divided into three diagnostic categories based on performance in cognitive testing and clinical evaluation. The Normal Cognition (NC) group had normal age-, sex-, and education-adjusted scores on the Mini-Mental State Examination (MMSE) [41] and Montreal Cognitive Assessment (MoCA) [42] based on standardized norms that account for age, sex, and education [43]. Subjective Cognitive Impairment (SCI) criteria were the same as that for NC, except patients reported persistent decline in cognition that was not explainable by an acute event, and answered yes to both, “Do you feel like your memory or thinking is becoming worse?” and “Does this concern you?”. As described in the study protocol and following work [36,44], Mild Cognitive Impairment (MCI) was based on Petersen criteria [9] and included 1) a score of 0.5 on the Clinical Dementia Rating (CDR) Scale, 2) subjective cognitive complaints, 3) measured cognitive impairment in memory, executive function, attention, and/or language defined as scores 1.5 SD below expected performance based on published norms for age, sex, and education [43], 4) intact Lawton-Brody Activities of Daily Living, and 5) absence of dementia determined by a specialized clinician and based on Diagnostic and Statistical Manual of Mental Disorders version IV-TR criteria.

The Alzheimer’s Disease Neuroimaging Initiative

The Alzheimer’s Disease Neuroimaging Initiative (ADNI) is a multi-phase study that began in 2003 as a public-private study partnership with the primary goal of testing whether neuroimaging, biological, and clinical assessments can be combined to measure progression from MCI to early AD (adni.loni.usc.edu). Study sites are located throughout North America. We used data from the first phase, ADNI1; detailed information on ADNI1 can be found at www.adni-info.org. Because ADNI1 collected the ADAS-Cog as well as several of the same cognitive measures administered in the Gait and Brain Study, ADNI1 data were used to develop a predictive model that estimates ADAS-Cog scores (ADAS-Cog-Proxy) that would have been collected in the Gait and Brain Study. ADNI1 data was downloaded on October 26, 2016. General inclusion criteria: Hachinski Score less than or equal to 4, age 55–90 years, stability of ADNI-permitted medications for 4 weeks, Geriatric Depression Scale under 6, study partner with 10 hours or more per week contact to accompany the participant to visits, visual and auditory acuity adequate for neuropsychological testing, good general health, sterile or two years past childbearing potential for women, willing and able to complete a three year imaging study, completed six grades of education or has good work history, fluent English or Spanish speaking ability, commitment to Neuroimaging and no medical contraindications to MRI, agrees to DNA for ApoE testing and banking and to blood and urine for biomarkers, and not enrolled in other trials or studies [45].

Measures

Cognition

The process from ADAS-Cog-Proxy model development in ADNI1 to score estimation for participants in the Gait and Brain Study is described in detail in the S1 Appendix. Briefly, a predictive model developed with ADNI1 data was used to obtain ADAS-Cog-Proxy scores. The outcome was ADAS-Cog score; candidate covariates were cognitive tests present in both ADNI1 and the Gait and Brain Study (MMSE, Rey Auditory Verbal Learning Test, CDR- Sum of Boxes, Digit Span Forward, Digit Span backward, Trail Making Test A, and Trail Making Test B). Linear Regression and Generalized Additive Model (GAM) predictive models were considered [46,47]. Candidate models were constructed in a development subset of baseline ADNI1 data, using different combinations of candidate covariates. Preliminary accuracy was assessed in the development subset as the percentage of participants with a predicted ADAS-Cog-Proxy score within three points of their observed ADAS-Cog score. Three points is considered a clinically significant change [48]. The best candidate ADAS-Cog-Proxy predictive model was selected based on preliminary accuracy estimates and on similarity of the measurement domains captured by covariates to the ADAS-Cog. Final model accuracy was estimated in a separate testing subset of ADNI1 data.

To allow all participants in the Gait and Brain Study to have an ADAS-Cog-Proxy GAM score, Multivariate Imputation by Chained Equations (MICE) was used to impute missing GAM covariate values [49,50]. Five imputed datasets were created; the ADAS-Cog-Proxy GAM was applied to each, and the mean of the five estimated scores for each participant was taken as their final ADAS-Cog-Proxy GAM score. MICE and estimation of ADAS-Cog-Proxy GAM scores for participants in the Gait and Brain Study was repeated for baseline, 6, 12, 24, 36, and 48 month follow-up visits. Additional details are in Table B of the S1 Appendix.

Motor function

We evaluated quantitative gait performance using an electronic walkway system (GAITRite™) [51]; participants walk along the walkway and several gait measurements are taken. To avoid measuring acceleration and deceleration phases, start and end points were marked one metre away from the boundaries of a six metre recording distance. Participants were asked to walk as they usually would, and average values across the recording distance were used. Four testing conditions were performed, including one single-task and three dual-task conditions (see below). The following spatio-temporal gait parameters were considered based on prominence and utility in previous research: velocity, stride time, step time, stride length, step length, double support time, swing time, stride width, stride velocity, and cadence. The coefficient of variation (StandardDeviation(SD)Mean×100) allows direct comparison of variability across variables measured using different units, [52] and was calculated for all parameters except velocity.

Motor-cognitive performance

Simultaneous assessment of motor and cognitive functioning was done using the same electronic walkway as for motor function measurements, but participants were asked to perform cognitive tasks while walking. We assessed the three dual-task gait conditions of walking while: i) counting backwards from 100 by ones, ii) counting backwards from 100 by sevens, and iii) naming animals. Participants were not instructed to prioritize either the cognitive or the walking task. DTC was calculated for velocity, stride time, and stride time coefficient of variation, using the formula [(Singletaskconditiondualtaskcondition)singletaskcondition]×100 [36]. These three parameters were selected based on literature supporting their importance in dementia and pre-dementia syndromes [25,33,36,40].

Baseline descriptive statistics

Participant characteristics summarized for the two datasets included demographics (age, education, sex), medication count, comorbidity count, depressive symptoms (Geriatric Depression Scale [53], physical activity, Activities of Daily Living (Lawton-Brody Scale, Instrumental and Basic [54], Cognition [1] or ADAS-Cog-Proxy, MoCA [42], MMSE [41], CDR-SB [55], Rey Auditory Verbal Learning Test [56], Trail Making Test A and B [57], gait (velocity, stride time, strive time coefficient of variation [25,33,2]), and dual-task gait (gait velocity with counting, stride time with serial sevens, stride time with naming animals [25, 33, 2]).

Analyses

Pooled index development

Our outcome measure was developed as a Pooled Index, which allows source variables with different scoring ranges to be combined into a single summary score while maintaining the ability to use each of the source variables individually [5860]. Including up to six component variables with low pairwise correlations in a PI is recommended for covering important measurement domains and reducing variability of final PI scores [1719]. We split our candidate components into three categories that have evidence of importance for pre-dementia and dementia syndromes and required at least one variable from each of the categories to be included in the Pooled Index: cognition, motor function, and motor-cognitive performance. After recoding variables so that higher values indicated greater dysfunction, Pooled Index scores were obtained by calculating Z-scores for each variable [Z=(observationgoupmean)SD], and then averaging those Z-scores [58,60]. When individual Pooled Index variables have low pairwise correlations, the SD of the combined score decreases as the number of source variables increases, which increases the detectable signal relative to noise or variability [59,60]. Variable selection for our Pooled Index was thus guided by low pairwise correlation coefficients, aided by theoretical considerations. An overview of our process is in Fig 1. To create the Pooled Index we first assessed pairwise correlations between the ADAS-Cog-Proxy GAM scores and each of the single-task gait and DTC variables. Variables were retained when |rho|<0.2 or when |rho| = 0.2 to 0.4 with evidence supporting that parameter’s involvement in dementia or pre-dementia syndromes.

Fig 1. Overview of pooled index development.

Fig 1

NC = Normal Cognition, SCI = Subjective Cognitive Impairment, MCI = Mild Cognitive Impairment.

Pairwise correlation coefficients were calculated for all retained single-task gait and DTC variables. In looking for at least one weakly intercorrelated pair, when numerical considerations were similar, we consulted literature on the involvement of the candidate variables in pre-dementia or dementia syndromes. When numerical and theoretical criteria were similar, box plots were created to assess which, if any, of the contending gait and DTC variables demonstrated a stepwise progression from NC to SCI to MCI diagnostic categories. Scatterplots were used to rule out strong non-linear relationships. Because most of the reduction in pooled SD occurs up to about six variables and diminishes thereafter, we focused our attention on finding a relatively small number of variables rather than the largest number possible. Finally, ease of assessment was a final pragmatic consideration for both individual variables and the Pooled Index as a whole.

Responsiveness

To assess whether motor function assessments improve responsiveness to changes in pre-dementia syndromes, we compared the Pooled Index to the ADAS-Cog-Proxy, which is standing in for the ‘gold standard’ original version of this cognitive outcome measure. Larger responsiveness test statistics suggest better detection of change.

Due to skewness and small sample sizes non-parametric tests were used to evaluate responsiveness to baseline discrimination. Kruskal-Wallis tests were used to assess whether the ADAS-Cog-Proxy and Pooled Index could detect a significant difference among the diagnostic categories of NC, SCI, and MCI. Mann-Whitney U tests were used to assess all pairwise comparisons.

Standardized Response Means [SRM=meandifferencescoreSDofthedifferencescore] were used to assess responsiveness to longitudinal change over 6 to 48 months of follow-up for the Pooled Index, the ADAS-Cog-Proxy, and the ADAS-Cog-Proxy combined with individual Pooled Index components. Standardization was always performed with respect to the baseline distribution of participants present at the follow-up visit of interest. Ninety-five percent bootstrap confidence intervals based on 1000 iterations of sampling with replacement were computed for the ADAS-Cog-Proxy GAM scores and for the Pooled Index.

Secondary analysis. SRMs and 95% bootstrap confidence intervals were calculated for the MMSE as a secondary analyses and used as a final comparison metric.

All analyses were conducted with RStudio, version 1.0.136 [61].

Results

A total of 109 participants from the Gait and Brain Study were used to develop our Pooled Index and assess responsiveness. Baseline Gait and Brain Study characteristics can be found in Table 1. One participant with SCI did not have single-task gait recorded at baseline and was omitted from analyses. Participants who converted to dementia were included for time points prior to their dementia diagnosis. Two participants converted by six months of follow-up, one by 12 months, four by 24 months, and one by 36 months.

Table 1. Gait and Brain Study baseline characteristics.

Characteristic Mean (SD) Minimum, Maximum Number of missing values (if applicable) unless otherwise specified Overalla Normal Cognitionb Subjective Cognitive Impairmentc Mild Cognitive Impairmentd
Age (years) 74.22 (6.33) 73.50 (4.58) 70.00 (4.59) 75.36 (6.52)
63.00, 92.00 67.00, 82.00 65.00, 85.00 63.00, 92.00
Education (years) 13.85 (2.92) 16.33 (3.06) 14.42 (2.81) 13.33 (2.74)
6.00, 20.00 10.00, 20.00 10.00, 20.00 6.00, 20.00
Sex
Female n (%) 58 (53) 7 (58) 15 (79) 36 (49)
Male 51 5 4 42
Medications (#) 7.62 (4.52) 6.42 (4.06) 6.53 (5.26) 8.06 (4.37)
0, 21 2, 16 0, 21 0, 21
Comorbidities (#) 6.06 (2.85) 4.33 (1.44) 4.79 (2.02) 6.64 (2.98)
0, 13 2, 7 1, 8 0, 13
Geriatric Depression Scale 2.35 (2.14) 1.60 (1.14) 2.25 (1.89) 2.40 (2.21)
0, 10 0, 3 1, 5 0, 10
22 7 15 0
General Physical Activity Level
Vigorous: n (%) 63 (58) 6 (50) 13 (68) 44 (56)
Moderate: n (%) 29 (27) 5 (42) 4 (21) 20 (26)
Seldom: n (%) 16 (25) 1 (8) 2 (11) 13 (17)
Missing: n 1 0 0 1
Instrumental Activities of Daily Living 7.69 (0.94) 8.00 (0.00) 7.75 (0.50) 7.67 (0.99)
2, 8 8, 8 7, 8 2, 8
22 7 15 0
Basic Activities of Daily Living 0.42 (0.97) 0.80 (0.84) 0.75 (0.96) 0.38 (0.98)
0, 5 0, 2 0, 2 0, 5
22 7 15 0
ADAS-Cog-Proxy 9.46 (2.34) 7.59 (1.32) 7.96 (1.93) 10.11 (2.24)
3, 16 4, 9 3, 12 5, 16
Montreal Cognitive Assessment 24.45 (3.82) 27.25 (1.48) 27.89 (2.45) 23.18 (3.60)
12, 30 24, 30 21, 30 12, 30
Mini-Mental State Examination 27.74 (2.52) 28.83 (1.80) 28.89 (1.45) 27.29 (2.69)
18, 30 24, 30 24, 30 18, 30
Clinical Dementia Rating Scale 0.99 (0.89) 0.0 (0.0) NA 1.07 (0.88)
0.0, 4.0 0.0,0.0 NA 0.0, 4.0
68 9 19 40
Rey Auditory Verbal Learning Test (3 trials) 17.20 (5.35) 23.40 (5.18) 24.75 (6.65) 16.34 (4.71)
8.0, 33.0 19.0, 32.0 17.0, 33.0 8.0, 28.0
29 7 15 7
Gait Velocity (cm/s) 108.40 124.80 114.10 104.60
21.27 15.78 17.59 21.47
57.27, 165.2 99.65, 155.80 82.17, 141.00 57.27, 165.20
1 0 0 0
Stride Time (s) 1.14 (0.10) 1.11 (0.08) 1.10 (0.08) 1.16 (0.10)
0.93, 1.41 0.95, 1.20 0.97.0, 1.26 0.93, 1.41
1 0 1 0
Stride Time Coefficient of Variation (CV) (%) 2.47 (1.48) 2.08 (0.76) 2.49 (2.02) 2.53 (1.43)
0.62, 9.73 1.14, 4.04 1.16, 9.73 0.62, 7.89
1 0 1 0
Dual-Task Gait Velocity Cost with Counting (%) 5.51 (10.68) 3.10 (11.52) 2.58 (5.48) 6.55 (11.35)
-16.04, 34.61 -8.16, 34.61 -11.05, 10.82 -16.04, 31.12
1 0 1 0
Dual-Task Stride Time Cost with Serial Sevens (%) -16.93 (18.42) -24.06 (29.08) -8.23 (9.87) -17.86 (17.37)
-75.93, 6.30 -75.93, 3.74 -38.54, 2.74 -69.50, 6.30
3 0 1 2
Dual-Task Stride Time CV Cost with Naming Animals (%) -133.40 -214.80 -44.54 -141.30
(270.66) (416.11) (80.73) (269.59)
-1382.00, 77.58 -1382.0, 63.87 -240.3, 53.55 -1200.00, 77.58
1 0 1 0

SD = Standard Deviation, NA = Not Applicable. Number of participants

a = 109

b = 12

c = 19

d = 78.

To develop the ADAS-Cog-Proxy, 573 participants from ADNI 1 were used; baseline ADNI1 characteristics can be found in Table 2.

Table 2. Alzheimer's Disease neuroimaging initiative baseline characteristics.

Characteristic Mean (SD) Minimum, Maximum Number of missing values (if applicable) unless otherwise specified
Age (years) 75.17 (6.56)
54.40, 89.60
Education (years) 15.84 (2.94)
6.00, 20.00
Sex
Female n (%) 228 (40%)
Male 345
Alzheimer’s Disease Assessment Scale-Cognitive Subscale 9.51 (4.63)
0, 28
Mini-Mental State Examination 27.78 (1.84)
23, 30
Rey Auditory Verbal Learning Test (3 trials) 18.34 (5.64)
5, 38
Clinical Dementia Rating Scale-Sum of Boxes 1.02 (1.03)
0.00, 4.50
Trail Making Test A 41.41 (20.08)
17.00, 188.00
4
Trail Making Test B 114.8 (65.62)
34.0, 348.0
5
Digit Span Forward Test 6.64 (1.05)
4, 8
Digit Span Backward Test 4.71 (1.19)
0, 7

Number of participants = 573, all from ADNI1. Five ADNI 1 participants were missing at least one covariate value for the fifth ADAS-Cog-Proxy candidate model and were excluded solely from analyses pertaining to that model.

ADAS-Cog-Proxy

The best candidate model was a GAM with three covariates: Rey Auditory Verbal Learning Test (RAVLT), the MMSE, and the CDR-Sum of Boxes. Cognitive domains assessed by these measures overlap with the ADAS-Cog; adding more less similar cognitive assessments did not meaningfully improve GAM performance. Model accuracy on the ADNI1 testing subset was 69% of participant scores predicted within three points and 88% within five points of observed ADAS-Cog scores. A summary of ADAS-Cog-Proxy GAM development is in the S1 Appendix.

A summary of the number of missing GAM covariates imputed using MICE is in Table B of the S1 Appendix.

Gait and cognition Pooled Index

Variables selected for the Pooled Index included the ADAS-Cog-Proxy, gait velocity, and DTC for gait velocity with the secondary task of counting backwards from 100 by ones (Fig 1). Pairwise correlation coefficients ranged from 0.27 to 0.32.

Baseline discrimination

Both the ADAS-Cog-Proxy and the Pooled Index showed an overall statistically significant difference in mean ranks across the three diagnostic categories (ADAS-Cog-Proxy: H(2) = 24.13; PI: H(2) = 22.36, both P<0.001). Statistically significant pairwise comparisons were found for SCI versus MCI (ADAS-Cog-Proxy: U = 331, P = 0.0002; PI: U = 348, P = 0.0009) and NC versus MCI (ADAS-Cog-Proxy: U = 153, P = 0.0002; PI: U = 148, P = 0.0001), but not NC versus SCI diagnostic categories (ADAS-Cog-Proxy: U = 93, P = 0.41; PI: U = 75, P = 0.17).

Longitudinal change

All SRMs are in Table 3. In terms of point estimates, the full Pooled Index demonstrated better responsiveness than the ADAS-Cog-Proxy GAM scores for 6 and 48 months, but not 36 months of follow-up. For 12 and 24 months the ADAS-Cog-Proxy GAM scores detected overall improvement, while the Pooled Index detected almost no change. Adding only gait velocity to the ADAS-Cog-Proxy using a Pooled Index approach consistently increased responsiveness to decline (less negative or more positive SRM), while adding only DTC to the ADAS-Cog-Proxy showed mixed results. For each time point, the 95% bootstrap confidence intervals for the Pooled Index capture a higher range of SRMs than for the ADAS-Cog-Proxy GAM scores; however, for each timepoint the intervals overlap. Secondary analysis of the MMSE show that it had the highest SRM point estimate for all time points except six months (S1 Table).

Table 3. Standardized response means to assess responsiveness to longitudinal change in the Gait and Brain Study.

n Months of follow-up ADASp (95% CI) ADASp + GV ADASp + DTC ADASp + GV + DTC (95% CI)
86 6 0.14 (-0.08, 0.34) 0.17 0.18 0.23 (0.01, 0.47)
73 12 -0.05 (-0.31, 0.17) -0.02 0.03 0.06 (-0.18, 0.31)
55 24 -0.24 (-0.49, 0.03) -0.11 -0.07 0.01 (-0.27, 0.26)
35 36 0.23 (-0.08, 0.55) 0.34 0.11 0.18 (-0.15, 0.56)
24 48 0.60 (0.22, 1.04) 0.68 0.59 0.65 (0.31, 1.2)

ADASp = Alzheimer’s Disease Assessment Scale-Cognitive Subscale-Proxy, CI = bootstrap Confidence Interval, DTC = Dual Task Cost, GV = Gait Velocity, m = months, n = sample size, + indicates variables were combined using a pooled index approach.

Due to the nature of using data from an on-going cohort study, not all participants had a chance to reach all follow-up time points, which contributed to our small sample sizes especially at the longest points of follow-up. Additional reasons for not having a visit at all time points include conversion to dementia and drop out due to health conditions or death. We assessed baseline differences between participants who did versus did not have each follow-up visit. Participants with 24- and 48-month visits had statistically significantly faster gait speed than those who did not have those visits. There were no statistically significant differences in baseline gait velocity for the other lengths of follow-up, or for any follow-up length in age, education, DTC, or ADAS-Cog-Proxy scores.

Discussion

This proof of principle study suggests a Pooled Index approach combining assessments of motor function with ADAS-Cog-Proxy GAM scores may have comparable or increased responsiveness to changes in pre-dementia syndromes compared to ADAS-Cog-Proxy GAM scores alone.

More specifically, the Pooled Index and the ADAS-Cog-Proxy demonstrated similar responsiveness to baseline discrimination. Both detected statistically significant differences between NC and MCI, and SCI and MCI, but not between NC and SCI categories. For all but one follow-up period the Pooled Index trended towards greater responsiveness to longitudinal decline than the ADAS-Cog-Proxy, but 95% bootstrap confidence intervals always overlapped. For two follow-up periods the point estimate for the ADAS-Cog-Proxy GAM scores detected improvement while the Pooled Index detected worsening; 95% bootstrap confidence intervals for both outcomes cross the point of no change so interpretation of these estimates must be done with caution. Estimates suggesting group-level improvement may be capturing the fact that the trajectory from NC to dementia is not linear, and not all participants are expected to progress to dementia. Motor function decline may not follow the same trajectory as cognitive decline, and has been found to occur in advance of cognitive decline and further disease progression [25,2831]. An additional possibility to explain improvement on ADAS-Cog-Proxy SRMs while Pooled Index SRMs suggest worsening is practice effects or other inconsistency due to multiple versions of the RAVLT [62,63], which is one of the GAM covariates—this may artificially improve scores. Importantly, changes in cognitive or motor function abilities alone, or in the ability to engage in motor and cognitive tasks simultaneously, are all important aspects of functionality. Further research is needed to assess whether including gait assessments provide a more realistic assessment of changes in overall disease severity than purely cognitive measures.

The improvements in longitudinal change responsiveness demonstrated by the Pooled Index were made without including explicit tests of delayed recall or executive function. These cognitive abilities decline in pre-dementia syndromes but are not included on the original ADAS-Cog [1,12]; previous ADAS-Cog modifications incorporating them have found improvements to responsiveness in pre-dementia studies [11,12,19,20]. Given previously found associations between gait velocity and DTC with cognitive abilities, especially executive function, we suspect part of the responsiveness of the Pooled Index is due to gait assessments capturing changes in executive function in addition to motor function aspects of disease progression; assessments of motor and cognitive function are not mutually exclusive [39,64,65].

The results from this study are weaker than expected when viewed alongside the larger body of literature demonstrating associations of gait and dual-task cost with cognitive decline and dementia [2530,33,34]. Within the last decade, a new predementia syndrome, motoric cognitive risk syndrome, including both cognitive and motor deficits was introduced; it has been found to be prevalent internationally and have an association with conversion to dementia, suggesting the relationship between cognitive and motor decline is widespread [35,66]. In terms of baseline discrimination, gait and dual-task parameters similar to those used in the present study have been found to distinguish between subtypes of MCI (amnestic vs. non-amnestic) and dementia [33,34,36]. In terms of longitudinal change, motor function, assessed with gait and dual task cost, has been found to occur in advance of cognitive decline and to predict future cognitive decline and conversion to dementia [1,2630,32,37]. Further, the combination of gait and cognitive measures has been found to better predicted dementia than either test alone [31]. Despite these advancements in understanding the natural history of cognitive and motor decline there is less research on developing an outcome measure that aligns with these advancements and optimizes different types of responsiveness; the present study provides a starting point—given the findings are not as clear as anticipated this further highlights a need for further research.

Outcome measures that assess motor and cognitive abilities at the same time may reduce inefficiencies in testing protocols and better detect meaningful changes in functionality or overall disease severity. Further practical advantages of using quantitative gait assessments for outcome measurement include language independence, non-invasive administration procedures, measurement precision, and for the DTC paradigm each participant serves as their own control. Advantages of gait velocity specifically are that it can be easily measured using only a stopwatch.

Similar advantages of using gait measures in a research setting apply to a clinical setting where ease and comprehensiveness of measurement are a priority. For example, gait velocity is a marker of overall health and the amount of dementia pathology, e.g. beta amyloid in the brain, is not necessarily associated with cognitive or functional decline. Adding gait measures to neuropsychological testing in a clinical setting is both feasible and may provide a more accurate picture of progression to dementia and of overall health more generally. To get to this point more research is needed to develop a valid and reliable measure that includes both cognitive and motor assessments and is associated with clinically relevant outcomes. Testing of the measure would need to happen to assess responsiveness in both research and clinical settings as good performance in one setting does not always transfer to another [1416].

An additional contribution of this study is our framework for developing the ADAS-Cog-Proxy. The process outlined in the S1 Appendix may be followed when there is an appropriate research question but not all necessary variables present in a single available database. Using a predictive model to obtain estimates of a missing variable allows preliminary tests of hypotheses without the time and resources that would be required to collect new data.

Main limitations of our study include small sample sizes, which may have contributed to some of the inconsistency in responsiveness across time points, missing data, and not using the original ADAS-Cog. The large proportion of missing data for the RAVLT and CDR-SB especially at longer follow-up visits may reduce the validity of imputation. There is a possible censoring bias given not all participants reached all time points, which is further suggested by the time points where participants who have the visit had faster baseline gait velocity (associated with overall health) than those who did not reach that time point. Our results should be replicated when a dataset with both ADAS-Cog and gait parameters collected under a common protocol becomes available. Two ADAS-Cog-Proxy GAM covariates were collected one month prior to ADAS-Cog administration, which may have contributed extra noise to the GAM development and led to an underestimate of accuracy. Restricting our Pooled Index to only gait velocity single and DTC with the ADAS-Cog-Proxy represents the trade-off in information value between practicality and measurement intensiveness. The derived units of the Pooled Index are also difficult to interpret and not directly comparable to ADAS-Cog scores.

In conclusion, our study used a proof of principle approach to explore whether adding motor tests to the ADAS-Cog would increase responsiveness to cognitive status and longitudinal changes. Our findings indicate a need for future research and researchers who are planning pre-dementia studies or developing new outcome measures may consider including gait assessments as part of a comprehensive test battery. Future steps include replicating the Pooled Index using the original ADAS-Cog, assessing responsiveness with larger subsamples of converters across all levels of disease severity from NC to dementia, further investigating direction of change identified by motor and cognitive measures, and assessing responsiveness to treatment effects in pre-dementia populations.

Supporting information

S1 Appendix. ADAS-Cog-Proxy development details.

(DOCX)

S1 Table. Secondary analysis: Mini mental state examination responsiveness to group-level within-person measured change over time.

(DOCX)

Acknowledgments

Portions of this study used data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Thus, data collection and sharing for this project was partially funded by ADNI (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute of Biomedical Imaging and Bioengineering, and through contributions from: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc,; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; Eurolmmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merk & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

A listing of all ADNI investigators, their role with ADNI, and affiliation can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf The principal investigator is Michael W. Weiner, MD and e-mail addresses categorized by type of question can be found at: http://adni.loni.usc.edu/about/contact-us/

Data Availability

Data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) may be requested through the ADNI website (http://adni.loni.usc.edu/data-samples/access-data/). Data from the Gait and Brain study cannot be shared publicly because the study is still ongoing and the data contain personal health information. This restriction has been imposed by the ethics board of the University of Western Ontario. Data access queries can be directed to Yanina Sarquis-Adamson (contact via Yanina.SarquisAdamson@sjhc.london.on.ca). The authors of the present study had no special access privileges in accessing the underlying data which other researchers would have.

Funding Statement

This work was supported by the Early Research Award (PI MMO) and the Alzheimer Foundation of London and Middlesex Master’s Scholarship in Alzheimer-Related Research to JKK. The Gait and Brain Study is funded by an operating grant from the Canadian Institutes of Health and Research (CIHR, MOP 211220) and a CIHR Project Grant (PJT 153100). Dr. Montero-Odasso’s program in “Gait and Brain Health” is supported by grants from the CIHR, the Ontario Ministry of Research and Innovation, the Ontario Neurodegenerative Diseases Research Initiative, the Canadian Consortium on Neurodegeneration in Aging, and by the Department of Medicine Program of Experimental Medicine Research Award, the University of Western Ontario. The funders involved in setting up and maintaining the ADNI national database, where we obtained our data, are listed in the Acknowledgements. Neither ADNI nor its funders, some of which were commercial, have had any communication with us related to or influencing the specifics of our project. Their funding of ADNI does not alter our adherence to PLOS ONE policies on sharing data and materials.

References

  • 1.Rosen WG, Mohs RC, Davis KL. A New Rating Scale for Alzheimer ‘ s Disease. Am J Psychiatry. 1984;141(11):1356–64. [DOI] [PubMed] [Google Scholar]
  • 2.Mielke MM, Roberts RO, Savica R, et al. Assessing the Temporal Relationship Between Cognition and Gait: Slow Gait Predicts Cognitive Decline in the Mayo Clinic Study of Aging. J Gerontol A Biol Sci Med Sci 2013; 68:929–37. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Verghese J, Robbins M, Holtzer R, et al. Gait dysfunction in mild cognitive impairment syndromes. J Am Geriatr Soc 2008;56:1244–51 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Aggarwal NT, Wilson RS, Beck TL, Bienias JL, Bennett DA. Motor dysfunction in mild cognitive impairment and the risk of incident Alzheimer disease. Arch Neurol 2006;63:1763–9. [DOI] [PubMed] [Google Scholar]
  • 5.Buchman AS, Bennett DA. Loss of motor function in preclinical Alzheimer's disease. Expert Rev Neurother 2011;11:665–76. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Liu-Ambrose TY, Ashe MC, Graf P, et al. Increased risk of falling in older community-dwelling women with mildcognitive impairment. Phys Ther. 2008; 88:1482–1491. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Waite LM, Grayson DA, Piguet O, et al. Gait slowing as a predictor of incident dementia: 6-year longitudinal data from the Sydney Older Persons Study. J Neurol Sci. 2005;229–230:89–93. [DOI] [PubMed] [Google Scholar]
  • 8.Albert MS, DeKosky ST, Dickson D, Dubois B, Feldman HH, Fox NC, et al. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimer’s & Dementia. 2011. May 1;7(3):270–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Petersen RC, Caracciolo B, Brayne C, Gauthier S, Jelic V, Fratiglioni L. Mild cognitive impairment: A concept in evolution. J Intern Med. 2014;275(3):214–28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Llano D a, Laforet G, Devanarayan V. Derivation of a new ADAS-cog composite using tree-based multivariate analysis: prediction of conversion from mild cognitive impairment to Alzheimer disease. Alzheimer Dis Assoc Disord. 2011;25(1):73–84. [DOI] [PubMed] [Google Scholar]
  • 11.Raghavan N, Samtani MN, Farnum M, Yang E, Novak G, Grundman M, et al. The ADAS-Cog revisited: Novel composite scales based on ADAS-Cog to improve efficiency in MCI and early AD trials. Alzheimer’s Dement. 2013;9(1 SUPPL.):1–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Skinner J, Carvalho JO, Potter GG, Thames A, Zelinski E, Crane PK, et al. The Alzheimer’s Disease Assessment Scale-Cognitive-Plus (ADAS-Cog-Plus): An expansion of the ADAS-Cog to improve responsiveness in MCI. Brain Imaging Behav. 2012;6(4):489–501. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Kueper JK, Speechley M, Montero-Odasso M. The Alzheimer’s Disease Assessment Scale-Cognitive Subscale (ADAS-Cog): Modifications and Responsiveness in Pre-Dementia Populations. A Narrative Review. J Alzheimer’s Dis. 2018;63(2):423–44. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Beaton DE, Bombardier C, Katz JN, Wright JG. A taxonomy for responsiveness. J Clin Epidemiol. 2001;54(12):1204–17. [DOI] [PubMed] [Google Scholar]
  • 15.Krishner B, Guyatt G. A methodological framework for assessing health indices. J Chron Dis. 1985;38(1):27–36. [DOI] [PubMed] [Google Scholar]
  • 16.Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness. J Clin Epidemiol. 2000;53(5):459–68. [DOI] [PubMed] [Google Scholar]
  • 17.De Bruin AF, Diederiks JPM, De Witte LP, Stevens FCJ, Philipsen H. Assessing the responsiveness of a functional status measure: The Sickness Impact Profile versus the SIP68. J Clin Epidemiol. 1997;50(5):529–40. [DOI] [PubMed] [Google Scholar]
  • 18.Wouters H, Gool WAVW van, Schmand B, Lindeboom R. Revising the ADAS-cog for a more accurate assessment of cognitive impairment. Alzheimer Dis. 2008;22(3):236–44. [DOI] [PubMed] [Google Scholar]
  • 19.Lowe DA, Balsis S, Benge JF, Doody RS. Adding Delayed Recall to the ADAS-cog Improves Measurement Precision in Mild Alzheimer’s Disease: Implications for Predicting Instrumental Activities of Daily Living. Psychol Assess. 2015;27(4):1234–40. [DOI] [PubMed] [Google Scholar]
  • 20.Sano M, Raman R, Emond J, Thomas RG, Petersen R, Schneider JLS, et al. Adding Delayed Recall to the Alzheimer Disease Assessment Scale is Useful in Studies of Mild Cognitive Impairment But Not Alzheimer Disease. Alzheimer Dis Assoc Disord. 2011;25(2):122–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Wessels A, Siemers E, Andersen S, Holdridge K, Sims J, Sundell K, et al. A combined measure of cognition and function for clinical trials: The integrated Alzheimer’s Disease Rating Scale (iADRS). J Prev Alzheimers Dis. 2015;2(4):227–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Huang Y, Ito K, Billing CB, Anziano RJ. Development of a straightforward and sensitive scale for MCI and early AD clinical trials. Alzheimer’s Dement. 2015;11(4):404–14. [DOI] [PubMed] [Google Scholar]
  • 23.Wang J, Logovinsky V, Hendrix SB, Stanworth SH, Perdomo C, Xu L, et al. ADCOMS: a composite clinical outcome for prodromal Alzheimer’s disease trials. J Neurol Neurosurg Psychiatry. 2016;jnnp-2015-312383. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Montero-Odasso M. Gait as a biomarker of cognitive impairment and dementia syndromes. Quo vadis? Eur J Neurol. 2016;23(3):437–8. [DOI] [PubMed] [Google Scholar]
  • 25.Montero-Odasso M, Verghese J, Beauchet O, Hausdorff JM. Gait and Cognition: A Complementary Approach to Understanding Brain Function and the Risk of Falling. J Am Geriatr Soc. 2012;60(11):2127–36. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Deshpande N, Metter EJ, Bandinelli S, Guralnik J, Ferrucci L. Gait speed under varied challenges and cognitive decline in older persons: a prospective study. Age Ageing. 2009;38(5):509–14. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Camicioli R, Howieson D, Oken B, Sexton G, Kaye J. Motor slowing precedes impaiment in the oldest old. Neurology. 1998;50(5):1496–8. [DOI] [PubMed] [Google Scholar]
  • 28.Buracchio T, Dodge HH, Howieson D, Wasserman D, Kaye J. The Trajectory of Gait Speed Preceding Mild Cognitive Impairment. Arch Neurol. 2010;67(8):980–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Beauchet O, Annweiler C, Callisaya ML, De Cock AM, Helbostad JL, Kressig RW, et al. Poor Gait Performance and Prediction of Dementia: Results From a Meta-Analysis. J Am Med Dir Assoc. 2016; [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Kueper JK, Speechley M, Lingum NR, Montero-Odasso M. Motor function and incident dementia: A systematic review and meta-analysis. Age Ageing. 2017;46(5):729–38. [DOI] [PubMed] [Google Scholar]
  • 31.Montero-Odasso MM, Barnes B, Speechley M, Muir Hunter SW, Doherty TJ, Duque G, et al. Disentangling Cognitive-Frailty: Results From the Gait and Brain Study. J Gerontol A Biol Sci Med Sci. 2016;71(11):1476–1482. [DOI] [PubMed] [Google Scholar]
  • 32.Beauchet O, Allali G, Montero-Odasso M, Sejdić E, Fantino B, Annweiler C. Motor phenotype of decline in cognitive performance among community-dwellers without dementia: Population-based study and meta-analysis. PLoS One. 2014;9(6). [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Montero-Odasso M, Oteng-Amoako A, Speechley M, Gopaul K, Beauchet O, Annweiler C, et al. The motor signature of mild cognitive impairment: Results from the gait and brain study. Journals Gerontol—Ser A Biol Sci Med Sci. 2014;69(11):1415–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Allali G, Annweiler C, Blumen H, Callisaya M, De Cock A-M, Kressig R, et al. Gait phenotype from MCI to moderate dementia: Results from the GOOD initiative. Eur J Neurol. 2016;23(3):527–41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Verghese J, Annweiler C, Ayers E, Bennett DA, Stephanie A, Buchman AS, et al. Motoric cognitive risk syndrome Multicountry prevalence and dementia risk. Neurology. 2014;83(8):718–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Montero-Odasso M, Muir SW, Speechley M. Dual-task complexity affects gait in people with mild cognitive impairment: The interplay between gait variability, dual tasking, and risk of falls. Arch Phys Med Rehabil. 2012;93(2):293–9. [DOI] [PubMed] [Google Scholar]
  • 37.Montero-Odasso MM, Sarquis-Adamson Y, Speechley M, Borrie MJ, Hachinski VC, Wells J, et al. Association of Dual-Task Gait With Incident Dementia in Mild Cognitive Impairment. JAMA Neurol. 2017;54(3):S154. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Hausdorff JM, Schweiger A, Herman T, Yogev-Seligmann G, Giladi N. Dual task decrements in gait among healthy older adults: Contributing factors. J Gerontol A Biol Sci Med Sci. 2008;63(12):133501343. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Hausdorff JM, Yogev G, Springer S, Simon ES, Giladi N. Walking is more like catching than tapping: Gait in the elderly as a complex cognitive task. Exp Brain Res. 2005;164(4):541–8. [DOI] [PubMed] [Google Scholar]
  • 40.Montero-Odasso M, Casas A, Hansen KT, Bilski P, Gutmanis I, Wells JL, et al. Quantitative gait analysis under dual-task in older people with mild cognitive impairment: a reliability study. J Neuroeng Rehabil. 2009;6(1):35. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–198. doi: 0022-3956(75)90026-6 [DOI] [PubMed] [Google Scholar]
  • 42.Nasreddine ZS, Phillips NA, Bedirian V, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for Mild Cognitive Impairment. JAGS. 2005;53:695–699. 10.1111/j.1532-5415.2005.53221.x [DOI] [PubMed] [Google Scholar]
  • 43.Strauss E, Sherman EMS Spreen O. A compendium of neuropsychological tests: Administration, norms, and commentary. 3rd ed. Oxford: Oxford University Press; 2006. [Google Scholar]
  • 44.Annweiler C., Beauchet O., Bartha R. et al. Slow gait in MCI is associated with ventricular enlargement: results from the Gait and Brain Study. J Neural Transm 120, 1083–1092 (2013). https://doi-org.proxy1.lib.uwo.ca/10.1007/s00702-012-0926-4 [DOI] [PubMed] [Google Scholar]
  • 45.ADNI Procedures Manual [Internet]. 2005. Available from: http://adni.loni.usc.edu/wp-content/uploads/2010/09/ADNI_GeneralProceduresManual.pdf
  • 46.Wood SN. Generalized Additive Models: An Introduction with R. Boca Raton: Taylor & Francis Group; 2006. Available from: http://doi.wiley.com/10.1111/j.1541-0420.2007.00905_3.x [Google Scholar]
  • 47.Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. 2010;33(1):1–22. [PMC free article] [PubMed] [Google Scholar]
  • 48.Schrag A, Schott JM. What is the clinically relevant change on the ADAS-Cog? J Neurol Neurosurg Psychiatry. 2012;83(2):171–3. [DOI] [PubMed] [Google Scholar]
  • 49.van Buuren S, Groothuis-Oudshoorn K. Mice: Multivariate imputation by chained equations. J Stat Softw. 2011;45(3):1–67. [Google Scholar]
  • 50.White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med. 2011;30(4):377–99. [DOI] [PubMed] [Google Scholar]
  • 51.GAITRite | World Leader in Temporospatial Gait Analysis [Internet]. [cited 2020 Apr 16]. Available from: https://www.gaitrite.com/
  • 52.Abdi H. Coefficient of variation. Encylopedia Res Des. 2010;2. [Google Scholar]
  • 53.Burke WJ, Houston MJ, Boust SJ, Roccaforte WH: Use of the Geriatric Depression Scale in dementia of the Alzheimer type. J Am Geriatr Soc 1989;37:856–860. [DOI] [PubMed] [Google Scholar]
  • 54.Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969. Autumn;9(3):179–86. [PubMed] [Google Scholar]
  • 55.Hughes CP, Berg L, Danziger WL, Coben LA, Martin RL. A new clinical scale for the staging of dementia. Br J Psychiatry. 1982;140:566–572 [DOI] [PubMed] [Google Scholar]
  • 56.Bean J. (2011) Rey Auditory Verbal Learning Test, Rey AVLT In: Kreutzer J.S., DeLuca J., Caplan B. (eds) Encyclopedia of Clinical Neuropsychology. Springer, New York, NY [Google Scholar]
  • 57.Bowie C., Harvey P. Administration and interpretation of the Trail Making Test. Nat Protoc 1, 2277–2281 (2006). 10.1038/nprot.2006.390 [DOI] [PubMed] [Google Scholar]
  • 58.Smythe H, Helewa A, Goldsmith C. “Independent assessor” and “pooled index” as techniques for measuring treatment effects in rheumatoid arthritis. J Rheumatol. 1977;4(2):144–52. [PubMed] [Google Scholar]
  • 59.Goldsmith C, Smythe H, Helewa A. Interpretation and power of a pooled index. J Rheumatol. 1993;20(3):575–8. [PubMed] [Google Scholar]
  • 60.Speechley M, Forchuk C, Hoch J, Jensen E, Wagg J. Deriving a mental health outcome measure using the pooled index: An application to psychiatric consumer—Survivors in different housing types. Heal Serv Outcomes Res Methodol. 2009;9(2):133–43. [Google Scholar]
  • 61.Team RS. R Studio [Internet]. Boston; 2016. Available from: http://www.rstudio.com/
  • 62.Hawkins KA, Dean D, Pearlson GD. Alternative Forms of the Rey Auditory Verbal Learning Test: A Review. Behav Neurol. 2004;15(3–4):99–107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 63.Duff K, Beglinger LJ, Moser DJ, Schultz SK, Paulsen JS. Practice Effects and Outcome of Cognitive Training: Preliminary Evidence from a Memory Training Course. Am J Geriatr Psychiatry. 2010. Jan;18(1):91. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 64.Montero-Odasso M, Bergman H, Phillips NA, Wong CH, Sourial N, Chertkow H. Dual-tasking and gait in people with Mild Cognitive Impairment. The effect of working memory. BMC Geriatr. 2009;9(1):41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 65.Montero-Odasso M, Hachinski V. Preludes to brain failure: executive dysfunction and gait disturbances. Neurol Sci. 2014;35(4):601–4. [DOI] [PubMed] [Google Scholar]
  • 66.Verghese J, Wang C, Lipton RB, Holtzer R. Motoric cognitive risk syndrome and the risk of dementia. J Gerontol A Biol Sci Med Sci. 2013;68(4):412–418. 10.1093/gerona/gls191 [DOI] [PMC free article] [PubMed] [Google Scholar]

Decision Letter 0

Antony Bayer

15 Nov 2019

PONE-D-19-25160

Cognition and Motor Function: The Gait and Cognition Pooled Index

PLOS ONE

Dear Dr. Montero-Odasso,

Thank you for submitting your manuscript to PLOS ONE. As you will see below, the four reviewers all had significant concerns and therefore, after careful consideration, we feel that although it has merit it does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to consider submitting a revised version of the manuscript should you feel able to address the points raised during the review process.

We would appreciate receiving your revised manuscript by Dec 30 2019 11:59PM. When you are ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter.

To enhance the reproducibility of your results, we recommend that if applicable you deposit your laboratory protocols in protocols.io, where a protocol can be assigned its own identifier (DOI) such that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). This letter should be uploaded as separate file and labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. This file should be uploaded as separate file and labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. This file should be uploaded as separate file and labeled 'Manuscript'.

Please note while forming your response, if your article is accepted, you may have the opportunity to make the peer review history publicly available. The record will include editor decision letters (with reviews) and your responses to reviewer comments. If eligible, we will contact you to opt in or out.

We look forward to receiving your revised manuscript.

Kind regards,

Antony Bayer

Academic Editor

PLOS ONE

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.  Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

3. We understand that you use data from ADNI in your study. Please update your Data Availability Statement to indicate how these data can be obtained.

4. Thank you for stating the following in the Financial Disclosure section:

This work was supported by the Early Research Award (PI MMO) and the Alzheimer Foundation of London and Middlesex Master’s Scholarship in Alzheimer-Related Research to JKK.  

The Gait and Brain Study is funded by an operating grant from the Canadian Institutes of Health and Research (CIHR, MOP 211220) and a CIHR Project Grant (PJT 153100).

Dr. Montero-Odasso’s program in “Gait and Brain Health” is supported by grants from the CIHR, the Ontario Ministry of Research and Innovation, the Ontario Neurodegenerative Diseases Research Initiative, the Canadian Consortium on Neurodegeneration in Aging, and by the Department of Medicine Program of Experimental Medicine Research Award, the University of Western Ontario.

Portions of this study used data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Thus, data collection and sharing for this project was partially funded by ADNI (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute of Biomedical Imaging and Bioengineering, and through contributions from: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc,; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; Eurolmmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merk & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

We note that you received funding from a commercial source: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc,; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; Eurolmmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merk & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics.

Please provide an amended Competing Interests Statement that explicitly states this commercial funder, along with any other relevant declarations relating to employment, consultancy, patents, products in development, marketed products, etc.

Within this Competing Interests Statement, please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your amended Competing Interests Statement within your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

5. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

6.  We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

7. One of the noted authors is a group or consortium: The Alzheimer’s Disease Neuroimaging Initiative

In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Partly

Reviewer #3: Yes

Reviewer #4: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

Reviewer #4: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The study tried to increase the responsiveness of clinical assessments of dementia through combining cognitive assessment and gait assessments. The idea is very interesting, however, the hypothesis and methodology have drawbacks.

Major points

1) The responsiveness of the measurement is off course important, however, it should also reflect some pathological changes, not physiological aging process. Therefore, in this case the measurement should be sensitive to conversion to AD or worse outcomes, not general changes over time. However, the methodology of the current study did not consider pathological changes at all.

2) The authors used new cognitive index of ADAS-Cog-Proxy instead of well-established ADAS-Cog, which has no established evidence of accuracy nor responsiveness.

Minor point

1) Table 1 and 2 seems to be exchanged.

Reviewer #2: Thank you for the opportunity to review the manuscript ‘Cognition and Motor Function: The Gait and Cognition Pooled Index by Keuper et al.

This study set out to investigate if the addition of motor function assessment to a cognitive test, the ADAS-Cog, could increase detection of cognitive deterioration in older adults, with a proof of principle approach. Since a database with complete set of variables was not found, the study used one database (the ADNI) to develop a predictive model for estimating an ADAS-cog proxy, which was then applied to the second database (the GABS) after imputing missing with multiple imputation. The outcome measure was a Pooled Index, developed from combining the ADAS-cog proxy with a motor function test (gait velocity) and a motor-cognitive test (dual-task). The Pooled Index was compared with the ADAS-cog proxy for responsiveness, that is, if able to detect a difference between diagnostic categories (NC, SCI and MCI) in the GABS sample (n=109). Calculation of effects size used to assess responsiveness to longitudinal change over 6 to 48 months. The study concluded that the Pooled index had comparable or increased responsiveness to changes compared to the ADAS-cog proxy. The concept of this study was interesting, and potential implications for the evaluation of effects from interventions preventing or delaying incident dementia important. The method was innovative, however, in my opinion, major revision is needed to improve description of the method and statistical analyses, and appropriate conclusions supported by the findings, which are tenuous at best.

Please find below major and minor comments.

Major:

Introduction

1. The rationale for using ADAD-cog could be better explained in the introduction, to support the development of a proxy of the test. Why not use MMSE for example, since it was available in GABS?

2. In the introduction it is suggested that the recognition of the relation between motor function and cognition function has impacted the suitability of ADAS-cog. Is there evidence of it being unsuitable? Is it not only a method of increasing responsiveness of the test pre-dementia?

3. Line 86: How is the backwards-compatibility of the pooled index and the ADAS-cog analysed?

Method:

4. The development of the ADAS-cog proxy takes much space in the method considering the aim. Could perhaps parts of it be lifted into supplement to aid readability?

5. Line 113: References for MMSE, MOCA, CDR, Lawton, DSM-TRI-IV and V missing.

6. Line 119: What tests and cut-offs for ‘measured cognitive impairments in memory, executive, function, attention, and/or language were used?

7. Line 120: DSM-V criteria for diagnoses differs from DSM-TR-IV. Did it differ between studies? How was this handled?

8. Line 125: What were inclusion/exclusion criteria, and informed consent and ethics for ADNI? Were the two datasets comparable with regards to age, physical and cognitive status?

9. Line 153: In Fig 1 (stage 2) the preliminary accuracy of M4 was 65.6%, while M5 was 68.4%? Why was M4 chosen as the best candidate model and not M5? Similarities of covariates perhaps?

10. Line 155: The progression from M4 (MMSE, RAVLT, CDR-SB) to the ADAS-cog proxy GAM in ADNI is not clear. A consistent terminology, for example, the ADAS-cog proxy GAM, may help. Did the final GAM differ from M4?

11. Line 164: ‘missing GAM covariate values’ confusing, which actual variables does this term refer to?

12. Line 165: What covariates were used in the multiple imputation model to predict RAVLT, CDR-SB, CDR-SB&RAVLT?

13. Line 163: Considering that missing on the covariates RAVLT, CDR-SB, CDR-SB&RAVLT was generally over 50% at most follow-ups for CDR-SB, and as much as 80% at 48 months, why use only 5 imputed datasets? The large proportion of missing values may reduce the validity of the imputation, which could also be discussed, and mentioned as a limitation. Furthermore, are data available with regards to reasons participants had missing values, particularly the CDR-SB?

14. Line 225: Were individual pooled index components added for the purpose of sensitivity analysis?

Results:

15. Line 236: Table 1 (and Table 2). Values with 2 decimals appear to be used as standard, but does not make much sense for variables measured in years for example.

16. Line 244: Further description of baseline measurements, with references, in the methods section would aid understanding of variables included in Table 2 (and Table 1). For example, with which test was Instrumental ADLs, and Basic ADLs assessed?

17. Line 270: Is the differences in responsiveness between the ADAS-cog proxy and the Pooled index tested statistically? Larger responsiveness suggests better detection of change, but differences seem very small. May the differences be due to chance? Since this is one of the main aims of this study, and on which main inferences are based, it would strengthen inferences if the difference was tested statistically. What size of change could be considered important?

18. Line 283: Is not having reached longer follow-up time points yet (due to staggered inclusion assumingly) the only reason for loss of data in later follow-ups? How do you explain participants with the longest follow-ups having faster gait speed compared with participants that have not yet reached those follow-up lengths?

19. Line 285: Was this comparison at baseline?

Discussion:

20. It is mentioned in the introduction that other studies have combined cognitive and motor components. How does method and results compare?

21. Line 291: This is a very generous (and also perhaps a little misleading) interpretation of results from the study. May be improved by being more specific for results for baseline discrimination (comparable results) and for responsiveness to changes (suggested results).

22. Line 297: This interpretation seems different from that in results section?

23. Line 299: The ADASp being the golden standard, is it not a big problem that the Pooled index detected worsening when the ADASp is suggesting improvement?

24. Line 300: While gait deterioration may have a different trajectory to cognitive decline, why does the ADASp+GV (i.e. only gait velocity added) follow a similar pattern to the ADASp?

25. Line 325: Specify how including motor function was a valuable contribution?

26. Since the Pooled index and the ADAS-cog proxy showed comparable baseline discrimination, and differences between tests on responsiveness to longitudinal changes did not appear to be statistically analysed, the inferences from this study seem tenuous?

Minor:

Title page:

27. Line 3: What is the order of authors? The order on the title page differs compared to page 1.

Abstract:

28. This being a proof of principle study could be mentioned.

Introduction:

29. Line 71: Can the order of references be arranged in sequence?

30. Line 79: Does the reference #33 compare dual-task in relation to ADL activities?

Methods:

31. Line 129: Does ADNI and ADNI 1 refer to the same study? (see also Line 133, 134, 138, 237, 241).

32. Line 143: How was similarity of covariates assessed?

Discussion:

33. Line 321: Is this short paragraph intended to be part of the previous?

Reviewer #3: Using data from ADNI, this study developed a pooled index combining gait assessment and ADAS-Cog, and further compared the responsiveness of pooled index to ADAS-Cog-Proxy both cross-sectionally and longitudinally in Gait and Brain Study (GABS). Authors showed comparable responsiveness to baseline discrimination. The pooled index, adding motor function assessment to ADAS-Cog, improved the responsiveness at 6- and 48-month follow-up but not at 36-month follow-up.

The rationale and analytical approaches are sound. I have several comments.

Major:

1. Regarding motor measures examined, a stronger rationale is needed. This study included one single-task and three dual-task conditions and included mean performance and variability of spatiotemporal gait parameters. It may be helpful to have a summary lead sentence or a vocabulary of various gait measures.

2. Regarding longitudinal analyses, a potential censoring bias exists because not all were followed up for 48 months. Although authors briefly mentioned this limitation, a sensitivity analysis should be considered to confirm findings.

3. Authors observed that ADAS-Cog-Proxy detected improvement and discussed a potential non-linear change. I wonder whether this reflects a learning effect from the memory test since the proxy included RAVLT. Perhaps authors can test this using GABS data.

Minor:

Line 112-113: please specify “normal” scores on MMSE and MoCA.

Line 115-116: please specify “persistent decline in cognition”. Between two consecutive assessments? Which cognitive function measure?

Line 149: were samples randomly divided into the testing and development subsets?

Line 153: it is not clear how the best candidate model was determined. Step 3 indicates the best candidate model is M4 and its preliminary accuracy is not the highest.

Line 164: what’s the criteria for imputation? It would be helpful to indicate missing here in the text, <5% for instance.

Line 165: please clarify “five datasets”

Line 188: It would be helpful to demonstrate pooled index development as a figure. I think that Figure 1 is a good illustration to show how ADAS-Cog proxy was developed in GABS.

Line 242: Please clarify Table 2 shows sample characteristics in GABS. Perhaps add “GABS” prior to “baseline”.

Reviewer #4: This is a technical article describing a pooled index approach to improve responsiveness in the ADAS-Cog in pre-dementia populations. This article could be improved by making it more interesting and clinically relevant to clinicians as well as researchers. Portions of the abstract, Introduction and discussion should be revised to describe the problem. Why is it important to create a tool that is more responsive in individuals with pre-dementia syndrome? Why is it important to distinguish between SCI and MCI.

In the abstract and introduction the authors take a historical approach ("The shift in focus in dementia research......"). However, this "shift" is not new and has been evolving in the research for over 15 years.

The first few lines for the abstract should be revised to reflect our current state of the science and the need for a more responsive measures. Revising the approach and tone of the introduction section should also be considered.

The introduction references studies related to 'Shifted attention" in research related to pre-dementia. Instead, I recommend defining key terms/conditions related to pre-dementia and summarize seminal studies that have demonstrated relationships between cognitive and motor function.

The discussion would also benefit from a discussion of how this work relates to prior studies, such a Verghese motot cognitive syndrome.

Also, what needs to be done to make this a clinically useful tool? What is the clinical relevance of this study?

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

Reviewer #3: No

Reviewer #4: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files to be viewed.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email us at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Sep 11;15(9):e0238690. doi: 10.1371/journal.pone.0238690.r002

Author response to Decision Letter 0


17 Jun 2020

Journal Requirements:

1. When submitting your revision, we need you to address these additional requirements.

Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

http://www.journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and http://www.journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

Response: We have updated the formatting to meet these PLOS ONE style requirements.

2. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

Response: We have updated Supporting Information files and their referencing.

3. We understand that you use data from ADNI in your study. Please update your Data Availability Statement to indicate how these data can be obtained.

Response: We have revised the Data Availability section of the submission portal to have statements for both the Gait and Brain Study data and for ADNI. Since only one drop down menu option is allowed, we maintained ‘No-some restrictions apply’ as this is true for the Gait and Brain Study data (see below), however we added to the description box a link to the data access application page on ADNI’s website. (http://adni.loni.usc.edu/data-samples/access-data/)

4. Thank you for stating the following in the Financial Disclosure section:

This work was supported by the Early Research Award (PI MMO) and the Alzheimer Foundation of London and Middlesex Master’s Scholarship in Alzheimer-Related Research to JKK.

The Gait and Brain Study is funded by an operating grant from the Canadian Institutes of Health and Research (CIHR, MOP 211220) and a CIHR Project Grant (PJT 153100).

Dr. Montero-Odasso’s program in “Gait and Brain Health” is supported by grants from the CIHR, the Ontario Ministry of Research and Innovation, the Ontario Neurodegenerative Diseases Research Initiative, the Canadian Consortium on Neurodegeneration in Aging, and by the Department of Medicine Program of Experimental Medicine Research Award, the University of Western Ontario.

Portions of this study used data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). Thus, data collection and sharing for this project was partially funded by ADNI (National Institutes of Health Grant U01 AG024904) and DOD ADNI (Department of Defense award number W81XWH-12-2-0012). ADNI is funded by the National Institute of Biomedical Imaging and Bioengineering, and through contributions from: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc,; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; Eurolmmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merk & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics. The Canadian Institutes of Health Research is providing funds to support ADNI clinical sites in Canada. Private sector contributions are facilitated by the Foundation for the National Institutes of Health (www.fnih.org). The grantee organization is the Northern California Institute for Research and Education, and the study is coordinated by the Alzheimer’s Therapeutic Research Institute at the University of Southern California. ADNI data are disseminated by the Laboratory for Neuro Imaging at the University of Southern California.

We note that you received funding from a commercial source: AbbVie, Alzheimer’s Association; Alzheimer’s Drug Discovery Foundation; Araclon Biotech; BioClinica, Inc.; Biogen; Bristol-Myers Squibb Company; CereSpir, Inc.; Cogstate; Eisai Inc,; Elan Pharmaceuticals, Inc.; Eli Lilly and Company; Eurolmmun; F. Hoffmann-La Roche Ltd and its affiliated company Genentech, Inc.; Fujirebio; GE Healthcare; IXICO Ltd.; Janssen Alzheimer Immunotherapy Research & Development, LLC.; Johnson & Johnson Pharmaceutical Research & Development LLC.; Lumosity; Lundbeck; Merk & Co., Inc.; Meso Scale Diagnostics, LLC.; NeuroRx Research; Neurotrack Technologies; Novartis Pharmaceuticals Corporation; Pfizer Inc.; Piramal Imaging; Servier; Takeda Pharmaceutical Company; and Transition Therapeutics.

Please provide an amended Competing Interests Statement that explicitly states this commercial funder, along with any other relevant declarations relating to employment, consultancy, patents, products in development, marketed products, etc.

Response: These commercial sources of funding were to ADNI investigators and none of the authors on this manuscript are ADNI investigators. The authors on this manuscript do not have any direct connection to this funding.

Within this Competing Interests Statement, please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests). If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your amended Competing Interests Statement within your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

5. PLOS requires an ORCID iD for the corresponding author in Editorial Manager on papers submitted after December 6th, 2016. Please ensure that you have an ORCID iD and that it is validated in Editorial Manager. To do this, go to ‘Update my Information’ (in the upper left-hand corner of the main menu), and click on the Fetch/Validate link next to the ORCID field. This will take you to the ORCID site and allow you to create a new iD or authenticate a pre-existing iD in Editorial Manager. Please see the following video for instructions on linking an ORCID iD to your Editorial Manager account: https://www.youtube.com/watch?v=_xcclfuvtxQ

Response: The ORCID ID has been updated.

6. We note that you have indicated that data from this study are available upon request. PLOS only allows data to be available upon request if there are legal or ethical restrictions on sharing data publicly. For information on unacceptable data access restrictions, please see http://journals.plos.org/plosone/s/data-availability#loc-unacceptable-data-access-restrictions.

In your revised cover letter, please address the following prompts:

a) If there are ethical or legal restrictions on sharing a de-identified data set, please explain them in detail (e.g., data contain potentially identifying or sensitive patient information) and who has imposed them (e.g., an ethics committee). Please also provide contact information for a data access committee, ethics committee, or other institutional body to which data requests may be sent.

b) If there are no restrictions, please upload the minimal anonymized data set necessary to replicate your study findings as either Supporting Information files or to a stable, public repository and provide us with the relevant URLs, DOIs, or accession numbers. Please see http://www.bmj.com/content/340/bmj.c181.long for guidelines on how to de-identify and prepare clinical data for publication. For a list of acceptable repositories, please see http://journals.plos.org/plosone/s/data-availability#loc-recommended-repositories.

We will update your Data Availability statement on your behalf to reflect the information you provide.

Response: We have added this information to the cover letter, submission portal, and are repeating it here for ease; the rationale for not publicly data availability is mainly 3 fold:

1-The Gait and Brain study is in ongoing study and there are several research questions than need to be responded by PI and co-PIs before data can be released and available for researchers.

2-The Gait and Brain study started in 2007 and the original protocol and funding did not considered resources for releasing data.

3-Since 2014 it has been considered by the Pi and co-Pis the data can be released by per researchers’ request when study is finished, expected by 2025. This is upon resources can be secured to make them available to other researchers.

7. One of the noted authors is a group or consortium: The Alzheimer’s Disease Neuroimaging Initiative

In addition to naming the author group, please list the individual authors and affiliations within this group in the acknowledgments section of your manuscript. Please also indicate clearly a lead author for this group along with a contact email address.

Response: ADNI maintains a list of investigators and their affiliations as well as a webpage with current e-mail addresses to contact ADNI dependent on the type of inquiry. Direct links to these locations have been added to an acknowledgements section; copying the information into our manuscript would occupy over five pages of space.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: No

Reviewer #2: Partly

Reviewer #3: Yes

Reviewer #4: Yes

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

Reviewer #3: Yes

Reviewer #4: No

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

Reviewer #3: Yes

Reviewer #4: Yes

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: The study tried to increase the responsiveness of clinical assessments of dementia through combining cognitive assessment and gait assessments. The idea is very interesting, however, the hypothesis and methodology have drawbacks.

Major points

1) The responsiveness of the measurement is off course important, however, it should also reflect some pathological changes, not physiological aging process. Therefore, in this case the measurement should be sensitive to conversion to AD or worse outcomes, not general changes over time. However, the methodology of the current study did not consider pathological changes at all.

Response: While our research questions hypotheses were centred around change over time at a pre-dementia stages, we understand the hesitation to draw strong conclusions without a ‘harder’ outcome, especially given the absence of original ADAS-Cog scores. To get an additional outcome that is well-understood we performed secondary analyses with original MMSE scores. These results were added to the supplementary material.

2) The authors used new cognitive index of ADAS-Cog-Proxy instead of well-established ADAS-Cog, which has no established evidence of accuracy nor responsiveness.

Response: the ADAS-Cog-Proxy scores are intended to estimate ADAS-Cog scores for the purposes of this proof of principle study; psychometric properties of the ADAS-Cog-Proxy depend on the performance of our predictive model. We do not advocate substituting the ADAS-Cog-Proxy for the ADAS-Cog in future study protocols. Rather, using a predictive model was a way to gain a preliminary look at our hypothesis before investing resources in a new study and may assist with future study measure selection. Nonetheless, we understand the concern about not having an original well-established cognitive measure to compare results to and have added a secondary analysis that uses MMSE scores for all time point. Since this is a secondary analysis we added it to the Supplementary Material, but reference it in the methods and provide a statement of the overall findings in the result section of the main manuscript.

Minor point

1) Table 1 and 2 seems to be exchanged.

Response: Thank you for catching this error. Table locations have been fixed.

Reviewer #2: Thank you for the opportunity to review the manuscript ‘Cognition and Motor Function: The Gait and Cognition Pooled Index by Keuper et al.

This study set out to investigate if the addition of motor function assessment to a cognitive test, the ADAS-Cog, could increase detection of cognitive deterioration in older adults, with a proof of principle approach. Since a database with complete set of variables was not found, the study used one database (the ADNI) to develop a predictive model for estimating an ADAS-cog proxy, which was then applied to the second database (the GABS) after imputing missing with multiple imputation. The outcome measure was a Pooled Index, developed from combining the ADAS-cog proxy with a motor function test (gait velocity) and a motor-cognitive test (dual-task). The Pooled Index was compared with the ADAS-cog proxy for responsiveness, that is, if able to detect a difference between diagnostic categories (NC, SCI and MCI) in the GABS sample (n=109). Calculation of effects size used to assess responsiveness to longitudinal change over 6 to 48 months. The study concluded that the Pooled index had comparable or increased responsiveness to changes compared to the ADAS-cog proxy. The concept of this study was interesting, and potential implications for the evaluation of effects from interventions preventing or delaying incident dementia important. The method was innovative, however, in my opinion, major revision is needed to improve description of the method and statistical analyses, and appropriate conclusions supported by the findings, which are tenuous at best.

Please find below major and minor comments.

Major:

Introduction

1. The rationale for using ADAD-cog could be better explained in the introduction, to support the development of a proxy of the test. Why not use MMSE for example, since it was available in GABS?

Response: We have revised the introduction with increased explanations of why the ADAS-Cog is used. Examples include stating early on that it is still in use today and that using it maintains compatibility with a large body of previously conducted research. We also added a sentence about how our methodological approach can be an example for other researchers who want to explore similar questions in other contexts.

2. In the introduction it is suggested that the recognition of the relation between motor function and cognition function has impacted the suitability of ADAS-cog. Is there evidence of it being unsuitable? Is it not only a method of increasing responsiveness of the test pre-dementia?

Response: This comment raises concerns about the framing around the ADAS-Cog; we have revised the introduction to be more precise. For example, the ADAS-Cog is still suitable for its original purpose in dementia populations; it is not suitable for where the focus of present research is, but given its strong history may be able to maintain an important place.

3. Line 86: How is the backwards-compatibility of the pooled index and the ADAS-cog analysed?

Response: Backwards compatibility is the ability to obtain the original measure from a new version. In our case, this can be achieved by using only the data collected about the ADAS-Cog without the data collected for other components of the pooled index. We specified how this is possible by extending the first sentence of the Analyses section to include, “while maintaining the ability to use each of the source variables individually”.

In revising the introduction, we further expanded upon backwards compatibility and why it is important with the ADAS-Cog.

Method:

4. The development of the ADAS-cog proxy takes much space in the method considering the aim. Could perhaps parts of it be lifted into supplement to aid readability?

Response: we have moved the figure describing the ADAS-Cog-Proxy GAM to Supplementary material, including the longer description that was below it.

5. Line 113: References for MMSE, MOCA, CDR, Lawton, DSM-TRI-IV and V missing.

Response: References have been added for MMSE and MoCA, which are mentioned on Line 113, as well as a reference for the source that was used to define NC/SCI and that includes the standardized norms used as cut offs. In clarifying the MCI definition, the other tests are no longer mentioned individually and instead there is increased description references to the source of the definitions are provided.

6. Line 119: What tests and cut-offs for ‘measured cognitive impairments in memory, executive, function, attention, and/or language were used?

Response: As per the study protocol and explanation in the following papers (1,2) MCI was defined following Petersen criteria and scores 1.5 SD below expected performance based on norms for age, sex, and education published in (3)

1. Montero-Odasso M, Muir SW, Speechley M. Dual-task complexity affects gait in people with mild cognitive impairment: The interplay between gait variability, dual tasking, and risk of falls. Arch Phys Med Rehabil. 2012;93(2):293–9.

2. Annweiler, C., Beauchet, O., Bartha, R. et al. Slow gait in MCI is associated with ventricular enlargement: results from the Gait and Brain Study. J Neural Transm 120, 1083–1092 (2013). https://doi-org.proxy1.lib.uwo.ca/10.1007/s00702-012-0926-4

3. Strauss, E., Sherman, E. M. S., Spreen, O., & Spreen, O. (2006). A compendium of neuropsychological tests: Administration, norms, and commentary. Third Edition. Oxford: Oxford University Press.

7. Line 120: DSM-V criteria for diagnoses differs from DSM-TR-IV. Did it differ between studies? How was this handled?

Response: Thank you for identifying this. The Gait and Brain Study began using the DSM-V in 2013; the data for the present study comes from before then and so all diagnoses are based on DSM-IV. We have made this amendment in the manuscript.

8. Line 125: What were inclusion/exclusion criteria, and informed consent and ethics for ADNI? Were the two datasets comparable with regards to age, physical and cognitive status?

Response: We added a summary of eligibility criteria for ADNI and a direct reference to the study protocol manual in the ADNI methods subsection. Given that both ADNI and GABS contain older adults along the pre- dementia disease continuum from NC to MCI, their cognitive abilities are expected to be similar. To ensure this was the case the range of ADAS-Cog-Proxy GAM covariates were compared between the ADNI data used to build the GAM and observed GABS data. The table used to compare this has been added to Supplementary Material.

9. Line 153: In Fig 1 (stage 2) the preliminary accuracy of M4 was 65.6%, while M5 was 68.4%? Why was M4 chosen as the best candidate model and not M5? Similarities of covariates perhaps?

Response: Yes, M4 was chosen because it was a simpler model without much worse performance. A key aspect of this similarity is that it includes covariates that measure similar domains to the ADAS-Cog whereas M5 includes assessments that measure additional domains; the generalizability of M4 is expected to be greater than M5. Clarification of these points were added to the Supplementary Material.

10. Line 155: The progression from M4 (MMSE, RAVLT, CDR-SB) to the ADAS-cog proxy GAM in ADNI is not clear. A consistent terminology, for example, the ADAS-cog proxy GAM, may help. Did the final GAM differ from M4?

Response: Terminology has been updated so that ADAS-Cog-Proxy GAM (scores) is used throughout the manuscript to indicate predictions of ADAS-Cog scores in GABs.

11. Line 164: ‘missing GAM covariate values’ confusing, which actual variables does this term refer to?

Response: This refers to all of the covariates for M4. A reference to Table 1 of the Supplementary material, which lists these variables and the amount of missingness, has been added to Line 164.

12. Line 165: What covariates were used in the multiple imputation model to predict RAVLT, CDR-SB, CDR-SB&RAVLT?

MICE was not performed on the entire GABS database due to multicollinearity and computational restrictions and because Research suggests that there is little improvement in accuracy when imputation considers more than 15-25 predictors. In accordance with published guidelines, predictor matrices included all GAM covariates, predictors of the outcome ADAS-Cog scores, variables that include a lot of variance as roughly identified by correlation with the target variables to be imputed, and no variables that had a lot of missing values within the subgroup of people with missing RAVLT and CDR-SB scores. It has also been suggested to include variables related to non-response. The main reason the CDR-SB scores are missing is if no collaborator was present to report on behalf of the patient; however, there was not a variable in the dataset expected to provide indication of this. The final list of included covariates for MICE was: Baseline Diagnosis, MMSE, MoCA, MoCAMIS, MoCAEIS, MoCAVIS, MoCALIS, MoCAAIS, MoCAOIS, CDR, Trail A, Trail B, Digit Forward, Digit Backward, Letter Number, RAVLT, BNT, FAB, number of falls in past 6 months, IADL, RSEO (balance), Gait Velocity, and Gait Velocity while counting backwards by ones, from the time point of interest, as well as CDR and RAVLT scores from the previous visit (T6 to T48 visit imputations) or a future visit (baseline visit imputations). The MICE procedure was performed separately for each time point to allow for the exclusion of observations that were missing simply because the corresponding participants did not have the follow-up visit. We added these details to Supplementary material.

1. van Buuren S, Groothuis-Oudshoorn K. mice: Multivariate imputation by chained equations in R. J Stat Softw. 2011;45(3). doi:10.18637/jss.v045.i03.

2. van Buuren S, Oudshoorn K. Flexible Multivariate Imputation by MICE. Netherlands Organization for Applied Science Research;1999.

3. White IR, Royston P, Wood AM. Multiple imputation using chained equations: issues and guidance for practice. Stat Med. 2011;30(4):377-399. doi:10.1002/sim.4067.

13. Line 163: Considering that missing on the covariates RAVLT, CDR-SB, CDR-SB&RAVLT was generally over 50% at most follow-ups for CDR-SB, and as much as 80% at 48 months, why use only 5 imputed datasets? The large proportion of missing values may reduce the validity of the imputation, which could also be discussed, and mentioned as a limitation. Furthermore, are data available with regards to reasons participants had missing values, particularly the CDR-SB?

Response: Numbers of missing values for the CDR-SB due to absence of a collaborator at appointments to fill out the assessment are available (the majority of cases) and have been added to Table 1 in Supplementary material. Reasons for missing RAVLT data are not available. High levels of missingness has been added as a limitation in the Discussion section.

14. Line 225: Were individual pooled index components added for the purpose of sensitivity analysis?

Response: analyses with sub-components of the pooled index were added to gain better insight into what types of change component measurement domains could capture and whether the trend across domains was similar.

Results:

15. Line 236: Table 1 (and Table 2). Values with 2 decimals appear to be used as standard, but does not make much sense for variables measured in years for example.

Response: decimals were included for variables measured in years as changes in cognition and motor function may occur on a monthly scale. We decided to keep it in in order to maintain a higher level of detail for comparison between subgroups.

16. Line 244: Further description of baseline measurements, with references, in the methods section would aid understanding of variables included in Table 2 (and Table 1). For example, with which test was Instrumental ADLs, and Basic ADLs assessed?

Response: Thank you for this suggestion to improve details. We added a “Baseline Descriptive Statistics” subsection to the Measures part of the methods that provides an overview of the baseline measurements with references.

17. Line 270: Is the differences in responsiveness between the ADAS-cog proxy and the Pooled index tested statistically? Larger responsiveness suggests better detection of change, but differences seem very small. May the differences be due to chance? Since this is one of the main aims of this study, and on which main inferences are based, it would strengthen inferences if the difference was tested statistically. What size of change could be considered important?

Response: We agree with you that additional information would be beneficial to better interpretation of results and computed bootstrap confidence intervals for the ADAS-Cog-Proxy GAM scores and for the Pooled Index at each time point, as well as for the secondary analysis of the MMSE outcome. Corresponding methods and findings have been incorporated into the manuscript, including tempering the final conclusions.

18. Line 283: Is not having reached longer follow-up time points yet (due to staggered inclusion assumingly) the only reason for loss of data in later follow-ups? How do you explain participants with the longest follow-ups having faster gait speed compared with participants that have not yet reached those follow-up lengths?

Response: Reasons for not reaching longer follow-up time points includes not being in the study long enough to reach all time-points, conversion to dementia, and drop out due to health conditions or death. These additional reasons have been added around Line 283.

it is known that gait velocity is a good global marker of overall health so a possible explanation for participants with the longest follow-ups having faster gait speed compared with participants who have not yet reached those follow-up lengths is that those with higher gait speed at baseline have better health and are less likely to develop dementia (at which point they are taken out of the GABS) or drop out due to health conditions or death; they are more likely to remain healthy enough to stay in the study until longer follow-up points.

19. Line 285: Was this comparison at baseline?

Response: yes; we have added ‘baseline’ as a descriptor to the sentence to clarify this.

Discussion:

20. It is mentioned in the introduction that other studies have combined cognitive and motor components. How does method and results compare?

Response: Thank you for this suggestion to use our discussion section to compare our results with the broader literature base. We have added a paragraph to the discussion that describes in more detail some of the findings from key studies cited in the introduction that motivated the present study.

21. Line 291: This is a very generous (and also perhaps a little misleading) interpretation of results from the study. May be improved by being more specific for results for baseline discrimination (comparable results) and for responsiveness to changes (suggested results).

Response: We have updated this sentence to mention this is a proof of principle study with non-conclusive results.

22. Line 297: This interpretation seems different from that in results section?

Response: We double checked the discussion points with the results section and are not sure where the discrepancy is. We have removed “which were based on a primarily cognitive conceptualization of the disease” because in re-reviewing this may be a source of confusion.

23. Line 299: The ADASp being the golden standard, is it not a big problem that the Pooled index detected worsening when the ADASp is suggesting improvement?

Response: This is a valid concern that highlights a ‘chicken-and-egg’ type of challenge. The ADASp became a ‘gold standard’ based on its performance in dementia trials; this status has been carried into other research study including observational and pre-dementia studies where it does not perform optimally. It additionally does not account for the now well-known motor components that accompany cognitive decline. So, rather than viewing this as a ‘problem’ we view it as a finding that needs to be further investigated. In conjunction with your point 20 suggestion, we have added to the discussion that our findings are not as clear as we expected and that more research is needed. Additionally, the now included 95% bootstrap confidence intervals show that in instances where the direction of change was opposite, the CI crosses the point of no change; the findings may be due to error and future research is needed to better understand this.

24. Line 300: While gait deterioration may have a different trajectory to cognitive decline, why does the ADASp+GV (i.e. only gait velocity added) follow a similar pattern to the ADASp?

Response: This is an excellent point which requires further research to untangle. We have added to our conclusions a future direction step to further investigate direction of change by motor and cognitive measures.

25. Line 325: Specify how including motor function was a valuable contribution?

Response: the contribution of investigating the including motor function is addressed throughout the manuscript as it is now understood to be associated with cognitive decline and dementia and there is a need for more responsive measures that align with this conceptualization. The paragraph beginning on Line 325 is focused on our predictive model work; the topic sentence has been updated to not mention motor function and avoid confusion.

26. Since the Pooled index and the ADAS-cog proxy showed comparable baseline discrimination, and differences between tests on responsiveness to longitudinal changes did not appear to be statistically analysed, the inferences from this study seem tenuous?

Response: to better understand the response to longitudinal changes 95% bootstrap confidence have been added for the ADAS-Cog-Proxy GAM scores and the Pooled Index at each time point. Throughout the manuscript we have tried to increase emphasis that this as a proof of principle study and to temper conclusions.

Minor:

Title page:

27. Line 3: What is the order of authors? The order on the title page differs compared to page 1.

Response: Thank you for identifying this inconsistency. The order on the title page is the correct one. We have updated the PlosOne information.

Abstract:

28. This being a proof of principle study could be mentioned.

Response: We have modified the descriptor in line three of the abstract from ‘pilot study’ to be ‘proof of principle study’.

Introduction:

29. Line 71: Can the order of references be arranged in sequence?

Response: Thank you for identifying this discrepancy; we will fix the reference order.

30. Line 79: Does the reference #33 compare dual-task in relation to ADL activities?

Response: Reference 33 investigates walking, tapping, and catching. Part of their hypothesis is that walking uses cognitive resources and will be more strongly correlated with the complex task of catching than the simpler/automatic task of tapping.

Methods:

31. Line 129: Does ADNI and ADNI 1 refer to the same study? (see also Line 133, 134, 138, 237, 241).

Response: ADNI is a multi-phase study; ADNI1 is the first phase. We have clarified this early in the ADNI section (line 123) and referred to ADNI as ADNI1 from thereon).

32. Line 143: How was similarity of covariates assessed?

Response: We have added that ‘similarity is based on measurement domains’. Additionally, the Supplementary Material now includes more details about the ADAS-Cog-Proxy model development including selection of the final model covariates.

Discussion:

33. Line 321: Is this short paragraph intended to be part of the previous?

Response: We intended for this short paragraph to stand on its own; we now moved the last sentence of the previous paragraph to this one.

Reviewer #3: Using data from ADNI, this study developed a pooled index combining gait assessment and ADAS-Cog, and further compared the responsiveness of pooled index to ADAS-Cog-Proxy both cross-sectionally and longitudinally in Gait and Brain Study (GABS). Authors showed comparable responsiveness to baseline discrimination. The pooled index, adding motor function assessment to ADAS-Cog, improved the responsiveness at 6- and 48-month follow-up but not at 36-month follow-up.

The rationale and analytical approaches are sound. I have several comments.

Major:

1. Regarding motor measures examined, a stronger rationale is needed. This study included one single-task and three dual-task conditions and included mean performance and variability of spatiotemporal gait parameters. It may be helpful to have a summary lead sentence or a vocabulary of various gait measures.

Response: Three categories to include in the PI were selected based on prior research suggesting importance for pre-dementia and dementia syndromes: cognition, motor function, and motor-cognitive performance. Including up to six component variables with low pairwise correlations in a PI is recommended for covering important measurement domains and reducing variability of final PI scores (9-11). So, we set out to include at least three variables from the aforementioned categories. All conditions assessed in GABS were considered; due to study restrictions and concerns about burden especially for an older population, conditions were selected based on previous research. Specific measures to use were selected based on presence and suggested importance in previous research; key studies are cited at the end of the DTC section.

Additional information has been added to the motor, motor-cognitive performance, and pooled index development sections of the methods to increase rationale and description of the measurement procedures. We also added a reference to the GAITRite webpage where readers may find detailed information about the system and measurements.

2. Regarding longitudinal analyses, a potential censoring bias exists because not all were followed up for 48 months. Although authors briefly mentioned this limitation, a sensitivity analysis should be considered to confirm findings.

Response: Thank you for this suggestion. We agree that there is potential bias that exists and have addressed this tangentially in another response—for some timepoints, those who remain in the study had faster gait velocity at baseline compared to those who did not and gait velocity is associated with overall better health. To help readers understand this we added additional reasons for not reaching a time point. We additionally added explicit mention of censoring bias to the limitation section. However, we decided not to perform a formal sensitivity analysis because that would require selecting expected outcomes for people who did not reach time points and it is unclear what the best or most unbiased method to do this would be, especially since we already know our primary results.

3. Authors observed that ADAS-Cog-Proxy detected improvement and discussed a potential non-linear change. I wonder whether this reflects a learning effect from the memory test since the proxy included RAVLT. Perhaps authors can test this using GABS data.

Response: this is in interesting observation and we found literature suggesting it is plausible. To avoid performing too many secondary analyses we have added this point to the discussion along with the references but did not test for this effect using GABS data.

Minor:

Line 112-113: please specify “normal” scores on MMSE and MoCA.

Response: we have added that ‘normal’ scores were based on standardized norms for age, sex, and education and provided a reference to the book these norms were taken from.

Line 115-116: please specify “persistent decline in cognition”. Between two consecutive assessments? Which cognitive function measure?

Response: this was a patient-reported outcome and not based on a cognitive function measure; SCI classification still required ‘normal’ scores on the MMSE and MoCA as defined for the NC group.

Line 149: were samples randomly divided into the testing and development subsets?

Response: Yes; we added ‘randomly’ before the word divided.

Line 153: it is not clear how the best candidate model was determined. Step 3 indicates the best candidate model is M4 and its preliminary accuracy is not the highest.

Response: we have added additional details about how the best candidate model was determined to the supplementary material.

Line 164: what’s the criteria for imputation? It would be helpful to indicate missing here in the text, <5% for instance.

Response: All missing values were imputed. Additional details have been added to the Supplementary Material under Table S2, as requested by another reviewer.

Line 165: please clarify “five datasets”

Response: We are not sure what this comment is referring to; line 165 reads “Five imputed datasets”.

Line 188: It would be helpful to demonstrate pooled index development as a figure. I think that Figure 1 is a good illustration to show how ADAS-Cog proxy was developed in GABS.

Response: Thank you for this suggestion which we agree may provide additional helpful explanation to the Pooled Index development; we added a new figure.

Line 242: Please clarify Table 2 shows sample characteristics in GABS. Perhaps add “GABS” prior to “baseline”.

Response: We have clarified for Table 1 and 2, ‘GABS’ and ‘ADNI1’ characteristics.

Reviewer #4: This is a technical article describing a pooled index approach to improve responsiveness in the ADAS-Cog in pre-dementia populations. This article could be improved by making it more interesting and clinically relevant to clinicians as well as researchers. Portions of the abstract, Introduction and discussion should be revised to describe the problem. Why is it important to create a tool that is more responsive in individuals with pre-dementia syndrome? Why is it important to distinguish between SCI and MCI.

Response: Thank you for this excellent suggestion to broaden the audience of our manuscript. Although the manuscript is targeted mainly at researchers we agree that adding functional or physical performance testing to global evaluation is of clinical relevance. It is known that the amount of dementia pathology, for example beta amyloid, in the brain is not necessarily associated with impeded cognition or functional performance. In fact. The strength of this association lessens with age. In other words, you may have a person in their 80s with lots of beta amyloid pathology without cognitive impairment. Therefore adding to neuropsychological testing functional or physical performance testing is expected to provide a more accurate picture of those with declining pathology and going to dementia. So, adding a physical test to assessments in a clinical setting may provide a better view of a patient’s health.

Distinguishing between SCI and MCI was done to increase the rigour of testing for the Pooled Index—a measure that is capable of distinguishing NC, SCI, and MCI is expected to be able to detect more subtle changes than a measure that can only distinguish NC and MCI. It then becomes a choice for whoever is using the finalized measure about what differences are meaningful to detect.

In the abstract and introduction the authors take a historical approach ("The shift in focus in dementia research......"). However, this "shift" is not new and has been evolving in the research for over 15 years.

The first few lines for the abstract should be revised to reflect our current state of the science and the need for a more responsive measures. Revising the approach and tone of the introduction section should also be considered.

Response: The abstract has been reframed to begin with a need for more responsive outcome measures, mention cognitive and motor decline, and state that this is a proof of principle study about the impact on responsiveness when motor function is added to a cognitive measure. The introduction has also been revised, as per your next point and suggestions from other reviewers.

The introduction references studies related to 'Shifted attention" in research related to pre-dementia. Instead, I recommend defining key terms/conditions related to pre-dementia and summarize seminal studies that have demonstrated relationships between cognitive and motor function.

Response: We have revised the introduction, including an explanation of pre-dementia syndromes and removing the emphasis on ‘shifts’ to stating where the research is now and why we need research projects around motor and cognitive measurement.

The discussion would also benefit from a discussion of how this work relates to prior studies, such a Verghese motot cognitive syndrome.

Response: This is a great idea and have added a paragraph to the discussion about prior studies, including MCRS.

Also, what needs to be done to make this a clinically useful tool? What is the clinical relevance of this study?

Response: Continuing off of our response to your above suggestions about clinical relevance, we have added to the discussion a paragraph about clinical relevance of combining cognitive and motor measures and key check points before this type of measure would be ready for clinical use.

Attachment

Submitted filename: Response To Reviewers.docx

Decision Letter 1

Antony Bayer

18 Aug 2020

PONE-D-19-25160R1

Cognition and motor function: The gait and cognition pooled index

PLOS ONE

Dear Dr. Montero-Odasso,

Thank you for submitting your revised manuscript to PLOS ONE and for your detailed attention to the previous reviewers' comments. After careful consideration, we feel that it has considerable merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the few outstanding points raised during the latest review process.​

Please submit your revised manuscript by Oct 02 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

  • A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

  • A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

  • An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Antony Bayer

Academic Editor

PLOS ONE

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #4: (No Response)

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #4: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #4: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #4: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #4: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #4: The introduction and discussion of this paper are substantially improved. Recommended changes:

Abstract:

Line 36/37: incomplete sentence. Please revise.

Introduction:

Line 52: I disagree with the use of term “emerging research." Instead, “multiple studies have reported relationships between cognitive and motor function in pre-dementia syndromes". Citations are needed.

Here are just a few studies that could be cited:

Aggarwal NT, Wilson RS, Beck TL, Bienias JL, Bennett DA. Motor dysfunction in mild cognitive impairment and the risk of incident Alzheimer disease. Arch Neurol 2006;63:1763-9.

Mielke MM, Roberts RO, Savica R, et al. Assessing the Temporal Relationship Between Cognition and Gait: Slow Gait Predicts Cognitive Decline in the Mayo Clinic Study of Aging. J Gerontol A Biol Sci Med Sci 2013; 68:929-37.

Verghese J, Robbins M, Holtzer R, et al. Gait dysfunction in mild cognitive impairment syndromes. J Am Geriatr Soc 2008;56:1244-51

Buchman AS, Bennett DA. Loss of motor function in preclinical Alzheimer's disease. Expert Rev Neurother 2011;11:665-76.

Liu-Ambrose TY, Ashe MC, Graf P, et al. Increased risk of falling in older community-dwelling women with mild

cognitive impairment. Phys Ther. 2008; 88:1482–1491.

Waite LM, Grayson DA, Piguet O, et al. Gait slowing as a predictor of incident dementia: 6-year longitudinal data from the Sydney Older Persons Study. J Neurol Sci. 2005;229–230:89–93.

Line 53-55: The sentences "There is a need……” is vague. I recommend replacing with a clear and accessible statement.

Line 56: Mild Cognitive Impairment - No need to capitalize

Line 65: ….Natural history. add “of disease progression” or “of MCI” (Natural history of disease progression)

Line 87-88 Unclear sentence - “Our literature review found…

Line 91-93 Remove this statement: “ While we focus on the ADAS-Cog given its prominence and strong history in the field, but our methodological approach can serve as an example for researchers to follow for other contexts and measures as well.”

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #4: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

PLoS One. 2020 Sep 11;15(9):e0238690. doi: 10.1371/journal.pone.0238690.r004

Author response to Decision Letter 1


20 Aug 2020

We are thankful to Reviewer 4 for going through our manuscript again and providing suggestions to improve its clarity. Please find our point by point responses below.

Reviewer #4: The introduction and discussion of this paper are substantially improved. Recommended changes:

Abstract:

Line 36/37: incomplete sentence. Please revise.

Response: We have revised this sentence to: Final selected variables for the Pooled Index include gait velocity, dual-task cost of gait velocity, and an ADAS-Cog-Proxy (statistical approximation of the ADAS-Cog using similar cognitive tests).

Introduction:

Line 52: I disagree with the use of term “emerging research." Instead, “multiple studies have reported relationships between cognitive and motor function in pre-dementia syndromes". Citations are needed.

Here are just a few studies that could be cited:

Aggarwal NT, Wilson RS, Beck TL, Bienias JL, Bennett DA. Motor dysfunction in mild cognitive impairment and the risk of incident Alzheimer disease. Arch Neurol 2006;63:1763-9.

Mielke MM, Roberts RO, Savica R, et al. Assessing the Temporal Relationship Between Cognition and Gait: Slow Gait Predicts Cognitive Decline in the Mayo Clinic Study of Aging. J Gerontol A Biol Sci Med Sci 2013; 68:929-37.

Verghese J, Robbins M, Holtzer R, et al. Gait dysfunction in mild cognitive impairment syndromes. J Am Geriatr Soc 2008;56:1244-51

Buchman AS, Bennett DA. Loss of motor function in preclinical Alzheimer's disease. Expert Rev Neurother 2011;11:665-76.

Liu-Ambrose TY, Ashe MC, Graf P, et al. Increased risk of falling in older community-dwelling women with mild

cognitive impairment. Phys Ther. 2008; 88:1482–1491.

Waite LM, Grayson DA, Piguet O, et al. Gait slowing as a predictor of incident dementia: 6-year longitudinal data from the Sydney Older Persons Study. J Neurol Sci. 2005;229–230:89–93.

Response: We agree with the reviewer’s idea and are thankful for the suggested citations; these have been incorporated.

Line 53-55: The sentences "There is a need……” is vague. I recommend replacing with a clear and accessible statement.

Response: We revised the sentence to read, “There is a need for outcome measures that reflect these advancements and are more responsive for present research settings, while maintaining compatibility with historical measurement techniques.”

Line 56: Mild Cognitive Impairment - No need to capitalize

Response: we removed the capitalization.

Line 65: ….Natural history. add “of disease progression” or “of MCI” (Natural history of disease progression)

Response: we added ‘of disease progression’.

Line 87-88 Unclear sentence - “Our literature review found…

Response: The sentence has been revised to: Our literature review of modifications made to the ADAS-Cog since its development did not find any revisions whereby motor function or DTC assessments were added to the ADAS-Cog

Line 91-93 Remove this statement: “ While we focus on the ADAS-Cog given its prominence and strong history in the field, but our methodological approach can serve as an example for researchers to follow for other contexts and measures as well.”

Response: We removed the sentence.

Attachment

Submitted filename: Response to Reviewers.docx

Decision Letter 2

Antony Bayer

24 Aug 2020

Cognition and motor function: The gait and cognition pooled index

PONE-D-19-25160R2

Dear Dr. Montero-Odasso,

Thank you for you manuscript with further revisions. We’re pleased to inform you that this has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Antony Bayer

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Acceptance letter

Antony Bayer

2 Sep 2020

PONE-D-19-25160R2

Cognition and motor function: The gait and cognition pooled index

Dear Dr. Montero-Odasso:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Professor Antony Bayer

Academic Editor

PLOS ONE

Associated Data

    This section collects any data citations, data availability statements, or supplementary materials included in this article.

    Supplementary Materials

    S1 Appendix. ADAS-Cog-Proxy development details.

    (DOCX)

    S1 Table. Secondary analysis: Mini mental state examination responsiveness to group-level within-person measured change over time.

    (DOCX)

    Attachment

    Submitted filename: Response To Reviewers.docx

    Attachment

    Submitted filename: Response to Reviewers.docx

    Data Availability Statement

    Data from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) may be requested through the ADNI website (http://adni.loni.usc.edu/data-samples/access-data/). Data from the Gait and Brain study cannot be shared publicly because the study is still ongoing and the data contain personal health information. This restriction has been imposed by the ethics board of the University of Western Ontario. Data access queries can be directed to Yanina Sarquis-Adamson (contact via Yanina.SarquisAdamson@sjhc.london.on.ca). The authors of the present study had no special access privileges in accessing the underlying data which other researchers would have.


    Articles from PLoS ONE are provided here courtesy of PLOS

    RESOURCES