Outcome Measures for Parkinson's Disease Dementia: A Systematic Review

Abstract

Background

Parkinson's disease (PD) dementia (PDD) is a major cause of morbidity and mortality in PD, which severely affects patient functioning and quality of life and increases the risk for nursing home admission. Unfortunately, current treatment options for PDD are limited and have only marginal therapeutic effects. As novel treatments are developed, there will be a need to assess their efficacy in well‐designed, randomized, controlled trials. However, there is no consensus on the optimal outcome measures for use in PDD clinical trials.

Methods

A systematic review of PDD clinical trials and empirical studies of outcome measures used in PDD was performed. Outcome measures were divided into 5 categories: (1) cognitive; (2) behavioral and mood; (3) activities of daily living and quality of life; (4) global; and (5) caregiver burden.

Findings

A total of 20 PDD pharmacological clinical trials were identified. These trials incorporated a broad array of outcome measures, which were used inconsistently across trials. We summarize the psychometric properties and other relevant data on outcome measures used, including their diagnostic utility, inter‐rater reliability, test‐retest reliability, responsiveness, clinically meaningful change, and availability of alternate forms.

Conclusions

We have identified the best‐evidenced PDD outcome measures in each domain. Further research is needed to assess the validity, reliability, and clinically meaningful change of these measures in PDD to inform the design of future clinical trials and enhance the ability of clinicians, researchers, and policy makers to interpret study results. In addition, the development of outcome measures specific to PDD may be warranted.

Parkinson's disease (PD) is the second‐most common neurodegenerative illness after Alzheimer's disease (AD), affecting over 5 million persons worldwide with an anticipated doubling of prevalence in the next 20 yr.1 PD dementia (PDD) is a major cause of morbidity and mortality in PD, affecting function, quality of life, and the risk for nursing home admission.2 PDD has a point prevalence of approximately 31% and a mean time from onset of PD to diagnosis of dementia of 11 yr.3 For patients surviving 20 yr or more, the chances of developing PDD may be as high as 75%.3

Although several effective therapies exist for PD motor symptoms, there are few treatment options available for PDD and they are of marginal clinical benefit. Currently, the only U.S. Food and Drug Administration approved treatment for PDD is the cholinesterase inhibitor, rivastigmine. A recent Cochrane review supports the use of other cholinesterase inhibitors for PDD, but acknowledges their limited clinical efficacy and the need for more‐effective treatments.4 The N‐methyl‐d‐aspartate receptor antagonist, memantine, may also be beneficial in PDD; however, a large randomized, controlled trial (RCT) demonstrated its effectiveness only for dementia with Lewy bodies and not PDD.5 Clearly, there is a need to develop and investigate more‐effective therapies for PDD.

Two critical issues in designing RCTs for PDD are defining the condition and choosing appropriate outcome measures. In regard to the former, an International Parkinson and Movement Disorder Society Task Force (MDS‐TF) developed clinical diagnostic criteria for PDD, with 2 levels of testing: level I, an abbreviated assessment for clinical use, and level II, more‐comprehensive testing intended for research studies and pharmacological trials.6, 7 Since their publication in 2007, the MDS‐TF diagnostic criteria have been shown to be more useful and accurate than the previous Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition standard.8 Work continues in the application and validation of these criteria, with results indicating good agreement between MDS‐TF diagnostic criteria and more‐exhaustive clinical and neuropsychological testing.9

Concerning outcome measures, there are few empirical studies examining their performance in PDD and these are limited in terms of scales studied and psychometric properties assessed.10 One of the more challenging aspects of assessing outcomes is defining clinically meaningful change (CMC), given that small, but statistically significant, improvements in test scores in RCTs often do not translate into any perceived benefit on the part of patients, caregivers, or clinicians.11 An ideal outcome measure would be psychometrically valid and reliable in a PDD population, be responsive to interventions, have well‐defined clinically meaningful change, and contribute to the diagnosis of PDD. Though change in cognitive test scores are clearly important, these may need to be supplemented with measures of function, quality of life, and/or caregiver burden, depending on study goals.

In this article, we review the outcome measures that have been used to date in PDD clinical trials. To evaluate the reliability and validity of the utilized measures, we summarize the data on their psychometric properties as evaluated in dementia populations. We conclude by highlighting areas where further research is needed.

Methods

Using PubMed, a literature review of clinical trials in patients with PDD from January 1965 to February 2015 was performed, using combinations of the following search terms: “Parkinson's disease dementia”; “treatment”; “intervention”; “therapy”; and “clinical trial.” The outcome measures utilized in the clinical trials were then classified into 5 domains: (1) cognitive; (2) behavioral and mood; (3) activities of daily living and quality of life; (4) global; and (5) caregiver burden. For all outcome measures, we used PubMed to perform an additional search of empirical studies assessing their validity and reliability, using the search terms: “clinical significance”; “clinically meaningful change”; “validity”; “validation”; “reliability”; and “sensitivity to change.” We included studies of participants with PDD, or AD, PD psychosis, and/or cognitive impairment (CI) when PDD studies were not available.

The statistical properties evaluated for each outcome measure include diagnostic utility, inter‐rater reliability, test‐retest reliability, responsiveness, and CMC values. Diagnostic utility was defined as the sensitivity and specificity of an outcome measure for diagnosis of PDD. If unavailable, concurrent validity for the measure with validated tests is reported. Inter‐rater reliability is the degree of concordance between multiple raters, usually defined as Cohen's kappa. Test‐retest reliability refers to the consistency of results in the same subject over a short time interval.

In a purely statistically sense, sensitivity to change refers to the magnitude of change that has occurred over time, as compared to a gold standard. This is slightly different than the related concept of responsiveness, which is the ability of a scale to detect meaningful changes over time, though the 2 terms are often used interchangeably in clinical research. To be as accurate as possible, we will use the term responsiveness throughout this review. Given that the true drug effect is unknown and cannot be evaluated for a single measure in isolation, we looked at the effects of the intervention for multiple measures. If the outcome of interest improved in line with other outcomes, we rated it as responsive, and if it failed to improve when other outcomes improved (or had significantly smaller effect size), we rated it as unresponsive. Responsiveness to disease progression was also looked at when data were available as a measure of responsiveness. CMC is usually defined in terms of the minimal clinically important difference, defined as “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side‐effects and excessive cost, a change in the patient's management.”12 The presence of alternate forms of the outcome measure and their susceptibility to practice effect are also reported, when relevant.

Based on these data, we make endorsements regarding the use of the studied outcome measures in future trials by the following criteria: (1) “recommended”: the outcome measure has been applied to PDD patients and has been found valid, reliable, and responsive in psychometric studies in studies of PDD or other dementias; (2) “suggested”: the outcome measure has been applied to PDD patients and psychometric studies are promising (“moderate” to “high” parameters), but not yet complete in dementia populations and/or some psychometric properties are suboptimal (“low”); and (3) “listed”: the outcome measure has been applied to PDD patients, but psychometric studies have demonstrated disappointing data (“low” parameters) and/or have not yet been performed in dementia populations. These criteria were modeled after similar reviews of scales in PD and the appendix of ancillary scales to complement the MDS‐UPDRS from the MDS.13, 14

Results

A total of 20 clinical trials evaluating the efficacy of interventions in PDD were identified.5, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33 All of these trials assessed pharmacological treatments. Twenty‐three unique outcome measures were utilized within the identified treatment trials (Tables 1 and 2).

Table 1.

Cognitive outcome measures

Outcome measure	Reference for studies used	Diagnostic utility	Inter‐rater reliability	Test‐retest reliability	Responsiveness	Clinically meaningful change	Endorsement
ADAS‐cog	25, 26, 27, 28, 33, 34, 36, and 37	NE	High	Moderate to high	Yes	4 points	Recommended
MDRS	23, 31, 32, and 37	High	High	High	Yes	NE	Recommended
MoCA	Not used	Moderate	High	High	Yes	NE	Suggested
CDR	25, 27, and 39	NE	NE	Moderate	Yes	NE	Suggested
FAB	33 and 34	Moderate	High	High	NE	NE	Suggested
D‐KEFS VF	25, 26, 27, 34, and 36	Low to moderate	NE	Moderate to high	NE	NE	Suggested
HVLT	31	High	NE	NE	NE	NE	Suggested
TPCT	23, 27, and 33	High	NE	NE	NE	NE	Suggested
MMSE	22, 23, 24, 25, 26, 27, 28, 31, 32, 33, 34, 35, 36, and 37	Low to moderate	High	High	Low for small changes	1.4 points	Listed
QTCS	22	NE	NE	NE	NE	NE	Listed
VMI	31	NE	NE	NE	NE	NE	Listed
BTA	26 and 31	NE	NE	NE	NE	NE	Listed

Endorsement definitions: (1) “Recommended”: The outcome measure has been applied to PDD patients and has been found valid, reliable, and responsive in psychometric studies in studies of PDD or other dementias; (2) “Suggested”: The outcome measure has been applied to PDD patients and psychometric studies are promising (“moderate” to “high” parameters), but not yet complete in dementia populations and/or some psychometric properties are suboptimal (“low”); and (3) “Listed”: The outcome measure has been applied to PDD patients, but psychometric studies have demonstrated disappointing data (“low” parameters) and/or have not yet been performed in dementia populations.

QTCS, Quick Test of Cognitive Speed; VMI, Developmental Test of Visual‐Motor Integration; BTA, Brief Test of Attention; PDD, Parkinson's disease dementia; NE, Not established.

Table 2.

Other outcome measures

Outcome measure	Reference for studies used	Inter‐rater reliability	Test‐retest reliability	Sensitivity to change	Clinically meaningful change	Endorsement
Mood/behavioral
NPI	6, 22, 23, 24, 25, 26, 27, 31, 32, 33, 34, and 36	High	Moderate to high	Yes	8 points	Recommended
CSDD	31	High	High	Yes	NE for dementia	Recommended
BPRS	37	Moderate	NE for dementia	NE for dementia	NE for dementia	Suggested
ADLs and QOL
ADCS‐ADL	6, 25, 27, 38, and 36	High	High	Yes	2 points	Recommended
QOL‐AD	7	High	High	Yes	NE for dementia	Recommended
DAD	22, 26, 33, and 34	High	High	Yes	NE for dementia	Recommended
SE‐ADL	26	Moderate	NE for dementia	NE for dementia	NE for dementia	Suggested
PDQ‐39	Not used	NE for dementia	Moderate	NE for dementia	Depends on subsection, but has been established	Suggested
UPDRS‐ADL	31 and 37	NE for dementia	NE for dementia	NE for dementia	NE for dementia	Listed
Global
CGI	22, 29, 35, and 37	Moderate	Moderate	Yes	NE for dementia	Recommended
ADCS‐CGIC	6, 25, and 27	NE for dementia	High	Yes	1–2 points	Suggested
CIBIC+	24, 26, and 32	Low	Moderate	Yes	NE for dementia	Suggested
Caregiver burden
ZBI	6 and 30	NE for dementia	High	Yes	13 points	Suggested

Endorsement definitions: (1) “Recommended”: The outcome measure has been applied to PDD patients and has been found valid, reliable, and responsive in psychometric studies in studies of PDD or other dementias; (2) “Suggested”: The outcome measure has been applied to PDD patients and psychometric studies are promising (“moderate” to “high” parameters), but not yet complete in dementia populations and/or some psychometric properties are suboptimal (“low”); and (3) “Listed”: The outcome measure has been applied to PDD patients, but psychometric studies have demonstrated disappointing data (“low” parameters) and/or have not yet been performed in dementia populations. NE, Not established.

Cognitive Outcome Measures

Cognitive scales were the most common (11 of 23) type of outcome measure identified (Table 1). The Mini–Mental State Examination (MMSE) is one of the oldest cognitive screening tools available and is the most frequently employed cognitive outcome measure in PDD trials, used in 14 studies,22, 23, 24, 25, 26, 27, 28, 31, 32, 33, 34, 35, 36, 37 despite validity concerns in PDD.17, 34 Though validated for assessing dementia severity,10 its sensitivity and specificity for diagnosing PDD are suboptimal (80% and 74%, respectively) when tested against the most recent MDS‐TF clinical diagnostic criteria.35 Moreover, the MMSE may have floor effects in subjects with severe dementia and ceiling effects in subjects with mild cognitive impairment38 and lacks adequate testing of executive functions.36 In PDD, the MMSE demonstrates moderate concurrent validity with the Alzheimer's Disease Assessment Scale‐Cognitive Subscale (ADAS‐Cog; r = −0.6)10 and low concurrent validity with the Mattis Dementia Rating Scale (MDRS; r = 0.29).37 Though not established in PDD, the MMSE demonstrates high inter‐rater reliability among cognitively impaired elderly individuals (k = 0.82–0.91)39 and high test‐retest reliability in AD (r = 0.87 and 0.89).40, 41 It lacks responsiveness to small changes, especially in patients with early PDD.30 In the Donepezil and Memantine in Moderate to Severe Alzheimer's Disease (DOMINO) trial, CMC was defined as 1.4 points on the MMSE, based on triangulation of expert opinion and distribution‐based values.42 CMC has not been established for PDD. Although short‐term practice effects have been reported in AD,16 2 versions of the MMSE are available for repeat testing.43

The ADAS‐cog was utilized in 8 studies.25, 26, 27, 28, 33, 34, 36, 37 Though originally designed for AD, it is commonly used in PDD.32 ADAS‐cog scores demonstrate statistically significant differences between PDD patients of mild and moderate severity of CI, but its utility in diagnosis of PDD remains untested.10 In PDD, it demonstrates moderate concurrent validity with the MMSE (r = −0.60).10 The inter‐rater reliability for PDD is not established, but it demonstrates high inter‐rater reliability in AD (k = 0.91).44 Test‐retest reliability is moderate in PDD of both mild (r = 0.65) and moderate disease severity (r = 0.71).10 The ADAS‐cog is responsive in both PDD and AD, as determined by analysis of data from 3 large rivastigmine treatment trials.45 CMC was defined as 4 points at 6 mo for the ADAS‐cog in AD, though this is only meaningful for groups and not individuals.46 Although learning effects have been reported on this exam in PDD,30 alternate forms are available.

The MDRS47 was used in 4 studies.17, 25, 26, 32 The MDRS has been validated for PDD diagnosis, with a total cut‐off score of 123 yielding high sensitivity (93%) and specificity (91%).48 However, this study has been criticized for using a sample with a low mean educational achievement (8.9 yr). A subsequent study, using subjects with a mean education of 14.8 yr, suggests this cut‐off score is inadequate for detecting PDD, correctly classifying only 60.7% of the sample, with high specificity (100%), but very low sensitivity (20%).49 This study suggests a total cut‐off score of 133 instead.49 Overall, the MDRS has been shown to be superior to the MMSE in assessing cognition50 and, particularly, in detecting frontal lobe dysfunction in PDD.11 Though not established for PDD, studies in AD show high inter‐rater reliability (k = 0.93)51 and high test‐retest reliability (r = 0.97).52 The MDRS is generally responsive to change in PDD trials, with the exception of 1 donepezil RCT in which it did not reveal change despite improvements in other measures.18 CMC has not been established. An alternate form is available.53

The Cognitive Drug Research (CDR) system, a computer‐based set of cognitive tests, was used in 3 trials.19, 21, 33 Diagnostic utility of the CDR system in PDD is not established, nor is inter‐rater reliability, though the computerized nature of the test mitigates inter‐rater variability. For all‐cause dementia, the CDR demonstrated moderate concurrent validity with the MMSE (r = 0.5–0.65).54 The CDR‐Power of Attention test demonstrated moderate test‐retest reliability in PDD patients with mild dementia (r = 0.63) and moderate dementia (r = 0.46).10 The CDR system demonstrated responsiveness in PDD pharmacological trials.55 CMC is not established in PDD or AD. Over 20 alternate forms of the CDR are available.56

The Frontal Assessment Battery57 (FAB) was used in 2 studies.27, 28 The FAB is not reliable as a sole test for diagnosing PDD (sensitivity 66% and specificity 72%58).11 In PD and atypical parkinsonian syndromes, the FAB demonstrated high concurrent validity with the MDRS (r = 0.82) and high inter‐rater reliability (k = 0.87).57 For AD, it shows high test‐retest reliability (r = 0.82) and inter‐rater reliability (k = 0.98).59 The FAB demonstrated responsiveness in both trials in which it was used. CMC is not established and there are no alternate forms.

Data on the psychometric properties of the following outcome measures for PDD are currently lacking: the Delis‐Kaplan Executive Function System (D‐KEFS)‐Verbal Fluency Test60 (D‐KEFS VF); the Hopkins Verbal Learning Test Revised (HVLT); and the Ten Point Clock Drawing Test61 (TPCT). The D‐KEFS shows high test‐retest reliability for PDD patients with mild disease severity (r = 0.79) and moderate test‐retest reliability for moderate disease severity (r = 0.55).10 The HVLT demonstrates high sensitivity (83%) and specificity (83%) for distinguishing all dementias from controls using a cut‐off score of 16.62 There are 6 alternate forms of the HVLT.63 The TPCT shows high concurrent validity (r = 0.71–0.73) with the MMSE in AD64 and a score of less than 8/10 identified 71% of patients with mild AD.65 CMC has not been established for any of these measures. No data on the psychometric properties of the following outcome measures are available: A Quick Test of Cognitive Speed66; the Developmental Test of Visual‐Motor Integration (VMI); and the Brief Test of Attention.

Although the Montreal Cognitive Assessment67 (MoCA) has not yet been used in any trials, it is commonly used clinically and has adequate psychometric properties as a screening tool for PDD.68 The optimal diagnostic cut‐off score for the MoCA in PDD is <21/30, yielding high sensitivity (81%) and specificity (95%).68, 69 In AD and frontotemporal dementia, the MoCA shows high concurrent validity with the MMSE (r = 0.82).70 It demonstrates high inter‐rater reliability (k = 0.81) and high test‐retest reliability (r = 0.79) in PD of varying cognitive involvement.71 In a 3‐yr longitudinal study of PD patients with varying levels of cognition, no significant change on the MoCA was found, even when subjects were stratified by age, MMSE score, and disease duration, suggesting that the MoCA lacks responsiveness.72 However, in a recent trial of early AD patients, the MoCA was shown to be capable of detecting small‐to‐moderate cognitive change over time.73 CMC for this measure is not established. Three alternate English versions are available.

Mood/Behavioral Outcome Measures

Mood and behavioral outcome measures are frequently employed in PDD trials (Table 2), with the Neuropsychiatric Inventory74 (NPI) being the most commonly utilized, in 12 studies.6, 22, 23, 24, 25, 26, 27, 31, 32, 33, 34, 36 The NPI dysphoria subscore demonstrated moderate concurrent validity (r = 0.62) with the Hamilton Rating Scale for Depression when administered to caregivers of patients with all‐cause dementia.74 This study also showed high inter‐rater reliability for each subscale of the NPI (k = 0.89–1.0).74 The NPI shows moderate test‐retest reliability for PDD of mild (r = 0.66) and moderate severity (r = 0.73).10 The NPI is commonly used in PD psychosis studies, with some antipsychotic treatment trials suggesting that this outcome measure may not be adequately responsive in PD populations.75, 76, 77 However, the NPI‐2, being the sum of scores for the delusions and hallucinations subscales, did reveal statistically significant changes in 1 trial of quetiapine.75 Although CMC has not been established for PDD, the statistical analysis plan from the DOMINO trial (AD) determined that an 8‐point change in the NPI was clinically meaningful, taking into account both expert opinion and distribution‐based values.42

The Brief Psychiatric Rating Scale78 (BPRS) was used in 1 dementia treatment trial.32 The Brazilian version of the BPRS shows high concurrent validity (r = 0.73) of the delusion subscale and moderate concurrent validity (r = 0.43) of the hallucination subscale with the NPI.79 Although not established for PDD, inter‐rater reliability of the BPRS for AD varies significantly (range = 0.13–1.0; median = 0.45).58 Test‐retest reliability for the BPRS has not been established. Responsiveness of the BPRS has been validated in patients with mental illness80 and PD psychosis.75 Although not established in PDD, CMC for the BPRS in a clinical trial for PD psychosis was suggested as a change of 25%; no explanation was provided on how this value was chosen.81

The Cornell Scale for Depression in Dementia82 (CSDD) was used in 1 trial.25 The CSDD is based on observation of both the patient and informant and is validated for diagnosing depression in patients with PD of varying cognitive impairment.83 A cut‐off score of ≥8 yields sensitivity of 75% and specificity of 82% for diagnosis of depression in PD.83 The Korean version of the CSDD in AD showed high concurrent validity with the Hamilton Depression Rating Scale (r = 0.91) and the Geriatric Depression Scale (r = 0.75).84 Although data are not available in PDD, the CSDD has shown high inter‐rater reliability in an elderly cohort with various dementias (k = 0.84)85 and high test‐retest reliability in AD (r = 0.91).84 CMC has not been established.

Activities of Daily Living and Quality of Life Outcome Measures

We identified 5 activities of daily living (ADLs) and quality of life (QOL) measures (Table 2), with the Alzheimer's Disease Cooperative Study‐Activities of Daily Living Scale (ADCS‐ADL) used most frequently in 5 trials.5, 19, 21, 30, 31 The ADCS‐ADL demonstrates moderate concurrent validity with the ADAS‐cog for PDD (r = −0.47).10 Individual components of the ADCS‐ADL show moderate concurrent validity with the MMSE in an AD population (r = 0.4–0.7).86 High inter‐rater reliability of the ADCS‐ADL has been noted,87 but we did not find any empiric study assessing this property. The ADCS‐ADL shows high test‐retest reliability for PDD patients with mild (r = 0.94) and moderate disease severity (r = 0.92).10 The ADCS‐ADL demonstrated responsiveness in detecting a treatment effect in 4 of 5 trials in which it was employed.19, 21, 30, 31 The ADCS‐ADL is sensitive to disease progression in AD, but this property has not been established for PDD.86 It is suggested that a change of 2 points is clinically meaningful (ie, a loss of dressing or bathing independently), but CMC has not been examined psychometrically.87 A modified version of the ADCS‐ADL, which includes a subset of 19 items, is intended for patients with moderate‐to‐severe dementia.

The Disability Assessment for Dementia (DAD) scale was used in 4 trials16, 20, 27, 28 and is a subscale of the Clinician's Interview Based Impression of Change scale (CIBIC).27, 28 In AD, the DAD shows high concurrent validity with the Rapid Disability Rating Scale‐2 (r = −0.85)88 and with the Japanese version of the Clinician Interview Based Impression of Change with caregiver input (CIBIC+) (r = 0.91).50 The DAD demonstrates high test‐retest (r = 0.96) and inter‐rater reliability (k = 0.95) in AD, but has not been established for PDD.88 The DAD demonstrated responsiveness in 2 of 4 trials in which it was utilized.27, 28 One study suggests the DAD is not sensitive to AD progression, but this property has not been assessed for PDD.89 CMC has not been established for the DAD.

The UPDRScale‐ADL Scale is not specific to dementia, but was included as an outcome measure in 2 trials.25, 32 This measure shows moderate sensitivity (67%) and specificity (79%) in discriminating PDD from PD‐normal cognition when using a cutoff of 15.5.90 The UPDRS‐ADL demonstrates high inter‐rater reliability (k ≥ 0.80)91 in PD patients with moderate disease severity8 and high test‐retest reliability (r = 0.85) in early‐stage PD patients,92 though not established for PDD. The UPDRS‐ADL did not demonstrate responsiveness in detecting a treatment effect in either of the 2 trials in which it was utilized. The UPDRS‐ADL has shown responsiveness in disease progression for PD patients, but this property has not been evaluated in PDD.93 One study suggests that 4 points represents CMC for the UPDRS‐ADL.92

The Schwab and England‐ADL (SE‐ADL) scale was used as a secondary outcome measure in 1 trial.20 The SE‐ADL demonstrates moderate sensitivity (71%) and specificity (77%) in detecting PDD when using a cutoff of 75.90 A study nonspecific to dementia suggests that the scale has moderate inter‐rater reliability between physicians, patients, and caregivers (k = 0.6) and slightly higher inter‐rater reliability between physicians and patients (k = 0.65).55 A study of PD patients that excluded dementia suggests that the SE‐ADL has moderate test‐retest reliability (r = 0.70).94 SE‐ADL demonstrates responsiveness for disease progression of early PD, but this property has not been assessed specifically for PDD.95 In the trial in which it was used, the SE‐ADL did not detect a treatment effect.20 One study suggests that a score of 12.33 represents CMC for PD patients.94

The Quality of Life in Alzheimer's Disease Scale (QOL‐AD) was used in secondary analysis of 1 clinical trial.15 For AD, the QOL‐AD shows moderate concurrent validity with the Dementia Quality of Life scale (DQOL; r = 0.69) and the Euroqol‐5D scale (r = 0.54).96 Inter‐rater reliability for PDD is not established, but the QOL‐AD demonstrates moderate inter‐rater reliability (k > 0.7) in AD.96 The Chinese version of the QOL‐AD shows high test‐retest reliability for patient and caregiver reports (r = 0.84 and 0.90, respectively) in an AD population.97 The QOL‐AD demonstrated responsiveness in detecting a treatment effect in the trial in which it was used.15 CMC for the QOL‐AD has not been established.

Although the 39‐item Parkinson's Disease Questionnaire98 (PDQ‐39) is a commonly used outcome measure in PD, it has not been used to date in PDD trials. The PDQ‐8, a short form of the PDQ‐39, was used in 1 trial.24 In a PD population, the PDQ‐39 shows moderate concurrent validity with the Beck depression and anxiety inventories (r = 0.73) and the Royal Postgraduate Medical School severity scale (r = 0.66)99 Information on inter‐rater reliability of the PDQ‐39 is not available for PDD. The PDQ‐39 demonstrates moderate‐to‐high test‐retest reliability in a PD population (r = 0.68–0.94),100 and the PDQ‐8 demonstrates high test‐retest reliability as well (r = 0.82).94 One study suggests that the PDQ‐39 is responsive in PD patients.99 The PDQ‐39 may be insensitive to detecting change in early PD progression.95 Minimally clinically important difference values have been determined for each subsection of the PDQ‐39 in PD without dementia (mobility: −1.5, −3.2; ADL: −0.7, −4.4; emotional well‐being: 0.3, −4.2; stigma: 0.8, −5.6; social support: −1.2, −11.4; cognition: 0.4, −1.8; Communication: −0.8, −4.2; bodily discomfort: 1.3, −2.1).100

Global Outcome Measures

Three global measures were used in the reviewed trials (Table 2). The Alzheimer's Disease Cooperative Study–Clinician's Global Impression of Change (ADCS‐CGIC) was used as a primary outcome measure in 2 trials19, 21 and in another as a secondary outcome measure.5 In AD, the ADCS‐CGIC demonstrated low concurrent validity with the Global Deterioration Scale (GDS; r = 0.15) and the CDR (r = 0.15), and no significant correlation with the Functional Assessment Staging scale (FAST).101 Inter‐rater reliability of the ADCS‐CGIC has not been established. The ADCS‐CGIC demonstrated high short‐term test‐retest reliability, with 90% to 94% of subjects rated as having not changed or only minimally changed using the scale between month 1 and 2.101 The ADCS‐CGIC demonstrated responsiveness in detecting a treatment effect in 2 of the 3 trials in which it was used.19, 21 A change in score of 1 or 2 points on a 7‐point scale from the ADCS‐CGIC indicates CMC for AD patients.21

The CIBIC+, an alternate version of the CIBIC nonspecific to dementia including caregiver input, was used in 3 trials.18, 20, 26 The Japanese version of the CIBIC+ demonstrated high concurrent validity in AD with the DAD (r = 0.91) and the Mental Function Impairment scale (MENFIS; r = 0.99).50 This study also showed low inter‐rater reliability (k = 0.45), but improved inter‐rater reliability when collapsed into a 3‐point scale (improved, no change, or worsened; k = 0.89).102 For AD, test‐retest reliability of the CIBIC+ is moderate (range = 0.4–0.6).103 The CIBIC+ demonstrated responsiveness in detecting a treatment effect in 2 trials18, 20 and a trend toward improvement in another.26 CMC is intrinsic to the scale, which may be used to rate patient, caregiver, and clinician impressions of change and has been validated in AD.101

The Clinical Global Impression scale (CGI) was used in 4 trials.16, 23, 29, 32 The CGI, which is not specific to dementia, consists of 3 items: the CGI‐Severity scale; CGI‐Improvement scale; and CGI‐Efficacy index. In patients with depression and panic disorder, the CGI‐Severity scale showed moderate concurrent validity (r = 0.44–0.65) and the CGI‐Improvement scale showed low concurrent validity (r = 0.26–0.46) with the Hamilton Depression Rating Scale.104 For patients with a diagnosis of dementia, the CGI‐Improvement subscale demonstrates moderate inter‐rater reliability (k = 0.51), whereas the inter‐rater reliability of the CGI‐Severity subscale was slightly higher (k = 0.66).105 Test‐retest reliability of the CGI‐Severity is moderate (r = 0.65).105 The CGI demonstrated responsiveness in detecting a statistically significant treatment effect in all 4 trials in which it was used.16, 23, 29, 32 CMC is intrinsic to the scale, which may be used to rate patient, caregiver, and clinicians impressions of change, but has not been validated in dementia.

Caregiver Burden

Only 1 caregiver burden outcome measure (Table 2) has been used in PDD trials to date, the Zarit Burden Interview‐Caregiver Burden Assessment106 (ZBI), which was used in 2 studies.5, 24 The ZBI demonstrates moderate‐to‐high concurrent validity with other measures of caregiver burden: the Burden Assessment Scale (BAS; r = 0.73) and the General Health Questionnaire (GHQ‐28; r = 0.62).107 Inter‐rater reliability has not been established for the ZBI in PDD. Among caregivers of AD patients, the ZBI shows high test‐retest reliability (r = 0.89).107 Responsiveness of the ZBI in PDD is questionable, given that it detected a treatment effect in 1 trial,24 but not in the other, though this study also did not show improvement in most cognitive test scores or the ADSC‐ADL.5 Responsiveness of the ZBI to specific interventions for caregivers of dementia patients has been demonstrated.108 For CMC, a study of caregivers of PDD patients suggests that a ZBI score of 13 or higher in caregivers of dementia patients represents a clinically significant burden.109

Discussion

Twenty clinical trials evaluating the efficacy of pharmacological interventions for PDD were identified in this systematic review. Overall, the trials inconsistently employed a wide variety of outcome measures. Additionally, many of these trials use outcome measures that have not been adequately assessed for reliability or validity in PDD, though many have data in other forms of dementia. In Tables 1 and 2, we have listed endorsements of the 23 identified measures based on the available psychometric data.

In the assessment of treatments for PDD, cognitive outcome measures are of the utmost importance, both for screening of individuals for trial participation and for evaluation of the efficacy of pharmacological interventions. Because almost all of these scales were developed for use in AD, it is important to pursue validation studies in PDD, particularly because the pattern of cognitive deficits observed in PDD differs from that of AD, with more executive and visuopatial dysfunction in PDD, as opposed to the more severe episodic memory impairment in AD. Cognitive measures that are biased more toward memory tasks may not be appropriate for use in PDD. The ADAS‐Cog and MDRS both received “recommended” ratings. The MMSE is the most frequently utilized scale in existent PDD treatment trials, yet received a “listed” endorsement based on the available psychometric data. Further work must be done to establish inter‐rater and test‐retest reliability for all of these measures specifically in PDD. Future treatment trials may also consider including the MoCA in place of the MMSE, given that this scale gains traction in both clinical and research settings for PD, but further studies of responsiveness in PDD are needed.

Although cognitive outcomes have traditionally been the target of dementia treatments, noncognitive outcomes, and particularly neuropsychiatric symptoms, may have greater functional impact on patients, caregivers, and risk for nursing home placement and are considered a clinical indication for currently available agents.110 Of the mood and behavioral outcome measures, the NPI and CSDD are the best‐evidenced scales available in dementia, receiving “recommended” endorsements. An MDS‐TF also designated the NPI a “recommended” scale in PD.111

Few ADL and QOL measures have been validated for use in PDD, though there are more data in other dementias. MDS‐TF diagnostic criteria for PDD require demonstrable impairment in ADLs owing to cognitive deficits, yet there is no validated scale to assess the functional impact of cognitive impairment in PD.6 In addition, current ADL scales may be subject to the influence of confounding variables, such as motor worsening and major depression,90 and studies have demonstrated significant differences between patient and caregiver subjective reporting of functional disability and objective performance ratings.112 Performance‐based measures of functionality would be difficult to incorporate into treatment trials owing to equipment and time constraints, but it would be helpful to validate subjective report‐based ADL scales against more objective measures. Regardless #bib2 ADL scales received “recommended” endorsements, the ADCS‐ADL and the DAD. QOL measures are rarely used in PDD trials, with only 1 trial employing a QOL outcome measure, that being the QOL‐AD, which received a “recommended” endorsement. This is a significant gap in the field, given that the overarching goal of all disease treatment should be improvement in a patient's functionality and quality of life. The PDQ‐39, which addresses both ADLs and QOL, could be a useful outcome measure in PDD treatment trials and should further study prove it valid and reliable in this population.

Global impression scales are useful in capturing minor treatment effects that may be missed by less‐comprehensive measures. However, given that global measures are largely subjective, they tend to have relatively low inter‐rater reliability. None of the utilized global scales have complete psychometric data in a PDD population, but the CGI does have adequate data to receive a “recommended” endorsement.

Only 2 trials incorporated a measure of caregiver burden, that being the ZBI. The ZBI is a promising measure in this domain given that it is a dementia‐specific measure, though its psychometric properties have not yet been adequately evaluated, leading to a “suggested” endorsement. Regardless, we suggest that future PDD trials include the ZBI, given that caregiver burden is substantial in dementia and is an important predictor of patient institutionalization.

None of the outcome measures used in treatment trials demonstrate complete psychometric data on validity, reliability, and responsiveness in PDD, though several have complete data in dementia. Further work must be done to obtain such data in PDD. In the design of future PDD trials, we stress the importance of using dementia‐specific scales that have been validated in PDD populations. Moreover, future trials should consider prioritizing outcome measures relevant to functionality and quality of life as related to cognition for PDD patients, namely, measures of cognitive functional ability. Purely cognitive measures are frequently used as primary endpoints in PDD clinical trials, yet even statistically significant improvements on such neuropsychological tests are of debatable clinical significance. As a result, many experts and even health care systems (eg, United Kingdom113) do not agree on the relevance of the findings of these treatment trials. From a patient‐centered approach, it can be argued that the effectiveness of potential treatments for cognitive impairment should be evaluated in terms of their effect on functionality, independence, and QOL, rather than on improvements in neuropsychological testing scores. However, there is no validated diagnostic procedure currently available for establishing functional impairment related to cognition, which recent114, 115 and future studies seek to rectify.

Conclusion

Owing to the lack of empirical studies assessing the psychometric properties of dementia outcome measures, further research is needed to assess their validity and reliability in PDD populations. We propose that the outcome measures we have identified as “recommended” receive top priority for rigorous empirical validation in PDD. We also suggest that other measures commonly used in clinical practice be examined and considered for use as outcome measures in treatment trials, including the MoCA and PDQ‐39. Finally, we raise the question of shifting the focus of PDD treatment trials away from neuropsychological testing scores and toward measures of functionality and QOL as related to cognition. By doing so, we could ensure that potential future treatments of PDD lead to meaningful clinical and personal improvements for patients and caregivers.

Author Roles

(1) Research Project: A. Conception, B. Organization, C. Execution; (2) Literature Search and Review: A. Design, B. Execution, C. Review and Critique; (3) Manuscript Preparation: A. Writing of the First Draft, B. Review and Critique.

S.K.H.: 1B, 1C, 2B, 2C, 3A, 3B

W.E.J.: 1C, 2B, 2C, 3B

K.A.B.: 1C, 2B, 2C, 3A, 3B

I.M.B.: 1C, 2B, 2C, 3B

B.M.K.: 1A, 1B, 1C, 2A, 2B, 2C, 3B

Disclosures

Funding Sources and Conflicts of Interest: This work was funded, in part, by the National Institutes of Health (to K.A.B.; grant no.: MH080859), the University of Colorado School of Medicine Research Track (to K.A.B.), and the National Center for Advancing Translational Sciences (NCATS; to B.M.K.; 8 KL2 TR000156‐05), and National Institute of Neurological Disorders and Stroke (to B.M.K.; 1 K02 NS080885‐01A1). The authors report no conflicts of interest.

Financial Disclosures for previous 12 months: Dr. Baker received funding from the University of Colorado School of Medicine Research Track, through the NIH (grant no.: MH080859). Dr. Kluger received funding from the NIH/NCATS (8 KL2 TR000156‐05), U.S. Army Research Laboratory and U.S. Army Research Office (contract no.: W911NF‐10‐0192), the Michael J Fox Foundation, the Parkinson's Association of the Rockies (speaker), and the National Parkinson's Foundation (speaker) and served as a medical‐legal expert for individual cases.