Outcome measures for Parkinson’s disease dementia: a systematic review

Abstract

Background

Parkinson’s disease dementia (PDD) is a major cause of morbidity and mortality in Parkinson’s disease (PD), which severely affects patient functioning and quality of life and increases the risk for nursing home admission. Unfortunately, current treatment options for PDD are limited and have only marginal therapeutic effects. As novel treatments are developed, there will be a need to assess their efficacy in well-designed randomized controlled trials. However, there is no consensus on the optimal outcome measures for use in PDD clinical trials.

Methods

A systematic review of PDD clinical trials and empiric studies of outcome measures used in PDD was performed. Outcome measures were divided into five categories: 1) cognitive; 2) behavioral and mood; 3) activities of daily living and quality of life; 4) global; and 5) caregiver burden.

Findings

A total of 20 PDD pharmacologic clinical trials were identified. These trials incorporated a broad array of outcome measures, which were used inconsistently across trials. We summarize the psychometric properties and other relevant data on outcome measures used, including their diagnostic utility, inter-rater reliability, test-retest reliability, responsiveness, clinically meaningful change, and availability of alternate forms.

Conclusions

We have identified the best-evidenced PDD outcome measures in each domain. Further research is needed to assess the validity, reliability, and clinically meaningful change of these measures in PDD to inform the design of future clinical trials and enhance the ability of clinicians, researchers and policy-makers to interpret study results. In addition, the development of outcome measures specific to PDD may be warranted.

Introduction

Parkinson’s disease (PD) is the second most common neurodegenerative illness after Alzheimer’s disease (AD), affecting over 5 million persons worldwide with an anticipated doubling of prevalence in the next 20 years.¹ Parkinson’s disease dementia (PDD) is a major cause of morbidity and mortality in PD affecting function, quality of life and the risk for nursing home admission.² PDD has a point prevalence of approximately 31% and a mean time from onset of PD to diagnosis of dementia of 11 years.³ For patients surviving 20 years or more, the chances of developing PDD may be as high as 75%.³

While several effective therapies exist for PD motor symptoms, there are few treatment options available for PDD and they are of marginal clinical benefit. Currently, the only FDA approved treatment for PDD is the cholinesterase inhibitor rivastigmine. A recent Cochrane review supports the use of other cholinesterase inhibitors for PDD, but acknowledges their limited clinical efficacy and the need for more effective treatments.⁴ The NMDA receptor antagonist memantine may also be beneficial in PDD; however, a large randomized controlled trial (RCT) demonstrated its effectiveness only for dementia with Lewy Bodies (DLB) and not PDD.⁵ Clearly, there is a need to develop and investigate more effective therapies for PDD.

Two critical issues in designing randomized control trials for PDD are defining the condition and choosing appropriate outcome measures. In regards to the former, a Movement Disorder Society Task Force (MDS-TF) developed clinical diagnostic criteria for PDD, with two levels of testing: level I, an abbreviated assessment for clinical use and level II, more comprehensive testing intended for research studies and pharmacological trials.^{6, 7} Since their publication in 2007, the MDS-TF diagnostic criteria have been shown to be more useful and accurate than the prior DSM-IV standard.⁸ Work continues in the application and validation of these criteria, with results indicating good agreement between MDS-TF diagnostic criteria and more exhaustive clinical and neuropsychological testing.⁹

Concerning outcome measures, there are few empiric studies examining their performance in PDD and these are limited in terms of scales studied and psychometric properties assessed.¹⁰ One of the more challenging aspects of assessing outcomes is defining clinically meaningful change (CMC), as small but statistically significant improvements in test scores in RCTs often do not translate into any perceived benefit on the part of patients, caregivers or clinicians.¹¹ An ideal outcome measure would be psychometrically valid and reliable in a PDD population, be responsive to interventions, have well-defined clinically meaningful change, and contribute to the diagnosis of PDD. While change in cognitive test scores are clearly important, these may need to be supplemented with measures of function, quality of life and/or caregiver burden depending on study goals.

In this paper, we review the outcome measures that have been used to date in PDD clinical trials. To evaluate the reliability and validity of the utilized measures, we summarize the data on their psychometric properties as evaluated in dementia populations. We conclude by highlighting areas where further research is needed.

Methods

Using PubMed, a literature review of clinical trials in patients with PDD from January 1965 to February 2015 was performed, using combinations of the following search terms: “Parkinson’s disease dementia”; “treatment”; “intervention”; “therapy” and “clinical trial”. The outcome measures utilized in the clinical trials were then classified into five domains: 1) cognitive; 2) behavioral and mood; 3) activities of daily living and quality of life; 4) global; and 5) caregiver burden. For all outcome measures, we used PubMed to perform an additional search of empiric studies assessing their validity and reliability, using the search terms: “clinical significance”, “clinically meaningful change”, “validity”, “validation”, “reliability” and “sensitivity to change”. We included studies of participants with PDD, or AD, PD psychosis and/or cognitive impairment (CI) when PDD studies were not available.

The statistical properties evaluated for each outcome measure include diagnostic utility, inter-rater reliability, test-retest reliability, responsiveness and clinically meaningful change values. Diagnostic utility was defined as the sensitivity and specificity of an outcome measure for diagnosis of PDD. If unavailable, concurrent validity for the measure with validated tests is reported. Inter-rater reliability is the degree of concordance between multiple raters, usually defined as Cohen’s kappa. Test-retest reliability refers to the consistency of results in the same subject over a short time interval.

In a purely statistically sense, sensitivity to change refers to the magnitude of change that has occurred over time as compared to a gold standard. This is slightly different than the related concept of responsiveness, which is the ability of a scale to detect meaningful changes over time, though the two terms are often used interchangeably in clinical research. To be as accurate as possible, we will use the term responsiveness throughout this review. As the true drug effect is unknown and cannot be evaluated for a single measure in isolation, we looked at the effects of the intervention for multiple measures. If the outcome of interest improved in line with other outcomes, we rated it as responsive and if it failed to improve when other outcomes improved (or had significantly smaller effect size) we rated it as unresponsive. Responsiveness to disease progression was also looked at when data was available as a measure of responsiveness. Clinically meaningful change (CMC) is usually defined in terms of the minimal clinically important difference, defined as “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side-effects and excessive cost, a change in the patient’s management”.¹² The presence of alternate forms of the outcome measure and their susceptibility to practice effect are also reported when relevant.

Based on this data, we make endorsements regarding the use of the studied outcome measures in future trials by the following criteria: (1) “recommended”: the outcome measure has been applied to PDD patients and has been found valid, reliable and responsive in psychometric studies in studies of PDD or other dementias; (2) “suggested”: the outcome measure has been applied to PDD patients and psychometric studies are promising (“moderate” to “high” parameters), but not yet complete in dementia populations and/or some psychometric properties are suboptimal (“low”); (3) “listed”: the outcome measure has been applied to PDD patients but psychometric studies have demonstrated disappointing data (“low” parameters) and/or have not yet been performed in dementia populations. These criteria were modeled after similar reviews of scales in Parkinson’s disease and the appendix of ancillary scales to complement the MDS-UPDRS (Unified Parkinson Disease Rating Scale) from the MDS.^13,14

Results

A total of twenty clinical trials evaluating the efficacy of interventions in PDD were identified.^{5, 15–33} All of these trials assessed pharmacologic treatments. Twenty-three unique outcome measures were utilized within the identified treatment trials (Tables 1 and 2).

Table 1.

Cognitive Outcome Measures

OUTCOME MEASURE	REFERENCE FOR STUDIES USED	DIAGNOSTIC UTILITY	INTER- RATER RELIABILITY	TEST- RETEST RELIABILITY	RESPONSIVENESS	CLINICALLY MEANINGFUL CHANGE	ENDORSEMENT
ADAS-cog	25–27, 28, 33, 34, 37 & 38	Not Established	High	Moderate to high	Yes	4 points	Recommended
MDRS	23, 31, 32 & 38	High	High	High	Yes	Not established	Recommended
MoCA	Not used	Moderate	High	High	Yes	Not established	Suggested
CDR	25, 27 & 39	Not established	Not established	Moderate	Yes	Not established	Suggested
FAB	33 & 34	Moderate	High	High	Not established	Not established	Suggested
D-KEFS VF	25–27, 34 & 37	Low to moderate	Not established	Moderate to high	Not established	Not established	Suggested
HVLT	31	High	Not established	Not established	Not established	Not established	Suggested
TPCT	23, 27 & 33	High	Not established	Not established	Not established	Not established	Suggested
MMSE	22–28, 31–35, 37 & 38	Low to Moderate	High	High	Low for small changes	1.4 points	Listed
QTCS	22	Not established	Not established	Not established	Not established	Not established	Listed
VMI	31	Not established	Not established	Not established	Not established	Not established	Listed
BTA	26 & 31	Not established	Not established	Not established	Not established	Not established	Listed

Endorsement definitions:

1. “Recommended”: the outcome measure has been applied to PDD patients and has been found valid, reliable and responsive in psychometric studies in studies of PDD or other dementias
2. “Suggested”: the outcome measure has been applied to PDD patients and psychometric studies are promising (“moderate” to “high” parameters), but not yet complete in dementia populations and/or some psychometric properties are suboptimal (“low”)
3. “Listed”: the outcome measure has been applied to PDD patients but psychometric studies have demonstrated disappointing data (“low” parameters) and/or have not yet been performed in dementia populations

Abbreviations: MMSE: Mini-Mental Status Examination; ADAS-cog: Alzheimer’s Disease Assessment Scale – Cognitive Subscale; MDRS: Mattis Dementia Rating Scale; MoCA: Montreal Cognitive Assessment; CDR: Cognitive Drug Research system; FAB: Frontal Assessment Battery; D-KEFS VF: Delis-Kaplan Executive Function System Verbal Fluency test; TPCT: Ten Point Clock Drawing Test; QTCS: Quick Test of Cognitive Speed; HVLT: Hopkins Verbal Learning Test Revised; VMI: Developmental Test of Visual-Motor Integration; BTA: Brief Test of Attention

Table 2.

Other Outcome Measures

OUTCOME MEASURE	REFERENCE FOR STUDIES USED	INTER-RATER RELIABILITY	TEST-RETEST RELIABILITY	SENSITIVITY TO CHANGE	CLINICALLY MEANINGFUL CHANGE	ENDORSEMENT
Mood/Behavioral
NPI	6, 22–27, 31–34 & 37	High	Moderate to high	Yes	8 points	Recommended
CSDD	31	High	High	Yes	Not established for dementia	Recommended
BPRS	38	Moderate	Not established for dementia	Not established for dementia	Not established for dementia	Suggested
Activities of Daily Living and Quality of Life
ADCS-ADL	6, 25, 27, 36 & 37	High	High	Yes	2 points	Recommended
QOL-AD	7	High	High	Yes	Not established for dementia	Recommended
DAD	22, 26, 33 & 34	High	High	Yes	Not established for dementia	Recommended
SE-ADL	26	Moderate	Not established for dementia	Not established for dementia	Not established for dementia	Suggested
PDQ-39	Not used	Not established for dementia	Moderate	Not established for dementia	Depends on subsection, but has been established	Suggested
UPDRS-ADL	31 & 38	Not established for dementia	Not established for dementia	Not established for dementia	Not established for dementia	Listed
Global
CGI	22, 29, 35 & 38	Moderate	Moderate	Yes	Not established for dementia	Recommended
ADCS-CGIC	6, 25 & 27	Not established for dementia	High	Yes	1–2 points	Suggested
CIBIC+	24, 26 & 32	Low	Moderate	Yes	Not established for dementia	Suggested
Caregiver Burden
ZBI	6 & 30	Not established for dementia	High	Yes	13 points	Suggested

Endorsement definitions:

1. “Recommended”: the outcome measure has been applied to PDD patients and has been found valid, reliable and responsive in psychometric studies in studies of PDD or other dementias
2. “Suggested”: the outcome measure has been applied to PDD patients and psychometric studies are promising (“moderate” to “high” parameters), but not yet complete in dementia populations and/or some psychometric properties are suboptimal (“low”)
3. “Listed”: the outcome measure has been applied to PDD patients but psychometric studies have demonstrated disappointing data (“low” parameters) and/or have not yet been performed in dementia populations

Abbreviations: NPI: Neuropsychiatric Inventory; BPRS: Brief Psychiatric Rating Scale: CSDD: Cornell Scale for Depression in Dementia; ADCS-ADL: Alzheimer’s Disease Cooperative Study-Activities of Daily Living Scale; QOL-AD: Quality of Life in Alzheimer’s Disease Scale; DAD: Disability Assessment for Dementia; SE-ADL: Schwab and England ADL scale; PDQ-39: Parkinson’s Disease Questionnaire-39; UPDRS-ADL: Unified Parkinson Disease Rating Scale-Activities of Daily Living; CGI: Clinical Global Impression scale; ADCS-CGIC: Alzheimer’s Disease Cooperative Study – Clinician’s Global Impression of Change; CIBIC+: Clinician Interview Based Impression of Change with caregiver input; ZBI: Zarit Burden Interview-Caregiver Burden Assessment

Cognitive Outcome Measures

Cognitive scales were the most common (11 of 23) type of outcome measure identified (Table 1). The MMSE is one of the oldest cognitive screening tools available and is the most frequently employed cognitive outcome measure in PDD trials, used in 14 studies^{22–28,31–35,37,38}, despite validity concerns in PDD.^17,34 Though validated for assessing dementia severity¹⁰, its sensitivity and specificity for diagnosing PDD are sub-optimal (80% and 74%, respectively) when tested against the most recent MDS-TF clinical diagnostic criteria.³⁵ Moreover, the MMSE may have floor effects in subjects with severe dementia and ceiling effects in subjects with mild cognitive impairment³⁶, and lacks adequate testing of executive functions.³⁷ In PDD, the MMSE demonstrates moderate concurrent validity with the ADAS-cog (r=−0.6)¹⁰ and low concurrent validity with the MDRS (r=0.29).³⁸ Though not established in PDD, the MMSE demonstrates high inter-rater reliability among cognitively-impaired elderly individuals (k=0.82–0.91)³⁹ and high test-retest reliability in AD (r=0.87 and 0.89).^40,41 It lacks responsiveness to small changes, especially in patients with early PDD.³⁰ In the Donepezil and Memantine in Moderate to Severe Alzheimer’s Disease (DOMINO) trial, CMC was defined as 1.4 points on the MMSE, based on triangulation of expert opinion and distribution-based values.⁴² CMC has not been established for PDD. Although short-term practice effects have been reported in AD¹⁶, two versions of the MMSE are available for repeat testing.⁴³

The ADAS-cog was utilized in eight studies.^{25–27,28,33,34,37,38} Though originally designed for AD, it is commonly used in PDD.³² ADAS-cog scores demonstrate statistically significant differences between PDD patients of mild and moderate severity of CI, but its utility in diagnosis of PDD remains untested.¹⁰ In PDD, it demonstrates moderate concurrent validity with the MMSE (r=−0.60).¹⁰ The inter-rater reliability for PDD is not established, but it demonstrates high inter-rater reliability in AD (k=0.91).⁴⁴ Test-retest reliability is moderate in PDD of both mild (r=0.65) and moderate disease severity (r=0.71).¹⁰ The ADAS-cog is responsive in both PDD and AD, as determined by analysis of data from three large rivastigmine treatment trials.⁴⁵ CMC was defined as 4 points at 6 months for the ADAS-cog in AD, though this is only meaningful for groups and not individuals.⁴⁶ Although learning effects have been reported on this exam in PDD³⁰, alternate forms are available.

The Mattis Dementia Rating Scale⁴⁷ (MDRS) was used in four studies.^{17, 25, 26, 32} The MDRS has been validated for PDD diagnosis, with a total cutoff score of 123 yielding high sensitivity (93%) and specificity (91%).⁴⁸ However, this study has been criticized for using a sample with a low mean educational achievement (8.9 years). A subsequent study, using subjects with a mean education of 14.8 years, suggests this cutoff score is inadequate for detecting PDD, correctly classifying only 60.7% of the sample, with high specificity (100%) but very low sensitivity (20%).⁴⁹ This study suggests a total cutoff score of 133 instead.⁴⁹ Overall, the MDRS has been shown to be superior to the MMSE in assessing cognition⁵⁰ and particularly in detecting frontal lobe dysfunction in PDD.¹¹ While not established for PDD, studies in AD show high inter-rater reliability (k=0.93)⁵¹ and high test-retest reliability (r=0.97).⁵² The MDRS is generally responsive to change in PDD trials, with the exception of one donepezil RCT in which it did not reveal change despite improvements in other measures.¹⁸ CMC has not been established. An alternate form is available.^53,54

The Cognitive Drug Research (CDR) System, a computer-based set of cognitive tests, was used in three trials.^{19, 21, 33} Diagnostic utility of the CDR system in PDD is not established, nor is inter-rater reliability, though the computerized nature of the test mitigates inter-rater variability. For all-cause dementia, the CDR demonstrated moderate concurrent validity with the MMSE (r=0.5–0.65).⁵⁵ The CDR-Power of Attention test demonstrated moderate test-retest reliability in PDD patients with mild dementia (r=0.63) and moderate dementia (r=0.46).¹⁰ The CDR system demonstrated responsiveness in PDD pharmacologic trials.⁵⁶ CMC is not established in PDD or AD. Over 20 alternate forms of the CDR are available.⁵⁷

The Frontal Assessment Battery⁵⁸ (FAB) was used in two studies.^{27, 28} The FAB is not reliable as a sole test for diagnosing PDD (sensitivity 66% and specificity 72%⁵⁹).¹¹ In PD and atypical parkinsonian syndromes, the FAB demonstrated high concurrent validity with the MDRS (r=0.82) and high inter-rater reliability (k=0.87).⁵⁸ For AD, it shows high test-retest reliability (r=0.82) and inter-rater reliability (k=0.98).⁶⁰ The FAB demonstrated responsiveness in both trials in which it was used. CMC is not established and there are no alternate forms.

Data on the psychometric properties of the following outcome measures for PDD is currently lacking: the Delis-Kaplan Executive Function System-Verbal Fluency Test⁶¹ (D-KEFS), the Hopkins Verbal Learning Test Revised (HVLT), and the Ten Point Clock Drawing Test⁶² (TPCT). The D-KEFS shows high test-retest reliability for PDD patients with mild disease severity (r=0.79) and moderate test-retest reliability for moderate disease severity (r=0.55).¹⁰ The HVLT demonstrates high sensitivity (83%) and specificity (83%) for distinguishing all dementias from controls using a cutoff score of 16.⁶³ There are 6 alternate forms of the HVLT.⁶⁴ The TPCT shows high concurrent validity (r=0.71–0.73) with the MMSE in AD⁶⁵ and a score of less than 8/10 identified 71% of patients with mild AD.⁶⁶ CMC has not been established for any of these measures. No data on the psychometric properties of the following outcome measures are available: A Quick Test of Cognitive Speed,⁶⁷ the Developmental Test of Visual-Motor Integration (VMI), and the Brief Test of Attention.

While the Montreal Cognitive Assessment⁶⁸ (MoCA) has not yet been used in any trials, it is commonly used clinically and has adequate psychometric properties as a screening tool for PDD.⁶⁹ The optimal diagnostic cutoff score for the MoCA in PDD is < 21/30, yielding high sensitivity (81%) and specificity (95%).^{69, 70} In AD and frontotemporal dementia, the MoCA shows high concurrent validity with the MMSE (r=0.82).⁷¹ It demonstrates high inter-rater reliability (k=0.81) and high test-retest reliability (r=0.79) in PD of varying cognitive involvement.⁷² In a three-year longitudinal study of PD patients with varying levels of cognition, no significant change on the MoCA was found, even when subjects were stratified by age, MMSE score, and disease duration, suggesting that the MoCA lacks responsiveness.⁷³ However, in a recent trial of early AD patients, the MoCA was shown to be capable of detecting small to moderate cognitive change over time.⁷⁴ CMC for this measure is not established. Three alternate English versions are available.

Mood/Behavioral Outcome Measures

Mood and behavioral outcome measures are frequently employed in PDD trials (Table 2), with the Neuropsychiatric Inventory⁷⁵ (NPI) being the most commonly utilized, in 12 studies.^{6,22–27,31–34,37} The NPI dysphoria sub-score demonstrated moderate concurrent validity (r=0.62) with the Hamilton Rating Scale for Depression when administered to caregivers of patients with all-cause dementia.⁷⁵ This study also showed high inter-rater reliability for each subscale of the NPI (k=0.89–1.0).⁷⁵ The NPI shows moderate test-retest reliability for PDD of mild severity (r=0.66) and moderate severity (r=0.73).¹⁰ The NPI is commonly used in PD psychosis studies, with some antipsychotic treatment trials suggesting that this outcome measure may not be adequately responsive in PD populations.^76–78 However, the NPI-2, being the sum of scores for the delusions and hallucinations subscales, did reveal statistically significant changes in one trial of quetiapine.⁷⁶ Although CMC has not been established for PDD, the statistical analysis plan from the DOMINO trial (AD) determined that an eight point change in the NPI was clinically meaningful, taking into account both expert opinion and distribution-based values.⁴²

The Brief Psychiatric Rating Scale⁷⁹ (BPRS) was used in one dementia treatment trial.³² The Brazilian version of the BPRS shows high concurrent validity (r=0.73) of the delusion subscale and moderate concurrent validity (r=0.43) of the hallucination subscale with the NPI.⁸⁰ Although not established for PDD, the inter-rater reliability of the BPRS for AD varies significantly (range=0.13–1.0, median=0.45).⁵⁹ Test-retest reliability for the BPRS has not been established. The responsiveness of the BPRS has been validated in patients with mental illness⁸¹ and PD psychosis.⁷⁶ Although not established in PDD, the CMC for the BPRS in a clinical trial for PD psychosis was suggested as a change of 25%; no explanation was provided on how this value was chosen.⁸²

The Cornell Scale for Depression in Dementia⁸³ (CSDD) was used in one trial.²⁵ The CSDD is based on observation of both the patient and informant and is validated for diagnosing depression in patients with PD of varying cognitive impairment.⁸⁴ A cutoff score of ≥8 yields sensitivity of 75% and specificity of 82% for diagnosis of depression in PD.⁸⁴ The Korean version of the CSDD in AD showed high concurrent validity with the Hamilton Depression Rating Scale (r=0.91) and the Geriatric Depression Scale (r=0.75).⁸⁵ Although data is not available in PDD, the CSDD has shown high inter-rater reliability in an elderly cohort with various dementias (k=0.84)⁸⁶ and high test-retest reliability in AD (r=0.91)⁸⁵. CMC has not been established.

Activities of Daily Living and Quality of Life Outcome Measures

We identified five activities of daily living (ADL) and quality of life (QOL) measures (Table 2), with the Alzheimer’s Disease Cooperative Study-Activities of Daily Living Scale (ADCS-ADL) used most frequently in five trials.^{5, 19, 21, 30, 31} The ADCS-ADL demonstrates moderate concurrent validity with the ADAS-cog for PDD (r=−0.47).¹⁰ Individual components of the ADCS-ADL show moderate concurrent validity with the MMSE in an AD population (r=0.4–0.7).⁸⁷ High inter-rater reliability of the ADCS-ADL has been noted,⁸⁸ but we did not find any empiric study assessing this property. The ADCS-ADL shows high test-retest reliability for PDD patients with mild (r=0.94) and moderate disease severity (r=0.92).¹⁰ The ADCS-ADL demonstrated responsiveness in detecting a treatment effect in 4 of 5 trials in which it was employed.^{19, 21, 30, 31} The ADCS-ADL is sensitive to disease progression in AD, but this property has not been established for PDD.⁸⁷ It is suggested that a change of 2 points is clinically meaningful, i.e. a loss of dressing or bathing independently, but CMC has not been examined psychometrically.⁸⁸ A modified version of the ADCS-ADL, which includes a subset of 19 items, is intended for patients with moderate to severe dementia.

The Disability Assessment for Dementia (DAD) scale was used in four trials^{16, 20, 27, 28} and is a subscale of the Clinician’s Interview Based Impression of Change scale (CIBIC).^27,28 In AD, the DAD shows high concurrent validity with the Rapid Disability Rating Scale-2 (r=−0.85)⁸⁹ and with the Japanese version of the Clinician Interview Based Impression of Change with caregiver input (CIBIC+) (r=0.91).⁵⁰ The DAD demonstrates high test-retest (r=0.96) and inter-rater reliability (k=0.95) in AD, but has not been established for PDD.⁸⁹ The DAD demonstrated responsiveness in 2 of 4 trials in which it was utilized.^27,28 One study suggests the DAD is not sensitive to AD progression, but this property has not been assessed for PDD.⁹⁰ CMC has not been established for the DAD.

The Unified Parkinson Disease Rating Scale-ADL Scale (UPDRS-ADL) is not specific to dementia but was included as an outcome measure in two trials.^25,32 This measure shows moderate sensitivity (67%) and specificity (79%) in discriminating PDD from PD-normal cognition when using a cutoff of 15.5.⁹¹ The UPDRS-ADL demonstrates high inter-rater reliability (k≥0.80)⁹² in PD patients with moderate disease severity⁸ and high test-retest reliability (r=0.85) in early stage PD patients⁹³, though not established for PDD. The UPDRS-ADL did not demonstrate responsiveness in detecting a treatment effect in either of the two trials in which it was utilized. The UPDRS-ADL has shown responsiveness in disease progression for PD patients, but this property has not been evaluated in PDD.⁹⁴ One study suggests 4 points represents CMC for the UPDRS-ADL.⁹³

The Schwab and England-ADL (SE-ADL) scale was used as a secondary outcome measure in one trial.²⁰ The SE-ADL demonstrates moderate sensitivity (71%) and specificity (77%) in detecting PDD when using a cutoff of 75.⁹¹ A study non-specific to dementia suggests the scale has moderate inter-rater reliability between physicians, patients, and caregivers (k=0.6) and slightly higher inter-rater reliability between physicians and patients (k=0.65).⁵⁶ A study of PD patients that excluded dementia suggests the SE-ADL has moderate test-retest reliability (r=0.70).⁹⁵ SE-ADL demonstrates responsiveness for disease progression of early PD, but this property has not been assessed specifically for PDD.⁹⁶ In the trial in which it was used, the SE-ADL did not detect a treatment effect.²⁰ One study suggests a score of 12.33 represents CMC for PD patients.⁹⁵

The Quality of Life in Alzheimer’s Disease Scale (QOL-AD) was used in secondary analysis of one clinical trial.¹⁵ For AD, the QOL-AD shows moderate concurrent validity with the Dementia Quality of Life scale (DQOL) (r=0.69) and the Euroqol-5D scale (r=0.54).⁹⁷ Inter-rater reliability for PDD is not established, but the QOL-AD demonstrates moderate inter-rater reliability (k>0.7) in AD.⁹⁷ The Chinese version of the QOL-AD shows high test-retest reliability for patient and caregiver reports (r=0.84 and 0.90 respectively) in an AD population.⁹⁸ The QOL-AD demonstrated responsiveness in detecting a treatment effect in the trial in which it was used.¹⁵ CMC for the QOL-AD has not been established.

Although the Parkinson’s Disease Questionnaire⁹⁹ (PDQ-39) is a commonly used outcome measure in PD, it has not been used to date in PDD trials. The PDQ-8, a short form of the PDQ-39, was used in one trial.²⁴ In a PD population, the PDQ-39 shows moderate concurrent validity with the Beck depression and anxiety inventories (r=0.73) and the Royal Postgraduate Medical School severity scale (r=0.66)¹⁰⁰ Information on the inter-rater reliability of the PDQ-39 is not available for PDD. The PDQ-39 demonstrates moderate to high test-retest reliability in a PD population (r=0.68–0.94)¹⁰¹ and the PDQ-8 demonstrates high test-retest reliability as well (r=0.82).⁹⁵ One study suggests the PDQ-39 is responsive in PD patients.¹⁰⁰ The PDQ-39 may be insensitive to detecting change in early PD progression.¹⁰² Minimally clinically important difference values have been determined for each subsection of the PDQ-39 in PD without dementia (Mobility: −1.5, −3.2; ADL: −0.7, −4.4; Emotional well-being: 0.3, −4.2; Stigma: 0.8, −5.6; Social support: −1.2, −11.4; Cognition: 0.4, −1.8; Communication: −0.8, −4.2; Bodily discomfort:1.3, −2.1).¹⁰¹

Global Outcome Measures

Three global measures were used in the reviewed trials (Table 2). The Alzheimer’s Disease Cooperative Study–Clinician’s Global Impression of Change (ADCS-CGIC) was used as a primary outcome measure in two trials^{19, 21} and in another as a secondary outcome measure.⁵ In AD, the ADCS-CGIC demonstrated low concurrent validity with the Global Deterioration Scale (GDS) (r=0.15) and the Clinical Dementia Rating Scale (CDR) (r=0.15), and no significant correlation with the Functional Assessment Staging scale (FAST).¹⁰³ The inter-rater reliability of the ADCS-CGIC has not been established. The ADCS-CGIC demonstrated high short-term test-retest reliability, with 90–94% of subjects rated as having not changed or only minimally changed using the scale between month 1 and 2.¹⁰³ The ADCS-CGIC demonstrated responsiveness in detecting a treatment effect in two of the three trials in which it was used.^{19, 21} A change in score of 1 or 2 points on a 7-point scale from the ADCS-CGIC indicates CMC for AD patients.²¹

The Clinician Interview Based Impression of Change with caregiver input (CIBIC+), an alternate version of the CIBIC non-specific to dementia including caregiver input, was used in three trials.^{18, 20, 26} The Japanese version of the CIBIC+ demonstrated high concurrent validity in AD with the DAD (r=0.91) and the Mental Function Impairment scale (MENFIS) (r=0.99).⁵⁰ This study also showed low inter-rater reliability (k=0.45), but improved inter-rater reliability when collapsed into a 3-point scale (improved, no change, or worsened; k=0.89).¹⁰⁴ For AD, the test-retest reliability of the CIBIC+ is moderate (range=0.4–0.6).¹⁰⁵ The CIBIC+ demonstrated responsiveness in detecting a treatment effect in two trials^{18, 20} and a trend toward improvement in another.²⁶ CMC is intrinsic to the scale, which may be used to rate patient, caregiver and clinicians impressions of change and has been validated in AD.¹⁰³

The Clinical Global Impression scale (CGI) was used in four trials.^16,23,29,32 The CGI, which is not specific to dementia, consists of three items: the CGI-Severity scale, CGI-Improvement scale, and CGI-Efficacy index. In patients with depression and panic disorder, the CGI-Severity scale showed moderate concurrent validity (r=0.44–0.65) and the CGI-Improvement scale showed low concurrent validity (r=0.26–0.46) with the Hamilton Depression Rating Scale.¹⁰⁶ For patients with a diagnosis of dementia, the CGI-Improvement subscale demonstrates moderate inter-rater reliability ((k=0.51), while the inter-rater reliability of the CGI-Severity subscale was slightly higher (k=0.66).¹⁰⁷ The test-retest reliability of the CGI-Severity is moderate (r=0.65).¹⁰⁷ The CGI demonstrated responsiveness in detecting a statistically significant treatment effect in all four trials in which it was used.^{16,23, 29,32} CMC is intrinsic to the scale which may be used to rate patient, caregiver and clinicians impressions of change, but has not been validated in dementia.

Caregiver Burden

Only one caregiver burden outcome measure (Table 2) has been used in PDD trials to date, the Zarit Burden Interview-Caregiver Burden Assessment¹⁰⁸ (ZBI), which was used in two studies.^{5, 24} The ZBI demonstrates moderate to high concurrent validity with other measures of caregiver burden: the Burden Assessment Scale (BAS) (r=0.73) and the General Health Questionnaire (GHQ-28) (r=0.62).¹⁰⁹ Inter-rater reliability has not been established for the ZBI in PDD. Among caregivers of AD patients, the ZBI shows high test-retest reliability (r=0.89).¹⁰⁹ The responsiveness of the ZBI in PDD is questionable, as it detected a treatment effect in one trial²⁴, but not in the other, though this study also did not show improvement in most cognitive test scores or the ADSC-ADL.⁵ The responsiveness of the ZBI to specific interventions for caregivers of dementia patients has been demonstrated.¹¹⁰ For CMC, a study of caregivers of PDD patients suggests that a ZBI score of 13 or higher in caregivers of dementia patients represents a clinically significant burden.¹¹¹

Discussion

Twenty clinical trials evaluating the efficacy of pharmacologic interventions for PDD were identified in this systematic review. Overall, the trials inconsistently employed a wide variety of outcome measures. Additionally, many of these trials use outcome measures that have not been adequately assessed for reliability or validity in PDD, though many have data in other forms of dementia. In Tables 1 and 2, we have listed endorsements of the 23 identified measures based on the available psychometric data.

In the assessment of treatments for Parkinson’s disease dementia, cognitive outcome measures are of the utmost importance, both for screening of individuals for trial participation and for evaluation of the efficacy of pharmacologic interventions. As almost all of these scales were developed for use in AD, it is important to pursue validation studies in PDD, particularly because the pattern of cognitive deficits observed in PDD differs from that of AD, with more executive and visuopatial dysfunction in PDD, as opposed to the more severe episodic memory impairment in AD. Cognitive measures that are biased more towards memory tasks may not be appropriate for use in PDD. The ADAS-Cog and MDRS both received “recommended” ratings. The MMSE is the most frequently utilized scale in existent PDD treatment trials, yet received a “listed” endorsement based on the available psychometric data. Further work must be done to establish inter-rater and test-retest reliability for all of these measures specifically in PDD. Future treatment trials may also consider including the MoCA in place of the MMSE, as this scale gains traction in both clinical and research settings for PD, but further studies of responsiveness in PDD are needed.

While cognitive outcomes have traditionally been the target of dementia treatments, non-cognitive outcomes, and particularly neuropsychiatric symptoms, may have greater functional impact on patients, caregivers and risk for nursing home placement and are considered a clinical indication for currently available agents.¹¹² Of the mood and behavioral outcome measures, the NPI and CSDD are the best-evidenced scales available in dementia, receiving “recommended” endorsements. An MDS-TF also designated the NPI a “recommended” scale in PD.¹¹³

Few ADL and QOL measures have been validated for use in PDD, though there is more data in other dementias. The MDS-TF diagnostic criteria for PDD require demonstrable impairment in activities of daily living due to cognitive deficits, yet there is no validated scale to assess the functional impact of cognitive impairment in PD.⁶ In addition, current ADL scales may be subject to the influence of confounding variables such as motor worsening and major depression⁹¹, and studies have demonstrated significant differences between patient and caregiver subjective reporting of functional disability and objective performance ratings.¹¹⁴ Performance-based measures of functionality would be difficult to incorporate into treatment trials due to equipment and time constraints, but it would be helpful to validate subjective report-based ADL scales against more objective measures. Regardless, two ADL scales received “recommended” endorsements, the ADCS-ADL and the DAD. QOL measures are rarely used in PDD trials, with only one trial employing a quality of life outcome measure, being the QOL-AD, which received a “recommended” endorsement. This is a significant gap in the field, as the overarching goal of all disease treatment should be improvement in a patient’s functionality and quality of life. The PDQ-39, which addresses both ADLs and QOL, could be a useful outcome measure in PDD treatment trials, should further study prove it valid and reliable in this population.

Global impression scales are useful in capturing minor treatment effects that may be missed by less comprehensive measures. However, since global measures are largely subjective, they tend to have relatively low inter-rater reliability. None of the utilized global scales have complete psychometric data in a PDD population, but the CGI does have adequate data to receive a “recommended” endorsement.

Only two trials incorporated a measure of caregiver burden, being the ZBI. The ZBI is a promising measure in this domain given that it is a dementia-specific measure, though its psychometric properties have not yet been adequately evaluated, leading to a “suggested” endorsement. Regardless, we suggest that future PDD trials include the ZBI, as caregiver burden is substantial in dementia and is an important predictor of patient institutionalization.

None of the outcome measures used in treatment trials demonstrate complete psychometric data on validity, reliability and responsiveness in PDD, though several have complete data in dementia. Further work must be done to obtain such data in PDD. In the design of future PDD trials, we stress the importance of using dementia-specific scales that have been validated in PDD populations. Moreover, future trials should consider prioritizing outcome measures relevant to functionality and quality of life as related to cognition for PDD patients, namely measures of cognitive functional ability. Purely cognitive measures are frequently used as primary endpoints in PDD clinical trials, yet even statistically significant improvements on such neuropsychological tests are of debatable clinical significance. As a result, many experts and even health care systems (e.g. United Kingdom¹¹⁵) do not agree on the relevance of the findings of these treatment trials. From a patient-centered approach, it can be argued that the effectiveness of potential treatments for cognitive impairment should be evaluated in terms of their effect on functionality, independence and quality of life, rather than on improvements in neuropsychological testing scores. However, there is no validated diagnostic procedure currently available for establishing functional impairment related to cognition, which recent^{116, 117} and future studies seek to rectify.

Conclusion

Due to the lack of empiric studies assessing the psychometric properties of dementia outcome measures, further research is needed to assess their validity and reliability in PDD populations. We propose that the outcome measures we have identified as “recommended” receive top priority for rigorous empiric validation in PDD. We also suggest that other measures commonly used in clinical practice be examined and considered for use as outcome measures in treatment trials, including the MoCA and PDQ-39. Finally, we raise the question of shifting the focus of PDD treatment trials away from neuropsychological testing scores and towards measures of functionality and quality of life as related to cognition. By doing so, we could ensure potential future treatments of PDD lead to meaningful clinical and personal improvements for patients and caregivers.