Psychometric validation of the Perceived Deficits Questionnaire-Depression (PDQ-D) instrument in US and UK respondents with major depressive disorder

Abstract

Background

Although depression and cognitive dysfunction are connected, limited tools exist to capture the patient’s perspective on cognitive dysfunction and its impact on major depressive disorder (MDD). We report results of a psychometric validation of the Perceived Deficits Questionnaire-Depression (PDQ-D), a self-report measure of cognitive dysfunction for use in MDD.

Methods

A non-interventional, prospective, panel-recruited, online survey was conducted using the PDQ-D in adults with and without MDD in the US and UK. Respondents were assessed at baseline and after 6 weeks (MDD only) (baseline: US n=418, UK n=437, 49% MDD; follow-up: US n=169, UK n=153, all MDD). The criterion measures included: Medical Outcomes Study Cognitive Functioning Scale-Revised-acute form (MOS COG-R), Patient Health Questionnaire-9 (PHQ-9), Patient Global Impression of Severity scale (PGI-Severity), Sheehan Disability Scale (SDS), Work Productivity and Activity Impairment Questionnaire: Specific-Health Problem (WPAI:SHP), and modified Lam Employment Absence and Productivity Scale (LEAPS). US and UK data were analyzed separately.

Results

Internal consistency was high for PDQ-D total scale and four subscales (Cronbach’s alpha 0.81–0.96). Convergent validity was good, with strong concordance with MOS COG-R and moderate/small correlations with PHQ-9, SDS, WPAI:SHP, LEAPS, and PGI-Severity. Significant differences (all P<0.001) existed for all PDQ-D subscale and total scores between MDD/non-MDD samples. The PDQ-D was responsive to changes in depression symptom severity. Confirmatory factor analysis supported scoring of a global overall scale for perceived cognitive dysfunction.

Conclusion

The PDQ-D provides a reliable and valid measure of subjective cognitive dysfunction in patients with MDD.

Introduction

MDD is a common, chronic, and recurrent disorder characterized by one or more major depressive episodes.¹ In 2010, the global point prevalence for MDD was estimated at 4.4% (close to 300 million cases).² MDD was also a leading cause of disability-adjusted life-years and the second leading cause of years lived with disability globally.³ Recent estimates of 12-month prevalence rates for MDD were 6.9% in both Europe and the US.^4,5

In addition to MDD having a significant negative impact on work, social relationships, and health-related quality of life,⁶ patients with MDD also exhibit a reduction in cognitive functioning domains, including attention, psychomotor speed, memory, and executive function, relative to healthy controls,^7–9 and a relationship between MDD and cognitive impairment has been recognized for many years.¹⁰ Using the Central Nervous System Vital Signs assessment battery, evidence of cognitive dysfunction has been reported in 21%–31% of patients with depression compared with just 4%–5% of healthy controls.^7,11 Furthermore, although not the only aspect of cognition, the DSM (fifth edition) diagnostic criteria for MDD include diminished concentration as a symptom of MDD.¹ Deficits in several cognitive subscales have also been reported in patients with MDD, including executive function,^9–12 mnemonic function,¹² memory,^9,10,13 attention,^9,10 and processing speed.¹¹ In a recent review of studies in adult patients aged ≤65 years with MDD, McIntyre et al reported pronounced deficits (≥1 SD below the normative mean) in executive function in 20%–30% of patients,¹⁴ and ability to concentrate was reported as a problem by 89.6% of patients with depression in the STAR^*D study.¹⁵ In addition, ongoing persistent cognitive decline, specifically impairments in immediate memory and attention, has been reported in some patients with previous MDD, suggesting that MDD may have long-lasting effects on cognitive performance.⁸ Thus, the presence of cognitive dysfunction has broad implications for assessment, treatment and prognosis for psychosocial recovery in patients with MDD.¹⁶

Currently, there is no well-validated, widely accepted, patient-reported outcome instrument available to specifically assess cognitive dysfunction in patients with MDD. The PDQ¹⁷ was developed to measure patient-reported cognitive symptoms associated with MS and forms part of a series of generic and disease-specific instruments that comprise the Multiple Sclerosis Quality of Life Inventory.¹⁸ The PDQ has also been used for the assessment of cognitive symptoms in patients with a number of other conditions including MDD.¹⁹ Furthermore, a modified version of the PDQ, referred to as the PDQ-D, has been adapted for use by adults with MDD to measure cognitive dysfunction, specifically attention/concentration, retrospective memory, prospective memory, and planning/organization.²⁰

The current article reports the results of a prospective study designed to conduct a psychometric validation of the PDQ-D in adults with MDD in the US and in the UK. A key objective of this study was to provide a valid and practical tool for the measurement of cognitive dysfunction in adults with MDD that could be completed by patients in outpatient settings and would also be suitable for use in clinical trials and/or observational studies. The primary objective was to evaluate the reliability, acceptability, convergent and discriminant validity, responsiveness, and underlying factor structure of the PDQ-D when used to measure cognitive dysfunction in respondents with MDD. We hypothesized that the PDQ-D psychometric results would significantly differentiate between respondents with and without physician-diagnosed MDD, and according to MDD severity and evolution.

Methods

This was a non-interventional, prospective, epidemiological online survey conducted in the US and UK. In addition to the PDQ-D, the study included a number of other relevant patient-reported instruments for the purpose of assessing convergent validity and responsiveness. The study population included a sample of respondents with and without MDD, comprising men and women aged ≥18 years (Figure 1). The survey questionnaires were administered in both samples at baseline and also at 6 weeks’ follow-up in the MDD sample only.

Figure 1.

Study design and selection of participants, including initial target for number of respondents.

Abbreviations: BL, baseline; LEAPS, modified Lam Employment Absence and Productivity Scale; MDD, major depressive disorder; MOS COG-R, Medical Outcomes Study Cognitive Functioning Scale-Revised-acute form; PDQ-D, Perceived Deficits Questionnaire-Depression; PGI-S, Patient Global Impression of Severity scale; PHQ-9, Patient Health Questionnaire-9; SDS, Sheehan Disability Scale; WPAI, Work Productivity and Activity Impairment Questionnaire.

Respondents/sample

US participants were recruited through the KnowledgePanel^® maintained by GfK Custom Research, LLC, USA. UK participants were recruited through MrOps (London, UK; see Supplementary materials for further details). Both market research panels required consent from each member when joining, with the expectation that they would participate in surveys and their anonymized data could be used in publications. Each participant also signed an online consent form to participate in this study. Due to the nature of the study, which involved market research panels with informed consent, ethical approval was not required (for further information see Supplementary materials).

The medical history and demographic characteristics of the panel members were documented in advance of the study. In particular, all panel members reported their age, sex, previous diagnosis for depression, and whether they had been under a physician’s care for depression in the 6 months before entry into the panel. Panel members were initially invited to participate in the study via an email invitation based on their pre-existing panel data. The follow-up survey was sent to all respondents with MDD who completed the baseline survey and were still available for study participation.

Male and female panel members aged ≥18 years were eligible for inclusion in the general population sample if they provided informed consent to participate in the study and indicated that they were able to complete the designated surveys. On reaching the target number of respondents for the general population sample, those panel respondents who met the same criteria and, according to their pre-existing panel data, had reported being diagnosed with depression by a physician were randomly selected for potential inclusion in the MDD sample. Eligibility for inclusion in the MDD sample required evidence at screening of ongoing symptoms of depression as evaluated by the PHQ-9 (score ≥10).²¹ Respondents were excluded from the study if they were pregnant or had given birth in the preceding 6 months, or if they had ever been diagnosed with bipolar disorder, Parkinson’s disease, debilitating stroke, MS, or traumatic brain injury.

Throughout the study, respondents received the standard care provided by their physician; no specific treatment interventions were made in the context of the study.

Measures

The online survey comprised the PDQ-D and a series of other validated and established patient self-report instruments, which were used to assess cognitive functioning (MOS COG-R),^22,23 severity of depression (PHQ-9²¹ and PGI-Severity [used at baseline]²⁴) and work-related outcomes and social functioning (SDS, WPAI:SHP, and LEAPS^25–28). Detailed descriptions of the criterion measures are provided in the Supplementary materials.

Validation measure: PDQ-D

The PDQ-D is a 20-item, patient-reported questionnaire with a 7-day recall period. Scores for four subscales of the PDQ-D (attention/concentration, retrospective memory, prospective memory, and planning/organization) are computed based on responses to five constituent items in each subscale. All 20 items use the same 5-point ordinal categorical response scale to reflect the frequency of experiencing a specific cognitive problem in the past 7 days. Scores for each of the four measured subscales are calculated by assigning a value of 0 (“never in the past 7 days”), 1 (“rarely – once or twice”), 2 (“sometimes – 3–5 times”), 3 (“often – about once a day”), or 4 (“very often – more than once a day”) to each item, then summing the five items of that subscale, to produce a score ranging from 0 to 20. A total global score for overall cognitive dysfunction (range 0–80) is calculated by summing the four subscale scores. Higher scores for each subscale and for the total score indicate greater perceived cognitive dysfunction.

Data collection

Data were collected using the self-administered online survey. Respondents in the MDD and general population samples completed the PHQ-9 at screening and the remaining survey questions at baseline. Only respondents in the MDD population were invited to complete the survey again at 6 weeks (±7 days) after baseline (Figure 1).

To prevent long delays in completing items in the survey, potential respondents were asked to complete the surveys online at their selected location within a 24-hour window. If a respondent failed to complete the baseline survey within 24 hours of starting, their data were excluded from the study analyses.

Before the study, the survey was pre-tested using 30 respondents each from the US and UK samples who met all MDD inclusion criteria and none of the exclusion criteria. The purpose of the pre-test was to confirm that the consent, screening, and survey content had been scripted correctly and to determine whether respondents had sufficient time to complete the survey. No problems or concerns were identified during pre-testing, so no changes were made to any study materials. Pre-test data were not included in the analysis, and pre-test respondents were not eligible for the main study sample.

To ensure that data collection was as consistent as possible in both the US and UK, only a single data-collection instrument was programmed for use with both samples, and both surveys were hosted by GfK Custom Research, LLC.

Analyses

It should be noted that the study was not an opinion poll survey where the objective is to extrapolate to a specific target population via weighting of observations. As a consequence, no weighting was applied to the patient observations.

Analyses conducted included assessment of internal consistency reliability, convergent validity, known-groups validity, responsiveness to change, and CFA. With the exception of the known-groups validity, responsiveness to change analyses, and the CFA, all analyses were conducted using baseline PDQ-D scores from the MDD samples only. With the exception of the CFA, all analyses were conducted separately for the US and UK samples. The CFA sample included respondents from both MDD and general population samples pooled across both countries.

Data were analyzed using Statistical Analysis Software (SAS^®; SAS Institute Inc., Cary, NC, USA) Version 9.2, Statistical Package for the Social Sciences (SPSS^®; SPSS Inc., Chicago, IL, USA) Version 17.0, or MPlus^® (Muthén & Muthén, Los Angeles, CA, USA) Version 5. No missing-data imputation methods were used. Patient-reported instrument scores were only calculated for respondents who completed all constituent items.

Descriptive statistics, including median, mean, and SD of observed values and percentage of respondents with missing scores, were determined for each item, subscale, and the total score of the PDQ-D at baseline for the US and UK MDD samples. To determine any potential differences between sample populations, sex distributions of the US and UK MDD samples were compared using Fisher’s exact test, and mean age and mean scores of all patient-reported measures were compared using independent-sample t-tests. The frequency of floor effects (frequency of respondents with the lowest possible score) and ceiling effects (frequency of respondents with the highest possible score) were also examined for the PDQ-D subscales and total score. Floor and ceiling effects for subscales and the total score were considered to be present if >15% of respondents achieved the highest or lowest possible score.²⁹

Internal consistency for each PDQ-D subscale and the total scale was estimated using Cronbach’s alpha (with values ≥0.9 considered excellent and ≥0.7 considered acceptable)³⁰ and the average inter-item correlation (for most self-reported outcome tools this should be ≥0.3 and preferably >0.4 for good reliability).³¹ A substantial increase in the internal consistency (Cronbach’s alpha) of the parent subscale or total scale following removal of an item indicates that the item reduces the reliability of the scale.

Convergent validity was tested using Pearson’s correlation coefficients to explore the direction and magnitude of associations between subscale and total scores of the PDQ-D and those of other patient-reported instruments (PHQ-9, SDS, WPAI:SHP, LEAPS, PGI-Severity, and MOS COG-R). A correlation coefficient with an absolute value of 0.5 indicates a large correlation, and coefficients with absolute values of 0.3 and 0.1 indicate moderate and small correlations, respectively.³² It was hypothesized that the strongest correlation would be between PDQ-D scores and scores from the cognitive functioning measure MOS COG-R, with this being a negative correlation as higher scores on the MOS COG-R indicate better functional outcomes (eg, less cognitive dysfunction). It was also hypothesized that PDQ-D scores would show moderate (r≥0.3) positive correlations with scores from all other criterion measures, given that higher scores for each of these measures reflect worse functional outcomes (eg, more severe depression, greater impairment in functioning).

Known-groups discriminant validity was tested by comparing the PDQ-D subscale and total scores between respondents in the MDD and general population samples at baseline. Within each country sample, PDQ-D scores in the two groups were compared using Cohen’s d effect size for differences between estimated means;³² an ANCOVA model was used for testing, with PDQ-D score sample as a fixed factor and subject age and sex as covariates. It was hypothesized that respondents with MDD would have higher (worse) PDQ-D scores compared with the general population.

Responsiveness to change in the MDD samples was evaluated by comparing changes in PDQ-D subscale and total scores with respondent scores on the PHQ-9. For this comparison, respondents in the US and UK MDD samples were categorized into three groups (worsened, no change, or improved) based on the magnitude of the difference between their baseline and 6-week follow-up scores on the PHQ-9 relative to the accepted minimal important change threshold for the PHQ-9 of 5 points.²¹ Change scores for each PDQ-D subscale and total score were calculated by subtracting each respondent’s baseline score from his/her follow-up score. The “no change” group therefore included subjects whose PHQ-9 varied by ≤4 points between baseline and follow-up. For both the US and UK MDD samples, ANCOVA models for each PDQ-D change score, with severity-change group as a fixed factor and respondent age and sex as covariates, were used to test for statistically significant differences in estimated mean PDQ-D change scores across categories. In models for which the omnibus (ie, global) effect of severity change reached statistical significance, post hoc analyses (with Bonferroni-adjusted P-values to control for inflated Type I error due to multiplicity) examined differences in estimated mean PDQ-D change scores for each pairwise comparison among severity-change groups. It was hypothesized that PDQ-D scores for respondents whose depression severity remained unchanged would be statistically significantly lower than for those whose depression severity worsened and statistically significantly higher than for those whose depression severity improved.

The factor structure of the PDQ-D was investigated using CFA. The proposed conceptual model that guided the development of the PDQ, from which the PDQ-D was modified, comprises four related but separable factors: attention/concentration, retrospective memory, prospective memory, and planning/organization. However, a previous factor analysis of this instrument in a population of patients with MS supported a single-factor model.³³ To test which, if either, of these structures best captured the observed interrelations among items in the PDQ-D, CFA was applied to baseline item scores of the total sample (ie, pooled across depressive status and country). The CFA models were tested using methods for categorical data (polychoric correlations) and the weighted least squares mean and variance adjustment estimator in the Mplus statistical modeling program.³⁴ The CFI³⁵ and the RMSEA³⁶ were used to assess the overall fit of the two models with the data. Adequate fit is indicated by a CFI value of >0.95 (range 0–1) and an RMSEA value of <0.08 (range 0–1).³⁷ In case of a suboptimal fit for the four-factor and single-factor models, testing of a two-factor model,^38,39 comprising four subscale factors with five items each plus a common factor with all 20 items loaded on it, was also planned.

Results

Sample/respondents

The first respondent was recruited to the study on November 28, 2012, and the last respondent completed the study on March 21, 2013.

The MDD sample comprised 421 respondents (US, n=206; UK, n=215) and the general population sample comprised 434 respondents (US, n=212; UK, n=222) (Figure 2). In the population with MDD, the proportion of female respondents was not statistically significantly different in the US and UK samples (64.6% and 72.6%, respectively; P>0.05); however, respondents in the US MDD sample were significantly older than those in the corresponding UK sample (mean age 50.8 vs 46.9 years, respectively; P<0.01). Comparison of the general population samples showed the opposite trend, with a significantly lower proportion of female respondents in the US vs UK populations (50.0% vs 75.2%, respectively; P<0.001) and no significant difference between the two countries for mean age (49.0 and 50.0 years, respectively; P>0.05). A total of 322 respondents with MDD (US, n=169; UK, n=153) entered the follow-up phase.

Figure 2.

Subject disposition.

Abbreviation: MDD, major depressive disorder.

Comparison of patient-reported instrument scores between US and UK MDD samples

There was no evidence of a difference in the PHQ-9 total score and PGI-Severity score between the US and UK samples, suggesting comparable levels of depression severity and perception of overall health (Table 1). However, significant differences were observed for all other outcomes (PDQ-D, MOS COG-R, SDS, WPAI:SHP, and LEAPS) between the US and UK samples, suggesting greater cognitive and functioning impairment in respondents included in the UK sample.

Table 1.

Comparison of baseline demographic characteristics and baseline patient-reported scores for US and UK MDD samples

Characteristic	US MDD (n=206)		UK MDD (n=215)		Difference (US–UK)
	N	Mean (SD)	N	Mean (SD)
Female sex (%)a	133	64.6	156	72.6	−8.0
Age, years	206	50.8 (14.2)	215	46.9 (13.9)	3.9
Educational status (%)a
Less than high school	206	5.8	–	–	–
High school	206	35.9	–	–	–
Some college	206	30.6	–	–	–
Bachelor’s degree or higher	206	27.7	–	–	–
≤5 grade C GCSEs (or equivalent)	–	–	215	29.8	–
>5 grade C GCSEs (or equivalent)	–	–	215	9.3	–
Advanced school/college qualificationsb	–	–	215	27.9	–
Undergraduate degree or equivalent	–	–	215	13.5	–
Postgraduate degree or equivalent	–	–	215	7.0	–
Professional qualification	–	–	215	12.6	–
PDQ-D
Total score	186	37.7 (19.4)	191	42.1 (18.3)	−4.4
Attention/concentration	198	10.6 (5.3)	208	11.7 (4.8)	−1.2
Retrospective memory	199	9.1 (5.6)	204	10.0 (5.5)	−0.9
Prospective memory	199	7.6 (4.7)	209	8.7 (4.8)	−1.2
Planning/organization	198	10.4 (5.3)	202	11.4 (5.0)	−0.9
PHQ-9 Total score	206	15.7 (4.9)	215	16.6 (5.0)	−0.9
MOS COG-R Total score	205	35.0 (12.6)	215	31.5 (12.4)	3.5
SDS Total score	206	16.0 (8.1)	215	18.7 (7.7)	−2.8
WPAI:SHP score
Absenteeism	92	5.7 (14.4)	77	9.2 (20.7)	−3.5
Presenteeism	91	33.7 (26.0)	75	49.3 (21.1)	−15.6
Overall work impairment	91	36.5 (27.7)	77	53.1 (24.1)	−16.6
Activity impairment	204	58.8 (26.3)	215	64.3 (24.9)	−5.5
LEAPS Total score
Total score	182	9.4 (8.3)	195	12.0 (9.1)	−2.6
Productivity	186	3.3 (3.6)	200	4.6 (4.0)	−1.3
PGI-Severity score	204	3.6 (1.5)	214	3.8 (1.5)	−0.2

Notes:

^aPercentage of respondents is presented;

^bAdvanced or Advanced Subsidiary levels/Scottish Highers/National Vocational Qualification levels 3 and 4. “–” indicates not applicable.

Abbreviations: GCSE, General Certificate of Secondary Education; LEAPS, modified Lam Employment Absence and Productivity Scale; MDD, major depressive disorder; MOS COG-R, Medical Outcomes Study Cognitive Functioning Scale-Revised-acute form; PDQ-D, Perceived Deficits Questionnaire-Depression; PGI-Severity, Patient Global Impression of Severity scale; PHQ-9, Patient Health Questionnaire-9; SDS, Sheehan Disability Scale; WPAI:SHP, Work Productivity and Activity Impairment Questionnaire: Specific-Health Problem.

Item- and scale-level descriptive statistics

Mean scores across all PDQ-D items were generally in the range of 2.5–3.5 for both the US and UK MDD samples, and SDs were generally between 1.2 and 1.4 for both MDD samples (Table 2). Floor and ceiling effects for the US and UK MDD samples were minimal for the PDQ-D total score and for each of the four subscales (Table 3), with values generally below 5%.

Table 2.

Response frequency distributions and descriptive statistics for PDQ-D items at baseline

Subscale	Item no	US MDD sample					UK MDD sample
		N	Missing (%)	Mean score (SD)	Percentage of responders		N	Missing (%)	Mean score (SD)	Percentage of responders
					“Never”	“Very often”				“Never”	“Very often”
Attention/concentration	1	204	1.0	3.17 (1.29)	9.7	21.5	215	0.0	3.36 (1.25)	6.3	25.7
	5	202	1.9	3.06 (1.32)	14.5	18.3	215	0.0	3.41 (1.21)	7.9	22.5
	9	202	1.9	3.05 (1.32)	15.6	17.2	213	0.9	3.39 (1.22)	7.3	22.5
	13	205	0.5	3.62 (1.26)	8.1	32.3	212	1.4	3.98 (1.11)	2.6	44.0
	17	205	0.5	2.63 (1.42)	28.0	16.1	213	0.9	2.60 (1.34)	27.2	11.0
Retrospective memory	2	205	0.5	2.92 (1.39)	21.0	18.8	212	1.4	3.02 (1.37)	18.3	17.8
	6	201	2.4	2.74 (1.20)	15.1	9.7	213	0.9	2.94 (1.24)	14.1	12.0
	10	202	1.9	2.81 (1.39)	22.0	16.1	214	0.5	3.14 (1.35)	16.2	19.9
	14	203	1.5	2.70 (1.43)	28.5	17.2	211	1.9	2.81 (1.42)	26.2	16.2
	18	205	0.5	3.05 (1.34)	14.0	19.9	213	0.9	3.14 (1.28)	11.5	18.3
Prospective memory	3	203	1.5	3.21 (1.30)	10.8	22.0	215	0.0	3.53 (1.20)	5.8	25.7
	7	202	1.9	2.09 (1.21)	41.9	5.9	212	1.4	2.24 (1.29)	39.8	8.4
	11	204	1.0	3.02 (1.29)	11.3	19.4	213	0.9	3.33 (1.26)	8.4	22.5
	15	205	0.5	2.04 (1.19)	43.5	5.4	214	0.5	2.42 (1.27)	29.3	8.9
	19	205	0.5	2.28 (1.22)	34.9	5.9	214	0.5	2.27 (1.26)	36.6	7.3
Planning/organization	4	203	1.5	3.23 (1.36)	14.5	25.3	213	0.9	3.34 (1.19)	6.8	21.5
	8	202	1.9	2.79 (1.37)	22.0	16.7	211	1.9	3.04 (1.30)	15.2	16.2
	12	205	0.5	3.63 (1.25)	8.1	30.1	214	0.5	3.87 (1.14)	3.7	38.7
	16	204	1.0	2.78 (1.30)	18.8	12.9	211	1.9	3.07 (1.32)	14.1	18.3
	20	204	1.9	2.91 (1.32)	18.3	15.6	212	1.4	3.23 (1.35)	13.6	23.0

Abbreviations: MDD, major depressive disorder; PDQ-D, Perceived Deficits Questionnaire-Depression.

Table 3.

Internal consistency reliability: PDQ-D scale-level properties at baseline for US and UK MDD samples

Measure	n	Mean	Median	SD	Floor (%)	Ceiling (%)	Cronbach’s alpha	Average inter-item correlation
US MDD sample (N=206)
Total score	186	37.7	37.5	19.4	2.2	1.1	0.96	0.53
Attention/concentration	198	10.6	11.0	5.3	2.5	4.0	0.86	0.54
Retrospective memory	199	9.1	9.0	5.6	4.5	3.0	0.89	0.64
Prospective memory	199	7.6	7.0	4.7	4.0	1.5	0.81	0.47
Planning/organization	198	10.4	10.5	5.3	3.0	3.0	0.86	0.56
UK MDD sample (N=215)
Total score	191	42.1	43.0	18.3	0.5	1.0	0.95	0.50
Attention/concentration	208	11.7	11.0	4.8	0.5	5.8	0.84	0.53
Retrospective memory	204	10.0	10.0	5.5	2.9	3.9	0.89	0.62
Prospective memory	209	8.7	9.0	4.8	1.4	1.4	0.81	0.47
Planning/organization	202	11.4	11.5	5.0	0.5	6.4	0.84	0.53

Abbreviations: MDD, major depressive disorder; PDQ-D, Perceived Deficits Questionnaire-Depression.

There was a good level of acceptability of the PDQ-D in both the US and UK samples, with 90% and 89% of respondents, respectively, completing all 20 items. At the item level, the proportion of missing values varied between 0.5% and 2.4% in the US sample and between 0.0% and 1.9% in the UK sample (Table 2).

Internal consistency reliability

As shown in Table 3, Cronbach’s alpha was ≥0.95 for the PDQ-D total scores in the US and UK MDD samples, indicating excellent reliability; Cronbach’s alpha was >0.80 for each of the subscales for both samples, indicating good reliability. For both the US and UK MDD samples, the averages of the inter-item correlations within the total scale (both ≥0.50) and within each of the four subscales (all ≥0.47) also provide evidence of good internal consistency reliability (Table 3). Removal of most PDQ-D items in turn resulted in a reduction in Cronbach’s alpha of each associated subscale in the US and UK MDD samples indicating appropriate reliability for each item. The reductions were 0.01–0.06 and 0.01–0.07 for the US and UK samples, respectively. Exceptions to this were an increase of 0.01 in the alpha coefficient in both the US and UK sample following removal of item 2 of the retrospective memory subscale, and no change in alpha coefficient in the US sample following removal of item 16 of the planning/organization subscale.

Convergent validity

In both the US and UK populations, with the exception of the WPAI:SHP absenteeism subscale for which correlations with PDQ-D scores were close to zero, all correlations with criterion measures were in the hypothesized direction. PDQ-D baseline subscale and total scores correlated positively with scores from measures for which higher scores indicate worse outcomes (PHQ-9, SDS, WPAI:SHP, LEAPS, and PGI-Severity) and correlated negatively with MOS COG-R for which a lower score indicates worse outcomes (P<0.001 for the magnitude of all correlations between PDQ-D scores and criterion measures being greater than 0) (Table 4).

Table 4.

Convergent validity: Pearson correlation coefficients between baseline PDQ-D subscale and total scores and baseline scores from other survey measures in the US and UK study populations with MDD

Scale	Measure	Pearson correlation at baseline
		PDQ-D Total score	Attention	Retrospective memory	Prospective memory	Planning
US MDD sample
PHQ-9	Total score	0.54	0.51	0.45	0.44	0.48
MOS COG-R	Total score	−0.78	−0.72	−0.69	−0.70	−0.71
SDS	Total score	0.32	0.28	0.22	0.33	0.35
WPAI:SHP	Absenteeism	0.06	0.04	0.00	0.04	0.08
	Presenteeism	0.34	0.27	0.22	0.33	0.37
	Overall work impairment	0.33	0.26	0.21	0.31	0.35
	Activity impairment	0.32	0.28	0.23	0.32	0.34
LEAPS	Total score	0.38	0.35	0.30	0.33	0.38
	Productivity	0.37	0.33	0.29	0.33	0.35
PGI-Severity	Score	0.31	0.28	0.20	0.26	0.37
UK MDD sample
PHQ-9	Total score	0.62	0.55	0.51	0.52	0.60
MOS COG-R	Total score	−0.80	−0.76	−0.69	−0.71	−0.74
SDS	Total score	0.53	0.47	0.43	0.47	0.55
WPAI:SHP	Absenteeism	0.05	−0.01	0.10	0.07	0.03
	Presenteeism	0.47	0.46	0.46	0.46	0.36
	Overall work impairment	0.41	0.38	0.42	0.41	0.32
	Activity impairment	0.53	0.47	0.44	0.43	0.59
LEAPS	Total score	0.51	0.48	0.40	0.49	0.48
	Productivity	0.52	0.46	0.41	0.49	0.49
PGI-Severity	Score	0.43	0.39	0.34	0.30	0.48

Abbreviations: LEAPS, modified Lam Employment Absence and Productivity Scale; MDD, major depressive disorder; MOS COG-R, Medical Outcomes Study Cognitive Functioning Scale-Revised-acute form; PDQ-D, Perceived Deficits Questionnaire-Depression; PGI-Severity, Patient Global Impression of Severity scale; PHQ-9, Patient Health Questionnaire-9; SDS, Sheehan Disability Scale; WPAI:SHP, Work Productivity and Activity Impairment Questionnaire: Specific-Health Problem.

As predicted, the strongest correlations were observed between the PDQ-D and the MOS COG-R. In the US sample, the Pearson correlation coefficient was −0.78 for PDQ-D total score and ranged from −0.69 to −0.72 for PDQ-D sub-scales; in the UK sample, the Pearson correlation coefficient was −0.80 for PDQ-D total score and ranged from −0.69 to −0.76 for PDQ-D subscales. Correlations between the PHQ-9 and the PDQ-D were generally moderate to large, with correlations for the PDQ-D total score approaching 0.60, and correlations for the PDQ-D subscales ranging from 0.44 to 0.60. Correlations with measures of depression severity (PGI-Severity), functioning and productivity (SDS, WPAI:SHP, and LEAPS) were generally small to moderate in the US sample (range 0.20–0.38) and moderate to large in the UK sample (range 0.30–0.59).

Known-groups validity

For both US and UK study populations, statistically significant mean differences (all P<0.001) were observed for all PDQ-D subscale scores and the total score between the MDD samples and their corresponding general population samples (Figure 3). For both countries, mean PDQ-D total and subscale scores for respondents in the MDD group were at least twice as high (worse) than those in the general population, as indicated by effect-size estimates exceeding 1.0 for all scores (US sample, ≥1.28; UK sample, ≥1.13 [Table 5]), suggesting a large effect of MDD on each PDQ-D score.

Figure 3.

Known-groups validity: mean baseline PDQ-D subscale and total scores for the MDD and general populations: (A) US sample; (B) UK sample. ^*P<0.001 vs general population sample.

Abbreviations: MDD, major depressive disorder; PDQ-D, Perceived Deficits Questionnaire-Depression.

Table 5.

Mean comparison across MDD and general populations for testing known-groups validity of baseline PDQ-D total and subscale scores

	MDD samples			General population sample			Mean difference	da
	N	Estimated mean	SEM	N	Estimated mean	SEM
US sample
Total score	186	37.6	1.22	198	11.5	1.19	26.1	1.56
Attention/concentration	198	10.5	0.34	203	3.4	0.33	7.1	1.50
Retrospective memory	199	9.0	0.34	205	2.7	0.33	6.3	1.33
Prospective memory	199	7.5	0.28	207	2.5	0.27	5.0	1.28
Planning/organization	198	10.4	0.32	205	2.7	0.32	7.7	1.69
UK sample
Total score	191	41.9	1.28	199	17.1	1.25	24.7	1.40
Attention/concentration	208	11.7	0.33	209	4.9	0.32	6.8	1.44
Retrospective memory	204	10.0	0.35	216	3.9	0.34	6.1	1.23
Prospective memory	209	8.7	0.30	218	3.8	0.29	4.9	1.13
Planning/organization	202	11.3	0.34	213	4.2	0.33	7.0	1.48

Note:

^aCohen’s d effect size for differences among estimated means calculated from the ANCOVA model.

Abbreviations: ANCOVA, analysis of covariance; MDD, major depressive disorder; PDQ-D, Perceived Deficits Questionnaire-Depression; SEM, standard error of the mean.

Responsiveness to change

For the US sample, differences in mean changes from baseline in PDQ-D total scores and scores for all PDQ-D subscales were statistically significant as a function of their classification based on whether a respondent showed worsening, improvement, or no change from baseline in the severity of their depression based on the PHQ-9 minimal important change (all P<0.01) (Table 6). Further, post hoc pairwise comparisons between the three categories (worsened, no change, and improved) showed that respondents with meaningful improvements in depression severity from baseline to follow-up had statistically significantly greater improvements in all PDQ-D scores than respondents who experienced worsening or no change in PHQ-9 scores (all Bonferroni-adjusted P<0.05 for differences), with no statistical differences between the latter two groups for any PDQ-D scores.

Table 6.

Responsiveness to change: Comparison of mean changes from baseline in PDQ-D subscale and total scores for respondents with MDD as a function of their classification based on worsening, no change, or improvement from baseline to follow-up in PHQ-9 score

Measure	Worseneda			No changea			Improveda			P-valueb
	n	Estimated mean	SEM	n	Estimated mean	SEM	n	Estimated mean	SEM
US MDD sample
Total score	15	1.6*	3.19	83	−2.7*	1.35	49	−10.5#	1.76	0.0004
Attention/concentration	16	0.6*	0.96	87	−0.8*	0.41	55	−2.5#	0.51	0.0049
Retrospective memory	17	0.4*	0.95	88	−0.7*	0.42	56	−2.6#	0.52	0.0039
Prospective memory	17	0.5*	0.75	87	−0.3*	0.33	56	−1.8#	0.41	0.0045
Planning/organization	16	0.4*	0.95	86	−1.1*	0.41	56	−3.4#	0.50	0.0001
UK MDD sample
Total score	19	−1.4	3.52	85	−5.4	1.67	22	−12.0	3.28	0.0798
Attention/concentration	22	0.6*	0.91	99	−1.4*,#	0.43	24	−3.1#	0.87	0.0148
Retrospective memory	23	0.3*	0.96	94	−1.5*,#	0.47	23	−3.1#	0.96	0.0499
Prospective memory	23	−0.7	0.85	97	−1.1	0.41	24	−2.2	0.83	0.3823
Planning/organization	22	0.3*	0.94	94	−1.7*,#	0.45	23	−3.7#	0.92	0.0111

Notes:

^*,#Estimated means within a row with different superscripts (^* or ^#) are significantly different (P<0.05) based on post hoc paired comparison using Bonferroni’s correction for multiple comparisons.

^a“Improved” respondents’ baseline PHQ-9 score exceeded follow-up score by ≥5 points; “No change” respondents’ PHQ-9 scores at baseline and follow-up were within 5 points of each other (in either direction); “Worsened” respondents’ follow-up PHQ-9 score exceeded baseline score by ≥5 points.

^bDifference in estimated mean PDQ-D scores across PHQ-9 categories.

Abbreviations: MDD, major depressive disorder; PDQ-D, Perceived Deficits Questionnaire-Depression; PHQ-9, Patient Health Questionnaire-9; SEM, standard error of the mean.

In the UK sample, changes in mean scores for three of the PDQ-D subscales, although not for the Prospective-memory subscale or the PDQ-D total score, differed statistically significantly as a function of whether a respondent showed worsening, improvement, or no change in the severity of their depression (P<0.05 for differences). For the three subscales with statistically significant omnibus differences, post hoc pairwise comparisons of change status showed that respondents with a meaningful improvement in depression severity from baseline to follow-up showed statistically significantly greater improvements than respondents with worsening of PHQ-9 scores during this period (all Bonferroni-adjusted P<0.05 for differences); however, scores for respondents with no change in depression severity did not differ significantly from those of respondents in either of the other two groups (Table 6).

Factor analysis

Fit indices for the one-factor model were poor, with both the CFI (0.878) and RMSEA (0.113) failing to meet the conventional thresholds established for these parameters. The subsequent CFA of the four-factor model, with the addition of some post hoc modifications (items 1 and 2 were allowed to correlate with item 3; item 10 was allowed to correlate with item 18, and item 12 was regressed on the depression group [ie, MDD or general population]), provided a better fit, with the RMSEA at the conventional threshold for adequate fit (0.080); however, the CFI was just below the relevant threshold (0.938). The majority of items loaded most strongly on their hypothesized factor (the item’s parent subscale): the only exceptions were for item 13 (an attention/concentration item that loaded most strongly with planning/organization items) and item 16 (a planning/organization item that loaded most strongly with attention/concentration items). For this model, the association among factors was strong, with all inter-factor correlations being >0.85 and the majority close to 0.95, indicating that these factors were capturing either a single construct or at least very closely related constructs. Given the suboptimal fit of these models and the apparent relatedness of the four factors, the two-factor model was tested, while applying the same modifications as in the four-factor model. This two-factor model showed the best fit, with both RMSEA (0.073) and CFI (0.95) meeting or exceeding the conventional threshold values. The high loadings on the global factor relative to those on the specific subfactors strongly support scoring of the global scale. These factor analyses suggest that the global scale will have higher reliability and greater overall statistical power than the subscales in most analyses.

Discussion

This report describes results from a psychometric validation of the PDQ-D using an online survey of respondents with and without MDD (defined as having received a diagnosis of depression by a physician and a PHQ-9 score ≥10) in the US and UK. The study was conducted in the US and the UK to allow elements of psychometric validation for the original English version of the PDQ-D to be assessed, and then tested in a US and EU setting. To our knowledge, this is one of only a small number of studies to validate a patient-reported instrument for the assessment of cognitive dysfunction in patients with depression. In recent years, both the Massachusetts General Hospital Cognitive and Physical Functioning Questionnaire and the British Columbia Cognitive Complaints Inventory have been validated in patients with depression, although not as extensively as the PDQ-D in the current study.^40,41

The PDQ-D was adapted from the PDQ, a patient-reported outcome measure of cognitive function that was originally developed for use in patients with MS. The PDQ was selected and adapted for use in MDD based on feedback and input from patients with MDD (n=50) participating in focus groups and cognitive debriefing interviews.²⁰ Similar to the PDQ, the PDQ-D comprises 20 items, but has a shorter recall period of 7 days (vs 4 weeks for the PDQ) and incorporates expanded response-category descriptions.²⁰ Based on the results for the US and UK MDD populations, the current study findings indicate that the PDQ-D is a reliable and valid measure of cognitive dysfunction in patients with MDD.

In the present study, analysis of PDQ-D items in the MDD samples showed homogenous variance and no meaningful floor or ceiling effects for PDQ-D subscales or the total score of the PDQ-D for individual items.

High internal consistency reliability was established for the PDQ-D total scale and good internal consistency was found for the four proposed individual subscales (attention, retrospective memory, prospective memory, and planning). Notably, the vast majority of PDQ-D items contributed positively to the internal consistency reliability of the PDQ-D subscales, and no items had a meaningful negative impact on the internal consistency reliability of the PDQ-D subscales.

The PDQ-D demonstrated good convergent validity, as indicated by strong concordance with an existing, validated measure of cognitive dysfunction (MOS COG-R) and generally moderate to small correlations with measures of constructs known to be associated with cognitive difficulties (PHQ-9, SDS, WPAI:SHP, LEAPS, and PGI-Severity); the strong correlation with the MOS COG-R and smaller correlations with other measures confirm that cognitive dysfunction is a concept different from that captured by these latter instruments. This observation was expected, given that both instruments were developed to measure the same underlying construct, namely cognitive dysfunction. The PDQ-D and MOS COG-R each have their own advantages and disadvantages. Specifically, the inclusion of a greater number of items in the PDQ-D (20 vs 6 items for MOS COG-R) affords the PDQ-D greater precision.²⁰ Also, the facility to determine PDQ-D subscale scores (not possible with the MOS COG-R), in addition to a total score, allows assessment of impairments specific to particular aspects of cognitive functioning. In contrast, the MOS COG-R is normative based, and the inclusion of fewer items means that it is associated with less patient burden connected with completing the assessment.²³

In terms of discriminant validity, the PDQ-D performed well. Large differences in scores were observed between the MDD samples and the general population samples, with the MDD groups scoring approximately 2–3 times higher (ie, worse) than the general population groups.

The PDQ-D also demonstrated responsiveness to changes in depression severity in respondents with MDD who showed a clinically significant decrease in their PHQ-9 score over a 6-week period. The PHQ-9 scale was selected for the responsiveness analysis as it is one of very few patient-reported outcome measures frequently used to assess depression severity. By using the PHQ-9 as the responsiveness criterion we were able to show a correlation between improvement in depression and reduction in cognitive dysfunction.

Validation of the PDQ-D independently in both US and UK populations provides further support for the scale. It should be noted that the US and UK samples (MDD and general population) differed in terms of demographic characteristics, specifically age and sex, with respondents in the US MDD sample being significantly older than those in the UK MDD sample and the US general population containing a significantly lower proportion of female respondents than the UK sample. These differences in age and sex distribution between the US and UK populations, although not large in practical terms, were unexpected and further examination of the study populations should be considered to determine whether the differences are a function of differential recruiting/sampling techniques or whether they reflect true differences. Health outcomes (PHQ-9 and PGI-Severity) were similar in the US and UK MDD samples, indicating similar levels of depression severity and perception of overall health; however, the US MDD sample reported less cognitive dysfunction and less impairment in work and activities compared with the UK sample. This difference was possibly attributable to a reporting bias as it is unlikely that depression would have a differential impact on these outcomes as a function of country; however, differences related to cultural aspects or health care system specificities cannot be excluded. Assuming that the relation between depression and objective cognitive dysfunction (eg, performance measured as part of a cognitive test battery) would be similar, there appears to be a tendency for the US MDD population to underestimate cognitive dysfunction, the UK MDD population to overestimate cognitive dysfunction, or a combination of both. This bias may not be limited to an MDD population and supports our approach of conducting separate analyses for each country.

CFA supported the scoring of a global overall scale for cognitive dysfunction rather than the four subscales of cognitive functioning. Although further study of the factor structure is needed, the results support greater confidence in the total score being predictive of overall cognitive dysfunction in MDD than in the subscale scores being predictive of specific aspects of cognitive dysfunction. In addition, our findings support the National Multiple Sclerosis recommendation that ‘caution should be exercised when interpreting PDQ subscale scores’.¹⁸

Limitations

This psychometric validation study has some limitations. First, a diagnostic interview was not conducted to confirm the diagnosis of MDD in respondents. Inclusion in the MDD sample was based on respondent self-report of a depression diagnosis by a physician and current symptoms meeting the PHQ-9 threshold. The PHQ-9 criterion used (score ≥10), however, has been shown in previous studies to have both sensitivity and specificity of 88% for the DSM (fourth edition)⁴² diagnosis of MDD (positive predictive value of 31% and 51% for PHQ-9 cutoff of 9 and 15, respectively),²¹ making it a reasonable proxy for an MDD diagnosis. A second limitation of the study is that the analyses took a very conservative approach to imputation of missing item scores for the PDQ-D, resulting in missing scale scores for a number of respondents, particularly for the total scale. However, the frequency of missing values was very low, indicative of good acceptability, with less than 2.4% of missing values per item and around 10% of missing total scores. For future studies, imputation approaches could be considered to reduce the number of missing scale scores, and the current analyses repeated to determine whether the same conclusions could be made. Finally, it is important to note that as an assessment tool for “perceived” cognitive dysfunction, the PDQ-D measures different outcomes to objective neuropsychological testing. Consistent with this, Lovera et al⁴³ reported a lack of correlation between the PDQ and objective cognitive tests in patients with MS. Further research is required on the assessment of perceived cognitive dysfunction in routine clinical practice. Regarding the need for cultural adaptation for worldwide use, a Korean and a Chinese version of the PDQ-D, incorporating only minor differences compared with English version, have been developed and recently validated.^44,45

Conclusion

Our study findings have shown that the PDQ-D has a good structure, internal consistency, convergent validity, discriminant validity, and responsiveness, and provides a reliable and valid measure of subjective cognitive dysfunction in patients with MDD. Neuropsychological tests can provide detailed assessment of cognitive performance but are not feasible for routine clinical use.⁴⁶ Patient-reported outcome measures can supplement the information from objective tests, and we believe that the PDQ-D will provide a practical tool for the assessment of cognitive dysfunction in adults with MDD in routine clinical practice to ensure that cognitive symptoms improve and remit with treatment alongside other depressive symptoms. Further research will be required to investigate the relationship between a comprehensive patient-reported outcome measure, such as the PDQ-D, and objective neuropsychological test performance.

Ethics approval and consent to participate

All patients enrolled in the study provided written informed consent. Due to the nature of the study, which involved market research panels with informed consent, ethical approval was not required (for further information see Supplementary materials).

Supplementary materials

Psychometric validation of the Perceived Deficits Questionnaire-Depression (PDQ-D) instrument in US and UK respondents with major depressive disorder

Background information on methods

Respondents/sample

The US and UK samples were provided by KnowledgePanel^® (GfK Custom Research, LLC, USA) and MrOps (London, UK), respectively. KnowledgePanel^® has been in continuous use in the US since 1998 and comprises more than 50,000 adult members recruited using a combination of random-digit-dial telephone and address-based sampling selected from the US Postal Service Computerized Delivery Sequence File. Profile information, with questions on health topics including depression, is collected and updated on an annual basis. MrOps is a specialist company with access to over 40 screened and validated sample sources worldwide, many of which are online panels.

Ethics approval

Data were collected by survey companies through their existing approved market-research panels, following their own procedures. When subjects are enrolled in such panels, they provide the panel owners with written informed consent, and agree to be contacted for participation in online surveys. For this survey, we ensured that the participants signed an additional informed consent form to participate.

In the US, this type of survey is exempt from Institutional Review Board approval according to guidelines from the US Office for Human Research Protections which stipulate that research involving the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures or observation of public behavior, are exempt unless:

1. Information obtained is recorded in such a manner that human subjects can be identified, directly or through identifiers linked to the subjects; and

2. Any disclosure of the human subjects’ responses outside the research could reasonably place the subjects at risk of criminal or civil liability or be damaging to the subjects’ financial standing, employability, or reputation.

For the UK, this survey does not fit any of the categories needing approval by a National Health Service Research Ethics Committee (NHR REC) (see http://www.hra.nhs.uk/resources/before-you-apply/is-nhs-rec-review-required/).

Criterion measures

The Patient Health Questionnaire-9 (PHQ-9) is a 9-item measure that assesses the frequency of experiencing symptoms of major depressive episode, as defined in the Diagnostic and Statistical Manual of Mental Disorders Fourth Edition, during the previous 2-week period plus one added item assessing the level of disturbance of these symptoms.¹ The Medical Outcomes Study Cognitive Functioning Scale-Revised-acute form (MOS COG-R) is a 6-item instrument that measures the frequency of problems in a range of day-to-day cognitive functions, such as memory, attention, and reasoning, in the previous 1-week period.^2,3 The Patient Global Impression of Severity scale (PGI-Severity) consists of a single item to assess the severity of the respondent’s depression at the time of assessment.⁴ The 6-item Work Productivity and Activity Impairment Questionnaire: Specific-Health Problem (WPAI:SHP) measures the impact of any specific health problem (in this case, depression) on respondents’ work productivity (including the duration of work time missed due to depression, as well as decreases in productivity when working while experiencing depression) and non-work related activities (eg, childcare) over the past 7 days.⁵ The modified Lam Employment Absence and Productivity Scale (LEAPS) is a 10-item questionnaire to assess occupational functioning and impairment over the previous 2-week period.⁶ The 5-item Sheehan Disability Scale (SDS) measures the degree to which mental disorder symptoms have disrupted a respondent’s work/school, social, and family life in the preceding week.^7,8

Abbreviations

ANCOVA

analysis of covariance

CFA

confirmatory factor analysis

CFI

comparative fit index

DSM

Diagnostic and Statistical Manual of Mental Disorders

LEAPS

modified Lam Employment Absence and Productivity Scale

MDD

major depressive disorder

MOS COG-R

Medical Outcomes Study Cognitive Functioning Scale-Revised-acute form

MS

multiple sclerosis

PDQ

Perceived Deficits Questionnaire

PDQ-D

Perceived Deficits Questionnaire-Depression

PGI-Severity

Patient Global Impression of Severity scale

PHQ-9

Patient Health Questionnaire-9

RMSEA

root mean square error of approximation

SDS

Sheehan Disability Scale

STAR^*D

Sequenced Treatment Alternatives to Relieve Depression

WPAI:SHP

Work Productivity and Activity Impairment Questionnaire: Specific-Health Problem.