Patient versus informant reporting of ICD symptoms in Parkinson’s disease using the QUIP: Validity and variability

Abstract

Questions exist regarding the validity of patient-reporting of psychiatric symptoms in Parkinson’s disease (PD). We assessed observer variability and validity in reporting of impulse control disorder (ICD) symptoms in PD by using the Questionnaire for Impulsive-Compulsive Disorders in Parkinson’s Disease (QUIP). PD patients and their informants (71 pairs) completed the QUIP to assess four ICDs (compulsive gambling, buying, sexual behavior, and eating) in patients. Trained raters then administered a diagnostic interview. Sensitivity of the QUIP for a diagnosed ICD was 100% for both patient- and informant-completed instruments, and specificity was 75% for both raters. Approximately 40% of patients without an ICD diagnosis had a positive QUIP, suggesting that many PD patients experience subsyndromal ICD symptoms that require ongoing monitoring. Agreement between patient- and informant-reporting of any ICD behaviors on the QUIP was moderate (kappa = 0.408), and for individual ICDs was highest for gambling (kappa = 0.550). Overall, a negative QUIP from either the patient or informant rules out the possibility of an ICD, while a positive QUIP requires a follow-up diagnostic interview and ongoing monitoring to determine if symptoms currently are, or in the future become, clinically significant.

1. Introduction

Case reporting [1] and large-scale cross-sectional studies [2] suggest that a range of impulse control disorders (ICDs), including compulsive gambling, buying, sexual behavior, and eating, are relatively common in Parkinson’s disease (PD), occurring in up to 15% of patients. ICDs, associated with dopamine replacement therapies (DRT) [2], can lead to significant impairments in psychosocial functioning, interpersonal relationships, and quality of life. Therefore, they can represent a significant problem in PD.

Health care professionals working with PD patients typically focus their assessment and treatment on motor symptoms, so a range of neuropsychiatric symptoms (NPS) in PD are under-diagnosed and under-treated in clinical practice [3,4]. Patients may not volunteer NPS for a variety of reasons. For ICDs specifically, patients may not report symptoms either due to embarrassment or not suspecting a possible association with their DRT. Thus, ICD behaviors are currently under-recognized and under-managed in clinical practice [5]. To improve the detection of ICDs and related disorders in PD, a patient-rated screening instrument called the Questionnaire for Impulsive-Compulsive Disorders in Parkinson’s Disease (QUIP) was recently developed and validated [6].

While questionnaires can be an effective method of obtaining information, patient-ratings and informant-ratings should not be used interchangeably unless they are shown to be equally valid. Although even high agreement between patients and informants can represent inaccurate representations of behaviors or symptoms, this is unlikely [7]. Multiple studies in various neurodegenerative diseases have highlighted the issue of patient-informant agreement. In a study of NPS in PD, agreement between patient- and informant-reports was low and quite variable, with agreement ranging from 6.7% to 45.8% across five different categories of NPS [8]. Low agreement between patients and informants in neurodegenerative diseases could be due to multiple reasons: patients may not have an objective view of their own behaviors or possess the mental capacity to analyze their situation; informants may have inaccurate information; or patients or informants may hide or distort information [9]. To examine the issue of validity of patient- versus informant reporting of ICD symptoms, as well as patient-informant agreement, we report on the subset of our QUIP validation sample that also had an informant rate the patient’s ICD symptoms by completing the QUIP and participating in the diagnostic interview.

2. Methods

2.1. Participants

The methodology used in the development and validation of the QUIP was reported previously [6]. For these analyses, data from the subset of patients who had an informant participate in the validation process were utilized (n = 71). Demographic and clinical characteristics for this sample are presented in Table 1.

Table 1.

Demographic and clinical characteristics

Variable	Patient	Informant
	Mean (SD) or %
Age (# years)	67.4 (9.1)	62.8 (10.7)
Sex (% male)	80.3%	21.1%
Race (% white)	98.6%	97.2%
Education (# years)	14.8 (2.6)	14.8 (2.9)
PD duration (# years)	9.2 (6.4)	N/A
Hoehn and Yahr stage	2.4 (0.7)	N/A
Levodopa dosage (mg/day)	598.1 (470.9)	N/A
Dopamine agonist use (% yes)	60.6%	N/A
History of DBS (% yes)	25.4%	N/A

2.2. Assessments

The Institutional Review Board at each participating institution approved the study, and written informed consent was obtained from subjects prior to study participation. After providing informed consent, patients and their informants completed the QUIP separately (but concurrently), and were instructed to answer questions based on patients’ behaviors that occurred anytime during PD that lasted at least four consecutive weeks. The full version of the QUIP includes 5 questions for each of the 4 ICDs most commonly reported in PD (compulsive gambling, buying, sexual behavior, and eating). Based on the results of the validation of the QUIP [6], the following cut-offs were used to represent a positive screen for both the patient-and informant-completed instrument: compulsive gambling (any 2 of the 5 items); compulsive sexual behavior (any 1 of the 5 items); compulsive buying (any 1 of the 5 items); compulsive eating (any 2 of the 5 items).

After patients and informants completed separate QUIPs in private, one of three psychiatric research coordinators trained by the PI administered a semi-structured, diagnostic interview for each disorder that was based upon either DSM-IV criteria or published proposed criteria for ICDs in PD and utilized the same time frame as in the QUIP [10–15]. The research coordinator, who was blinded to the patient and informant QUIP results, interviewed the patient and informant together until a consensus diagnosis for each ICD was reached. Nine participants (12.7%) had at least one ICD, and frequencies of individual ICDs were 7.0% (sexual behaviors), 3.2% (eating), 2.8% (buying), and 1.4% (gambling) (Fig. 1).

Fig. 1.

Frequencies of ICD behaviors based on the diagnostic interview.

2.3. Analyses

The kappa (κ) statistic was used to assess the extent of agreement between patient- and informant-based reporting on the QUIP. Kappa was used to assess agreement between the dichotomous ratings because kappa corrects for agreement that occurs by chance [16]. To assess the psychometric properties of a positive QUIP based on either patient or informant reporting, the sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) against the diagnostic interview were calculated. SPSS version 17.0 [17] was used for all analyses.

3. Results

3.1. Patient versus informant validity

The sensitivity of both the patient- and informant-rated QUIP was 100%, indicating that all patients who met diagnostic criteria for one or more ICDs (n = 9) were identified by both sources. As expected for a screening instrument, the specificity of the QUIP was lower than its sensitivity, and was 75.8% for the patient-rated and 74.2% for the informant-rated version. 40.3% of patients without an ICD by diagnostic interview were identified as having ICD symptoms by at least one individual (self or informant) on the QUIP, but patient and informant agreement occurred in only 24.0% of these cases.

The negative predictive value (NPV) for both the patient- and informant-rated versions of the QUIP was 100%, indicating that all negative QUIP results corresponded with a negative diagnostic interview. The positive predictive value (PPV) for the patient-rated version was 37.5%, similar to the 36.0% for the informant-rated QUIP. This indicates that a positive QUIP result represents a true ICD diagnosis approximately one-third of the time.

When patients were diagnosed with only one ICD (N = 7), both patients and informants identified the same ICD on the QUIP. When patients were diagnosed with more than one ICD (N = 2), the patient and informant both identified on the QUIP at least one of the diagnosed ICDs.

3.2. Patient and informant agreement on the QUIP

Agreement between patient- and informant-reporting for ICDs overall and for each specific ICD is shown in Table 2. Agreement between patients and informants for reporting any ICD behaviors was moderate (kappa = 0.408). Agreement for reporting of any ICD occurred in 52 of 71 patient-informant pairs, including endorsement of ICD symptoms in 15 pairs and lack of endorsement in 37 pairs. There was lack of source agreement in 19 cases, with 9 patients reporting ICD symptoms that their informant did not identify, and 10 informants reporting ICD symptoms that the patient did not endorse.

Table 2.

Agreement between patient- and informant-rated QUIPs for ICDs

	n	Kappa value	Total Agreement	Agreement (Patient +; Informant +)	Agreement (Patient −; Informant −)	Total Disagreement	Disagreement (Patient +; Informant −)	Disagreement (Patient −; Informant +)
Any ICD	71	0.408	52	15	37	19	9	10
Gambling	71	0.550	68	2	66	3	1	2
Sex	70	0.384	59	5	54	11	5	6
Buying	71	0.132	58	2	56	13	7	6
Eating	71	0.399	59	6	53	12	5	7

Examining each ICD separately, agreement was moderate (kappa = 0.550) for reporting of gambling, fair (kappa = 0.384) for reporting of sex, slight (kappa = 0.132) for reporting of buying, and fair (kappa = 0.399) for reporting of eating.

4. Discussion

Using a recently developed and validated screening instrument for ICDs in PD, we found that (1) the QUIP is a sensitive instrument for the detection of ICDs whether completed by a patient or informant, (2) subsyndromal ICD symptoms are common in PD, and (3) patient-informant agreement of ICD symptoms is high for patients with an ICD, but in a sample that includes both ICD and non-ICD patients the agreement is moderate.

In our sample, the sensitivity of the QUIP for the detection of an ICD was 100%, regardless if a patient or informant completed the instrument. This suggests that if the purpose of administering the QUIP is to make sure that an ICD is not overlooked, then it does not matter if it is completed by the patient or an informant who knows the patient well. Phrased slightly differently, a negative screen from either the patient or an informant essentially rules out the presence of an ICD.

In addition, we found that approximately 40% of patients without an ICD had a positive QUIP based on either patient- or informant-reporting. As the QUIP is worded to identify core ICD symptoms (e.g., thoughts, drives and behaviors), this suggests that a substantial percentage of PD patients without an ICD experience subsyndromal ICD symptoms. Whether such patients are at increased risk of developing an ICD over time is an important area for future research.

There was moderate agreement between patient- and informant-reporting of any ICD symptoms in this population. The varying degrees of agreement for each individual ICD suggest that it is easiest for patients and informants to agree on the clinical significance of gambling behaviors. Regarding the other ICDs (buying, sex and eating), either behaviors may be more likely to be concealed (only patient endorsement on the QUIP) or some patients may be unaware of the clinical significance of their ICD behaviors (only informant endorsement on the QUIP).

The moderate agreement suggests that patient- and informant-reporting of ICD symptoms in PD should not be used interchangeably, especially for patients with subsyndromal symptoms. One possible reason for a mismatch between patient- and informant-reporting include a lack of awareness or minimization of symptoms by patients. In addition, patients may be motivated to hide undesirable behaviors. Some ICD behaviors may be easy to conceal [9], and high agreement between patients and informants depends upon behavior observability [7]. Finally, ICD signs and symptoms include not only behaviors, but thoughts and drives, and the latter may not be observable to others.

Limitations of this study included the relatively small number of patients with an ICD and a gender imbalance (80.3% of participants and 21.1% of informants were male). Future research examining assessment instruments for ICDs in PD should involve larger samples, includes patients from different countries, and be diverse in terms of clinical and demographic characteristics. In addition, it is possible that both the QUIP and the diagnostic interview were inaccurate (e.g., if a patient with an ICD did not endorse symptoms on both the QUIP and the diagnostic interview), so clinicians must be open to the possibility that patients or informants may be reluctant to disclose ICD symptoms in a clinical setting.

Both patient- and informant-rated QUIPs perform well as a screening instrument for ICDs in PD. For patients with subsyndromal ICD symptoms, which appear to be common in PD, the data from patient- and informant-rated QUIPs may not be interchangeable. A positive QUIP requires a follow-up clinical interview to confirm the presence of clinically-significant ICD symptoms, and patients found to have subsyndromal symptoms should be closely monitored. Additional research is needed to determine if this relatively large percentage of PD patients are at an increased risk of developing one or more ICDs in the future.