Skip to main content
NCBI home page
Archives of Clinical Neuropsychology logoLink to Archives of Clinical Neuropsychology
. 2022 Nov 29;38(2):205–212. doi: 10.1093/arclin/acac088

Unpacking the NIH Toolbox Emotion Battery in Persons With Parkinson’s disease

Francesca V Lopez 1,#,✉, Rachel Schade 2,#, Adrianna Ratajska 3, Lauren Kenney 4, Katie Rodriguez 5, Alyssa Ray 6, Lauren Santos 7, Bonnie M Scott 8,3, Erin Trifilio 9, Dawn Bowers 10,11
PMCID: PMC9940110  PMID: 36446750

Abstract

Objective

Examine the relationship between the National Institutes of Health Toolbox Emotion Battery (Emotion Toolbox) and traditional measures in Parkinson’s disease (PD).

Method

Persons with PD (n = 30) and cognitively healthy older adults (OA; n = 40) completed the Emotion Toolbox consisting of Well-Being, Negative Affect, and Social Satisfaction scores along with traditional measures of depression (Beck Depression Inventory-II [BDI-II]), anxiety (State–Trait Anxiety Inventory [STAI]), and apathy (Apathy Scale [AS]); total raw scores).

Results

Separate bootstrapped analyses of covariance indicated that the PD group scored higher on BDI-II and STAI-State compared to OA (ps < .01); groups did not differ on Emotion Toolbox. In the PD group, bootstrapped partial correlations indicated that Negative Affect was positively related to BDI-II and STAI (ps ≤ .001). Social Satisfaction was negatively related to BDI-II and STAI-Trait (.05 < ps < .004). Psychological Well-Being was negatively related to BDI-II, AS, and STAI (p < .004). No relationships emerged in OA. In the PD group, separate binary logistic regressions showed that traditional measures (BDI-II, AS, and STAI-Trait) correctly classified 79.6% those with formal psychiatric diagnoses (presence vs. absence; p < .011), whereas Emotion Toolbox measures correctly classified 73.3% (p < .019).

Conclusions

The Emotion Toolbox showed moderate-strong correlations with traditional measures in persons with PD. Even so, it did not capture the group differences between PD and OA and had a somewhat lower classification accuracy rate for persons with PD who had a formal psychiatric diagnosis than traditional measures. Together, findings question the utility of the Emotion Toolbox as a stand-alone emotion screener in PD.

Keywords: Parkinson’s disease, Emotion health, Computerized assessment

Introduction

Persons with Parkinson’s disease (PD) have prevalence rates of significant neuropsychiatric symptoms, including apathy (40%), depression (35%), and anxiety (31%), throughout their disease course (Nagy & Schrag, 2019), which contribute to decreased quality of life and increased caregiver burden (Eichel et al., 2022). Thus, appropriate and timely diagnosis of mood disorders is important for this population. Rating scales (i.e., self, clinician, and informant) are often used to help identify and monitor neuropsychiatric symptoms in PD in clinical and research settings. These measures are used to assess the presence and severity of specific mood symptoms based on classical test theory (CTT). The CTT relies on the assumption that an observed score is dependent on a random error and a true score, or the expected score of a construct that the same individual would receive if administered with an infinite number of times without error (Cappelleri, Lundy, & Hays, 2014). Many of the commonly used traditional mood measures were validated via CTT, including the Beck Depression Inventory-II (BDI-II; Beck, Steer, & Brown, 1996), the State–Trait Anxiety Inventory (STAI; Spielberger, Gorsuch, Lushene, Vagg, & Jacobs, 1970), and the Apathy Scale (AS; Starkstein et al., 1992). Although these measures have been well validated in PD, there are multiple criticisms regarding CTT-based scales. First, CTT scores are based on the number-correct scale, which is a shared estimate of measurement that is the same for all individuals and does not take individual response patterns or attributes into account (Jabrayilov, Emons, & Sijtsma, 2016). Second, CTT assumes that each item equally contributes to total score and assumes the same response options per item, which may not accurately reflect a construct (Martinez-Martin & Forjaz, 2012). Third, all outcome variables (observed score and standard deviation of error) are sample dependent and rely on norms for score interpretation, which may not be representative, especially with population changes over time (Martinez-Martin & Forjaz, 2012). Criticisms unrelated to CTT also exist for these measures. Specifically, criticisms include the predominant focus on negative emotions without questions investigating positive emotions and the misuse of cutoff scores as sole criteria for diagnoses.

To address some of these criticisms, the National Institutes of Health (NIH) sponsored development of the NIH Toolbox for Assessment of Neurological and Behavioral Function. In addition to a Cognitive Battery, the NIH Toolbox contains an Emotion Battery, that is, the NIH Toolbox Emotion Battery (Emotion Toolbox), that assesses a broad range of both positive and negative emotions across the lifespan (age range: 8–85, N = 2,551; Salsman et al., 2013) and provides composite scores for negative affect, psychological well-being, stress and self-efficacy, and social relationships. These composite scores derive from distinct subscales that are based on a person’s rating of a series of emotion descriptor statements (e.g., “I felt worthless” or “I felt cheerful”). In contrast to traditional mood measures which rely on score totals to interpret symptom severity (CTT), the Emotion Toolbox is a computer adaptive test that is based on item response theory (IRT). In brief, IRT is an adaptive modeling technique that measures the probability that an individual with a certain trait level will respond a certain way to each item and considers this within this individual’s overall response pattern, thereby increasing measurement precision (Salsman et al., 2013; Zanon, Hutz, Yoo, & Hambleton, 2016). By incorporating trait values, some argue that IRT may be better at detecting subtle changes due to its emphasis on score pattern and not just score totals, particularly for tests with at least 20 items (Jabrayilov et al., 2016). Use of IRT can also result in shorter and more efficient assessments compared to traditional rating scales based on CTT (Salsman et al., 2013).

A recent factor analysis study of the Emotion Toolbox in a large cohort of English- (N = 1,036) and Spanish-speaking persons (N = 408) found support for three rather than four emotion domains (age range = 18–85). These three domains included: Psychological Well-Being (general life satisfaction, meaning and purpose, and positive affect); Negative Affect (perceived stress, anger-affect, anger-hostility, sadness, and fear-affect); and Social Satisfaction (friendship, emotional support, instrumental support, loneliness, and perceived rejection; Babakhanyan, McKenna, Casaletto, Nowinski, & Heaton, 2018). The remaining scales did not contribute to composite scores (perceived hostility, anger-physical aggression, fear-somatic arousal, and self-efficacy). This new framework has been validated in persons with stroke, traumatic brain injury, spinal cord injuries, and diffuse gliomas (Babakhanyan et al., 2019; Carlozzi et al., 2017; Lang et al., 2017).

Due to the high prevalence of mood symptoms and disorders in persons with PD and the need for efficient, brief assessments, the broad goal of this study is to determine the utility of the Emotion Toolbox in persons with PD compared to cognitively healthy older adults (OA) using the three composite score framework (Babakhanyan, McKenna, Casaletto, Nowinski, & Heaton, 2018). We examined whether composite scores from the Emotion Toolbox mapped onto three widely used clinical mood measures of depression, anxiety, and apathy. We hypothesized that higher scores on clinical measures of depression and anxiety would be associated with increased Negative Affect on the Emotion Toolbox and that lower scores on depression, anxiety, and apathy would be associated with greater Psychological Well-Being and Social Satisfaction on the Emotion Toolbox in persons with PD as well as cognitively healthy OA.

Method

Design

The current study used a cross-sectional design to examine the correspondence between traditional and experimental measures of emotional functioning in persons with PD as compared to cognitively healthy older adults. Data were obtained from two IRB approved studies at the University of Florida. All participants provided written, informed consent in accordance with University of Florida IRB guidelines and the Declaration of Helsinki.

Participants

Participants included 32 persons diagnosed with idiopathic PD and 40 cognitively healthy OA. Exclusion criteria for both groups included: evidence of significant cognitive impairment based on scores on either the Montreal Cognitive Assessment (MoCA; Nasreddine et al., 2005) or the Dementia Rating Scale-2 (DRS-2; Jurica, Leitten, & Mattis, 2001); history of significant, uncontrolled psychiatric disturbance that would interfere with study adherence (i.e., schizophrenia and bipolar disorder); significant medical diseases (e.g., active cancer); and history of neurologic disorders affecting the brain other than PD. All persons in the PD group were diagnosed by a board-certified neurologist specializing in movement disorders and were seen as part of a multi-disciplinary evaluation for the consideration of deep brain stimulation surgery at the University of Florida Fixel Institute of Neurological Diseases. As a part of this workup, all received standard measures for staging of the severity of their motor symptoms, including the Unified Parkinson’s disease Rating Scale Motor Scale (Part III; Fahn & Elton, 1987), a neuropsychological battery including a cognitive screener (DRS-2), clinical mood scales, and a formal psychiatric evaluation by a board-certified psychiatrist. All assessments were completed “on dopa medication.”

Persons in the OA group were drawn from an ongoing NIH-funded study (R01-AG064587). Inclusion criteria of the parent study were: family history of dementia, subjective memory complaints (defined as >16 on Cognitive Change Index; Rattanabannakit et al., 2016), no evidence of mild cognitive impairment or dementia (defined as Clinical Dementia Scale = 0; Weintraub et al., 2018) and within normal limits on the NACC Unified Dataset Neuropsychological Battery (Weintraub et al., 2018), and significant medical and psychiatric disturbances.

We used previously published conversion score tables to convert from DRS-2 to MoCA scores in the PD group (Van Steenoven et al., 2014), as PD and OA groups received different cognitive screeners. As a result, one participant with PD who initially met criteria was excluded after DRS-2 to MoCA score conversion. Thus, we used both previously published recommendations for PD (MoCA <21/22) and other dementias and clinical judgment to exclude for significant cognitive impairment or dementia (Dalrymple-Alford et al., 2010; Hoops et al., 2009).

Traditional Measures of Mood Functioning

Participants completed standardized rating scales of mood and motivation, including the AS (Starkstein et al., 1992), BDI-II (Beck et al., 1996), and STAI (Spielberger, Gorsuch, Lushene, Vagg, & Jacobs, 1970). The AS (Starkstein et al., 1992) assessed symptoms of apathy over the past 4 weeks. Total scores range from 0 to 42, with higher scores representing greater apathetic symptoms and a suggested clinical cutoff of ≥14 (Leentjens et al., 2008; Starkstein et al., 1992). The BDI-II (Beck et al., 1996) measures symptoms of depression over the past 2 weeks. Total scores range from 0 to 63, with higher scores representing greater depressive symptoms and a suggested clinical cutoff of ≥14 (Leentjens, Verhey, Luijckx, & Troost, 2000; Torbey, Pachana, & Dissanayaka, 2015). The STAI (Spielberger et al., 1970) is a measure of anxiety that includes a scale of “state” anxiety and a scale of more longstanding “trait” anxiety. Total scores for each of the two STAI scales range from 20 to 80, with higher scores representing greater levels of anxiety (Dissanayaka, Torbey, & Pachana, 2015).

Experimental Measures of Emotional Health

Participants completed the Emotion Toolbox, which is a 20–30 min, iPad-administered, and non-proprietary self-report assessment of emotional health and psychological function. All scales ask participants to rate their emotional health and psychological function over the past 7 days. Using previously published factor analytic summary scores (Babakhanyan et al., 2018), 13 scales from the NIH Toolbox were used to generate three composite scores: Psychological Well-Being (general life satisfaction, meaning and purpose, and positive affect), Social Satisfaction (friendship, emotional support, instrumental support, loneliness [reverse coded], and perceived rejection [reverse coded]), and Negative Affect (perceived stress, anger-affect, anger-hostility, sadness, and fear-affect). All composite scores are described as T scores (M = 50, SD = 10). Lower scores on Psychological Well-Being and Social Satisfaction represent lower levels of positive emotion, and higher scores on Negative Affect represent higher levels of negative emotion. Additional information about the Emotion Toolbox can be found at https://www.healthmeasures.net/explore-measurement-systems/nih-toolbox.

Statistical Analysis

Separate one-way analyses of variance (ANOVAs) or chi-square tests were used to examine the PD and OA group differences in demographic characteristics. Analyses of covariance (ANCOVAs) were used to examine the group differences on the traditional and experimental measures of emotional health. Partial correlations were used to examine the relationship between the traditional and experimental emotional health measures by group. Due to the non-normality of the data, all analyses were bootstrapped, bias-corrected using 10,000 samples (Erceg-Hurn & Mirosevich, 2008). In an exploratory set of analyses, the PD group was categorized according to absence or presence of formal psychiatric diagnosis by a board-certified psychiatrist. Using this dichotomous categorization, independent t-tests were used to examine the group differences in demographic characteristics between those with and without psychiatric diagnoses. Finally, binary logistic regressions were used to examine the relationship between psychiatric diagnoses (absence vs. presence) with the traditional and experimental emotional health measures. All statistical analyses were performed using Statistical Package for the Social Sciences Version 27.

Results

Study Sample Characteristics

The initial sample size was 72 and was reduced to 70 due to low cognitive screening scores of two persons in the PD group (MoCA < 21/22). The final sample included 30 persons with PD and 40 OA. As shown in Table 1, a chi-square revealed no significant differences in sex or race distribution between groups. Separate one-way ANOVAs indicated the PD group was significantly younger, completed fewer years of education, and had lower MoCA total scores than the OA group. As such, age and years of education were used as the covariates.

Table 1.

Study sample characteristics

PD group (n = 30) OA group (n = 40) Test Statistic
Demographic characteristics
 aAge (years) 63.83 (8.28) 71.10 (4.623) 21.63*
 aEducation (years) 15.30 (2.43) 16.35 (2.190) 5.17*
 bSex (M/F) 18/11 19/21 1.34
 bRace (n) 2.50
 Black 0 1
 Multiracial 0 2
 White 30 37
Disease duration (years) 7.90 (5.34) – –
 UPDRS Part III (on medication) 21.19 (10.18) – –
 Levodopa equivalency dosage 453 (222) – –
Cognitive screener
 aMoCA total score 24.90 (1.91)§ 26.95 (2.28) 19.37*
cTraditional measures (raw total scores)
 Beck Depression Inventory-II 11.62 (7.39) 6.30 (4.43) 6.55*
 Apathy Scale 12.14 (5.83) 9.18 (3.75) 3.00
 STAI-State 39.97 (11.91) 30.50 (6.54) 8.46*
 STAI-Trait 38.33 (11.59) 31.45 (6.13) 3.53
cExperimental measures (composite T scores)
 Psychological Well-Being 48.12 (7.76) 50.70 (7.68) .60
 Negative Affect 50.32 (7.84) 46.69 (6.29) 2.15
 Social Satisfaction 50.74 (8.75) 49.59 (6.41) .54
Open in a new tab

Note: OA = older adults; UPDRS = Unified Parkinson’s Disease Rating Scale; MoCA = Montreal Cognitive Assessment; STAI = State–Trait Anxiety Inventory; ANOVA = analysis of variance. aOne-way ANOVA. bChi-square test. cBootstrapped one-way ANOVA controlling for age and education. §Converted MoCA score (Van Steenoven et al., 2014); values are represented as mean (standard deviations).

* p < .01.

Group Differences on Traditional and Experimental Measures of Emotional Health

As shown in Table 1, results of ANCOVAs using traditional measures indicated a main effect of group for the BDI-II [η2p = .09] and STAI-State [η2p = .11] but not for AS [η2p = .05] or STAI-Trait [η2p = .05]. There were no significant differences between the PD and OA groups for any of the Emotion Toolbox composites (Negative Affect [η2p = .03], Psychological Well-Being [η2p = .01], or Social Satisfaction [η2p = .01]). Bonferroni bootstrapped, bias-corrected post hoc tests indicated that the PD group had significantly higher scores compared to the OA group on BDI-II (mean difference = 4.30, SE = 1.62, 95% confidence interval [CI] = 1.57–8.05, p = .017) and STAI-State (mean difference = 7.46, SE = 2.50, 95% CI = 2.91–12.29, p = .008).

Relationship Between Traditional and Experimental Measures of Emotional Health

As shown in Table 2, in the PD group, bootstrapped, bias-corrected partial correlations indicated that Negative Affect from the Emotion Toolbox was significantly and positively related to the BDI-II, STAI-State, and STAI-Trait but not to the AS. Psychological Well-Being was significantly and negatively related to BDI-II and STAI-Trait but not to the AS or STAI-State. Finally, Social Satisfaction was significantly and negatively related to the BDI-II, AS, STAI-State, and STAI-Trait. By contrast, no significant relationships were found in the OA group.

Table 2.

Bootstrapped partial Pearson correlations between traditional and experimental measures of emotional functioning in persons with PD and older adults (OA) controlling for age and years of education

Negative Affect 95% CI (Lower, Upper) Psychological Well-Being 95% CI (Lower, Upper) Social Satisfaction 95% CI (Lower, Upper)
PD group (n = 30)
 BDI-II .651*** .462, .880 −.507** −.753, −.220 −.498** −.812, −.235
 AS .293 .053, .665 −.355 −.705, −.155 −.455*** −.740, −.122
 STAI-State .703*** .394, .877 −.351 −.722, −.120 −.624*** −.840, −.426
 STAI-Trait .722*** .594, .879 −.380* −.739, −.270 −.567*** −.822, −.386
OA group (n = 40)
 BDI-II .240 −.108, .515 −.241 −.592, .121 −.240 −.516, .075
 AS .035 −.317, .328 −.112 −.430, .329 −.229 −.542, .204
 STAI-State .243 −.061, .482 −.223 −.477, −.006 −.170 −.481, .173
 STAI-Trait .084 −.355, .464 −.269 −.636, .216 −.292 −.603, .139
Open in a new tab

Note: 95% CI = 95% confidence interval; BDI-II = Beck Depression Inventory-II; AS = Apathy Scale; STAI = State–Trait Anxiety Inventory.

* p < .05; **p < .004; ***p ≤ .001.

Relationship With Psychiatric Diagnoses in the PD Group

Based on these findings, the question arose as to whether the traditional and experimental emotion health measures map onto formal psychiatric diagnoses. However, the current study did not have data to fully answer this question as only the PD group underwent formal psychiatric evaluation by board-certified psychiatrists at the time of completion of the mood and NIH Toolbox measures. Thus, we performed an exploratory set of analyses in the PD group: PD patients with and without formal psychiatric diagnoses.

Approximately, 56% of persons in the PD group received at least one formal psychiatric diagnosis. The most common psychiatric diagnoses were anxiety (n = 7), depression (n = 4), and combined depression-anxiety (n = 5, including 1 with post-traumatic stress disorder) followed by history of hallucinations (n = 1). All data were normal (Kolmogorov–Smirnov, Shapiro–Wilk; all ps < .05). Based on independent samples’ t-tests, PD patients with formal psychiatric diagnoses (n = 17) had significantly worse scores across all traditional and experimental measures than PD patients without (n = 13; all ps < .015), except for AS and Social Satisfaction. The PD subgroups did not differ in terms of demographic, or disease-related characteristics (all ps > .05). Results of a logistic regression indicated traditional measures (BDI-II, AS, and STAI-Trait) explained 44% of the variance in PD psychiatric diagnoses (absence vs. presence) and correctly classified 79.6% of cases (X2(3) = 11.07, p < .011, Nagelkerke R2 = .442). However, none of the traditional measures alone were significant predictors. See Table 3.

Table 3.

Binary logistic regressions using traditional measures to predict psychiatric diagnosis status in the PD group

β SE β Exp β p 95% confidence interval
Lower Upper
BDI-II .061 .146 1.063 .674 .799 1.415
AS −.083 .120 .920 .488 .727 1.164
STAI-Trait .143 .097 1.154 .139 .955 1.395
Open in a new tab

Note: BDI-II = Beck Depression Inventory-II; AS = Apathy Scale; STAI = State–Trait Anxiety Inventory.

Similarly, experimental measures (Negative Affect, Psychological Well-Being, and Social Satisfaction) explained 37% of the variance in PD psychiatric diagnoses (absence vs. presence) and correctly classified 73.3% of cases (X2(3) = 9.92, p < .019, Nagelkerke R2 = .378). However, none of the Emotion Toolbox composite scales were significant predictors alone. See Table 4.

Table 4.

Binary logistic regressions using experimental measures to predict psychiatric diagnosis status in the PD group

β SE β Exp β p 95% confidence interval
Lower Upper
Negative Affect .143 .083 1.153 .086 .980 1.357
Psychological Well-Being −.116 .079 .890 .143 .762 1.040
Social Satisfaction .043 .065 1.044 .509 .919 1.186
Open in a new tab

Discussion

This study aimed to learn how the Emotion Toolbox compared to traditional clinical measures in a sample of non-demented persons with PD and cognitively healthy OA. This question was driven, in part, by the premise that the Emotion Toolbox might have promise as a brief screener of emotional health and psychological function in clinical and research settings (Gershon et al., 2013). There were two prominent findings. First, persons with PD had significantly higher depression (BDI-II) and state anxiety (STAI) scores than OA, which is consistent with prior studies. By contrast, the two groups did not differ across Emotion Toolbox composites (Negative Affect, Psychological Well-Being, and Social Satisfaction). This finding raises the possibility that the Emotion Toolbox may be less sensitive in detecting traditional psychological distress than conventional mood measures of depression/state anxiety in PD. One test of this hypothesis might involve how well traditional and experimental measures can predict formal psychiatric diagnoses. We did not have data to fully test this hypothesis, as the OA in our sample did not receive formal evaluations by board-certified psychiatrists; by contrast, persons with PD did receive as a part of the routine clinical care. Over half of persons with PD (i.e., 56%) received formal psychiatric diagnoses, with 94% of diagnoses indicating anxiety and/or depression. Using binary logistic regression, we found that both traditional and experimental emotion measures each significantly predicted the presence/absence of a psychiatric diagnosis in PD, though percent classification was somewhat better for traditional measures (79.6%) than experimental measures (73.3%). Although promising, a stronger test of the clinical utility of the Emotion Toolbox would require a broader range of persons who received psychiatric evaluations beyond PD.

Broadly, previous studies using the Emotion Toolbox in neurological samples have been mixed. A previous study investigated the use of the Emotion Toolbox for individuals with gliomas using the three factor domains as described by Babakhanyan et al. (2018) without the inclusion of a demographically similar control group (Lang et al., 2017). By contrast, Carlozzi et al. (2017) found poorer scores on emotion constructs on the Emotion Toolbox compared to an age-matched normative group. However, neither study compared the Emotion Toolbox against the traditional or clinically validated measures of emotion health. One possible reason for our findings is the fact that traditional psychological measures are more targeted and focus more on negative emotion constructs than the Emotion Toolbox, which has a more holistic focus of emotional health. Together, by comparing against traditional clinical measures, the current study extends prior work by suggesting that the Emotion Toolbox may capture different aspects of emotion health in PD.

Second traditional measures of depression and anxiety were strongly related to negative affect on the Emotion Toolbox, though this relationship was only observed in persons with PD and not OA. This pattern varied slightly with positive emotion measures (Psychological Well-Being and Social Satisfaction) from the Emotion Toolbox but in the expected directions (i.e., higher Psychological Well-Being and higher Social Satisfaction were related to lower scores across most traditional measures). Failure to find a relationship in OA may, in part, simply reflects less variability in the degree of psychological distress experienced by this group as reflected in their scores on traditional and experimental measures. Another possible interpretation is that this study was underpowered to detect relationships in OA. Alternatively, the Emotion Toolbox composites may measure different latent constructs depending on the group. Prior studies have used factor analysis to investigate the underlying structure of the Emotion Toolbox across the lifespan (i.e., factor invariance; Babakhanyan et al., 2018; Paolillo et al., 2020), but it remains unknown whether these composites are measuring the same latent construct across clinical and non-clinical groups (i.e., measurement invariance). Addressing this would require larger cohorts of clinical and non-clinical populations to explore the factor and measurement invariance of the Emotion Toolbox.

This study has several limitations. First, our findings may have limited generalizability, given the educational status, socio-economic statuses, and/or minoritized ethno-racial identities of the participants in this study. Relatedly, our modest sample size may have been underpowered to detect relationships in OA, which further limits the generalizability. Nevertheless, group differences between the OA and PD groups reflected medium-to-large effect sizes. Further, partial correlations were strong in the PD group, whereas, by contrast, were weak in the OA group. Second, our PD participants had formal psychiatric diagnoses available and were drawn from a clinical sample (who consented to participate in research), whereas our OA group was drawn from a research sample. Further, we do not have access to information regarding the duration and treatment for psychiatric conditions in the PD group. Together, this likely increased the likelihood of differences in the psychopathology between groups. Third, Emotion Toolbox uses a 7-day time period for all scales and is shorter than the 2–4-week time period for the traditional clinical measures (apart from STAI-State), which may viewed as another potential limitation.

Taken together, findings add to the literature by highlighting that the Emotion Toolbox may not fully capture the emotional health in persons with PD relative to standard traditional clinical measures despite moderate to strong correlation between the two. Future work should continue to determine the utility of the Emotion Toolbox in a larger cohort of individuals with PD to determine the efficacy of this measure, which could be compared with other neurological populations. More broadly, future studies should examine the Emotion Toolbox with formal psychiatric diagnoses as well as factor and measurement invariance in larger cohorts of clinical and non-clinical populations. Lastly, longitudinal studies should be conducted to determine the utility of the Emotion Toolbox to detect the change in emotional health over time compared to traditional psychological assessments for both clinical and non-clinical populations.

Acknowledgements

We would like to thank all the participants for their contributions to this study, UF Norman Fixel Institute for Neurological Diseases, and members of the Cognitive Neuroscience Laboratory for their help in data collection.

Contributor Information

Francesca V Lopez, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA.

Rachel Schade, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA.

Adrianna Ratajska, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA.

Lauren Kenney, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA.

Katie Rodriguez, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA.

Alyssa Ray, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA.

Lauren Santos, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA.

Bonnie M Scott, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA.

Erin Trifilio, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA.

Dawn Bowers, Department of Clinical and Health Psychology, College of Public Health and Health Professions, University of Florida, Gainesville, FL 32610, USA; Department of Neurology, Norman Fixel Institute for Neurological Diseases, University of Florida, Gainesville, FL 32610, USA.

Funding

This work was supported by grants from the National Institutes of Health (T32NS082168 to DB and AR, R01AG064587 to DB, and F31AG071264 to FVL).

Conflict of Interest

None declared.

References

  1. Babakhanyan, I., Carlozzi, N. E., McKenna, B. S., Casaletto, K. B., Heinemann, A. W., & Heaton, R. K. (2019). National Institutes of Health toolbox emotion battery: application of summary scores to adults with spinal cord injury, traumatic brain injury, and stroke. Archives of Physical Medicine and Rehabilitation, 100(10), 1863–1871. 10.1016/j.apmr.2019.05.024. [DOI] [PubMed] [Google Scholar]
  2. Babakhanyan, I., McKenna, B. S., Casaletto, K. B., Nowinski, C. J., & Heaton, R. K. (2018). National Institutes of Health Toolbox Emotion Battery for English-and Spanish-speaking adults: Normative data and factor-based summary scores. Patient Related Outcome Measures, 9, 115–127. 10.2147/PROM.S151658. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Beck, A. T., Steer, R. A., & Brown, G. K. (1996). Beck depression inventory-II. San Antonio. 78(2), 490–498. [Google Scholar]
  4. Cappelleri, J. C., Lundy, J. J., & Hays, R. D. (2014). Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clinical Therapeutics, 36(5), 648–662. 10.1016/j.clinthera.2014.04.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Carlozzi, N. E., Goodnight, S., Casaletto, K. B., Goldsmith, A., Heaton, R. K., Wong, A. W. K., et al. (2017). Validation of the NIH toolbox in individuals with neurologic disorders. Archives of Clinical Neuropsychology, 32(5), 555–573. 10.1093/arclin/acx020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Dalrymple-Alford, J. C., MacAskill, M. R., Nakas, C. T., Livingston, L., Graham, C., Crucian, G. P., et al. (2010). The MoCA: Well-suited screen for cognitive impairment in Parkinson disease. Neurology, 75(19), 1717–1725. 10.1212/WNL.0b013e3181fc29c9. [DOI] [PubMed] [Google Scholar]
  7. Dissanayaka, N. N., Torbey, E., & Pachana, N. A. (2015). Anxiety rating scales in Parkinson's disease: A critical review updating recent literature. International Psychogeriatrics, 27(11), 1777–1784. 10.1017/S1041610215000885. [DOI] [PubMed] [Google Scholar]
  8. Eichel, H. V., Heine, J., Wegner, F., Rogozinski, S., Stiel, S., Groh, A., et al. (2022). Neuropsychiatric symptoms in Parkinson’s disease patients are associated with reduced health-related quality of life and increased caregiver burden. Brain Sciences, 12(1), 89. 10.3390/brainsci12010089. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Erceg-Hurn, D. M., & Mirosevich, V. M. (2008). Modern robust statistical methods: An easy way to maximize the accuracy and power of your research. American Psychologist, 63(7), 591–601. 10.1037/0003-066X.63.7.591. [DOI] [PubMed] [Google Scholar]
  10. Fahn, S., & Elton, R. L. (1987). UPDRS program members. Unified Parkinsons disease rating scale. Recent developments in Parkinson’s disease, 2, 153–163. [Google Scholar]
  11. Geisser, M. E., Roth, R. S., Theisen, M. E., Robinson, M. E., & Riley, J. L., III (2000). Negative affect, self-report of depressive symptoms, and clinical depression: relation to the experience of chronic pain. The Clinical Journal of Pain, 16(2), 110–120. 10.1097/00002508-200006000-00004. [DOI] [PubMed] [Google Scholar]
  12. Gershon, R. C., Wagster, M. V., Hendrie, H. C., Fox, N. A., Cook, K. F., & Nowinski, C. J. (2013). NIH toolbox for assessment of neurological and behavioral function. Neurology, 80(11 Supplement 3), S2–S6. 10.1212/WNL.0b013e3182872e5f. [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Hoops, S., Nazem, S., Siderowf, A. D., Duda, J. E., Xie, S. X., Stern, M. B., et al. (2009). Validity of the MoCA and MMSE in the detection of MCI and dementia in Parkinson disease. Neurology, 73(21), 1738–1745. 10.1212/WNL.0b013e3181c34b47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Jabrayilov, R., Emons, W. H., & Sijtsma, K. (2016). Comparison of classical test theory and item response theory in individual change assessment. Applied Psychological Measurement, 40(8), 559–572. 10.1177/0146621616664046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Jurica, P. J., Leitten, C. L., & Mattis, S. (2001). Dementia rating Scale-2: DRS-2: professional manual. Psychological Assessment Resources. [Google Scholar]
  16. Lang, S., Cadeaux, M., Opoku-Darko, M., Gaxiola-Valdez, I., Partlo, L. A., Goodyear, B. G., et al. (2017). Assessment of cognitive, emotional, and motor domains in patients with diffuse gliomas using the National Institutes of Health Toolbox Battery. World Neurosurgery, 99, 448–456. 10.1016/j.wneu.2016.12.061. [DOI] [PubMed] [Google Scholar]
  17. Leentjens, A. F., Dujardin, K., Marsh, L., Martinez-Martin, P., Richard, I. H., Starkstein, S. E., et al. (2008). Apathy and anhedonia rating scales in Parkinson's disease: Critique and recommendations. Movement Disorders, 23(14), 2004–2014. 10.1002/mds.22229. [DOI] [PubMed] [Google Scholar]
  18. Leentjens, A. F., Verhey, F. R., Luijckx, G. J., & Troost, J. (2000). The validity of the Beck Depression Inventory as a screening and diagnostic instrument for depression in patients with Parkinson's disease. Movement Disorders, 15(6), 1221–1224. . [DOI] [PubMed] [Google Scholar]
  19. Martinez-Martin, P., & Forjaz, M. J. (2012). How to evaluate validation data. Rating scales in Parkinson’s disease, 16–41. 10.1093/med/9780199783106.003.0041. [DOI] [Google Scholar]
  20. Nagy, A., & Schrag, A. (2019). Neuropsychiatric aspects of Parkinson’s disease. Journal of Neural Transmission, 126(7), 889–896. 10.1007/s00702-019-02019-7. [DOI] [PubMed] [Google Scholar]
  21. Nasreddine, Z. S., Phillips, N. A., Bédirian, V., Charbonneau, S., Whitehead, V., Collin, I., et al. (2005). The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. Journal of the American Geriatrics Society, 53(4), 695–699. 10.1111/j.1532-5415.2005.53221.x. [DOI] [PubMed] [Google Scholar]
  22. Paolillo, E. W., McKenna, B. S., Nowinski, C. J., Thomas, M. L., Malcarne, V. L., & Heaton, R. K. (2020). NIH Toolbox® emotion batteries for children: Factor-based composites and norms. Assessment, 27(3), 607–620. 10.1177/1073191118766396. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Rattanabannakit, C., Risacher, S. L., Gao, S., Lane, K. A., Brown, S. A., McDonald, B. C., et al. (2016). The cognitive change index as a measure of self and informant perception of cognitive decline: Relation to neuropsychological tests. Journal of Alzheimer's Disease, 51(4), 1145–1155. 10.3233/JAD-150729. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Salsman, J. M., Butt, Z., Pilkonis, P. A., Cyranowski, J. M., Zill, N., Hendrie, H. C., ... & Cella, D. (2013). Emotion assessment using the NIH Toolbox. Neurology, 80(11 Supplement 3), S76–S86. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Spielberger, C. D., Gorsuch, R. L., Lushene, R. E., Vagg, P. R., & Jacobs, G. A. (1970). Manual for the state-trait inventory. Palo Alto, CA: Consulting Psychologists. [Google Scholar]
  26. Starkstein, S. E., Mayberg, H. S., Preziosi, T., Andrezejewski, P., Leiguarda, R., & Robinson, R. G. (1992). Reliability, validity, and clinical correlates of apathy in Parkinson’s disease. The Journal of Neuropsychiatry and Clinical Neurosciences, 4(2), 134–139. 10.1176/jnp.4.2.134. [DOI] [PubMed] [Google Scholar]
  27. van  Steenoven, I., Aarsland, D., Hurtig, H., Chen-Plotkin, A., Duda, J. E., Rick, J., et al. (2014). Conversion between mini-mental state examination, Montreal cognitive assessment, and dementia rating scale-2 scores in Parkinson's disease. Movement Disorders, 29(14), 1809–1815. 10.1002/mds.26062. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Torbey, E., Pachana, N. A., & Dissanayaka, N. N. (2015). Depression rating scales in Parkinson's disease: A critical review updating recent literature. Journal of Affective Disorders, 184, 216–224. 10.1016/j.jad.2015.05.059. [DOI] [PubMed] [Google Scholar]
  29. Weintraub, S., Besser, L., Dodge, H. H., Teylan, M., Ferris, S., Goldstein, F. C., et al. (2018). Version 3 of the Alzheimer Disease Centers’ neuropsychological test battery in the Uniform Data Set (UDS). Alzheimer Disease and Associated Disorders, 32(1), 10–17. 10.1097/WAD.0000000000000223. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Zanon, C., Hutz, C. S., Yoo, H. H., & Hambleton, R. K. (2016). An application of item response theory to psychological test development. Psicologia: Reflexão e Crítica, 29. [Google Scholar]

Articles from Archives of Clinical Neuropsychology are provided here courtesy of Oxford University Press

RESOURCES