Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2019 Aug 1.
Published in final edited form as: Rehabil Psychol. 2018 Jul 19;63(3):365–371. doi: 10.1037/rep0000216

A Comparison of PHQ-9 and TBI-QOL Depression Measures among Individuals with Traumatic Brain Injury

Matthew L Cohen 1, James A Holdnack 2, Pamela A Kisala 3, David S Tulsky 4
PMCID: PMC6095809  NIHMSID: NIHMS978613  PMID: 30024199

Abstract

Purpose/Objective:

To compare and contrast how individuals with traumatic brain injury (TBI) are classified (positive or negative screen) by different cut-offs on two self-report measures of depressive symptoms: the PHQ-9, which assesses somatic symptoms, and the TBI-QOL Depression item bank, which does not.

Research Method/Design:

385 individuals with TBI were recruited from 6 rehabilitation hospitals in the U.S. as part of the calibration data collection for the TBI-QOL patient-reported outcome measurement system.

Results:

The TBI-QOL and PHQ-9 total scores correlated strongly (disattenuated r = .83). The correlation was even stronger (disattenuated r = .92) when the 4 PHQ-9 somatic items were removed from the total score. When the PHQ-9 was scored traditionally, the rate of agreement was approximately 80–85% using standard cut-offs for each scale. Depending on the cut-off score, 23–26% of participants screened positive on the PHQ-9, whereas 9–38% screened positive on the TBI-QOL Depression. Individuals who screened positive on the PHQ-9 alone reported more somatic symptoms than those who screened positive on the TBI-QOL alone. Individuals who screened positive on the TBI-QOL alone were at slightly greater risk for other negative psychological functioning than individuals who screened positive on the PHQ-9 alone.

Conclusions/Implications:

The PHQ-9 and TBI-QOL Depression performed similarly in screening for depressive symptoms among individuals with TBI. The PHQ-9 identified more individuals with somatic symptoms, which may overlap with other medical issues, while the TBI-QOL Depression instrument identified more individuals who reported other forms ofemotional distress.

Keywords: Traumatic brain injury, Depression, Psychological Assessment

The prevalence of depression among individuals with traumatic brain injury (TBI) is greater than in the general population (Alway, Gould, Johnston, McKenzie, & Ponsford, 2016a; Bombardier et al., 2010; Scholten et al., 2016). Cognitive symptoms of depression include perceived poor concentration and memory. Emotional symptoms include feelings of sadness, worthlessness, guilt, and hopelessness. Somatic symptoms include disrupted appetite and sleep cycles, physical sensations and pain, and psychomotor slowing. These symptoms are distressing in and of themselves, and post-TBI depression has been associated with a wide variety of negative outcomes (Hart et al., 2011; Whitnall, McMillan, Murray, & Teasdale, 2006)(Fann, Katon, Uomoto, & Esselman, 1995)(Rockhill et al., 2012; Wei, Sambamoorthi, Crystal, & Findley, 2005)(Koskinen, 1998).

The Patient Health Questionnaire [PHQ]-9 (Kroenke et al., 2001) is one of the most commonly used screening measures of depression after TBI and in general. It has been shown to have good psychometric properties with adults with TBI (Bombardier et al., 2010; Bombardier, Hoekstra, Dikmen, & Fann, 2016; Dyer et al., 2015; Fann et al., 2005), but there is also some disagreement in the literature about whether screening instruments for individuals with TBI should assess somatic symptoms, as the PHQ-9 does. On one hand, there is concern that somatic symptoms (e.g., fatigue, sleep disturbance) may be attributable to the TBI itself rather than to depression (Gass & Russell, 1991; Gass & Wald, 1997; Kim et al., 2007; Rosenthal, Christensen, & Ross, 1998). Indeed, emotional symptoms of depression may have more diagnostic utility after TBI than vegetative symptoms (Jorge, Robinson, & Arndt, 1993). On the other hand, it has been argued that somatic symptoms are part of a clinical diagnosis and there are strong data that items that assess somatic symptoms are not biased against individuals with TBI (Cook et al., 2011; Dyer, Williams, Bombardier, Vannoy, & Fann, 2015).

The Traumatic Brain Injury Quality of Life (TBI-QOL) measurement system (Tulsky, Kisala, et al., 2016) is a new set of assessment instruments (item banks) that were developed with item response theory (IRT) methods. It is psychometrically linked with two other modern measurement systems - the Patient Reported Outcome Measurement Information System (PROMIS)(Cella et al., 2010; Cella, Yount, et al., 2007) and the Neurological Quality of Life (Neuro-QOL)(Cella et al., 2012; Gershon et al., 2012) measurement system. For constructs like depression that are assessed with a TBI-QOL item bank and also with a PROMIS and/or Neuro-QOL item bank, the TBI-QOL score can be transformed to the PROMIS metric (Tulsky, Kisala, et al., 2016), which permits comparison against the PROMIS normative sample, which mirrors the demographics of the 2000 U.S. Census. Because of the uncertainty about assessing somatic symptoms of depression after TBI, the TBI-QOL Depression item bank only includes items related to the cognitive and emotional aspects of depression.

The purpose of the present investigation is to compare and contrast how the PHQ-9 and the TBI-QOL Depression item bank classify individuals with TBI. Specifically, we will examine the correlation between the two tests and identify the rates and characteristics of individuals who screen positive on only one test, both tests, or neither test, when applying different cut-offs. When individuals screen positive on only one test, individual items will be evaluated to determine which ones influence the discrepant classifications. The groups will also be compared on other assessments of emotional functioning to test whether they report other emotional problems. Ultimately, it is our goal to understand how the two tests differ in identifying levels of depressive symptoms after TBI, which may help clinicians and researchers use these tools more effectively.

Methods

Participants and Procedures

Participants were 385 adults with TBI, ages 18–85, who were recruited from six rehabilitation hospitals to participate a research study designed to calibrate and validate the TBI- QOL emotional health measures (Tulsky, Kisala, et al., 2016). The full demographic and injury characteristics of the sample are presented in Table 1. Although this sample was originally recruited to be the calibration sample for the TBI-QOL measures, it was diverse and well-suited to this analysis also. Although the time-since-injury was variable, depressive symptoms are often chronic/recurrent after TBI (Alway, Gould, Johnston, McKenzie, & Ponsford, 2016b). All study measures were administered in interview format by a trained examiner, either in person or over the phone, which have been shown to be equivalent forms of administration (Wang, Chen, Usinger, & Reeve, 2017). For full details of the study methodology the reader is referred to Tulsky et al. (2016). The Institutional Review Board at each site reviewed and approved this study.

Table 1.

Demographic Characteristics

Variable TBI Sample
(n=385)
Age (Years) Mean (SD) 39.46 (14.3)
Years Since Injury: Mean (SD) 5.2 (6.1)
Gender (%) Male 70.1
Female 29.1
Race (%) Caucasian 59.5
African American 20.8
Other 16.1
Unknown 3.6
Ethnicity (%) Not Hispanic or Latino 87.1
Hispanic or Latino 12.9
Education (%) High school or less 40.0
Some college 35.8
Bachelor’s degree or more 24.2
Severity (%) Mild 18.4
Moderate 8.3
Severe 54.3
Unknown 19.0
Open in a new tab

Measures

The primary measures were the PHQ-9 and the TBI-QOL Depression item bank. The PHQ-9 can be scored in different ways, either by using a simple sum score based upon adding responses to the items or by using an algorithm based on the DSM diagnostic criteria for Major Depressive Disorder (MDD) (Manea, Gilbody, & McMillan, 2012, 2015). Because the algorithm-based scoring method was shown to have worse sensitivity to MDD than a sum score cut-off of 10 (Manea et al., 2015), the algorithm-based scoring method is not evaluated here. Because a cut-off of 11 (i.e. ≥11) may have even better diagnostic accuracy (Manea et al.,2012), we evaluated cutoffs of both 10 and 11. Four of the nine PHQ-9 items assess somatic symptoms of depression: sleep disturbance, fatigue, changes in appetite, and psychomotor slowing. The remaining five items assess non-somatic (i.e., cognitive or emotional) symptoms of depression: anhedonia, depressed mood, low self-esteem, disturbed attention and thinking, and suicidal ideation. In addition to interpreting the PHQ-9 total score, analyses presented here interpret two subtotals based on somatic or non-somatic symptoms.

As also mentioned above, the TBI-QOL Depression item banks surveys the emotional and cognitive aspects of depression, but not the somatic aspects. The item bank includes 28 items, and here we administered the full item bank to participants. Each item has 5 response options: Never, Rarely, Sometimes, Often, and Always. Scores of 1 to 5 are assigned to each item and the total scale is reported on the PROMIS T metric where a mean of 50 represents the mean of the general population. While there currently are no studies validating specific cut-off scores for the TBI-QOL, here we tested common T score cut-offs for a positive screen: T≥55, T ≥60, and T≥65.

Additional measures from the TBI-QOL were used as concurrent validity measures, including the Anxiety, Grief/Loss, Self-Esteem, Resilience, and Positive Affect and Well Being item banks (Tulsky, Kisala, et al., 2016). These can be divided into two domains, negative psychological experiences (high scores indicate greater symptom burden) and positive psychological functioning (high scores indicate better functioning or adjustment) (Table 2).

Table 2.

Negative and Positive Emotion Domain Item Banks

TBI-QOL Item Bank Description of Content
Domain: Negative Emotion
Anxiety fearfulness, panic, anxious, misery, and hyperarousal
Grief and Loss emotional reactions of grief that occur in response to sustaining an TBI, such as anger, guilt, anxiety, sadness, and despair.
Domain: Positive Emotion
Self-Esteem emotional, evaluative, and cognitive perceptions of personal competence and worth.
Resilience motivation, coping, and acceptance
Positive Affect & Well-Being sense of well-being, life satisfaction, or an overall sense of purpose and meaning
Open in a new tab

Data Analysis

All data were analyzed using IBM SPSS version 23. The first analysis examines the construct overlap between the TBI-QOL and PHQ-9 using Pearson correlations that were adjusted for the unreliability of the measures, producing a “disattenuated” correlation coefficient (Charles, 2005). The second analysis identifies rates of positive screens using two previously published criteria for the PHQ-9 (Bombardier et al., 2012) (total scores of ≥10 and ≥11) alongside various cut-offs of the TBI-QOL depression measure (T≥5, 60, and 65).

The third set of analyses compares groups of examinees that screen positive on one or both instruments. Multivariate analysis of variance (MANOVA) are used to compare 4 groups of subjects: those who screen positive on both measures (PHQ+/TBI-QOL+), those who screen negative on both measures (PHQ-/TBI-QOL-), those who screen positive on the PHQ-9 only (PHQ+/TBI-QOL-), and those who screen positive on the TBI-QOL only (PHQ-/TBI-QOL+). The dependent measures for the analyses include the 9 item-level scores from the PHQ-9 and the additional emotional health outcome measures described above. Bonferroni corrections are used for post-hoc analyses of between-group differences.

For the final analysis, the TBI-QOL emotional heath outcome measures are classified into two domains of functioning: negative emotion (Anxiety, Grief/Loss, Stigma, and Psychological Trauma) and positive emotion (Resilience, Self-esteem, Positive Affect and Well Being). Each individual item bank score is converted to a dichotomous score (1 or 0) to indicate whether the score reflects problematic functioning (e.g., t-score ≥ 60 or t-score ≤ 40, depending on the scoring direction of the test). These scores are tallied within domains and base rates of elevated scores are identified for each of the four PHQ/TBI-QOL groups. This analysis demonstrates the risk for poor outcomes based on group membership.

Results

Correlation of TBI-QOL Depression and PHQ-9

The disattenuated correlation between the TBI-QOL Depression scale and the PHQ-9 total score was r = .83 (p;< 001), When the 4 somatic items were excluded, the disattenuated correlation with TBI-QOL was very strong (r = .92, p;< 001), These results demonstrated that the TBI-QOL and PHQ-9 have a high degree of construct overlap.

Base Rates of Screening Positive in this TBI Sample

Table 3 presents the rates of screening positive when applying two cut-offs for the PHQ-9 and the three cut-offs for the TBI-QOL. On the PHQ-9, applying a cut-off of 10 resulted in 26.0% of participants screening positive while a cut-score of 11 resulted in 22.6% screening positive. Applying cut-scores of T≥55, T≥60, and T≥65 on TBI-QOL Depression scale resulted in 37.9%, 22.0%, and 9.4% of participants screening positive, respectively. A commonly applied T-score of 60 identified nearly identical numbers of positive screens as a PHQ-9 cut-off of ≥11.

Table 3.

Base Rates of Depression and Agreement Rates Between the PHQ-9 and TBI-QOL Depression at Various Cut-offs

Base Rate TBI-QOL > 60 TBI-QOL > 55 TBI-QOL > 65
Base
Rate		 % agree P+/T- Base
Rate		 % agree P+/T- Base
Rate		 % agree P+T-
Total PHQ-9 score ≥ 10 26.0% 22% 82.9% 10.4% 37.0% 80.3% 3.9% 9.4% 82.3% 17.1%
Total PHQ-9 score ≥ 11 22.6% - 84.7% 7.5% - 81.0% 2.1% - 84.2% 14.5%
Open in a new tab

Note: % agree includes both depressed and non-depressed agreement rates, P+/T- = indicates the PHQ-9 was positive for depression but the TBI-QOL was negative for depression.

PHQ-9 and TBI-QOL Agreement

The rates of agreement between the two PHQ-9 cut-scores and the three TBI-QOL Depression scale cut-scores are reported in Table 3. Test agreement was defined as instances when an individual screened positive or negative on both measures. The agreement rate ranged from 82.9% to 84.7% when TBI-QOL T-score ≥ 60. The disagreement rates where PHQ (cut-offs 10 and 11) was positive for depression but TBI-QOL (T-Score ≥ 60) was negative were 10.4% and 7.5%, respectively. The overall disagreement rate was split approximately equally between TBI-QOL screening positive and screening negative compared to the PHQ-9. Therefore, disagreement between the measures is not due to a lack of TBI-QOL sensitivity.

When applying a more liberal criterion for the TBI-QOL (T-score ≥ 55), the rates of agreement between PHQ-9 and TBI-QOL dropped considerably. This was because many more individuals screened positive on the TBI-QOL alone. In general, the two tests agreed more often for negative screens than positive screens. The restrictiveness of the cut-offs that were applied played a significant role in the agreement rate.

Identifying Symptoms Related to Discrepant Classification

To identify which PHQ-9 items influenced disagreement between the tests, four groups of cases were created as described previously. For the purpose of this study, the groups were created using criteria of PHQ-9 ≥11 and TBI-QOL depression T-score ≥ 60. This combination of cut-offs had a good degree of agreement and also identified a sufficient number of positive screen cases for the remaining analyses.

[Table 4 presents descriptive data that compare the four groups. MANOVA of the four agreement/disagreement groups with the 9 items of the PHQ-9 was highly significant (Pillai’s trace=1.017, F(27, 1125) =21.36, p<.001). All univariate effects were also significant (p<.001). Post-hoc analyses (Bonferroni) identified that the PHQ+/TBI-QOL-group and PHQ-/TBI-QOL+ group had significantly different scores for PHQ-9 items 3, 4, 5, 7, and 8. A second MANOVA compared the 4 groups on the PHQ-9 total score, non-somatic items total score, and somatic items total score. The 4 groups were significantly different (Pillai’s trace=.865, F(6,762) =96.88, p<.001) at both the multivariate and univariate levels (p < .001). The PHQ+/TBI-QOL- and PHQ-/TBI-QOL+ groups were also significantly different on the PHQ-9 total score and somatic score, but not the non-somatic score. In both analyses, the results indicated that when the two tests disagreed, it related primarily to the items with somatic content resulting in higher PHQ-9 scores overall. The two agreement groups (i.e., PHQ+/TBI-QOL+ versus PHQ-/TBI-QOL-), were significantly different from each other and from the two disagreement groups.

Table 4.

Comparison of PHQ-9 Scores for Agreement/Disagreement Groups (Criteria PHQ ≥11 and TBI-QOL ≥ 60)
PHQ+ /TBI-QOL+ PHQ-/TBI-QOL- PHQ+ /TBI-QOL- PHQ-/TBI-QOL+
mean SD mean SD mean SD mean SD
PHQ-9 TotalA 15.42 3.45 3.51 2.90 13.53 2.27 7.41 1.96
Item 1 1.54 0.87 0.33 0.60 1.37 1.13 1.10 0.86
Item 2 2.02 0.88 0.31 0.56 1.13 0.94 1.21 0.82
Item 3A 2.07 1.10 0.75 0.93 2.40 0.89 0.79 1.08
Item 4A 2.14 0.97 0.69 0.82 2.53 0.78 1.00 0.80
Item 5A 1.44 1.18 0.37 0.67 1.93 1.08 0.76 0.95
Item 6 2.23 0.82 0.34 0.62 0.93 0.94 1.14 1.03
Item 7A 1.72 1.19 0.45 0.75 2.00 1.05 0.90 0.94
Item 8A 1.53 1.17 0.25 0.58 1.07 1.23 0.21 0.56
Item 9 0.74 0.95 0.02 0.15 0.17 0.59 0.31 0.60
Non Somatic 8.25 2.54 1.45 1.63 5.60 2.01 4.66 1.97
SomaticA 7.18 2.27 2.06 1.89 7.93 1.78 2.76 1.66
Open in a new tab

Note: n=PHQ+/TBI-QOL+ (57), PHQ-/TBI-QOL- (269), PHQ+/TBI-QOL- (30), PHQ-/TBI-QOL+ (29)

A

PHQ+/TBI-QOL- differs from PHQ-/TBI-QOL+

An additional ANOVA compared the 4 groups on the TBI-QOL Depression measure to determine if symptom severity differed when the two tests did not agree. The overall effect for group was significant (F(3,381)=184.39,P< .001). Post-hoc analysis (Bonferroni) indicated that each group was significantly different from one another on TBI-QOL Depression scale except the PHQ+/TBI-QOL+ and the PHQ-/TBI-QOL+ groups. The mean scores for the PHQ+/TBI-QOL and the PHQ-/TBI-QOL+ were 56.3 (3.5) and 63.4 (4.8), respectively.

Emotional Health Outcome Characteristics of PHQ-9/TBI-QOL Groups

The four agreement/disagreement groups were compared on additional measures to identify if there were specific trends on the overall functioning of participants that were differently classified by the two depression measures. Two MANOVAs were completed, one for the negative and one for the positive emotional outcomes with the 4 groups as the between-groups factor. There was a significant multivariate between-group effect for negative emotional experiences (Pillai’s trace=.496, F (6, 762) =41.88, p < .001). Table 5 displays the mean scores for each of the outcome measures for the 4 groups. All univariate ANOVAs were highly significant (p < .001), primarily due to the consistent finding of the PHQ+/ TBI-QOL+ being different than the PHQ-/ TBI-QOL-group. The largest effect was observed on the TBI-QOL Anxiety scale (F(3,381)=l12.7, p<.001). The individuals who screened positive on the TBI-QOL Depression scale alone did not report significantly different psychological outcomes compared to individuals who screened positive on the PHQ-9 alone. There was also significant multivariate between-group effect for positive emotional experiences (Pillai’s trace=.500, F (9,1143) =25.42, p <.001). screened positive on both measures had the lowest score (i.e., worst functioning) compared to all the other groups, except the PHQ-/TBI-QOL+ group, most notably on the TBI-QOL Self-Esteem measure. The PHQ-/TBI-QOL+ group also had significantly lower Self-Esteem scores than the PHQ+/TBI-QOL-group. There were no other statistically significant differences between the groups.

Table 5.

Descriptive Statistics Disagreement Groups for Other Emotion Measures
PHQ9+/TBI-QOL+
(1)		 PHQ9-/TBI-QOL-
(2)		 PHQ9+/TBI-QOL-
(3)		 PHQ9-/TBI-QOL+
(4)
n mean SD n mean SD n mean SD n mean SD
Anxietyb,c,d,e,f 57 66.3 7.1 269 48.4 7.7 30 58.9 6.2 29 61.3 7.3
Grief/Lossb,c,e,f 57 61.3 7.0 269 46.3 8.0 30 54.3 6.5 29 58.0 6.0
Self-Esteema,b,c,e,f 57 37.6 5.3 269 54.1 7.1 30 46.5 5.9 29 41.4 4.4
Resiliencec,e,f 57 41.5 7.7 269 53.3 8.7 30 46.4 8.3 29 44.0 7.3
Positive Affect and Well-Beingb,c,e,f 57 45.6 5.6 269 56.5 6.6 30 50.6 7.2 29 48.3 4.9
Open in a new tab

Note:

a

3 vs 4 significant,

b

3 vs 1 significant,

c

3 vs 2 significant,

d

4 vs 1 significant,

e

4 vs 2 significant,

f

1 vs 2 significant

Table 6 presents the base rates of group members having at least 1 other poor psychological outcome (i.e., aside from depression). Nearly every participant (93%) in the PHQ+/TBI-QOL+ group had at least 1 other score above the criterion for a negative emotional issue. And most (70%) of these same participants had at least one score below the criterion in a positive emotion domain. This group showed a moderate probability of multiple elevations in negative mood (43%) and a low probability of having multiple negative outcomes in positive emotional functioning (19%). Participants that screened negative on both tests had low rates of negative emotional issues; 7% had 1 elevation and 1% had more than 1.

Table 6.

Base Rates of Concomitant Symptoms by Domain

Negative Emotion Positive Emotion
1+ Elevated Score 2+ Elevated
Scores		 1+ Low Score 2+ Low Scores
PHQ9+ /TBI-QOL+ 93% 42% 70% 19%
PHQ9-/TBI-QOL- 7% 1% 6% 0%
PHQ9+ /TBI-QOL- 60% 10% 27% 0%
PHQ9- /TBI-QOL+ 72% 10% 48% 7%
Open in a new tab

Note: elevated is t-score ≥ 60 for negative outcome measures and t-score ≤40 for positive outcome measures.

The PHQ+/TBI-QOL-group had a moderate probability of additional negative emotional issues (60%) and a low probability of multiple areas of emotional distress (10%). In general, few of these individuals reported poor positive emotional outcomes; 27% had 1 low score and none had more than one low score. The PHQ-/ TBI-QOL+ group showed a high probability of one (72%) and a low probability of more than one (10%) concurrent negative emotional outcomes. There was a moderate probability of one low score in the positive emotional domain (48%). Overall, these analyses showed that individuals who screened positive on both depression instruments had the highest likelihood of experiencing poor emotional functioning across multiple domains. Individuals who screened positive on one test only had an elevated probability of additional emotional issues, but less than individuals who screened positive on both depression instruments.

Discussion

This investigation examined the relation between the PHQ-9 and the TBI-QOL Depression item bank, and identified the rates and characteristics of adults with TBI who screen positive on only one test, both tests, or neither test. Our analyses revealed that the two scales have a high level of correlation with one another, indicating that there was considerable overlap in the constructs that were assessed. When PHQ-9 somatic items were removed from the PHQ-9 total score, the correlation between the TBI-QOL items and the remaining PHQ-9 items subtotal was stronger (r = .92). This suggests that the two measures assessed similar cognitive and emotional constructs

The rates of screening positive, and the agreement rate between the two measures, varied as a function of the cut-off scores that were applied. The lowest agreement rate occurred when the most lenient criteria were applied to both measures. When common cut-offs were applied (PHQ-9 > 11; TBI-QOL > 60), the agreement rate was 84.6%, with 7.5% of participants screening positive on the PHQ-9 alone and 7.8% screening positive on the TBI-QOL alone. As expected, the more restrictive the PHQ-9 criteria, the percentage of cases that screened positive on the PHQ-9 decreased relative to those who screened positive on the TBI-QOL.

Individuals who were differently classified (i.e., those who screened positive on only one measure) differed on PHQ-9 items related to somatic symptoms. That is, individuals who screened positive on the PHQ-9 alone typically did so because of somatic symptoms, which may or may not be due to depression. These results are not surprising because the PHQ-9 was designed to assess all symptoms of depression, whereas the TBI-QOL Depression instrument was designed to identify only the emotional and psychological aspects of depression. Indeed, those who screened positive on the TBI-QOL alone had an increased probabiluty of negative psychological outcomes compared to the group identified by the PHQ-9 alone.

A significant limitation of the current study is the absence of clinical diagnoses (e.g., MDD) established by diagnostic interview. Because that information is not known, we can only discuss rates of classification agreement between the two measures, rather than diagnostic accuracy. The TBI-QOL identified individuals that had more psychological issues in general, but this does not necessarily indicate a diagnosis of MDD over other clinical conditions. Also, because the number of individuals who were differently classified was small, the observed differences between the groups may not generalize to a larger population of TBI cases. This study motivates and informs a future study comparing each measure’s diagnostic accuracy relative to clinical diagnoses. Until such a study has been conducted, however, it would be premature to use the TBI-QOL Depression item bank alone for clinical purposes (e.g., to screen for depression in this population).

As part of the TBI-QOL development, advanced IRT-based equating methods (Choi et al., 2014; Gibbons et al., 2011) have permitted the production of linking tables so that researchers who have used the PHQ-9 in longitudinal studies could convert group-level scores to the TBI-QOL metric if desired. In that case, the results presented here suggest that a total score based on the non-somatic PHQ-9 items has a stronger relation with TBI-QOL Depression than the full PHQ-9 score.

Conclusion

The TBI-QOL Depression instrument and PHQ-9 performed similarly when classifying individuals who reported emotional and cognitive symptoms of depression. Given the strong correlation between the TBI-QOL and a summed score of the 5 non-somatic PHQ items, an IRT link between the two measures could be developed and applied for score comparison if desired.

Patient-reported outcomes measures that are based in IRT have several advantages over measures based in classical test theory, for example, they can typically produce scores with fewer items (Cappelleri, Lundy, & Hays, 2014; Cella, Gershon, Lai, & Choi, 2007; Hays et al., 2000; Thomas, 2011). This report indicates that the IRT-based TBI-QOL Depression item bank classifies individuals with TBI similarly as the classical test theory-based PHQ-9. Further research is needed to support the interpretation of TBI-QOL Depression scores, for example, the accuracy of different cutoff scores in predicting confirmed diagnoses of MDD. However, if this instrument is determined to be at least as accurate as the PHQ-9, the advantages of IRT-based assessment would make the TBI-QOL Depression item bank worth considering for use with mindividuals with TBI.

Impact.

  • The TBI-QOL Depression item bank does not assess somatic symptoms of depression, and it is unclear how it will classify adults with TBI (positive vs. negative screen) compared with the PHQ-9, which does assess somatic symptoms. This work compares how these two measures classify a relatively large sample of adults with TBI.

  • The two measures classified adults with TBI very similarly. The PHQ-9 identified more individuals with somatic symptoms, while the TBI-QOL identified more individuals with other (non-depressive) psychological symptoms.

  • Further research is needed to support the interpretation of TBI-QOL Depression scores, However, if this instrument is at least as accurate as the PHQ-9, the advantages of IRT-based assessment make the TBI-QOL Depression item bank worth considering for use with individuals with TBI. The non-somatic items of the PHQ-9 correlated highly enough with the TBI-QOL Depression item bank that scores may be transformed between measures.

Acknowledgments

This study was supported by grant numbers H133G070138 from the National Institute on Disability and Rehabilitation Research and U01AR057929 from the NIH Common Fund/National Institute of Arthritis and Musculoskeletal and Skin Diseases. Support for the development of this manuscript was also provided by an Institutional Development Award (IDeA) from the National Institute of General Medical Sciences of the National Institutes of Health under grant number U54-GM104941(PI: Binder-Macleod). The authors would like to thank Dr. Keith Bredemeier for his very thoughtful comments on the manuscript.

Footnotes

The authors have no financial disclosures related to this project. Dr. Tulsky co-owns copyright of the TBI-QOL measures (with Kessler Foundation); however the scales are free to the public and he does not receive any remuneration of any kind for this copyright.

Contributor Information

Matthew L. Cohen, Department of Communication Sciences and Disorders, Department of Psychological and Brain Sciences, and Center on Assessment Research and Translation, University of Delaware

James A. Holdnack, Center on Assessment Research and Translation, University of Delaware

Pamela A. Kisala, Department of Physical Therapy and Center on Assessment Research and Translation

David S. Tulsky, Department of Physical Therapy, Department of Psychological and Brain Sciences, Center on Assessment Research and Translation, and Kessler Foundation Research Center.

References

  1. Alway Y, Gould K, Johnston L, McKenzie D, & Ponsford J (2016a). A prospective examination of Axis I psychiatric disorders in the first 5 years following moderate to severe traumatic brain injury. Psychological Medicine, 46(6), 1–11. doi: 10.1017/S0033291715002986 [DOI] [PubMed] [Google Scholar]
  2. Alway Y, Gould K, Johnston L, McKenzie D, & Ponsford J (2016b). A prospective examination of Axis I psychiatric disorders in the first 5 years following moderate to severe traumatic brain injury. Psychological medicine, 1–11. [DOI] [PubMed] [Google Scholar]
  3. Arroll B, Goodyear-Smith F, Crengle S, Gunn J, Kerse N, Fishman T, . . . Hatcher S (2010). Validation of PHQ-2 and PHQ-9 to screen for major depression in the primary care population. Annals of Family Medicine, 8(4), 348–353. doi: 10.1370/afm.1139 [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bombardier CH, Fann JR, Temkin NR, Esselman PC, Barber J, & Dikmen SS (2010). Rates of major depressive disorder and clinical outcomes following traumatic brain injury. Journal of the American Medical Association, 303(19), 1938–1945. doi: 10.1001/jama.2010.599 [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Bombardier CH, Hoekstra T, Dikmen S, & Fann JR (2016). Depression trajectories during the first year after traumatic brain injury. Journal of Neurotrauma doi: 10.1089/neu.2015.4349 [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Bombardier CH, Kalpakjian CZ, Graves DE, Dyer JR, Tate DG, & Fann JR (2012). Validity of the Patient Health Questionnaire-9 in assessing major depressive disorder during inpatient spinal cord injury rehabilitation. Archives of Physical Medicine and Rehabilitation, 93(10), 1838–1845. doi: 10.1016/j.apmr.2012.04.019 [DOI] [PubMed] [Google Scholar]
  7. Cameron IM, Crawford JR, Lawton K, & Reid IC (2008). Psychometric comparison of PHQ-9 and HADS for measuring depression severity in primary care. British Journal of General Practice, 58(546), 32–36. doi: 10.3399/bjgp08X263794 [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Cappelleri JC, Lundy JJ, & Hays RD (2014). Overview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measures. Clinical therapeutics, 36(5), 648–662. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Cella D, Gershon R, Lai JS, & Choi S (2007). The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res, 16 Suppl 1, 133–141. doi: 10.1007/s11136-007-9204-6 [DOI] [PubMed] [Google Scholar]
  10. Cella D, Lai JS, Nowinski CJ, Victorson D, Peterman A, Miller D, Moy C (2012). Neuro-QOL: brief measures of health-related quality of life for clinical research in neurology. Neurology, 78(23), 1860–1867. doi: 10.1212/WNL.0b013e318258f744 [DOI] [PMC free article] [PubMed] [Google Scholar]; Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, . . . Hays R (2010). The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. Journal of Clinical Epidemiology, 63(11), 1179–1194. doi: 10.1016/j.jclinepi.2010.04.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Cella D, Yount S, Rothrock N, Gershon R, Cook K, Reeve B, Group PC (2007). The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Medical Care, 45(5 Suppl 1), S3–S11. doi: 10.1097/01.mlr.0000258615.42478.55 [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Charles EP (2005). The correction for attenuation due to measurement error: clarifying concepts and creating confidence sets. Psychological Methods, 10(2), 206–226. doi: 10.1037/1082-989X.10.2.206 [DOI] [PubMed] [Google Scholar]
  13. Choi SW, Schalet B, Cook KF, & Cella D (2014). Establishing a common metric for depressive symptoms: linking the BDI-II, CES-D, and PHQ-9 to PROMIS depression. Psychological Assessment, 26(2), 513–527. doi: 10.1037/a0035768 [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Cook KF, Bombardier CH, Bamer AM, Choi SW, Kroenke K, & Fann JR (2011). Do somatic and cognitive symptoms of traumatic brain injury confound depression screening? Archives of Physical Medicine and Rehabilitation, 92(5), 818–823. doi: 10.1016/j.apmr.2010.12.008 [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Drayer RA, Piraino B, Reynolds CF 3rd, Houck PR, Mazumdar S, Bernardini J, . . . Rollman BL (2006). Characteristics of depression in hemodialysis patients: symptoms, quality of life and mortality risk. General Hospital Psychiatry, 28(4), 306–312. doi: 10.1016/j.genhosppsych.2006.03.008 [DOI] [PubMed] [Google Scholar]
  16. Dyer JR, Williams R, Bombardier CH, Vannoy S, & Fann JR (2015). Evaluating the Psychometric Properties of 3 Depression Measures in a Sample of Persons With Traumatic Brain Injury and Major Depressive Disorder. Journal of Head Trauma Rehabilitation, 31(3), 225–232. doi: 10.1097/HTR.0000000000000177 [DOI] [PubMed] [Google Scholar]
  17. Fann JR, Bombardier CH, Dikmen S, Esselman P, Warms CA, Pelzer E, . . . Temkin N (2005). Validity of the Patient Health Questionnaire-9 in assessing depression following traumatic brain injury. Journal of Head Trauma Rehabilitation, 20(6), 501511. [DOI] [PubMed] [Google Scholar]
  18. Fann JR, Katon WJ, Uomoto JM, & Esselman PC (1995). Psychiatric disorders and functional disability in outpatients with traumatic brain injuries. American Journal of Psychiatry, 152(10), 1493–1499. doi: 10.1176/ajp.152.10.1493 [DOI] [PubMed] [Google Scholar]
  19. Fries JF, Bruce B, & Cella D (2005). The promise of PROMIS: using item response theory to improve assessment of patient-reported outcomes. Clinical and Experimental Rheumatology, 23(5 Suppl 39), S53–57. [PubMed] [Google Scholar]
  20. Gass CS, & Russell EW (1991). MMPI profiles of closed head trauma patients: impact of neurologic complaints. Journal of Clinical Psychology, 47(2), 253–260. [DOI] [PubMed] [Google Scholar]
  21. Gass CS, & Wald HS (1997). MMPI-2 interpretation and closed-head trauma: cross-validation of a correction factor. Archives of Clinical Neuropsychology, 12(3), 199–205. [PubMed] [Google Scholar]
  22. Gershon RC, Lai JS, Bode R, Choi S, Moy C, Bleck T, . . . Cella D (2012). Neuro-QOL: quality of life item banks for adults with neurological disorders: item development and calibrations based upon clinical and general population testing. Quality of Life Research, 21(3), 475–486. doi: 10.1007/sm36-011-9958-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Gibbons LE, Feldman BJ, Crane HM, Mugavero M, Willig JH, Patrick D, . . . Crane PK (2011). Migrating from a legacy fixed-format measure to CAT administration: calibrating the PHQ-9 to the PROMIS depression measures. Quality of Life Research, 20(9), 1349–1357. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Grinnon ST, Miller K, Marler JR, Lu Y, Stout A, Odenkirchen J, & Kunitz S (2012). National Institute of Neurological Disorders and Stroke Common Data Element Project -approach and methods. Clinical Trials, 9(3), 322–329. doi: 10.1177/1740774512438980 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hart T, Brenner L, Clark AN, Bogner JA, Novack TA, Chervoneva I, . . . Arango-Lasprilla JC (2011). Major and minor depression after traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 92(8), 1211–1219. doi: 10.1016/j.apmr.2011.03.005 [DOI] [PubMed] [Google Scholar]
  26. Hays RD, Morales LS, & Reise SP (2000). Item response theory and health outcomes measurement in the 21st century. Medical care, 38(9 Suppl), II28. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Jorge RE, Robinson RG, & Arndt S (1993). Are there symptoms that are specific for depressed mood in patients with traumatic brain injury? Journal of Nervous and Mental Disorders, 181(2), 91–99. [DOI] [PubMed] [Google Scholar]
  28. Kendel F, Wirtz M, Dunkel A, Lehmkuhl E, Hetzer R, & Regitz-Zagrosek V (2010). Screening for depression: Rasch analysis of the dimensional structure of the PHQ-9 and the HADS-D. Journal of Affective Disorders, 122(3), 241–246. doi: 10.1016/j.jad.2009.07.004 [DOI] [PubMed] [Google Scholar]
  29. Kim E, Lauterbach EC, Reeve A, Arciniegas DB, Coburn KL, Mendez MF, . . . Research, A. C. o. (2007). Neuropsychiatric complications of traumatic brain injury: a critical review of the literature (a report by the ANPA Committee on Research). Journal of Neuropsychiatry and Clinical Neuroscience, 19(2), 106–127. doi: 10.1176/jnp.2007.19.2.106 [DOI] [PubMed] [Google Scholar]
  30. Kisala PA, Tulsky DS, Cohen ML, Holdnack JA, Slavin MD, Mulcahey MJ, & Jette AM (In preparation). Examining Interview Vs. Self-Report Administration of PROMIS Pediatric and Adult Measures.
  31. Koskinen S (1998). Quality of life 10 years after a very severe traumatic brain injury (TBI): the perspective of the injured and the closest relative. Brain Injury, 12(8), 631–648. [DOI] [PubMed] [Google Scholar]
  32. Kroenke K, Spitzer RL, & Williams JB (2001). The PHQ-9: validity of a brief depression severity measure. Journal of General Internal Medicine, 16(9), 606–613. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Kroenke K, Spitzer RL, & Williams JB (2002). The PHQ-15: validity of a new measure for evaluating the severity of somatic symptoms. Psychosomatic Medicine, 64(2), 258–266. [DOI] [PubMed] [Google Scholar]
  34. Lange RT, Brickell TA, Bailie JM, Tulsky DS, & French LM (2016). Clinical Utility and Psychometric Properties of the Traumatic Brain Injury Quality of Life Scale (TBI-QOL) in US Military Service Members. Journal of Head Trauma Rehabilitation, 31(1), 62–78. doi: 10.1097/HTR.0000000000000149 [DOI] [PubMed] [Google Scholar]
  35. Manea L, Gilbody S, & McMillan D (2012). Optimal cut-off score for diagnosing depression with the Patient Health Questionnaire (PHQ-9): a meta-analysis. Canadian Medical Association Journal, 184(3), E191–196. doi: 10.1503/cmaj.110829 [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Manea L, Gilbody S, & McMillan D (2015). A diagnostic meta-analysis of the Patient Health Questionnaire-9 (PHQ-9) algorithm scoring method as a screen for depression. General Hospital Psychiatry, 37(1), 67–75. doi: 10.1016/j.genhosppsych.2014.09.009 [DOI] [PubMed] [Google Scholar]
  37. Mulcahey MJ, Haley SM, Slavin MD, Kisala PA, Ni P, Tulsky DS, & Jette AM (2015). Ability of PROMIS Pediatric Measures to Detect Change in Children With Cerebral Palsy Undergoing Musculoskeletal Surgery. Journal of Pediatric Orthopaedics, 36(7), 749–756. doi: 10.1097/BPO.0000000000000533 [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. National Institute on Neurological Disorders and Stroke. (2016). Common Data Elements: Traumatic Brain Injury. Retrieved from https://commondataelements.ninds.nih.gov/TBI.aspx
  39. Pilkonis PA, Choi SW, Reise SP, Stover AM, Riley WT, Cella D, & Group PC(2011). Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS(R)): depression, anxiety, and anger. Assessment, 18(3), 263–283. doi: 10.77/1073191111411667 [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Pilkonis PA, Yu L, Dodds NE, Johnston KL, Maihoefer CC, & Lawrence SM (2014). Validation of the depression item bank from the Patient-Reported Outcomes Measurement Information System (PROMIS) in a three-month observational study. Journal of Psychiatry Research, 56, 112–119. doi: 10.1016/j.jpsychires.2014.05.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  41. Reddy P, Philpot B, Ford D, & Dunbar JA (2010). Identification of depression in diabetes: the efficacy of PHQ-9 and HADS-D. British Journal of General Practice, 60(575), e239–245. doi: 10.3399/bjgp10X502128 [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Reeve BB, Hays RD, Bjorner JB, Cook KF, Crane PK, Teresi JA, . . . Group PC (2007). Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Medical Care, 45(5 Suppl 1), S22–31. doi: 10.1097/01.mlr.0000250483.85507.04 [DOI] [PubMed] [Google Scholar]
  43. Rockhill CM, Jaffe K, Zhou C, Fan MY, Katon W, & Fann JR (2012). Health care costs associated with traumatic brain injury and psychiatric illness in adults. Journal of Neurotrauma, 29(6), 1038–1046. doi: 10.1089/neu.2010.1562 [DOI] [PubMed] [Google Scholar]
  44. Rosenthal M, Christensen BK, & Ross TP (1998). Depression following traumatic brain injury. Archives of Physical Medicine and Rehabilitation, 79(1), 90–103. [DOI] [PubMed] [Google Scholar]
  45. Scholten AC, Haagsma JA, Cnossen MC, Olff M, Van Beeck EF, & Polinder S (2016). Prevalence and risk factors of anxiety and depressive disorders following traumatic brain injury: a systematic review. Journal of Neurotrauma, 33(22), 1969–1994. doi: 10.1089/neu.2015.4252 [DOI] [PubMed] [Google Scholar]
  46. Stafford L, Berk M, & Jackson HJ (2007). Validity of the Hospital Anxiety and Depression Scale and Patient Health Questionnaire-9 to screen for depression in patients with coronary artery disease. General Hospital Psychiatry, 29(5), 417–424. doi: 10.1016/j.genhosppsych.2007.06.005 [DOI] [PubMed] [Google Scholar]
  47. Thomas ML (2011). The value of item response theory in clinical assessment: a review. Assessment, 18(3), 291–307. doi: 10.1177/1073191110374797 [DOI] [PubMed] [Google Scholar]
  48. Tulsky DS, Boulton AJ, Kisala PA, & Tyner CE (2016). Recommendations for Using Crosswalks in the TBI Model Systems. Retrieved from Report to the TBI Model Systems Centers: [Google Scholar]
  49. Tulsky DS, Kisala PA, Victorson D, Carlozzi N, Bushnik T, Sherer M, . . . Cella D (2016). TBI-QOL: Development and Calibration of Item Banks to Measure Patient Reported Outcomes Following Traumatic Brain Injury. Journal of Head Trauma Rehabilitation, 31(1), 40–51. doi: 10.1097/HTR.0000000000000131 [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wagner LI, Schink J, Bass M, Patel S, Diaz MV, Rothrock N, . . . Cella D (2015). Bringing PROMIS to practice: brief and precise symptom screening in ambulatory cancer care. Cancer, 121(6), 927–934. doi: 10.1002/cncr.29104 [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Wang M, Chen RC, Usinger DS, & Reeve BB (2017). Evaluating measurement invariance across assessment modes of phone interview and computer self-administered survey for the PROMIS measures in a population-based cohort of localized prostate cancer survivors. Qual Life Res. doi: 10.1007/s11136-017-1640-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Wei W, Sambamoorthi U, Crystal S, & Findley PA (2005). Mental illness, traumatic brain injury, and medicaid expenditures. Archives of Physical Medicine and Rehabilitation, 86(5), 905–911. doi: 10.1016/j.apmr.2004.09.026 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Whitnall L, McMillan TM, Murray GD, & Teasdale GM (2006). Disability in young people and adults after head injury: 5–7 year follow up of a prospective cohort study. Journal of Neurology, Neurosurgery, and Psychiatry, 77(5), 640–645. doi: 10.1136/jnnp.2005.078246 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES