Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2021 Oct 1.
Published in final edited form as: Psychiatry Res. 2020 Jul 20;292:113313. doi: 10.1016/j.psychres.2020.113313

TEST-RETEST & FAMILIAL CONCORDANCE OF MDD SYMPTOMS

Ariela J E Kaiser 1, Carter J Funkhouser 1, Vijay A Mittal 2, Sebastian Walther 3, Stewart A Shankman 4,*
PMCID: PMC7529979  NIHMSID: NIHMS1617939  PMID: 32738552

Abstract

Psychopathology research has increasingly sought to study the etiology and treatment of individual symptoms, rather than categorical diagnoses. However, it is unclear whether commonly used measures have adequate psychometric properties for assessing individual symptoms. This study examined the test-retest reliability and familial concordance (an indicator of validity) of the symptoms of Major Depressive Disorder (MDD), a disorder consisting of nine core symptoms, most of which are aggregated (e.g., symptom 7 of the DSM criteria for MDD is worthlessness or guilt). Lifetime MDD symptoms were measured in 504 young adults (237 sibling pairs) using the Structured Clinical Interview for DSM-5 (SCID). Fifty-one people completed a second SCID within three weeks of their first SCID. Results indicated that aggregated and unaggregated symptoms demonstrated moderate to substantial test-retest reliability and generally significant, but slight to fair familial concordance (with the highest familial concordance being for markedly diminished interest or pleasure and its unaggregated components - decreased interest and decreased pleasure). Given the increasing focus on the differential validity of individual MDD symptoms, the present study suggests that interview-based assessments of depression can assess most individual symptoms with adequate levels of reliability and validity.

Keywords: Depression, Psychometrics, Symptoms, Heterogeneity

1. Introduction

Major Depressive Disorder (MDD) is one of the most common psychiatric disorders, is a major cause of disability worldwide, and is among the leading disorders for global disease burden (Lopez et al., 2006). MDD often co-occurs with other psychiatric disorders (Kessler et al., 2005; Lewinsohn et al., 2004), is a significant predictor of suicide (Berman, 2009), and is highly recurring (e.g., more than half of all individuals diagnosed with MDD experience more than one episode in their lifetime [McClintock et al., 2010]).

MDD, like most psychiatric disorders, is a polythetic disorder, which means the disorder is defined by multiple symptoms, and not all symptoms need to be present in order for the syndrome to be present. In the fifth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-5; American Psychiatric Association, 2013), individuals meet criteria for a major depressive episode if they endorse five of nine symptoms, at least one of which must be either depressed mood or markedly diminished interest/pleasure. An underlying assumption of the operationalization of MDD as a categorical diagnosis is that the non-cardinal symptoms are interchangeable. In other words, the number of symptoms is emphasized rather than the nature of the symptoms.

The MDD categorical diagnosis has also shown questionable reliability across multiple studies (Markon et al., 2011; Bromet et al., 1986; Spitzer& Fleiss, 1974). In the field trials for DSM-5, clinician agreement for the MDD diagnosis was 0.28 (95% CI 0.20–0.35), which fell into in the questionable range of pooled intraclass Kappa (Regier et al., 2013). The poor reliability also likely contributes to difficulty clarifying the etiology of MDD. That is, if an assessment of a construct or disorder has poor reliability, then it will be extremely difficult to identify its etiology (Smoller, 2003).

One explanation for the poor reliability of the MDD diagnosis may be the phenotypic heterogeneity of depression. There are thousands of possible symptom presentations that meet diagnostic criteria for MDD (Zimmerman et al., 2014) - for example, one study identified 1,030 unique depression symptom profiles in 3,703 individuals with MDD (Fried & Nesse, 2015). What further compounds the problem of the polythetic criteria is that 7 of the 9 symptoms are aggregated, consisting of at least two different symptoms (e.g., worthlessness OR guilt). Moreover, many of the symptoms consist of contrasting features (e.g., increase OR decrease in weight/appetite). Importantly, studies show that many of the aggregated symptoms that are phrased as contrasting co-occur in some patients with MDD (e.g., psychomotor retardation OR psychomotor agitation [Parker et al., 1995]; insomnia OR hypersomnia [Soehner et al., 2014]).

Although the categorical diagnosis has demonstrated questionable reliability, it remains unclear if the individual symptoms comprising MDD are themselves reliable. One of the main challenges in symptom-based research is establishing reliable and valid tools to measure individual symptoms. Although most rating scales are not validated at the symptom level (and certainly not for unaggregated symptoms), one small study (N = 31) did examine inter-rater reliability and concurrent validity for individual symptoms of MDD (Mazure et al., 1986). Results highlighted that most symptoms of MDD can be reliably detected by clinicians during a semi-structured interview and that observable symptoms correlated with patients’ behaviors.

1.1. Differential Validity of Individual Depressive Symptoms

In addition to reliability, research has increasingly focused on the differential validity of individual depressive symptoms. For example, studies suggest that diverse etiological factors may lead to the occurrence of different depressive symptoms, further suggesting that different depressive symptoms are not interchangeable. Keller et al. (2007) found that chronic stress was associated with some but not all symptoms, such as fatigue and hypersomnia. Likewise, genetic disposition to specific symptoms is uncertain, as Kendler and Aggen (2017) detected only modest concordance of individual MDD symptoms among monozygotic twins, while there were separate environmental factors for different symptoms. Another study found that chronic stress, prior history of depression, higher number of stressful life events, childhood stress, and sex predicted worsening of only certain MDD symptoms (Fried et al., 2014). Finally, a subset of MDD symptoms seems to impact functional impairment, e.g. low mood, difficulty concentrating, fatigue, and loss of interest/pleasure (Fried & Nesse, 2014). These findings support the idea that symptom sum-scores disregard qualitative differences between particular depressive symptoms.

Besides research showing differential impact of certain MDD symptoms on impairment and etiological factors, several studies have shown that the presence of particular symptoms (e.g., sleep disturbances and hopelessness [Benca & Peterson, 2008; Dew et al., 1997]) predicted a worse outcome to antidepressant treatment (see also, McMakin et al., 2012; Uher et al.,2012). Thus, treating specific symptoms as interchangeable indicators of a latent disease ignores key prognostic variables in the course and treatment of depression.

Studying specific symptoms, rather than a latent disorder, is also consistent with the network theory of psychopathology, which states that disorders are the result of causal relationships between individual symptoms (Borsboom & Cramer, 2013; Boccaletti et al., 2006). Psychopathology network analysis has become extremely popular in psychopathology research (Funkhouser et al., 2020; McNally, 2019) suggesting that even more MDD research will focus on individual symptoms. Additionally, studying the psychometrics of individual symptoms would support the growing interest in depression measures guided by Item Response Theory, such as computerized adaptive tests that automatically administer items to examinees based on their prior responses (Fries et al., 2011).

In sum, individual symptoms of MDD have been and continue to be widely studied, and the exponentially increasing popularity of network theory and modeling suggests that even more MDD research will focus on individual symptoms. Despite the increasing focus on individual symptoms in MDD research, relatively little is known about the reliability of commonly used measures of individual MDD symptoms.

The goal of the study was to examine the psychometric properties for the (1) categorical diagnosis, (2) aggregated symptoms (e.g., sleep disturbance overall), and (3) unaggregated symptoms (e.g., hypersomnia and insomnia) of MDD. The study focused on two psychometric properties - test-retest reliability and familialness (i.e., the extent to which they run in families). Since Robins and Guze’s (1970) classic paper on psychiatric validity, familialness has long been considered an important validator for psychiatric disorders (and, by extension, individual symptoms) given that family history of a disorder is a robust correlate/risk factor for disorders in probands (Klein et al., 2003; Nierenberg et al., 2007). To our knowledge, no study has examined the test-retest reliability of all unaggregated and aggregated symptoms, and only one study examined the familialness of individual symptoms. Korszun et al. (2004) examined the familialness of depression symptom dimensions between sibling pairs and found the highest correlations between siblings for the symptoms of restlessness, anxiety, loss of libido, and irritability (rs = .26–.31; Korszun, et al., 2004). The present study will build off Korszun et al. by examining DSM-5 aggregated and unaggregated symptoms in a community sample of sibling pairs rather than in a sample with severe and recurrent MDD. This is an important advancement because individuals with subthreshold MDD are a clinically significant group (Shankman et al., 2008, 2009) and only focusing on those with severe and recurrent MDD may limit generalizability. Given the differential validity of individual MDD symptoms, certain symptoms may run in families more than others, further supporting the varied validity of individual MDD symptoms.

We hypothesize that both aggregated and unaggregated symptoms will exhibit significant test-retest reliability and familial concordance, but (given the heterogeneity of MDD symptoms) will vary substantially.

2. Methods

2.1. Participants

504 participants were recruited as part of a NIMH-funded family study (see Gorka et al., 2016; Katz et al., 2017 for additional details). Participants were 18 to 30 years old, nested within 274 families, and included 237 sibling pairs breaking down into 40 male-male dyads, 91 male-female dyads, and 106 female-female dyads (see Table 1 for additional participant demographics). Advertisements (fliers, internet postings, etc.) were used to recruit participants from the community and from mental health clinics. A Research Domain Criteria (RDoC) approach (Shankman & Gorka, 2015) was taken to participant recruitment such that recruitment screening was agnostic to DSM diagnostic categories (beyond the exclusion criteria listed below). However, participants with elevated symptoms of internalizing psychopathology were oversampled to ensure that the sample was clinically relevant. Specifically, the Depression, Anxiety, and Stress Scale (Lovibond & Lovibond, 1995) was administered during the initial phone screen to ensure that the severity of internalizing symptomology within the sample was normally distributed, but also was higher than the general population (M = 10.35 [SD = 10.07] vs. M =8.3 [SD = 9.8]; Crawford et al., 2011).

Table 1.

Participant demographics and characteristics.

Sibling Pairs (N = 237) Test-Retest Reliability (N = 51)
Age (Mean, SD) 22.3 (3.2) 22.3 (3.3)
Percent Female 64% 60%
Race/Ethnicity
 Percent Caucasian 41.1% 41.2%
 Percent Hispanic 22.6% 11.8%
 Percent African American 14.5% 23.5%
 Percent Asian 10.6% 9.8%
 Percent Middle eastern 4.1% 2.0%
 Percent Mixed race 6.1% 9.0%
 Percent Other .9% 2.0%
Open in a new tab

Inclusion criteria specified that participants had at least one full biological sibling that was also willing to participate in the study. Exclusion criteria included personal or family history of psychosis or mania at the time of the interview (given that psychosis and mania have been shown to be separable from internalizing and externalizing disorders; Caspi et al., 2014; Kotov et al., 2011; Krueger et al., 1998, Markon, 2010), being a twin, inability to read or write in English, history of serious head trauma, and left-handedness (to protect against confounds with the neurophysiological data collected for other aims of the larger study).

2.2. Structured Clinical Interview for Diagnostic and Statistical Manual of Mental Disorders (SCID)

Symptoms were assessed for participants’ worst lifetime depressive episode using the SCID (First et al., 2015), a widely used semi-structured clinical interview. The SCID used in the present study was modified slightly to better assess individual symptoms (Shankman et al., 2018). First, the separate parts of aggregated symptoms were coded independently (e.g., the MDD symptom “worthlessness or guilt” was split so that it yielded separate ratings for worthlessness and guilt). Second, to increase sensitivity to individuals with subthreshold psychopathology and facilitate the calculation of symptom severity scales, interviewers assessed all lifetime MDD symptoms even if the cardinal symptoms (depressed mood and loss of interest/pleasure) were not endorsed (note: if neither cardinal symptom was endorsed, interviewers assessed symptoms for the 2-week or longer period in their life during which participants were “most distressed or upset”). Doctoral students and bachelor’s level interviewers were supervised by a licensed clinical psychologist and trained to criterion on the SCID by viewing the SCID-101 training videos (SCID-101, 1998), overseeing two or three SCID interviews with an experienced interviewer, and completing three SCID interviews (observed by an advanced interviewer) in which diagnoses in were in full agreement with those of the observer. For this study, three sets of variables were pulled from the adapted SCID: the categorical diagnosis of MDD, the nine aggregated symptoms of MDD, and the 23 unaggregated symptoms of MDD. To assess test-retest reliability, a subset of participants (N = 51) were pseudo-randomly selected from the overall sample to complete a second SCID with a different interviewer within three weeks of their first SCID (M = 8.5 days, SD = 4.3; see Table 1 for subsample characteristics). As SCID ratings for each of the depressive items consist of 1 (absent), 2 (subthreshold) and 3 (full threshold), ratings were dichotomized to reflect the presence (3) or absence (1 or 2) of each symptom. Similarly, diagnoses of lifetime MDD reflected the presence or absence of MDD, and the number of depressive episodes was not considered.

2.3. Data Analyses

The test-retest reliability and familialness of the lifetime MDD diagnosis and each lifetime MDD symptom were evaluated using Cohen’s kappa using established conventions for agreement (Cohen, 1960). We used R (Version 1.2.1237; R Core Team, 2019) and the R-packages psych (Version 1.8.12; Revelle, 2018) and boot (Version 1.2–20; Canty & Ripley, 2017). For power analyses, we used package irr (Version 0.84.1; Gamer, Lemon & Fellows, 2019). We randomly assigned each subject within each sibling pair to be either sibling 1 or sibling 2 and then computed bootstrapped 95% confidence intervals around the estimated Kappa for each symptom (i.e., the 9 aggregated symptoms and 23 unaggregated symptoms). If the confidence interval of the Kappa did not contain a value of zero, it was considered significant.

Sex and age have been shown to impact the clinical presentation of MDD. Women are approximately 1.7 times as likely as men to report a lifetime history of MDD (Kessler et al., 1993) and research has shown that the presentation of MDD symptoms varies as a function of age (Kovacs, 1996; Lux & Kendler, 2010). In order to include age and sex as covariates, logistic regressions were run as kappa analyses do not allow for the inclusion of covariates. Thus, all models testing for familial concordance covaried for each sibling’s sex and age and the interaction of the two siblings’ sex and age. A randomly assigned sibling’s symptom (aggregated or unaggregated) was the independent variable, the other sibling’s symptom was the dependent variable, and the sexes and ages of the two siblings and their interactions were included as covariates (for a similar approach, see Khan et al., 2002; Moskvina et al., 2008). If the same familial effect was operative for both male and female siblings, logistic regressions covarying for the interaction of each sibling’s sex as well as their ages would remain significant. However, if a familial effect was age- and/or sex-dependent, then the results would no longer remain when including these covariates.

3. Results

3.1. Test-Retest Reliability

The lifetime diagnosis of MDD had substantial test-retest reliability (k = .68). The test-retest reliabilities for aggregated symptoms fell into the moderate range (see Figure 1a), and were highest for loss of interest/pleasure (k = .71) and depressed mood (k = .63) and lowest for suicidal thoughts and behaviors (k = .52) and inappropriate guilt and feelings of worthlessness (k = .52). At the unaggregated symptom level, nearly all unaggregated symptoms had fair to substantial test-retest reliability (see Figure 1b). The unaggregated symptoms with the strongest reliability (all of which fell into the substantial range) included middle insomnia (k = .75), hypersomnia (k = .72), psychomotor retardation (k = .72), and specific plan for a suicide attempt (k = .79). The unaggregated symptoms with the lowest test-retest reliability included weight loss (k = .17) and psychomotor agitation (k = .23).

Figure 1.

Figure 1.

Open in a new tab

Cohen’s Kappas for the test-retest reliability of (a) aggregated and (b) unaggregated MDD symptoms.

3.2. Familial Concordance

In the sample of 237 sibling pairs, there was fair familial concordance for the lifetime categorical MDD diagnosis (k = .21). The familial concordances for the aggregated symptoms are presented in Figure 2 and were highest for loss of interest/pleasure (k = .28) and depressed mood (k = .21). At the unaggregated symptom level, the symptoms that comprise anhedonialoss of interest/pleasure —loss of interest (k = .21) and loss of pleasure (k = .25)-had the highest familial concordance and were in the fair agreement range (see Figure 3). The unaggregated symptoms with the lowest familial concordance included sleep disturbance (k = .08) and psychomotor disturbance (k = .04).

Figure 2.

Figure 2.

Open in a new tab

Cohen’s Kappa for the familial concordance of aggregated MDD symptoms.

Figure 3.

Figure 3.

Open in a new tab

Cohen’s Kappa for the familial concordance of unaggregated MDD symptoms.

3.3. Impact of Sex and Age on Familial Concordance

Results of logistic regressions examining familial associations covarying for age and sex were consistent with the results of Cohen’s Kappa for all aggregated symptoms and the majority of unaggregated symptoms in MDD. However, the unaggregated symptoms of appetite loss (B = .80, p = .035) and suicidal ideation (B = .96, p = .025) were significant when controlling for sex and age but did not have significant Cohen’s Kappas in models that did not take into account sex and age differences.

4. Discussion

Research has increasingly studied the etiology, course, and treatment responsiveness of individual MDD symptoms rather than the overall disorder. Many of these studies have assessed symptoms using single items from structured interviews. However, the psychometric properties of these assessments are unclear. Therefore, it is necessary to study the reliability and validity of tools that can measure individual symptoms such as the SCID. The present study found that nearly all symptoms, both aggregated and unaggregated, had significant test-retest reliability - with kappas falling in the moderate to substantial range. While aggregated and unaggregated symptoms had similar familial concordance in the slight to fair range, aggregated symptoms had slightly better estimates of familial agreement. Importantly, results also indicated that the effects were generally not influenced by differences in sex or age.

4.1. Test-Retest Reliability

Due to the symptomatic heterogeneity of MDD, questions have been raised regarding whether the different sub-parts of each symptom (e.g., psychomotor agitation vs psychomotor retardation; Walther et al., 2019) have different psychometric properties (Mazure et al. 1986). Therefore, it is important to examine the test-retest reliability of aggregated and unaggregated MDD symptoms. Our results show that the test-retest reliability of individual MDD symptoms fell into the moderate to substantial range for both aggregated and unaggregated depressive symptoms. The results for the unaggregated symptom “specific plan for suicide” should be interpreted with caution as the sample had a low endorsed rate for this symptom. Future research should examine its test-retest reliability in samples with higher base rates of symptoms regarding suicidal behaviors, such as those with chronic or treatment resistant depression (Klein et al., 2008).

With the exception of worthlessness/guilt and weight loss/gain, the reliability of unaggregated symptoms was comparable to the corresponding aggregated symptoms, falling into the moderate to substantial range. Therefore, studies examining individual symptoms should consider the lower reliability of worthlessness/guilt and weight loss/gain when examining these symptoms separately relative to when they are aggregated. Importantly, these two aggregated symptoms may have had slightly better reliability not because their subcomponents are interchangeable, but simply because they aggregated multiple items (Sijtsma, 2009). Additionally, the poorer reliability of these (and other unaggregated) symptoms may have resulted from unclear definitions or idiosyncratic interpretations of symptoms. For example, participants/interviewers may have varied in their perception of what constitutes ‘inappropriate’ or ‘excessive’ guilt (Tilghman-Osborne et al., 2010 - see Zahn et al, 2015 for comparable discussion for worthlessness).

Our findings address questions about the reliability of single-item assessments that are increasingly being used in studies of internalizing disorders (Mitchell et al., 2010; Turon et al., 2019; Zimmerman et al., 2006). Since our results show that nearly all symptoms had fair to substantial test-retest reliability, we suggest that using items from the SCID is one way to assess individual symptoms reliably. Importantly, skip-outs were suspended, allowing for the assessment of each symptom regardless of the presence of the cardinal symptoms of MDD (or the full diagnosis of MDD). This is crucial when modeling the relationships between individual symptoms, as abiding by the skip-out rules can bias the symptom covariance estimates upon which both network models (Hoffman et al., 2019) and factor models (Kotov et al., 2018) are based.

Although these findings suggest that the SCID can assess individual MDD symptoms reliably, this study does not address other issues related to the measurement of individual symptoms. For example, the study only examined MDD symptoms as outlined in the DSM-5. While the DSM-5 criteria do a reasonable job of reflecting the prominent symptoms of depression, the list may only reflect a subset of depression-related signs and symptoms (Kendler et al., 2018; Fried et al, 2015). Indeed, non-MDD symptoms such as irritability and anger are linked to more severe and chronic depression (Judd et al., 2013). Thus, DSM-5 criteria remain an imperfect approximation of the broader syndrome. Kendler (2016) argues that diagnostic criteria are designed to index rather than describe the complete syndrome and aggregating disparate symptoms (as explored in this study) likely further impacts the reliability and validity of diagnosis. Thus, clinicians should make an effort to explore the diversity of the depressive symptoms that patients experience which includes aggregated, unaggregated, and non-DSM-5 items. Future research should also examine the reliability of individual symptoms of other, commonly comorbid disorders such as GAD, to facilitate transdiagnostic work on individual symptoms.

4.2. Familial Concordance

In addition to reliability, our results provide insight into the validity of the aggregated and unaggregated symptoms of MDD by examining the familial concordance of MDD symptoms. A well-replicated finding in the literature is that most (if not all) psychiatric disorders are moderately heritable (Zuk et al., 2012; Sullivan et al., 2000). Although a previous study estimated the familial concordance of factors of depression symptoms (identified via exploratory factor analysis, Korszun et al., 2004), to our knowledge this study is the first to examine the familial concordance of individual MDD symptoms. Results indicated that individual symptoms of MDD had low to fair concordance between siblings, suggesting that symptoms are majorly influenced by unique (that is, unshared with their biological sibling) environmental experiences. Aggregated and unaggregated symptoms were roughly equally familial. Additionally, the significance of each symptom’s familial concordance did not change when the interaction of each sibling’s sex and age was included in the model, suggesting that there likely is not a sex-specific factor (e.g., genetic, hormonal, social, environmental) that strongly impacts familial concordance.

While MDD diagnosis and numerous MDD symptoms were shown to be familial, the degree to which these effects were due to genetic versus environmental factors is uncertain. Twin studies, which can quantify the relative importance of genetic, shared environmental, and unique environmental factors on MDD, offer insight into this important question. Twin studies have shown that while genetics are an important factor for certain MDD symptoms (e.g., loss of appetite, loss of libido/sexual pleasure, feelings of guilt, and hopelessness; Jang et al., 2014), most of the variance is attributable to unique environmental factors (Kendler et al., 2013) and some symptoms associated with MDD may not be heritable at all (Jang et al., 2014). Several studies have sought to identify specific genetic or environmental factors that contribute to one or more individual depressive symptoms (e.g., Keller et al., 2007; Myung et al., 2012), but further research is needed to gain insight into the etiology of depressive symptoms.

4.3. Limitations

These findings should be interpreted in the light of several limitations. First, the bootstrapped CIs for the Kappa estimates are fairly large, possibly due to the small sample size and low prevalence rates for many of the symptoms, especially the unaggregated symptoms. Replication of these results is needed. However, a power analysis revealed that the N of 51 had 95% power to detect moderately endorsed (> 30%) symptoms. Second, the differential variability in depressive symptoms is a potential source of biased estimates, because heavily skewed symptoms that were infrequently endorsed are less likely to demonstrate significant statistical relationships (e.g., suicidal behaviors or agitation in our sample). Third, the degree of accuracy for recalling symptoms of lifetime psychiatric disorders, including MDD, has been questioned (Olino et al., 2012; Takayanagi et al., 2014). According to Wells & Horwood, only 44% of participants with a lifetime diagnosis of MDD recalled a cardinal symptom of MDD, which, although concerning, also speaks to the importance of examining all 32 aggregated and unaggregated symptoms when studying lifetime MDD. In the same vein, participants may have had idiosyncratic understandings of the unaggregated symptoms. For example, participants understanding of the symptom “suicide plan” may or may not have reflected an intent to die (Linehan, 1986). Fourth, the present study only looked at one component of validity (familial concordance), and future studies should examine other aspects of validity (e.g., treatment response, functional impairment, laboratory tasks). Fifth, analyses of familial concordance examined each symptom independently, but it is possible that some symptoms share a common genetic or environmental cause. It is also possible that some symptoms are causally related (e.g., insomnia may cause fatigue; Borsboom & Cramer, 2013), which would bias the estimated familial concordance of a given symptom to the extent that it is caused by other symptoms. Sixth, test-retest reliability was not examined for the same exact participant interview, rather, it was the reliability of the same MDD episode using a different interviewer. Test retest reliability was therefore confounded with inter rater reliability; nevertheless, this is a conservative estimate of the true reliability and validity of the SCID’s ability to assess individual symptoms

4.4. Conclusions

Psychopathology research has increasingly focused on individual symptoms of depression and other disorders. The majority of this work has utilized item-level analyses of assessments that were originally designed to measure latent disorders, and the psychometric properties of these single-item symptom measures are unclear. The present study found that individual MDD symptoms (as assessed by the SCID, a widely used diagnostic interview) demonstrated significant test-retest reliability in the fair to substantial range; and slight to fair familial concordance. Test-retest reliability and familial concordance was generally similar for unaggregated symptoms and their corresponding aggregated symptom. Taken together, these findings provide preliminary evidence that interview-obtained measures of individual MDD symptoms (either aggregated or unaggregated) generally have adequate test-retest reliability to assess certain lifetime depressive symptoms, but might not be able to assess all symptoms. However, further research examining the reliability and validity of both interview- and questionnaire-based measures of individual symptoms is needed.

Table 2.

Aggregated and Unaggregated Symptom Endorsement at Time 1, Time 2, Both, Only One Time

% Endorsed Time 1 % Endorsed Time 2 % Endorsed Time 1 and Time 2 % Endorsed at one time point only (but not both)
Depressed Mood 56.86 58.82 49.02 17.65
Low Mood 54.9 54.9 45.1 19.61
Emptiness 43.14 43.14 35.29 15.69
Hopelessness 33.33 37.25 25.49 19.61
Loss of Interest/Pleasure 54.9 60.78 50.98 13.73
Loss of Interest 50.98 56.86 45.1 17.65
Loss of Pleasure 50.98 50.98 43.14 15.69
Change in Weight 47.06 43.14 35.29 19.61
Appetite Gain 13.73 18 9.8 11.76
Appetite Loss 29.41 24 19.61 11.76
Weight Gain 11.76 14 5.88 13.73
Weight Loss 15.69 4 1.96 13.73
Sleep Disturbance 54.9 56.86 49.02 13.73
Early Insomnia 33.33 25.49 21.57 15.69
Middle Insomnia 29.41 23.53 21.57 9.8
Late Insomnia 17.65 17.65 9.8 15.69
Hypersomnia 19.61 25.49 17.65 9.8
Psychomotor Disturbance 19.61 33.33 19.61 13.73
Psychomotor
Agitation 5.88 7.84 1.96 9.8
Psychomotor
Retardation 15.69 27.45 15.69 11.76
Fatigue 52.94 56.86 47.06 15.69
Worthlessness/Guilt 39.22 47.06 31.37 23.53
Worthlessness 23.53 39.22 21.57 19.61
Guilt 27.45 19.61 11.76 23.53
Concentration Difficulty 41.18 39.22 31.37 17.65
Indecisiveness 21.57 17.65 11.76 15.69
Ability to
Concentrate 39.22 35.29 29.41 15.69
Suicidal Thoughts/Behaviors 29.41 19.61 15.69 17.65
Thoughts of Death 29.41 19.61 15.69 17.65
Suicidal Ideation 23.53 13.73 11.76 13.73
Specific Plan 5.88 3.92 3.92 1.96
Past Attempt 1.96 4 1.96 1.96
Open in a new tab

Highlights.

  • Examined psychometric properties of interview measures of individual MDD symptoms

  • MDD symptoms exhibited moderate to substantial test-retest reliability

  • Individual MDD symptoms ran in families, but to varying degrees.

Acknowledgment

This work was supported by grants from the National Institute of Mental Health (R01MH098093 [PI: Shankman] and R01MH118741 [PI: Shankman, Mittal, and Walther]). Other support for this work was provided by the University of Illinois at Chicago (UIC) Center for Clinical and Translational Science (CCTS) award number UL1RR029879 from the National Center for Research Resources.

Footnotes

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

Conflicts of interest

The authors have no conflicts of interest.

References

  1. American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). Arlington, VA. [Google Scholar]
  2. Angst J, 1999. Major depression in 1998: Are we providing optimal therapy? J. Clin. Psychiatry [PubMed] [Google Scholar]
  3. Angst J, Clayton P, 1986. Premorbid personality of depressive, bipolar, and schizophrenic patients with special reference to suicidal issues. Compr. Psychiatry 10.1016/0010-440X(86)90055-6 [DOI] [PubMed] [Google Scholar]
  4. Benca RM, Peterson MJ, 2008. Insomnia and depression. Sleep Med. 10.1016/S1389-9457(08)70010-8 [DOI] [PubMed] [Google Scholar]
  5. Berman AL 2009. Depression and suicide, in: Gotlib Ian H & Hammen CL (Eds.), Handbook of depression New York, NY: Guilford Press, 2nd ed., pp. 510–530. [Google Scholar]
  6. Boccaletti S, Latora V, Moreno Y, Chavez M, Hwang DU, 2006. Complex networks: Structure and dynamics. Phys. Rep 10.1016/j.physrep.2005.10.009 [DOI] [Google Scholar]
  7. Bollen K, Lennox R, 1991. Conventional Wisdom on Measurement: A Structural Equation Perspective. Psychol. Bull 10.1037/0033-2909.110.2.305 [DOI] [Google Scholar]
  8. Borsboom D, Cramer AOJ, 2013. Network Analysis: An Integrative Approach to the Structure of Psychopathology. Annu. Rev. Clin. Psychol 10.1146/annurev-clinpsy-050212-185608 [DOI] [PubMed] [Google Scholar]
  9. Bromet EJ, Dunn LO, Connell MM, Amanda Dew M, Schulberg HC, 1986. Long-term Reliability of Diagnosing Lifetime Major Depression in a Community Sample. Arch. Gen. Psychiatry 10.1001/archpsyc.1986.01800050033004 [DOI] [PubMed] [Google Scholar]
  10. Brown TA, Di Nardo PA, Lehman CL, Campbell LA, 2001. Reliability of DSM-IV anxiety and mood disorders: Implications for the classification of emotional disorders. J. Abnorm. Psychol 10.1037/0021-843X.110.1.49 [DOI] [PubMed] [Google Scholar]
  11. Cohen J, 1960. A Coefficient of Agreement for Nominal Scales. Educ. Psychol. Meas 10.1177/001316446002000104 [DOI] [Google Scholar]
  12. Colman I, Naicker K, Zeng Y, Ataullahjan A, Senthilselvan A, Patten SB, 2011. Predictors of long-term prognosis of depression. CMAJ 10.1503/cmaj.110676 [DOI] [PMC free article] [PubMed] [Google Scholar]
  13. Crawford J, Cayley C, Lovibond PF, Wilson PH, Hartley C, 2011. Percentile Norms and Accompanying Interval Estimates from an Australian General Adult Population Sample for Self-Report Mood Scales (BAI, BDI, CRSD, CES-D, DASS, DASS-21, STAI-X, STAI-Y, SRDS, and SRAS). Aust. Psychol 10.1111/j.1742-9544.2010.00003.x [DOI] [Google Scholar]
  14. Dew MA, Reynolds CF, Houck PR, Hall M, Buysse DJ, Frank E, Kupfer DJ, 1997. Temporal profiles of the course of depression during treatment: Predictors of pathways toward recovery in the elderly. Arch. Gen. Psychiatry 10.1001/archpsyc.1997.01830230050007 [DOI] [PubMed] [Google Scholar]
  15. Finlay-Jones R, Brown GW, 1981. Types of Stressful Life Event and the Onset of Anxiety and Depressive Disorders. Psychol. Med 10.1017/S0033291700041301 [DOI] [PubMed] [Google Scholar]
  16. First MB, Williams JBW, Karg RS, Spitzer RL, 2015. Structured clinical interview for DSM-5 research version Am. Psychiatr. Assoc Washingt. D.C. [Google Scholar]
  17. Freedman R, Lewis DA, Michels R, Pine DS, Schultz SK, Tamminga CA, Gabbard GO, Gau SSF, Javitt DC, Oquendo MA, Shrout PE, Vieta E, Yager J, 2013. The initial field trials of DSM-5: New blooms and old thorns. Am. J. Psychiatry 10.1176/appi.ajp.2012.12091189 [DOI] [PubMed] [Google Scholar]
  18. Fried EI, 2017. The 52 symptoms of major depression: Lack of content overlap among seven common depression scales. J. Affect. Disord 10.1016/j.jad.2016.10.019 [DOI] [PubMed] [Google Scholar]
  19. Fried EI, Epskamp S, Nesse RM, Tuerlinckx F, Borsboom D, 2016. What are “good” depression symptoms? Comparing the centrality of DSM and non-DSM symptoms of depression in a network analysis. J. Affect. Disord 10.1016/j.jad.2015.09.005 [DOI] [PubMed] [Google Scholar]
  20. Fried EI, Nesse RM, 2015. Depression is not a consistent syndrome: An investigation of unique symptom patterns in the STAR*D study. J. Affect. Disord 10.1016/j.jad.2014.10.010 [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Fried EI, Nesse RM, 2014. The impact of individual depressive symptoms on impairment of psychosocial functioning. PLoS One. 10.1371/journal.pone.0090311 [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Fries JF, Krishnan E, Rose M, Lingala B, Bruce B, 2011. Improved responsiveness and reduced sample size requirements of PROMIS physical function scales with item response theory. Arthritis Res. Ther 10.1186/ar3461 [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Funkhouser CJ, Chacko AA, Correa KA, Kaiser AJE, Shankman SA, 2020. Unique longitudinal relationships between symptoms of psychopathology in youth: A cross-lagged panel network analysis in the ABCD study. J. Child Psychol. Psychiatry Allied Discip 10.1111/jcpp.13256 [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Germain A, Kupfer DJ, 2008. Circadian rhythm disturbances in depression. Hum. Psychopharmacol 10.1002/hup.964 [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Gorka SM, Hee D, Lieberman L, Mittal VA, Phan KL, Shankman SA, 2016. Reactivity to uncertain threat as a familial vulnerability factor for alcohol use disorder. Psychol. Med 10.1017/S0033291716002415 [DOI] [PubMed] [Google Scholar]
  26. Gutman DA, Nemeroff CB, 2003. Persistent central nervous system effects of an adverse early environment: Clinical and preclinical studies, in: Physiology and Behavior. 10.1016/S0031-9384(03)00166-5 [DOI] [PubMed] [Google Scholar]
  27. Gilmer WS, McKinney WT, 2003. Early experience and depressive disorders: Human and non-human primate studies. J. Affect. Disord 10.1016/S0165-0327(03)00046-6 [DOI] [PubMed] [Google Scholar]
  28. Harald B, Gordon P, 2012. Meta-review of depressive subtyping models. J. Affect. Disord 10.1016/j.jad.2011.07.015 [DOI] [PubMed] [Google Scholar]
  29. Hoffman M, Steinley D, Trull TJ, Sher KJ, 2019. Estimating transdiagnostic symptom networks: The problem of “skip outs” in diagnostic interviews. Psychol. Assess 10.1037/pas0000644 [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Jang KL, Livesley WJ, Taylor S, Stein MB, Moon EC, 2004. Heritability of individual depressive symptoms. J. Affect. Disord 10.1016/S0165-0327(03)00108-3 [DOI] [PubMed] [Google Scholar]
  31. Janssen J, Beekman ATF, Comijs HC, Deeg DJH, Heeren TJ, 2006. Late-life depression: The differences between early- and late-onset illness in a community-based sample. Int. J. Geriatr. Psychiatry 10.1002/gps.1428 [DOI] [PubMed] [Google Scholar]
  32. Judd LL, Schettler PJ, Coryell W, Akiskal HS, Fiedorowicz JG, 2013. Overt irritability/anger in unipolar major depressive episodes: Past and current characteristics and implications for long-term course. JAMA Psychiatry 10.1001/jamapsychiatry.2013.1957 [DOI] [PubMed] [Google Scholar]
  33. Katz AC, Hee D, Hooker CI, Shankman SA, 2018. A family study of the DSM-5 section III personality pathology model using the personality inventory for the DSM-5 (PID-5). J. Pers. Disord 10.1521/pedi_2017_31_323 [DOI] [PubMed] [Google Scholar]
  34. Kaymaz N, Van Os J, Loonen AJM, Nolen WA, 2008. Evidence that patients with single versus recurrent depressive episodes are differentially sensitive to treatment discontinuation: A meta-analysis of placebo-controlled randomized trials. J. Clin. Psychiatry 10.4088/JCP.v69n0910 [DOI] [PubMed] [Google Scholar]
  35. Keller MC, Neale MC, Kendler KS, 2007. Association of different adverse life events with distinct patterns of depressive symptoms. Am. J. Psychiatry 10.1176/appi.ajp.2007.06091564 [DOI] [PubMed] [Google Scholar]
  36. Kendler KS, 2016. The phenomenology of major depression and the representativeness and nature of DSM criteria. Am. J. Psychiatry 10.1176/appi.ajp.2016.15121509 [DOI] [PubMed] [Google Scholar]
  37. Kendler KS, Aggen SH, 2017. Symptoms of major depression: Their stability, familiality, and prediction by genetic, temperamental, and childhood environmental risk factors. Depress. Anxiety 10.1002/da.22591 [DOI] [PubMed] [Google Scholar]
  38. Kendler KS, Aggen SH, Neale MC, 2013. Evidence for multiple genetic factors underlying DSM-IV criteria for major depression. JAMA Psychiatry 10.1001/jamapsychiatry.2013.751 [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Kendler KS, Baker JH, 2007. Genetic influences on measures of the environment: A systematic review. Psychol. Med 10.1017/S0033291706009524 [DOI] [PubMed] [Google Scholar]
  40. Kendler KS, Kuhn J, Prescott CA, 2004. The Interrelationship of Neuroticism, Sex, and Stressful Life Events in the Prediction of Episodes of Major Depression. Am. J. Psychiatry 10.1176/appi.ajp.161.4.631 [DOI] [PubMed] [Google Scholar]
  41. Kessler RC, McGonagle KA, Swartz M, Blazer DG, Nelson CB, 1993. Sex and depression in the National Comorbidity Survey I: Lifetime prevalence, chronicity and recurrence. J. Affect. Disord 10.1016/0165-0327(93)90026-G [DOI] [PubMed] [Google Scholar]
  42. Kessler RC, McGonagle KA, Swartz M, Blazer DG, Nelson CB, 1993. Sex and depression in the National Comorbidity Survey I: Lifetime prevalence, chronicity and recurrence. J. Affect. Disord 10.1016/0165-0327(93)90026-G [DOI] [PubMed] [Google Scholar]
  43. Kessler RC, Wai TC, Demler O, Walters EE, 2005. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch. Gen. Psychiatry 10.1001/archpsyc.62.6.617 [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Khan AA, Gardner CO, Prescott CA, Kendler KS, 2002. Gender differences in the symptoms of major depression in opposite-sex dizygotic twin pairs. Am. J. Psychiatry 10.1176/appi.ajp.159.8.1427 [DOI] [PubMed] [Google Scholar]
  45. Klein DN, Lewinsohn PM, Rohde P, Seeley JR, Shankman SA, 2003. Family study of co-morbidity between major depressive disorder and anxiety disorders. Psychol. Med 10.1017/S0033291703007487 [DOI] [PubMed] [Google Scholar]
  46. Klein DN, Shankman SA, Rose S, 2008. Dysthymic disorder and double depression: Prediction of 10-year course trajectories and outcomes. J Psychiat. Res, 42, 408–415. DOI: 10.1016/j.jpsychires.2007.01.009 [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Kotov R, Waszczuk MA, Krueger RF, Forbes MK, Watson D, Clark LA, Achenbach TM, Althoff RR, Ivanova MY, Bagby MR, Brown TA, Carpenter WT, Caspi A, Moffitt TE, Eaton NR, Forbush KT, Goldberg D, Hasin D, Hyman SE, Lynam DR, Samuel DB, South SC, Markon K, Miller JD, Morey LC, Mullins-Sweatt SN, Ormel J, Patrick CJ, Regier DA, Rescorla L, Ruggero CJ, Sellbom M, Simms LJ, Skodol AE, Slade T, Tackett JL, Waldman ID, Widiger TA, Wright AGC, Zimmerman M, 2017. The hierarchical taxonomy of psychopathology (HiTOP): A dimensional alternative to traditional nosologies. J. Abnorm. Psychol 10.1037/abn0000258 [DOI] [PubMed] [Google Scholar]
  48. Korszun A, Moskvina V, Brewster S, Craddock N, Ferrero F, Gill M, Jones IR, Jones LA, Maier W, Mors O, Owen MJ, Preisig M, Reich T, Rietschel M, Farmer A, McGuffin P, 2004. Familiality of Symptom Dimensions in Depression. Arch. Gen. Psychiatry 10.1001/archpsyc.61.5.468 [DOI] [PubMed] [Google Scholar]
  49. Linehan M, 1986. Suicidal people. One population or two? Ann. N. Y. Acad. Sci [DOI] [PubMed] [Google Scholar]
  50. Lovibond PF, Lovibond SH, 1995. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav. Res. Ther 10.1016/0005-7967(94)00075-U [DOI] [PubMed] [Google Scholar]
  51. Lewinsohn PM, Shankman SA, Gau JM, Klein DN, 2004. The prevalence and comorbidity of subthreshold psychiatric conditions. Psychol. Med 10.1017/S0033291703001466 [DOI] [PubMed] [Google Scholar]
  52. Lux V, Kendler KS, 2010. Deconstructing major depression: A validation study of the DSM-IV symptomatic criteria. Psychol. Med 10.1017/S0033291709992157 [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Markon KE, Chmielewski M, Miller CJ, 2011. The reliability and validity of discrete and continuous measures of psychopathology: A quantitative review. Psychol. Bull 10.1037/a0023678 [DOI] [PubMed] [Google Scholar]
  54. Mazure C, Nelson JC, Price LH, 1986. Reliability and validity of the symptoms of major depressive illness. Arch. Gen. Psychiatry 10.1001/archpsyc.1986.01800050053006 [DOI] [PubMed] [Google Scholar]
  55. McClintock SM, Husain MM, Greer TL, Cullum CM, 2010. Association between depression severity and neurocognitive function in major depressive disorder: A review and synthesis. Neuropsychology 10.1037/a0017336 [DOI] [PubMed] [Google Scholar]
  56. McMakin DL, Olino TM, Porta G, Dietz LJ, Emslie G, Clarke G, Wagner KD, Asarnow JR, Ryan ND, Birmaher B, Shamseddeen W, Mayes T, Kennard B, Spirito A, Keller M, Lynch FL, Dickerson JF, Brent DA, 2012. Anhedonia predicts poorer recovery among youth with selective serotonin reuptake inhibitor treatmentresistant depression. J. Am. Acad. Child Adolesc. Psychiatry 10.1016/j.jaac.2012.01.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. McNally RJ, 2019. The network takeover reaches psychopathology. Beh. Brain Sci, 42, 25–26. [DOI] [PubMed] [Google Scholar]
  58. Mueller TI, Leon AC, Keller MB, Solomon DA, Endicott J, Coryell W, Warshaw M, Maser JD, 2013. Recurrence after recovery from major depressive disorder during 15 years of observational follow-up, in: Depression: The Science of Mental Health. 10.1176/ajp.156.7.1000 [DOI] [PubMed] [Google Scholar]
  59. Murray CJL, & Lopez AD (1997). Global mortality, disability, and the contribution of risk factors: Global burden of disease study. Lancet. 10.1016/S0140-6736(96)07495-8 [DOI] [PubMed] [Google Scholar]
  60. Murray CJL, Lopez AD, 1997. Global mortality, disability, and the contribution of risk factors: Global burden of disease study. Lancet. 10.1016/S0140-6736(96)07495-8 [DOI] [PubMed] [Google Scholar]
  61. Moskvina V, Farmer A, Jones IR, Brewster S, Ferrero F, Gill M, Jones LA, Maier W, Mors O, Owen MJ, Perry J, Preisig M, Rietschel M, McGuffin P, Craddock N, Korszun A, 2008. Sex differences in symptom patterns of recurrent major depression in siblings. Depress. Anxiety 10.1002/da.20372 [DOI] [PubMed] [Google Scholar]
  62. Myung W, Song J, Lim SW, Won HH, Kim S, Lee Y, Kang HS, Lee H, Kim JW, Carroll BJ, Kim DK, 2012. Genetic association study of individual symptoms in depression. Psychiatry Res 10.1016/j.psychres.2011.12.037 [DOI] [PubMed] [Google Scholar]
  63. Nierenberg AA, Trivedi MH, Fava M, Biggs MM, Shores-Wilson K, Wisniewski SR, Balasubramani GK, Rush AJ, 2007. Family history of mood disorder and characteristics of major depressive disorder: A STAR*D (sequenced treatment alternatives to relieve depression) study. J. Psychiatr. Res 10.1016/j.jpsychires.2006.02.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Olino TM, Shankman SA, Klein DN, Seeley JR, Pettit JW, Farner RF, Lewinsohn PM, 2012. Lifetime rates of psychopathology in single versus multiple diagnostic assessments: Comparison in a community sample of probands and siblings. J. Psychiatr. Res, 46, 1217–1222. doi: 10.1016/j.jpsychires.2012.05.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Oquendo MA, Barrera A, Ellis SP, Li S, Burke AK, Grunebaum M, Endicott J, Mann JJ, 2004. Instability of Symptoms in Recurrent Major Depression: A Prospective Study. Am. J. Psychiatry 10.1176/appi.ajp.161.2.255 [DOI] [PubMed] [Google Scholar]
  66. Regier DA, Narrow WE, Clarke DE, Kraemer HC, Kuramoto SJ, Kuhl EA, Kupfer DJ, 2013. DSM-5 field trials in the United States and Canada, part II: Test-retest reliability of selected categorical diagnoses. Am. J. Psychiatry 10.1176/appi.ajp.2012.12070999 [DOI] [PubMed] [Google Scholar]
  67. Robins E, Guze SB, 1970. Establishment of diagnostic validity in psychiatric illness: its application to schizophrenia. Am. J. Psychiatry 10.1176/ajp.126.7.983 [DOI] [PubMed] [Google Scholar]
  68. Schmittmann VD, Cramer AOJ, Waldorp LJ, Epskamp S, Kievit RA, Borsboom D, 2013. Deconstructing the construct: A network perspective on psychological phenomena. New Ideas Psychol 10.1016/j.newideapsych.2011.02.007 [DOI] [Google Scholar]
  69. Soehner AM, Kaplan KA, Harvey AG, 2014. Prevalence and clinical correlates of cooccurring insomnia and hypersomnia symptoms in depression. J. Affect. Disord 10.1016/j.jad.2014.05.060 [DOI] [PMC free article] [PubMed] [Google Scholar]
  70. Shankman SA, Funkhouser CJ, Klein DN, Davila J, Lerner D, Hee D, 2018. Reliability and validity of severity dimensions of psychopathology assessed using the Structured Clinical Interview for DSM-5 (SCID). Int. J. Methods Psychiatr. Res 10.1002/mpr.1590 [DOI] [PMC free article] [PubMed] [Google Scholar]
  71. Shankman SA, Gorka SM, 2015. Psychopathology research in the RDoC era: Unanswered questions and the importance of the psychophysiological unit of analysis. Int J of Psychophys, 98, 330–337. doi: 10.1016/j.ijpsycho.2015.01.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  72. Shankman SA, Klein DN, Lewinsohn PM, Seeley JR, Small JW, 2008. Family study of subthreshold psychopathology in a community sample. Psychol. Med 10.1017/S0033291707001857 [DOI] [PubMed] [Google Scholar]
  73. Shankman SA, Lewinsohn PM, Klein DN, Small JW, Seeley JR, Altman SE, 2009. Subthreshold conditions as precursors for full syndrome disorders: A 15-year longitudinal study of multiple diagnostic classes. J. Child Psychol. Psychiatry Allied Discip 10.1111/j.1469-7610.2009.02117.x [DOI] [PMC free article] [PubMed] [Google Scholar]
  74. Shrout PE, 1998. Measurement reliability and agreement in psychiatry. Stat. Methods Med. Res 10.1191/096228098672090967 [DOI] [PubMed] [Google Scholar]
  75. Sijtsma K, 2009. On the use, the misuse, and the very limited usefullness of Cronbach’s alpha. Psychometrika, 74, 107–120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  76. Smoller JW, Kendler K, Craddock N, Lee PH, Neale BM, Nurnberger JN, Ripke S, Santangelo S, Sullivan PS, Neale BN, Purcell S, Anney R, Buitelaar J, Fanous A, Faraone SF, Hoogendijk W, Lesch KP, Levinson DL, Perlis RP, Rietschel M, Riley B, Sonuga-Barke E, Schachar R, Schulze TS, Thapar A, Kendler KK, Smoller JS, Neale M, Perlis R, Bender P, Cichon S, Daly MD, Kelsoe J, Lehner T, Levinson D, O’Donovan Mick, Gejman P, Sebat J, Sklar P, Daly M, Devlin B, Sullivan P, O’Donovan Michael, 2013. Identification of risk loci with shared effects on five major psychiatric disorders: A genome-wide analysis. Lancet. 10.1016/S0140-6736(12)62129-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
  77. Spitzer RL, Fleiss JL, 1974. A reanalysis of the reliability of psychiatric diagnosis. Br. J. Psychiatry 10.1192/bjp.125.4.341 [DOI] [PubMed] [Google Scholar]
  78. Spitzer RL, Forman JBW, Nee J, 1979. DSM-III field trials: I. Initial interrater diagnostic reliability. Am. J. Psychiatry 10.1176/ajp.136.6.815 [DOI] [PubMed] [Google Scholar]
  79. Sullivan PF, Neale MC, Kendler KS, 2000. Genetic epidemiology of major depression: Review and meta-analysis. Am. J. Psychiatry 10.1176/appi.ajp.157.10.1552 [DOI] [PubMed] [Google Scholar]
  80. Takayanagi Y, Spira AP, Roth KB, Gallo JJ, Eaton WW, Mojtabai R, 2014. Accuracy of Reports of Lifetime Mental and Physical Disorders. JAMA Psychiatry 10.1001/jamapsychiatry.2013.3579 [DOI] [PMC free article] [PubMed] [Google Scholar]
  81. Uher R, Perlis RH, Henigsberg N, Zobel A, Rietschel M, Mors O, Hauser J, Dernovsek MZ, Souery D, Bajs M, Maier W, Aitchison KJ, Farmer A, McGuffin P, 2012. Depression symptom dimensions as predictors of antidepressant treatment outcome: Replicable evidence for interest-activity symptoms. Psychol. Med 10.1017/S0033291711001905 [DOI] [PMC free article] [PubMed] [Google Scholar]
  82. Walther S, Bernard JA, Mittal VA, Shankman SA, 2019. The utility of an RDoC motor domain to understand psychomotor symptoms in depression. Psychol. Med 10.1017/S0033291718003033 [DOI] [PubMed] [Google Scholar]
  83. Wells JE, Horwood LJ, 2004. How accurate is recall of key symptoms of depression? A comparison of recall and longitudinal reports. Psychol. Med 10.1017/S0033291703001843 [DOI] [PubMed] [Google Scholar]
  84. Zahn R, Lythe KE, Gethin JA, Green S, Deakin JFW, Young AH, Moll J, 201, The role of self-blame and worthlessness in the psychopathology of major depressive disorder. J Affect Disorder, 186,337–341. doi: 10.1016/j.jad.2015.08.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
  85. Zimmerman M, Ellison W, Young D, Chelminski I, Dalrymple K, 2015. How many different ways do patients meet the diagnostic criteria for major depressive disorder? Compr. Psychiatry 10.1016/j.comppsych.2014.09.007 [DOI] [PubMed] [Google Scholar]
  86. Zuk O, Hechter E, Sunyaev SR, Lander ES, 2012. The mystery of missing heritability: Genetic interactions create phantom heritability. Proc. Natl. Acad. Sci. U. S. A 10.1073/pnas.1119675109 [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES