Clashing Diagnostic Approaches: DSM-ICD versus RDoC

Abstract

Since at least the middle of the past century, one overarching model of psychiatric classification, namely, that of the Diagnostic and Statistical Manual of Mental Disorders and International Classification of Diseases (DSM-ICD), has reigned supreme. This DSM-ICD approach embraces an Aristotelian view of mental disorders as largely discrete entities that are characterized by distinctive signs, symptoms, and natural histories. Over the past several years, however, a competing vision, namely, the Research Domain Criteria (RDoC) initiative launched by the National Institute of Mental Health, has emerged in response to accumulating anomalies within the DSM-ICD system. In contrast to DSM-ICD, RDoC embraces a Galilean view of psychopathology as the product of dysfunctions in neural circuitry. RDoC appears to be a valuable endeavor that holds out the long-term promise of an alternative system of mental illness classification. We delineate three sets of pressing challenges – conceptual, methodological, and logistical/pragmatic – that must be addressed for RDoC to realize its scientific potential, and conclude with a call for further research, including investigation of a rapprochement between Aristotelian and Galilean approaches to psychiatric classification.

“I can calculate the motion of heavenly bodies, but not the madness of people.”

-Sir Isaac Newton

The contentious field of psychiatric classification has long been marked by an analogous clash of Aristolelian and Galiliean worldviews (Carson, 1996). The approach enshrined in recent editions of the Diagnostic and Statistical Manual of Mental Disorders (DSM) and the International Classification of Diseases (ICD), which we heretofore dub the DSM-ICD model, adopts – at least implicitly – an Aristotelian model of categorization. In this approach, which has been dominant in American psychiatry over at least the past half century, psychiatric disorders are presumed to constitute largely discrete entities: They are commonly assumed to differ qualitatively from normality and from each other. For example, the influential framework for the validation of mental disorders advanced by Robins and Guze (1970) delineated five criteria for ascertaining whether a diagnostic entity is genuine. Among them was the “delimitation [of the disorder] from other disorders” (p. 108), the assumption being that a diagnosis that overlaps extensively with other diagnoses is of doubtful validity.

DSM-ICD and Kraepelin: Hints of a Paradigm Shift

The reigning DSM-ICD approach is commonly referred as “neo-Kraepelinian” (Blashfield, 2002) in recognition of the contributions of the pioneering German systematist Emil Kraepelin, who initially believed that superficially similar mental disorders, such as dementia praecox (now termed schizophrenia) and manic-depression (now termed bipolar disorder), could be differentiated by a detailed documentation of their (a) signs (observable manifestations), (b) symptoms (subjective reports), and (c) natural history (trajectory over time). Kraepelin, following in the footsteps of Linnean botanists (Compton & Guze, 1995), regarded different mental disorders as akin to differing species or subspecies that could be distinguished largely by their topography.

A key assumption of the neo-Kraepelian approach is that diagnostic signs and symptoms are sufficient to differentiate mental disorders. Nevertheless, as the history of medicine reminds us, this assumption may be unwarranted. For example, in the early 20^th century, physicians diagnosed dozens of different fevers – ranging alphabetically from blackwater fever to yellow fever – and distinguished them on the basis of other co-occurring signs and symptoms (Kihlstrom, 2002). Today, physicians recognize that fever is a nonspecific indicator of a plethora of underlying pathologies rather than a disorder per se, and instead attempt to identify the etiology of the fever prior to initiating treatment.

Despite the enormous impact of the DSM on everyday research and practice, there are growing indications that its hegemony may at last be beginning to wane. Over the past several years, rumblings of what some have described as a Kuhnian “paradigm shift” (Fu & Costafreda, 2013) or at least a “reboot” (Kendler, 2015) in psychiatric classification have become increasingly audible Among the first responders has been the NIMH’s Research Domain Criteria initiative (RDoC; Insel et al., 2010), Though more of promissory note than a formalized system at present, the RDoC aims to develop a system of psychiatric classification based not on signs, symptoms, and course, ala Kraepelin, but instead on markers of psychobiological systems linked to adaptive – and maladaptive – functioning.

In the pages to follow, we briefly examine the successes of the DSM-ICD approach, discuss the anomalies that have led scholars to question its scientific status, and present the raison d’etre of RDoC as a competing model. We focus on RDoC’s scientific potential, but also on the often unappreciated challenges standing in the way of realizing this potential. We conclude with recommendations for future directions in classification, including the possibility of a rapprochement between these two overarching approaches.

DSM and ICD: Origins and Assumptions

The history of the DSM and its revisions has been recounted in numerous sources (e.g., Lieberman & Ogas, 2015; Lilienfeld, Smith, & Watts, 2014; Widiger & Clark, 2000; Wilson, 1993), so we reprise it only very briefly here. In response to the perceived shortcomings with the first two DSMs, especially their often vague diagnostic descriptions and the low or best modest inter-rater reliabilities of their diagnostic categories, the American Psychiatric Association, with psychiatrist Robert Spitzer at the helm, released DSM-III in 1980 (American Psychiatric Association, 1980). DSM-III was influenced heavily by the seminal work of the St. Louis psychiatric group at Washington University, who had introduced preliminary diagnostic criteria and algorithms (decision rules) for 14 major mental disorders in the early 1970s (Feighner et al., 1972). Another noteworthy and more immediate precursor of DSM-III were the Research Diagnostic Criteria (RDC; Spitzer, Endicott, & Robins, 1978), which delineated diagnostic criteria and algorithms for approximately 20 major diagnostic categories, along with subtypes within several of these categories.

Like the Feighner and RDC criteria, DSM-III was considerably more neo-Kraepelinian than its DSM predecessors (Compton & Guze, 1995), as it emphasized the differentiation of conditions on the basis on their signs, symptoms, and natural history. In accord with its neo-Kraepelinian emphasis, DSM-III provided users with (a) standardized diagnostic criteria and (b) algorithms, for each diagnosis. Rather than merely describing each diagnosis, as DSM-I and DSM-II had done, DSM-III delineated the specific signs and symptoms comprising each diagnosis and the method by which they needed to be combined to establish it. In this way, it almost certainly boosted the inter-rater reliability of most diagnostic categories, although some critics have maintained that these increases were modest at best (Kirk & Kutchins, 1992). Although DSM-III-R and DSM-IV introduced a number of important changes to numerous diagnoses, they retained DSM-III’s basic structure and format.

In an effort to accommodate novel evidence that had emerged in the wake of the second millennium, DSM-5, spearheaded by David Kupfer and Darrel Regier, was published in May of 2013 amidst a host of searing criticisms, many of which centered on major alterations to a number of diagnostic categories in the absence of adequate data (see Frances & Widiger, 2012). Although the early phases of planning for DSM-5 were rife with speculations regarding drastic changes in its content and structure, such as a heightened focus on neuroscientific markers or the inclusion of a potential dimensional system for personality disorders (Skodol et al., 2011), DSM-5 by and large retained the principal format of DSM-IV, as well as most of its major diagnoses. Although DSM-5 is the predominant system of psychiatric classification around the world, Chapter V of the ICD, 10^th revision (ICD-10) of the World Health Organization (1993) is a neo-Krapelinian alternative to DSM-5 (which is similar to the DSM in most important respects) that has been adopted in numerous countries outside of the U.S (ICD-11 is under construction as of this writing).

The hue and cry regarding the most controversial changes in DSM-5, such as the deletion of the bereavement criterion for major depressive disorder or the jettisoning of hypochondriasis as a diagnosis (Frances, 2014), may have obscured a deeper point. With the potential exception of (a) the inclusion of dimensional scales to capture the functioning and impairment associated with major mental disorders and (b) the inclusion of a hybrid prototype-dimensional model for personality disorders, both of which appeared in Section III (“Emerging Measures and Models”), DSM-5 was every bit as neo-Kraepelinian as its predecessors. Thus, despite initial hopes – or fears, depending on one’s perspective – DSM-5 did not constitute a radical shift in classification or diagnosis, at least with respect to its central presuppositions.

DSM-ICD Successes

Extreme skeptics of the DSM-ICD system (e.g., Greenberg, 2015) have at times argued that this approach has yielded psychiatric categories with negligible validity. Similarly, in a widely publicized blog posting announcing the shift of the National Institute of Mental Health (NIMH) toward RDoC, NIMH Director Thomas Insel (2013) wrote that the DSM’s “major weakness is its lack of validity.”

The assertion that DSM categories “lack validity” paints with too broad a brush. Many DSM categories, such as major depression, panic disorder, bipolar disorder, obsessive-compulsive disorder, and schizophrenia, clearly display at least some construct validity, as demonstrated by consistent relations with laboratory indicators, biological correlates, natural history, and family history. For example, although the DSM diagnosis of schizophrenia almost surely encompasses a heterogeneous mélange of overlapping conditions, this diagnosis is associated with certain external correlates, such as smooth pursuit eye movement dysfunction, ventricular enlargement, a chronic and frequently relapsing course, and a family history of schizophrenia among biological relatives (e.g., Tsuang, Stone, & Faraone, 2000).

Moreover, the development of lists of empirically supported therapies (ESTs), which are psychological treatments that have been demonstrated to be efficacious for specific disorders (e.g., cognitive-behavioral therapy for major depression, panic control treatment for panic disorder; Chambless & Ollendick, 2001) is a testament to the treatment validity (Hayes, Nelson, & Jarrett, 1996) of at least some DSM-ICD categories. .If the DSM and ICD were entirely devoid of validity, one could not identify psychological interventions that are differentially efficacious for different conditions (Garb, Lilienfeld, Nezworski, Wood, & O’Donohue, 2009; but see “Inadequate treatment validity”). Moreover, by providing researchers and clinicians with a lingua franca for facilitating diagnostic communication, the DSM has greatly accelerated scientific progress regarding the correlates, etiology, and treatment of mental disorders (see Kendell, 1975, for a broader discussion). These impressive achievements are not to be dismissed lightly.

DSM-ICD Anomalies

At the same time, it has become evident that the DSM-ICD approach has been beset by a growing number of anomalies (Lilienfeld, 2014b), many of which have not been adequately resolved across DSM editions; we touch on the most salient of these conundrums here.

Scientifically arbitrary diagnostic cut-offs

The DSM is technically agnostic with regard to whether the conditions it contains are categorical (taxonic) or dimensional in nature. Still, the DSM adopts a categorical approach as a working model for measurement purposes. This model is debatable for two major reasons. First, data from taxometric studies, which allow investigators to ascertain whether an observed distribution can be decomposed into two or more independent distributions (Meehl & Golden, 1982), indicate that most DSM conditions, including the lion’s share of mood, anxiety, eating, personality, and externalizing disorders, appear to be underpinned by one or more dimensions. The potential exceptions are schizophrenia spectrum disorders, autism spectrum disorders, and some substance use disorders (Haslam, Holland, & Kuppens, 2012). The absence of a point of rarity (Sneath, 1957) demarcating most DSM conditions from normality raises questions regarding the Aristotelian assumption of discrete conditions. Second, even if it turns out that some DSM conditions are taxonic, it would not justify the imposition of a categorical measurement model. A categorical measurement model decreases reliability and validity relative to a dimensional model, because it forfeits valuable psychometric information (Markon, Chmielewski, & Miller, 2011). Even taxonic variables, dimensional measures almost always provide more sensitive indicators than do categorical measures, as they offer information regarding individuals’ proximity to the natural cut-point.

Heterogeneity

The polythetic model of DSM-III-R and later DSM editions, sometimes derided as the “Chinese menu” approach, provides criteria that are neither singly necessary nor jointly sufficient for a diagnosis. This model has generated marked phenotypic heterogeneity. In DSM-5, for example, 256 different sign/symptom combinations are compatible with a diagnosis of borderline personality disorder. As an even more extreme example, in DSM-5, there are 636,120 ways to meet diagnostic criteria for PTSD (Galatzer-Levy & Bryant, 2013). It is even possible for two people to meet DSM-5 criteria for obsessive-compulsive personality disorder and to share no diagnostic criteria. Although phenotypic heterogeneity does not necessarily imply etiological heterogeneity, it seems unlikely that two conditions that overlap minimally in their expression would stem from similar, let alone identical, causal influences. In addition, such heterogeneity renders the search for etiological agents more challenging (see Monroe & Anderson, 2015, for a discussion of this issue with respect to major depressive disorder).

Comorbidity

An ideal taxonomy yields categories that are largely mutually exclusive, with few or no intermediate cases (Frances, 1980). Yet across its multiple editions, the DSM has consistently been bedeviled by a vexing problem termed comorbidity. This term was coined by medical epidemiological Alvan Feinstein (1970) to denote the co-occurrence of a “distinct clinical entity that has existed or that may occur during the clinical course of a patient who has the index disease under study” (pp. 456-457). Although some scholars have voiced concerns about the use of the “comorbidity” concept in most cases of psychopathology research given that it is unclear how many DSM conditions are distinct clinical entities (Lilienfeld, Waldman, & Israel, 1994), extremely high levels of covariation among ostensibly separable conditions may raise questions concerning their etiological independence.

The extent of the comorbidity problem is difficult to overstate. In the Australian National Survey of Mental Health and Well-Being, 21% of participants with one DSM-IV disorder met criteria for three or more DSM-IV disorders (Andrews et al., 2002); comparable data derive from the United States National Comorbidity Study (Kessler et al., 1994). The comorbidity problem is especially glaring in the personality disorders domain. In one study of over 1000 patients (Stuart et al., 1998), patients with one DSM personality disorder met criteria for an additional 1.7 DSM-III-R personality disorders on average; 9.4% met criteria for 4 or more personality disorders. Notably, only 18% of patients with a personality disorder did not meet criteria for at least one other personality disorder, indicating that comorbidity is much more the rule than the exception. In one especially extreme case, a participant in a research study simultaneously met criteria for all 10 DSM-IV (and DSM-5) personality disorders (Lilienfeld, Smith, & Watts, 2013)!

In the eyes of many authors, the presence of rampant comorbidity is a red flag that the DSM system is not drawing the correct diagnostic borders. Other authors (e.g., Maj, 2005) go further, suggesting that such comorbidity reflects the propensity of the DSM to attach different names to slightly different manifestations of a shared predisposition, a logical error known as the jangle fallacy (Block, 1995). For example, many or most personality disorders appear to be complex constellations or configurations of normal-range personality dimensions (e.g., antagonism, low conscientiousness, introversion; see Widiger & Trull, 2007). If so, it is hardly surprising that many of these conditions, such as narcissistic and borderline personality disorders, display substantial covariation, as it likely that these disorders reflect highly overlapping densifications in multivariate space.

Large number of intermediate cases

As noted earlier, an optimal classification system consists of categories that yield few intermediate cases (Frances, 1980). Yet for most major classes of psychopathology, one of the most frequent diagnoses is NOS (Not Otherwise Specified), meaning that most patients with mental disorders do not fit into any extant category (Stein, Black, & Pienaar, 2000; Westen, 2012). Such patients must be diagnosed by means of a “wastebasket category” that encompasses individuals who are clearly disordered but whose signs and symptoms do not meet criteria for any extant diagnosis. For eating disorders, NOS has typically been the modal diagnosis in routine clinical settings (Fairburn & Bohn, 2005); in unstructured interviewed studies of personality disorders, NOS is similarly the most frequent diagnosis (Verheul & Widiger, 2004).

The high prevalence of NOS diagnoses probably derives from what Hyman (2010) termed the “problem of overspecification” (p. 166). For its diagnoses, the DSM-ICD prescribe strict criteria and algorithms that almost certainly exclude many individuals who suffer from the same dysfunction as do most individuals within the category, but who fall just short of the diagnostic threshold. For example, in a research study an individual who met all of the criteria for DSM-5 major depressive disorder (MDD) but whose episode lasted only 12 days (rather than the required two weeks) would be precluded from receiving a formal diagnosis of MDD. Nevertheless, it is unlikely that this person differs fundamentally from most people with MDD.

Inadequate treatment validity.

As also noted earlier, the presence of lists of ESTs suggests that at least some DSM categories possess treatment validity and clinical utility (Garb et al., 2009). Nevertheless, it clear that the linkage between diagnosis and treatment is not nearly as tight in psychiatry as it is in other domains of medicine. Increasing data suggest although at least some treatment specificity clearly exists (Lilienfeld, 2014a), common factors – those that cut across most or all efficacious treatments - account for hefty chunks of variance in therapeutic outcomes (Laska, Gurman, & Wampold, 2014). More broadly, the DSM era has not borne witness to large-scale reductions in the morbidity or mortality associated with major mental disorders; nor to decreases in suicide rates. These sobering facts stand in stark contrast to sharp declines in the death rates associated with heart disease, stroke, cancer, and a number of major medical disorders (Insel, 2009). Of course, it would be unfair to attribute all of the lack of progress in diminishing psychiatric morbidity and mortality to our contemporary system of classification. Nevertheless, this state of affairs raises the distinct possibility that fresh approaches to classification will be needed to achieve breakthroughs in intervention.

Genetic and environmental nonspecificity.

If DSM-ICD conditions were largely distinct, as would be expected from a neo-Kraepelinian framework, one might anticipate that their genetic and environmental architecture would be largely distinct. Yet the more we learn about the etiology of most DSM conditions, the more apparent it becomes that many of the influences impinging on them are nonspecific. For example, one recent genome-wide association study revealed that many single nucleotide polymorphisms (SNPs) of small effect contribute to risk for five mental disorders traditionally deemed to be etiologically disparate, namely schizophrenia, bipolar disorder, major depression, attention-deficit hyperactivity disorder (ADHD), and autism spectrum disorder. For example, the genetic correlation of SNPs in schizophrenia and bipolar disorder was.68 (Cross-Disorder Group of the Psychiatric Genetics Consortium, 2013). The genetic overlap in this study is especially striking given that several of these disorders (e.g., ADHD and schizophrenia) display little overt phenotypic resemblance.

Similarly, there is compelling evidence for substantial environmental nonspecificity in the etiology of mental disorders. For example, even after controlling statistically for childhood adversity, childhood sexual abuse has emerged as modest nonspecific antecedent, and perhaps a causal risk factor, for a host of different mental disorders, including multiple mood, anxiety, and substance use disorders (Molnar, Buka, & Kessler, 2001). Although not undermining the possibility of some disorder specificity, these well-replicated findings pose a challenge to the assumption that DSM-ICD conditions are associated with distinct specific etiologies (see Meehl, 1977, for a discussion of different forms of specific etiology).

DSM-ICD anomalies: Taking stock

In aggregate, the anomalies we have reviewed reveal that all is not well with the scientific health of the DSM. With each new DSM edition, concerted efforts have been directed to attempting to salvage the essence of the neo-Kraepelinian framework by a variety of ad hoc maneuvers (Popper, 1959), such as removing, adding, or revising diagnostic criteria; changing thresholds or age of onsets for diagnoses; adding new diagnoses or removing others; adding new disorder subtypes; and adding or removing hierarchical exclusion rules (which prohibit certain conditions from being diagnosed if they appear to be attributable to another condition). Yet repeated attempts to nibble around the edges of the DSM anomalies across editions of the manual have met with at best limited success. In the words of Kendler (2015), the DSM may have found itself in a “box canyon”: a deep gorge from which it cannot extricate itself. If so, small-scale revisions to the DSM-ICD, or, in the lingo of Lakatos (1978), modifications to the auxiliary hypotheses of the DSM-ICD research program, are unlikely to remedy its core shortcomings. In reacting to the lack of substantial scientific progress in the DSM-ICD research program, a growing chorus of critics has argued that this approach has failed to carve nature at its joints, to borrow Plato’s hallowed phrase (see Gangestad & Snyder, 1985). .

The view that the DSM-ICD model is defective beyond repair is by no means new. In 1989, eminent British psychiatrist R.E. Kendell (see also Widiger & Trull, 2007) wrote that:

Ninety years have now elapsed since Kraepelin first provided the framework of a plausible classification of mental disorders. Why then, with so many potential validators available, have we made so little progress since that time? …One important possibility is that the discrete clusters of psychiatric symptoms we are trying to delineate do not actually exist (Kendell, 1989, p. 313).

Ironically, a similar alarm call was sounded 13 years later by the two principal architects of DSM-5: “Research exclusively focused on refining DSM-defined syndromes may never be successful in uncovering their underlying etiologies. For that to happen, an as yet unknown paradigm shift may need to occur” (Kupfer et al., 2002, xix). RDoC may be heralding the leading edge of this very paradigm shift.

RDoC: Intellectual Antecedents

Many of the intellectual precursors of RDoC can be traced to early successes in the biomedical sciences, which unearthed the etiology of what we now recognize as physical disorders masquerading as mental disorders (Schildkrout, 2014). At the same time, many of the ensuring failures of this approach to traditional mental disorders, such as schizophrenia, imparted valuable sobering lessons that were to inform RDoC.

Laboratory methods in medicine

It is easy to forget that by the middle or even the end of 19^th century, medical diagnosis, much like modern-day psychiatric diagnosis, was based almost exclusively on a combination of careful history taking from patients and a physical examination, an approach known as bedside medicine (Ackerknect, 1967). Medical tests began to be introduced in the 18^th and 19^th centuries, with the advent of the stethoscope, opthalmoscope, and laryngoscope becoming routine in medical examinations by the 1850s (Kihlstrom, 2002). Nevertheless, it was not until the 20^th century that laboratory tests became de rigueur for diagnosing many medical conditions (Burke, 2002). The examination of cerebrospinal fluid to detect meningitis emerged around the turn of the 20^th century, and the Wasserman Test for the detection of syphilis was developed in 1906 (Berger, 1999). Today, over 3000 standardized laboratory tests are used in standard medical practice (Kapur, Phillips, & Insel, 2012).

Early triumphs in psychopathology research

The laboratory revolution in medicine led to two sensational triumphs in the psychopathology domain. Specifically, medical science demonstrated that two conditions long presumed to be largely or entirely psychogenic, namely general paresis and pellagra, were of purely organic etiology. General paresis, also called general paralysis of the insane, accounted for 5 to 10 percent of hospital admissions around the turn of the century (Tsay, 2013). Its primary psychological features included delusions, grandiosity, euphoria, concentration difficulties, and poor impulse control. In 1913, Noguchi and Moore isolated the syphilis spirochete (Treponema pallidum) in the brains of patients with general paresis, demonstrating that condition was an outcome of tertiary syphilis, in which the bacterium had invaded the central nervous system. Individuals suffering from pellagra, whose symptoms included irritability, depression, insomnia, paranoia, confusion, and memory problems, were also common fixtures in inpatient psychiatric hospitals around the turn of the 20^th century. Research later showed this condition to be a consequence of niacin deficiency (Sartorius, 1993).

The discovery of the medical causes of general paresis and pellagra were justifiably hailed as triumphs of biomedicine, especially because they promptly led to effective treatments (penicillin and niacin, respectively). Coming as they did against the backdrop of the discovery of mendelian genetics, they encouraged several generations of psychopathology researchers to believe that other major mental disorders, such as schizophrenia, bipolar disorder, and obsessive-compulsive disorder, would similarly be traceable to a single etiological agent (Kendler & Schaffner, 2011). In many respects, however, these successes were misleading; in the case of schizophrenia, dozens if not hundreds of candidates for the “schizochete” (Neimark, 2008) – a presumably unique biological cause of the disorder, such as dopamine hyperactivity, a dominant gene, or hypofrontality – were proposed over the years. Yet this “single bullet” approach ultimately failed, almost surely because the causes of these disorders turned out to be exceedingly multifactorial (Kendler, 2006). This approach also foundered because these causes are not unique to schizophrenia, but are shared across numerous DSM-ICD disorders. This belated realization was almost certainly one of many catalysts for the transdiagnostic approach embraced by RDoC.

Biological markers

The appreciation of multifactorial causality notwithstanding, the 1970s and 1980s saw keen interest in the identification of biological markers for psychopathology (Iacono, 1985). Sophisticated thinkers viewed these markers not as akin to the spirochete that causes general paresis, but rather as fallible but nonetheless useful indicators that could assist in etiological and perhaps diagnostic efforts. For example, the early 1980s bore witness to the development of the Dexamethasone Suppression Test (DST), an indicator of cortisol reactivity, to assist in the diagnosis of endogenous depression (Carroll, 1982), the discovery of reduced rapid eye movement (REM) latency as an indicator of major depression (Kupfer et al., 1978), and the development of measures of smooth pursuit eye movement dysfunction (Iacono, Tuason, & Johnson, 1981) as an indicator of schizophrenia.

Nevertheless, the much vaunted search for biological markers, which continues apace today, has met with at best qualified success. Although some of these putative markers offered fruitful leads to etiology, no marker displayed sufficiently high levels of both sensitivity and sensitivity (as well as positive and negative predictive power) to qualify as proxies for research diagnoses, let alone as screening or laboratory tests for routine clinical use (Insel, 2014; Kapur et al., 2012). For example, while the DST displayed reasonably high sensitivity for endogenous depression, its specificity was often poor, reflecting high levels of false positive identifications among patients with schizophrenia, alcoholism, anxiety disorders, and dementia (Coppen et al., 1983). As a consequence, a state of disenchantment regarding the DST and other biological markers gradually set in. As of today, the lone laboratory-based biological tests in the DSM are for a subset of sleep disorders, such as assay-based measures of hypocretin levels for narcolepsy (American Psychiatric Association, 2013), and even these tests are hardly infallible.

Nevertheless, it has long been unclear whether the failure to detect highly sensitive and specific markers of mental disorders was a reflection of the failure of the biological strategy per se rather than of the gross imprecision of diagnostic phenotypes themselves. As the 20^th century drew to a close, more and more researchers were placing their bets on the latter.

The slow progress of psychopharmacology

Another impetus for RDoC has been the lack of progress in the psychopharmacology industry over the past few decades. The 1950s, which were marked by “the explosive growth” of psychopharmacology, witnessed the discovery of the first tricyclic antidepressant, imipramine (Tofranil), the first monoamine oxidase inhibitor, iproniazid (Marsilid), the first benzodiazepine, chlordiazepoxide (Librium), and the first antipsychotic medication, chlorpromazine (Thorazine; see Lopez-Munoz & Alamo, 2009). The next two to three decades were awash in the development of scores of variants of these medications.

Yet for reasons that are still poorly understood, progress in psychopharmacology soon began to slow to a crawl. Although the advent of selective serotonin uptake inhibitors in the 1980s and atypical antipsychotic medications in the 1990s substantially ameliorated the problematic side effect profiles of major classes of psychiatric drugs, there is little evidence that these new classes of medications contributed to marked enhancements in overall therapeutic efficacy (Anderson, 2000; Chakos et al., 2014). As another example, recent enthusiasm concerning the potential of glutamate antagonists for schizophrenia gave way to disillusionment in the wake of several failed clinical trials (Carroll, 2012). In 2011, two premier drug companies, AstraZeneca and GlaxoSmithKline, announced that they were terminating their search for new psychiatric medications in light of the bleak outlook for major discoveries (Cowen, 2012), and other firms may soon be following suit. Although some of lack of progress may stem from failures of innovation and risk-taking in the psychopharmacology industry, much of it may also stem from the fact that the disorder phenotypes afforded by DSM-ICD are too crude and heterogeneous to allow for effective medication targets.

Precision medicine

A final intellectual cross-current that informed RDoC was the emergence of precision medicine, sometimes also called personalized medicine, a new branch of medicine that strives to harness the power of clinic-pathological laboratory measures, such as genetic tests, to tailor treatments to individuals (Mirnemazi, Nicholson, & Darzi, 2012). For example, researchers have developed a targeted drug treatment for a small subset (4%) of patients with cystic fibrosis who possess a specific mutation (Insel, 2014). Similarly, the treatment of breast cancer has been revolutionized by the development of oncotype testing, permitting physicians to move from a “one size fits all” intervention approach to treatment geared to specific genetic profiles (Afimos, Barthelemy, & Awada, 2014). The coming era of precision medicine has stirred hopes for a similar revolution in psychiatry and clinical psychology.

The Birth of RDoC: Fundamental Presuppositions and Structure

In many ways, RDoC has DSM to thank for its birth. During his tenure as NIMH director, Steven Hyman expressed his concern at seeing preclinical researchers attempting to develop animal models that would mimic DSM-based diagnostic criteria. As Hyman put it “The DSM system…created an unintended epistemic prison that was palpably impeding scientific progress…Even animal studies that purported to develop disease models…were often judged by how closely they approximated DSM disorders.” (Hyman, 2010, p. 157). Under Hyman’s successor, Thomas Insel, the NIMH sought to develop a new framework for organizing research that could liberate clinical and translational researchers from the“epistemic prison” of DSM criteria. The primary concern was not that the DSM criteria lacked validity or utility for clinical practice; rather, the issue centered on whether the de facto whole-sale appropriation of DSM criteria by the research community was impairing the elucidation of pathophysiology. In this sense, the birth of the RDoC initiative can be seen as inextricably linked to the NIMH’s growing emphasis on translational neuroscience to guide research priorities.

So-named as an homage to the RDC, the RDoC was formally launched in 2009 as a bold initiative to transform the current framework of psychiatric classification into an explicitly biological system (Cuthbert, 2014; Insel et al., 2010; Sanislow et al., 2010). RDoC’s aim was ambitious: to inaugurate nothing short of a paradigm shift in descriptive psychiatry. Rather than base psychiatric diagnosis on presenting signs and symptoms, as do DSM and ICD, RDoC strives to anchor psychiatric classification and diagnosis in a scientifically supported model of neural circuitry. RDoC conceptualizes mental disorders as dysfunctions in brain systems that bear important adaptive implications, such as systems linked to reward responsiveness and threat sensitivity (see also Harkness, Reynolds, & Lilienfeld, 2013).

An RDoC Primer

In a widely discussed and controversial blog posting issued several weeks prior to the release of DSM-5, NIMH Director Thomas Insel (2013) staked out a bold claim for RDoC: “…NIMH will be re-orienting its research away from DSM categories. Going forward, we will be supporting research projects that look across current categories – or sub-divide current categories – to begin to develop a better system.” Nevertheless, in subsequent communications, Insel made clear that NIMH would continue to fund grants that included data on DSM categories, but that it would prioritize those that emphasized transdiagnostic etiological processes.

At least for the foreseeable future, RDoC is not envisioned as a system of psychiatric classification in its own right. Instead, in the near term, RDoC and DSM-ICD are expected to coexist. Nevertheless, the RDoC is intended to provide scaffolding for a large-scale research program that will ultimately yield an alternative to DSM-ICD (MacDonald & Krueger, 2013). .

RDoC adopts no a priori stance on what form a new model of classification will eventually take (Insel, 2014). In the words of the current NIMH director and his colleagues, RDoC is “a vision for the future” (Insel et al., 2010, p. 749) rather than a fleshed out proposal. As of this writing, RDoC is largely fluid in its mission. Such flexibility is probably justifiable in the early phases of scientific investigation, in which hypothesis generation (“the context of discovery”) should often take precedence over hypothesis testing (“the context of justification”; Kell & Oliver, 2003).

Still, the RDoC research program is not entirely open-ended. RDoC provides researchers with an explicit rubric for guiding investigations. As can be seen in Figure 1, RDoC proposes that research efforts be conducted within a two dimensional matrix (Morris & Cuthbert, 2012; Weinberger & Goldberg, 2013). This matrix took shape following a series of workshop meetings held over an 18 month period that involved panels of experts in different domains of neural circuitry. On the horizontal axis are seven units of analysis organized roughly from more to less “basic”: genes, molecules, cells, circuits, physiology, behavior, and self-reports (the matrix also includes a column for “paradigms,” allowing investigators to indicate which tasks are useful for the research question at hand). On the vertical axis are five broad domains/constructs that correspond to brain-based circuits deemed relevant to psychopathology: negative valence systems (e.g., threat, loss), positive valence systems (e.g., approach motivation, responsiveness to reward); cognitive systems (e.g., attention, working memory), social processes (e.g., theory of mind, dominance), and arousal/regulatory systems.

Figure 1.

The Provisional Matrix for the Research Domain Criteria (RDoC)

RDoC rests on several assumptions, four of which are especially crucial (Cuthbert & Insel, 2013). First, RDoC is explicitly transdiagnostic in seeking markers of dysfunctional psychobiological circuitry that transcend multiple traditional disorder categories; it posits that many superficially different conditions are underpinned by similar psychobiological processes. Second, RDoC is translational in emphasis, encouraging researchers to apply the basic science of brain systems and behavior to an understanding of mental disorders. Third, RDoC adopts a dimensional framework in light of evidence that the activity of most brain circuits, such as reward and threat systems, is continuously distributed, with few or no clear-cut boundaries demarcating normality from abnormality. Fourth, RDoC strives to accord roughly equal weight to different levels of analysis, including the biological and behavioral (Cuthbert & Insel, 2013).

Consistent with the view of science as a self-correcting enterprise, RDoC is conceptualized as provisional and open to revision in light of new scientific data. As a consequence, novel brain-based constructs may be added to the matrix with the emergence of new neuroscience evidence.

The Scientific Promise of RDoC

As of this writing, there is considerable excitement in many quarters regarding RDoC’s scientific potential, and for good reason. RDoC has already begun to loosen the longstanding grip of the DSM-ICD system over research and grant funding. This hegemony has stifled investigators’ capacity to explore scientifically promising alternatives to the status quo model of psychiatric classification (see also Berenbaum, 2013; Harkness & Lilienfeld, 2013, and Markon, 2013). Moreover, RDoC’s adoption of a dimensional approach to psychopathology as a starting assumption accords with the bulk of the evidence derived from taxometric studies, which suggests that most conditions are underpinned by dimensions rather than by taxa, that is, natural categories (Haslam et al., 2012). In this respect, RDoC is more consistent with data on the latent structure of mental disorders than is the DSM. At the same time, RDoC allows for the possibility of threshold effects (“tipping points”) or other categorical effects (Cuthbert & Insel, 2013), whereby certain psychopathological phenomena differ qualitatively rather than quantitatively from normality.

Furthermore, RDoC promises to capitalize on the burgeoning corpus of knowledge concerning affective and cognitive neuroscience by applying it to the classification of mental disorders. In this respect, it may ultimately allow for closer linkages between assessment and diagnosis, on the one hand, and treatment and prevention, on the other. More ambitiously, RDoC may eventually shift psychiatry and clinical psychology closer to the goal of precision medicine (Insel, 2014), or at least the more modest goal of stratified medicine, in which interventions are tailored to individuals within certain well-defined subgroups defined by laboratory-based profiles (Kapur et al., 2012). With the enormous recent influx of research attention and grant funding to RDoC, there is renewed hope of uncovering laboratory-based assays with high levels of sensitivity and specificity for the proximal pathological processes that contribute to psychopathological signs and symptoms.

In all of these respects, RDoC appears to be a valuable alternative to the current approach to classification, especially given that the DSM-ICD’s scientific yield, which has been substantial despite its noteworthy defects (Regier, Narrow, Kuhl, & Kupfer, 2009), appears to be reaching an asymptote. We regard RDoC as holding out the promise of generating a competing, and ideally more valid, system of classification relative to that of DSM-ICD. At the same time, RDoC confronts a number of challenges, many of which have received short shrift in ongoing discussions (but see Berenbaum, 2012; Lilienfeld, 2014b; Shankman & Gorka, 2015).

In the following section, we examine three overarching sets of challenges to RDoC: (1) conceptual, (2) methodological, and (3) practical/logistical (see also Lilienfeld, 2014b). Although there is overlap among these three sets of challenges, we differentiate them for expository purposes. Given that we intend to be constructive, we devote considerably more space to RDoC’s potential weaknesses than to its potential strengths. Nevertheless, this differential space allotment should not be construed as implying that we view RDoC’s weaknesses as outweighing its strengths. To the contrary, we focus on these weaknesses in the hope that researchers and theorists, including RDoC’s architects, will pay them greater heed and thereby increase the likelihood that RDoC will bear scientific fruit. Because we do not perceive any of these challenges as insurmountable, we hope that our delineation of them plays a role in shaping the future direction of RDoC.

RDoC: Conceptual Challenges

Falsifiability

The philosopher of science Sir Karl Popper (1959) argued that for a theoretical model to be scientific, it must be capable in principle of being proven wrong. Although few contemporary philosophers of science accept Popper’s premise that falsifiability is the lone demarcation criterion separating science from nonscience (Pigliucci & Boudry, 2013), many concur that this feature is one key indicator of a model’s scientific status. Moreover, Popper maintained that science ideally progresses by means of “severe” or “risky” tests (see also Meehl, 1978), those that place theories at grave theoretical risk.

In the case of RDoC, it is not evident what findings would suggest that it is not mapping well onto the state of nature, or at least falling considerably short of its avowed goals. Nor is it entirely clear what would constitute “negative” findings – results that would indicate that RDoC is a flawed endeavor that is not leading to a feasible alternative model of classification. In our view, it is incumbent on RDoC’s developers to lay out a set of provisional but explicit benchmarks by which its progress, or lack of progress, can be gauged.

In a related vein, it will be critical to address the question of whether RDoC, especially once it is in better-developed form, is superior to the DSM-ICD model in predicting theoretically and pragmatically relevant external criteria, such as natural history and treatment response. Based on our conversations with current and would-be RDoC grantees, it is our impression that researchers who are submitting proposals within the RDoC framework are being discouraged from pitting RDoC-relevant markers against DSM diagnoses in their capacity to predict such criteria. Such advice would be ill-conceived, as it could hamper long-term efforts to ascertain whether RDoC is performing as well as, or ideally better than, the extant system.

A further issue that bears on the falsifiability of RDoC concerns the distinction between convergent and discriminant validity (Campbell & Fiske, 1959; Cole, 1987). As of this writing, the lion’s share of RDoC research focuses on convergent validity, viz., demonstrating that presumed markers are predictive of their hypothesized RDoC domains (Shankman & Gorka, 2015). For example, it is useful to demonstrate that excessive levels of fear-potentiated startle are related to intrusive memories and fear-related memories associated with a traumatic event (Norholm et al., 2014). This finding suggests that fear-potentiated startle is a valid indicator of the RDoC Negative Valence domain, especially its acute threat (fear) subconstruct. Nevertheless, moving forward, it will be at least equally crucial to demonstrate that putative markers are not associated, or at least less associated with, non-hypothesized RDoC domains. For example, it will be important to show that fear-potentiated startle is largely unassociated with markers of the RDoC Arousal/modulatory domain, which should theoretically be largely independent of fear, although it may be predictive of levels of baseline startle.

Potential overemphasis on biological “dysfunction” at the level of an individual

RDoC’s posits that mental disorders are “disorders of brain circuits” (Insel et al., 2013, p. 749). As a corollary, RDoC presumes that “the tools of clinical neuroscience…can be used to identify dysfunction in neural circuits” (Morris & Cuthbert, 2012, p. 33). At some level, this assumption is merely a truism given that all psychological disorders, and all psychological phenomena for that matter, must be mediated by the brain and the remainder of the central nervous system (Kendler, 2005). Still, researchers who adopt the RDoC framework must be careful not to confuse biological mediation with biological etiology.

In the phraseology of Graham (2013), all mental disorders are in the brain, not all are necessarily of the brain. That is, at least some conditions classically regarded as mental disorders may stem from neural systems reacting more or less normally to harsh or extreme environmental input. Such reactions may be acute–such as rapid, over-generalized fear responses to trauma–or gradual, such as the slow accumulation of glucocorticoid resistance and chronic inflammation following prolonged periods of life stress that may result in structure damage to key brain regions involved in regulation of mood (Macqueen & Frodl, 2010; Treadway et al., 2015). Similar phenomena are observed in other areas of medicine; hypothermia reflects the activity of the body reacting normally to extreme environmental input, and some forms of cardiovascular disease may be mediated in part by normal biological responses to a chronically unhealthy lifestyle. In other cases, biological mediation may be more akin to a lesion of some kind, in which there is a genuine loss of function due to an environmental insult, genetic predisposition, or their interaction. Many clinical neuroscience studies neglect to specify these different classes of hypothesized mechanisms clearly, sufficing to label observed differences between patient and healthy groups simply as “deficits”, “dysfunction” or “alterations.” A limitation of the RDoC project as it is currently implemented is that it makes few demands for better delineation of mechanisms at this level.

Although RDoC does not conflate biological mediation with etiology, it may inadvertently foster this error by placing considerably less emphasis on psychosocial than on biological variables (Hershenberg & Goldfried, 2015; Lilienfeld, 2014b). The RDoC matrix focuses almost exclusively on intra-individual variables, with little or no explicit coverage of extra-individual variables, such as the social or cultural context (Berenbaum, 2013; Shankman & Gorka, 2015; Whooley & Horwitz, 2014). This omission is significant given that the phenotypic expression of biological vulnerabilities may often be constrained by sociocultural factors. For example, religious beliefs, as well as regional differences in the pricing and availability of alcohol, are associated with – and probably causally linked to - risk for alcohol use disorder (Kendler, 2012). Hence, even individuals with a potent genetic propensity toward alcohol use disorder may display low rates of this condition if raised in a socially traditional environment.

Furthermore, five of the seven RDoC units of analysis focus explicitly on biological indicators, raising concerns that biological levels of analysis may receive undue attention by investigators (Berenbuam, 2013). Although several RDoC publications (e.g., Morris & Cuthbert, 2012) have acknowledged the importance of psychosocial variables and developmental considerations in the RDoC program, these processes are not explicitly represented in the matrix, an omission that RDoC may wish to remedy.

Fueling this concern is the fact that some researchers appear to have assumed erroneously that indicators drawn from one level of analysis (e.g., physiological, behavioral) are necessarily best suited for detecting abnormalities at that level. For example, in response to Berenbaum’s (2013) concern that RDoC underemphasizes the role of beliefs, emotions, and other potential “emergent” phenomena that are not readily reducible strictly to neural events (Franklin et al., 2014, O’Connor, 1994; Paris, 2013), Cuthbert and Kosak (2013) wrote that “Berenbaum is right in supposing that research that relies exclusively on self-report data would fall outside of the RDoC approach” (p. 933).

The reasons for this a priori exclusion are unclear. Such decisions should be adjudicated by data, not by fiat. There is no inherent reason why self-report measures, which can readily capitalize on aggregation across indicators of behavior, cognition, and emotion across diverse situations, cannot provide equally – or more – construct-valid measures of biological systems relative to biological markers of these systems. For example, Patrick et al. (2013) reported that scores on self-report measures of an externalizing propensity and aggression were better markers of a latent disinhibition dimension than were P300 responses, an event-related potential indicator of response to novelty. Hence, RDoC must remain open to the possibility that self-report measures alone may in some cases be the optimal indicators of relevant psychobiological systems. Put somewhat differently, RDoC should be guided exclusively by data, and should adopt no a priori stance on which measures or sets of measures will prove most valid for detecting individual differences in the behavioral manifestations of neural circuitry.

Biological predispositions versus their behavioral manifestations

A potentially crucial distinction that has received little attention in the RDoC literature is that between biological predispositions to psychopathology and their behavioral manifestations (Lilienfeld, 2014b). In this respect, the distinction between basic tendencies and characteristic adaptations in the personality literature provides a useful organizing framework (McCrae & Costa.1995; see also Harkness & Lilienfeld, 1997). Basic tendencies are personality traits, such as negative emotionality (NE), whereas characteristic adaptations are the behavioral expressions of these traits, such as an anxiety disorder. Wakefield’s (1992) influential harmful dysfunction framework, which posits that the definition of mental disorder is a conjunction of (a) a failure in, or breakdown of, a naturally selected psychological system (dysfunction) and (b) impairment (harm) is also broadly consistent with this distinction (but see Lilienfeld & Marino, 1998, for a critique of aspects of Wakefield’s framework). This model proposes that the presence of biological dysfunction alone is not sufficient for psychopathology; this dysfunction must also be manifested in social harm.

The distinction between basic tendencies and characteristic adaptations underscores the point that certain adaptations to personality traits may be largely adaptive, whereas others may be largely maladaptive. For instance, an individual with high levels of NE may manifest this predisposition in an anxiety disorder; alternatively, she may manifest it in artistic productivity, which is associated with high levels of NE (Sheldon, 1994). As another example, the mean sensation seeking scores of prisoners are essentially indistinguishable from those of firefighters (Zuckerman, 1994), suggesting that sensation seeking can be manifested in either (a) socially and personally destructive outlets (e.g., crime, substance abuse) or (b) socially and personally constructive outlets (e.g., firefighting, law enforcement) depending on psychosocial factors and

Data on discordant monozygotic (MZ) twins among individuals with r mental disorders afford another potential illustration of multifinality. For example, between 35% and 59% of the co-twins (Carndo & Gottesman, 2000) of MZ twins with schizophrenia do not develop the disorder. Some of this discordance may derive from epigenetic differences within MZ pairs, in turn perhaps stemming partly from differential twin exposure to environmental influences (Dempster et al., 2011; Kato et al., 2005; Petronis et al., 2003). More broadly, the literature on MZ twin discordance in mental disorders raises the possibility that individuals with similar biological liabilities, such as those reflecting a predisposition to schizophrenia, display this risk in markedly different exophenotypes, only some of which are associated with psychopathology.

The principles of (a) basic tendencies versus characteristic adaptations and (b) multifinality highlight the point that individuals with similar biological predispositions toward psychopathology may manifest these predispositions in different ways, in part as a consequence of developmental and psychosocial factors. If so, RDoC may be insufficient as a model for mental disorder, as it may often be unable to distinguish physiological risk factors for psychopathology from psychopathology per se (see also Wakefield, 2014). If so, RDoC, at least in its present form, may be better suited as a model of the predispositions toward mental illness than of mental illness itself.

RDoC: Methodological Challenges

Neglect of Measurement Error

Consistent with the RDoC’s mission, several authors have argued that psychiatric diagnosis should transition to a laboratory-based approach (Kihlstrom, 2002; see Widiger & Clark, 2000, for a review) and thereby bring psychiatry in line with the rest of medicine (Nemeroff, Kilts, & Berns, 1999). Nevertheless, Kilhstrom’s (2002) vision, echoed by RDoC, may be considerably more challenging to achieve than is commonly appreciated.

Laboratory measures are typically associated with largely unappreciated psychometric weaknesses. As Epstein (1980) noted, psychologists have long granted such measures an undeserved scientific cachet, often “giving them a pass” with respect to psychometric requirements (see also Tryon, 1973). Laboratory measures of psychological constructs frequently display low levels of temporal and cross-situational consistency, largely because they contain substantial elements of “situational uniqueness.” Performance on such measures can be affected by a plethora of contextual and situational factors, including the mood and alertness of the participant, time of day, experimental instructions, nature of the laboratory setting, perceived attitude of the experimenter, demand characteristics, and idiosyncrasies of the laboratory paradigm itself (Kendler & Neale, 2010).

With respect to the lattermost source of variance, Mischel (1968) observed that even seemingly trivial changes in experimental paradigms can lead to dramatic changes in measures’ intercorrelations and correlations with other measures. Ironically, Kidd, Palmeri, and Aslin (2013) demonstrated this point using the very paradigm that Mischel (1974) made famous: the marshmallow test of delay of gratification in children. They showed that a seemingly trivial alteration in the experimental set-up – namely, whether an adult who had promised to bring a set of attractive art supplies to the child immediately prior to the task actually did so – resulted in massive changes in outcomes. Specifically, children exposed to the “reliable” adult waited an average of 12 minutes, whereas children exposed to the “unreliable” adult waited an average of only 2 minutes. Kidd and colleagues interpreted this result as suggesting that children who encounter unpredictable environments are less likely than other children to delay gratification, as they have ample reason to doubt whether expected rewards will materialize.

Block (1977; see also Tellegen, 1991) similarly noted that T data (test data), that is, data derived from isolated laboratory indicators, are more unreliable and erratic in their relations with (a) each other and (b) other measures compared with S data (self-report data) and R data (rating data). Although S and R data possess their own psychometric limitations, these data are typically aggregated across multiple diverse situations. Such aggregation is often accomplished by summing items within scales, or in more sophisticated analyses, by creating latent variables using such techniques as structural equation modeling. Such aggregation minimizes situational error and yields more reliable and construct-valid composites of behavior across situations (Epstein, 1980; Rushton, Brainerd, & Pressley, 1983). In contrast, T data are rarely aggregated across situations, at best being combined only across multiple trials of the same measure.

These problems may be especially acute in the domain of neuroimaging. Few investigators have examined the test-retest reliability of measures of functional magnetic resonance imaging (fMRI; Bennett & Miller, 2010), even though this form of reliability is a basic expectation of measures in other psychological domains. In an analysis of 63 studies, Bennett and Miller (2010) found that the test-retest reliability of fMRI measures was typically modest, with intraclass correlations (ICCs) averaging 0.50 (see also Vul, Harris, Wienkielman, & Pashler, 2009). Furthermore, only 29% of activated voxels that were statistically significant in one study were significant in a second study. Although test-retest reliabilities for fMRI data were higher with briefer intervals, even back-to-back scans (taken within one hour) exhibited an average cluster overlap of only 33%.

At the same time, because some of the neural processes examined in these studies may been influenced substantially by state variables, these modest test-retest values may reflect inherent short-term fluctuations in the neural processes themselves rather than measurement error. Scanning is can be an unfamiliar experience for many participants, and initial anxiety or discomfort in the scanner may change over repeated exposures. Moreover, repeated exposures to cognitive tests will invariably lead to learning, even if only at the level of greater familiarity and fluency. It would therefore be surprising if these changes were not reflected in differences in neural responses. One recent study used a relatively large sample to examine how activity in the ventral striatum (VS) during the feedback phase of a rewarded instrumental conditioning task changed over time as compared with activity during the anticipation phase (Chase et al 2015). As would be predicted by temporal-discounting models of reinforcement learning (Schultz et al 1997), VS responses to better-than-expected outcomes during the first scanning session were robust, but were nearly absent during the second session. Conversely, ventral striatal responses to anticipation were absent during the first session, but significant during the second. Moreover the magnitude of feedback related VS activity at session predicted the magnitude of anticipatory VS activity at session 2. This outcome-to-cue transfer in VS activity patterns suggests a change due to learning, but also contributes to very low interclass-correlations when simply comparing the same condition across testing sessions.

These effects may also vary across different types of tasks and conditions. For example, Sauder, Hajcak, Angstadt, and Phan (2013) reported that the reliability of fMRI measures of amygdala activation was adequate in response to fearful faces (ICCs ranged from 0.32 to 0.43), but inadequate in response to angry faces (ICCs ranged from −0.24 to 0.11). In contrast, structural MRI measures appear to have considerably higher test-retest reliability (Kendler & Neale, 2010). For example, the stability of measures of cortical thickness as assessed by structural MRI is on the order of r=0.95 (Wonderlick et al., 2009).

Even the fMRI research center at which the study is conducted accounts for approximately 8 percent of the variance in fMRI blood oxygen-level dependent (BOLD) signal results, suggesting that the laboratory itself is a potential source of error in analyses (Costafreda et al., 2007). Another investigation revealed that the median ICC of fMRI findings across different imaging centers that contained identical hardware set-ups was only 0.22 (Friedman et al., 2008; see also Bennett & Miller, 2010).

All of these psychometric limitations may impede the replicability of functional imaging findings unless addressed analytically. Adding to these concerns are findings that the average statistical power of functional brain imaging studies is only about .20, which is considerably lower than in most other domains of psychological and psychiatric research (Button et al., 2013). Low power not only increases the chances of Type II errors (false negative results), but also boosts the likelihood of overestimating the effect sizes of statistically significant findings, a phenomenon known as the winner’s curse (Munafo et al., in press). Replicability concerns are not limited to functional brain imaging studies. Using a fresh sample of college students as participants, Boekel, Wagenmakers, Belay, Verhagen, Brown, and Forstmann (2014) attempted to replicate 17 published findings on the relation between structural brain imaging findings and psychological findings derived from self-reported data and other measures (e.g., a previously reported positive correlation between amygdala gray matter and number of Facebook friends). Using Bayesian methods, they found minimal support for any of the previous 17 findings.

The Limitations of Endophenotypes

Endophenotypes comprise a core intellectual progenitor for the RDoC’s dimensional conceptualization of brain circuit dysfunction, and many RDoC studies have borrowed heavily from endophenotype designs. Because they are presumed to be more proximal to genes compared with exophenotypes, endophenotypes should ideally provide purer and more construct-valid indicators of genetically influenced psychobiological systems. Moreover, because many endophenotypes cut across traditional disorder categories, endophenotypes accord well with RDoC’s transdiagnostic approach and emphasis on identifying etiological processes that contribute to multiple DSM conditions (Miller & Rockstroh, 2013). Nevertheless, RDoC is broader than the endophenotype approach, as it does not mandate that candidate biological markers be heritable.

Despite high initial expectations and provisional successes for certain conditions, such as schizophrenia (Cannon & Keller, 2006; Miller & Rockstroh, 2013), the endophenotypes identified to date have not necessarily proven to be more genetically informative than traditional exophenotypes, such as DSM criteria. Flint and Munafo (2007) examined this issue in meta-analyses of studies of catechol O-methyl transferase (COMT), an enzyme that metabolizes dopamine (among other neurotransmitters), and schizophrenia, a disorder associated with dopamine overactivity in mesolimbic brain regions. They tested whether the COMT genotype displayed higher effect sizes with candidate neuropsychological and psychophysiological endophenotypes of schizophrenia, such as performance on the Wisconsin Card Sorting Task, the N-Back Task, and P300 amplitude and latency, than with DSM schizophrenia itself. Flint and Munafo found no evidence that the ostensible endophenotypes were more highly related to the COMT genotype than was schizophrenia. Their findings suggest that investigators should not assume that candidate endophenotypes will necessarily yield higher effect sizes than do exophenotypes in genetic studies (but see Tan, Callicott, & Weinberger, 2008, for a more sanguine view of the status of endophenotypes for schizophrenia).

In contrast, Jonas and Markon (2014) reported more encouraging results. They examined the relation between (a) three polymorphisms in the dopamine and serotonin systems that are potentially relevant to impulsivity and (b) diagnostic (e.g., measures of attention-deficit hyperactivity disorder), trait (e.g., self-report measures of impulsivity), neuropsychological (e.g., continuous performance tasks), and neurobiological (e.g., functional imaging indices of right inferior prefrontal activity) phenotypes relevant to impulsivity. Neurobiological measures were the most highly associated with the genetic markers, with trait and neuropsychological measures roughly tied for a distant second, and diagnostic measures last. These analyses suggest that neurobiological measures, such as functional brain imaging indices, may be promising endophenotypes of impulsivity. Still, as the authors observed, neurobiological studies were characterized by considerably lower statistical power than studies from other domains, raising the possibility that the effect sizes of the former studies were inflated.

As noted earlier, one of two models of endophenotypes posits that they are intermediate phenotypes, operating as mediators between genes and exophenotypes. Nevertheless, the evidence that candidate endophenotypes mediate the relation between genes and behavioral phenotypes is slim. In in a twin sample, Kendler, Neale, Kessler, Heath, and Eaves (1993) found that although neuroticism was associated with elevated rates of major depression, it did not mediate the association between genetic risk and major depression (see also Kendler & Neale, 2010). Waldman (2005) reported inconsistent findings concerning whether scores on the Trail Making Test mediate relations between dopamine genes and attention-deficit hyperactivity disorder (ADHD), with partial mediation for Trails A but no significant mediation for Trails B.

Such mediation should not be presumed, as certain putative endophenotypes may lie causally downstream of the exophenotypes with which they are associated (Kendler & Neale, 2010). For example, P300 amplitude appears to be a valid endophenotype for a broad predisposition toward externalizing behavior and disinhibition (Patrick et al., 2009). Nevertheless, P300 amplitude is exquisitely sensitive to attention (Polich, 2012). Hence, this marker may be a consequence, not an antecedent, of the attentional and motivational deficits associated with externalizing disorders, such antisocial personality and substance use disorders, which display high levels of co-occurrence and covariation with ADHD (Lilienfeld & Waldman, 1990; Torgersen, Gjervan, & Rasmussen, 2006).

One crucial assumption guiding the search for endophenotypes is that the biological indicators in question are trait rather than state markers, as a measure of a biological vulnerability would be expected to be stable over time. Again, the trait status of such markers must be demonstrated empirically rather than assumed (Stoyanov & Kandilorova, 2014). For example, motion discrimination appears to be impaired both in patients with schizophrenia and in their nonaffected relatives, suggesting that it may be a useful trait marker of the disorder. In contrast, motion integration appears to be impaired in patients with schizophrenia but not in their nonffected relatives, suggesting that is influenced by state variables (Chen, Bidwell, & Norton, 2006). One major challenge for RDoC researchers will be to demonstrate that endophenotypes, as well as other ostensible vulnerability markers of RDoC domains are present even between disorder episodes or exacerbations.

RDoC: Logistical/Pragmatic Challenges

The dangers of premature reification of the RDoC matrix

As noted earlier, the RDoC matrix is intended to be provisional. It comprises cells directing investigators to well-supported neural circuits linked to psychopathology while leaving open the possibility of modifying extant cells or adding new ones on the basis of new evidence (Cuthbert & Insel, 2013). In this regard, this matrix appears to represent a reasonable compromise between excessive open-endedness and excessive prescriptiveness.

At the same time, there is a danger of premature reification of the matrix and of the RDoC endeavor more broadly; it would indeed be unfortunate if the march to freedom from the DSM’s “epistemic prison” (Hyman, 2010, p. 157) led merely to a padded cell in the matrix penitentiary. A number of scholars have cautioned against such reification in the frequent (mis)interpretation of DSM categories as “settled truths” (Kendler, 2014), and RDoC must be careful to avoid the same error. In fairness, the RDoC framers have repeatedly described their intentions for the matrix to be a “working document”. Good intentions, however, may afford inadequate protection against ossification, which can easily insinuate itself through the mundane, bureaucratic activities required by any large-scale administrative effort. For instance, several recent requests for applications (RFAs) from NIMH have noted that “applications must focus on at least one of the constructs that have been defined in these RDoC workshops” (NIMH RFA-MH-14-050; see Shankman & Gorka, 2015). Such wording appears to vitiate RDoC’s avowed goal of being self-correcting, as it will be difficult or impossible to ascertain whether to add novel cells to the matrix unless these cells are explicitly investigated. Social cognition research suggests that once mental categories are set in place, they can quickly become implacable (McCrae & Bodenhausen, 2000). To avoid the error of reification, RDoC should encourage researchers to examine psychobiological constructs that have the potential to bridge, transcend or challenge current matrix boundaries, as well as articulate an explicit process of matrix evaluation and revision. Additionally, future RFAs should offer as much consideration to proposals that seek to explicitly challenge matrix boundaries as they do to proposals that operate within them.

Although the RDoC matrix was thoughtfully constructed to be broad and flexible, a number of fruitful hypotheses that may be challenging to capture within its framework. For example, a small body of work has suggested that emotional numbing and alexithymia may be core facets of reduced approach related-behaviors, but only in the context of response to a traumatic experience, as in PTSD (Kashdan et al 2006). It is not entirely clear, however, how this type of etiology-by-symptom interaction might be examined within the RDoC framework.

Implications for grant funding in non-RDoC domains

The RDoC initiative developed out of an explicit desire to support translational neuroscience in mental health research. It is therefore perhaps unsurprising that the RDoC has hastened the already significant shift in NIMH extramural funding priorities away from psycho-social research, including research on psychotherapy. Although we welcome the emphasis on translational neuroscience, there is a legitimate concern that this approach may burn the candle too far at both ends. On the one hand, as a consequence of RDoC basic neuroscience efforts to understand circuit function more generally may be at risk for being underfunded, raising the concern of insufficient knowledge available for translation. On the other hand, the promise of neuroscience-guided treatment and diagnosis is widely acknowledged to be on the distant horizon (e.g., Frances & Widiger, 2012; Paulus, 2015), whereas research devoted to improvements to empirically-supported treatments for psychopathology could alleviate the suffering of thousands in the near-term. It would therefore be beneficial for RDoC to co-exist with research programs focused on (a) developing basic science knowledge without necessarily requiring immediate demonstration of clear translatability and (b) clinical outcome and process research that can yield important short-term breakthroughs in treatment.

Translating RDoC findings into a feasible classification system

One crucial challenge for RDoC will be to delineate how the large and diverse corpus of findings emanating from its investigations will be meaningfully synthesized into the rudiments of an alternative model of psychiatric classification. The RDoC matrix is a heuristic blueprint to guide research, but it is not a classification system in and of itself, nor is it intended to be. A key unanswered question, then, is how RDoC findings will be “translated” into a usable system that can ultimately guide clinical practice. Although it may too early to address this question, it will soon be incumbent on RDoC architects to sketch out an overarching plan for how RDoC research will be used to inform classification and diagnosis. In the absence of such a rubric, it may be difficult to convert the enormous wealth of data yielded by RDoC investigations into a concrete model that can supplement, or perhaps eventually substitute for, the DSM-ICD.

The DSM-ICD and RDoC: Quo Vadis?

The DSMs, especially DSM-III and its progeny, were noteworthy achievements in psychiatric classification, and they brought considerable order and diagnostic reliability to previously disorganized territory. In certain respects, the DSM-ICD model has served us well (cf., Greenberg, 2013), as it has greatly facilitated epidemiological and etiological research and contributed to the development of efficacious interventions that have alleviated the suffering of countless individuals. Moreover, by mapping onto signs and symptoms, the DSM does a serviceable and face-valid job of capturing the essence of individuals’ distress and impairment. It is difficult to imagine a comprehensive system of psychiatric classification bereft of any reference to the subjective and behavioral manifestations of people’s psychological suffering.

At the same time, it has become increasingly evident that many central features of the DSM-ICD model do not map adequately onto the state of nature (Sanislow et al., 2010). The high levels of covariation among putatively distinct categories, the large number of intermediate cases, and the substantial phenotypic and etiological heterogeneity of numerous diagnostic categories, among other vexing anomalies, suggest that something is deeply awry with at least some core presuppositions underpinning the neo-Kraepelian model of psychiatric classification. Moreover, these problems have proven stubbornly resistant to repeated efforts at amelioration across multiple DSM and ICD revisions. The shortcomings of the DSM-ICD edifice therefore appear to reflect an inherent deficiency in its architectural floorplan that cannot be fixed merely by adjusting some of its walls.

The RDoC initiative emerged in the new millennium to address these mounting anomalies (Insel, 2014), and it appears to be a valuable effort to ground psychopathology in well-supported biological systems that carry important implications for adaptation and maladaptation (Lilienfeld, 2014b). In this respect, RDoC has the potential to map more closely than the DSM-ICD onto psychobiological reality. Moreover, even its critics acknowledge that RDoC has already exerted one important salutary effect: loosening the stranglehold of the DSM over research and grant funding. Such hegemony has impeded the investigation of fruitful alternatives to classification and etiology, and we experience few qualms in bidding it a greatly belated adieu

In this respect, RDoC should take heed of the lessons of the past. One historical feature shared by DSM-ICD and RDoC is that neither was intended to become a fixed system in terms of guiding research priorities. Yet, the DSM system eventually acquired such sovereignty over psychiatric and psychological research that it became difficult for investigators to examine clinical problems that fell outside of traditional disorder boundaries (Hyman, 2010). RDoC must make concerted efforts to avoid the same error of reification.

As we have also noted, RDoC confronts a number of conceptual, methodological, and logistical/pragmatic challenges, none of which appear to be insuperable (Lilienfeld, 2014b). At the same time, many of the challenges appear to have received insufficient discussion in RDoC documents, and will require careful consideration for RDoC to realize its considerable scientific potential. In Table 1, we offer a baker’s dozen of recommendations for addressing these challenges in the coming years of RDoC research.

Table 1.

A Baker’s Dozen Recommendations for RDoC

(1) Lay out explicit benchmarks for ascertaining what findings, or patterns of findings, could falsify the RDoC research framework or at least suggest that it is not making adequate scientific progress

(2) Examine how RDoC is performing relative to DSM-ICD for statistically predicting important external criteria, such as response to treatment and natural history

(3) Explicate alternative models for dysfunction in psychobiological systems (e.g.., lesion, hyperactivity or hypoactivity in the functioning of these systems, a network model in which there are bidirectional causal relations among diagnostic signs and symptoms)

(4) Accord adequate attention to environmental (e.g., social and cultural context) and developmental (e.g., the effects of developmental timing) influences, and incorporate such influences explicitly into the RDoC matrix to encourage their investigation

(5) Adopt no a priori assumptions regarding what measures will be optimal or detecting individual differences in neural circuitry relevant to psychopathology, and be guided exclusively by data in this regard

(6) Accord adequate attention to the role of measurement error inherent to laboratory measures (“T data”) and capitalize on the power of statistical aggregation across indices.

(7) When conducting research on endophenotypes, test statistical models of pleiotropy and mediation, and do not presume that such phenotypes are necessarily more heritable or straightforward in their genetic architecture compared with exophenotypes

(8) Do not assume that endophenotypes or other candidate biomarkers are more trait-like than state-like; the assertion that such indicators are stable over time must be demonstrated empirically

(9) Remain cognizant of the possibility that psychobiological predispositions can be manifested in a host of diverse behavioral manifestations

(10) Avoid premature reification of the RDoC matrix and remain open to research that examines cells not explicitly represented in the current matrix

(11) Develop broad guidelines for beginning to translate the growing body of RDoC findings into a usable system of psychiatric classification

(12) Seek ways in which RDoC can co-exist with both (a) basic research on neural circuitry and (b) applied research on empirically supported treatments tied to DSM categories

(13) Explore the possibility of a combined system that incorporates both diagnostic sign/symptoms and psychobiological predispositions

One of the foremost challenges to RDoC’s effort to offer a viable alternative to DSM-ICD is the fact that psychobiological predispositions can be expressed in a host of diverse behavioral patterns, some of them largely healthy and others largely unhealthy (Harkness & Lilienfeld, 1997). In this regard, it may be unrealistic to expect RDoC to fully supplant a classification system, such as the DSM, that provides a remarkably rich compilation of the variegated ways in which human behavior, both subjective and observable, can break down. Signs and symptoms, their inevitable shortcomings notwithstanding, will always be necessary for a complete characterization of mental disorder.

In our view, the DSM-ICD will never provide a sufficient foundation for a comprehensive classification system, because psychiatric signs and symptoms, like fever, are inevitably nonspecific indicators of a host of psychobiological dysfunctions. Conversely, RDoC will similarly never be sufficient for a comprehensive classification system, because psychobiological dysfunctions can be manifested in a host of markedly diverse signs and symptoms as a function of innumerable moderating variables. As a consequence, a full characterization of psychopathology will require the DSM-ICD’s remarkably astute descriptive observations, informed by the best available basic research on neural circuitry relevant to psychopathology. If it attends carefully to the constructive challenges posed here, RDoC and the fruits of its labor hold the potential to complete the story that Emil Kraepelin, the St. Louis group, and Robert Spitzer started.