Skip to main content
NCBI home page
Wiley Open Access Collection logoLink to Wiley Open Access Collection
. 2018 Mar 25;35(4):346–352. doi: 10.1002/da.22741

Indicators to facilitate the early identification of patients with major depressive disorder in need of highly specialized care: A concept mapping study

F C W van Krugten 1,✉, M Goorden 1, A J L M van Balkom 2,3, J Spijker 4,5, W B F Brouwer 1, L Hakkaart‐van Roijen 1; on behalf of the Decision Tool Unipolar Depression Consortium†
PMCID: PMC6282616  PMID: 29575387

Abstract

Background

Early identification of the subgroup of patients with major depressive disorder (MDD) in need of highly specialized care could enhance personalized intervention. This, in turn, may reduce the number of treatment steps needed to achieve and sustain an adequate treatment response. The aim of this study was to identify patient‐related indicators that could facilitate the early identification of the subgroup of patients with MDD in need of highly specialized care.

Methods

Initial patient indicators were derived from a systematic review. Subsequently, a structured conceptualization methodology known as concept mapping was employed to complement the initial list of indicators by clinical expertise and develop a consensus‐based conceptual framework. Subject‐matter experts were invited to participate in the subsequent steps (brainstorming, sorting, and rating) of the concept mapping process. A final concept map solution was generated using nonmetric multidimensional scaling and agglomerative hierarchical cluster analyses.

Results

In total, 67 subject‐matter experts participated in the concept mapping process. The final concept map revealed the following 10 major clusters of indicators: 1‐depression severity, 2‐onset and (treatment) course, 3‐comorbid personality disorder, 4‐comorbid substance use disorder, 5‐other psychiatric comorbidity, 6‐somatic comorbidity, 7‐maladaptive coping, 8‐childhood trauma, 9‐social factors, and 10‐psychosocial dysfunction.

Conclusions

The study findings highlight the need for a comprehensive assessment of patient indicators in determining the need for highly specialized care, and suggest that the treatment allocation of patients with MDD to highly specialized mental healthcare settings should be guided by the assessment of clinical and nonclinical patient factors.

Keywords: depressive disorder, disease management, major, precision medicine, tertiary healthcare

1. INTRODUCTION

Timely selection of the best initial treatment for patients with major depressive disorder (MDD) is critical to the goal of improving remission rates (Simon & Perlis, 2010). The often applied stepped care approach in which patients indiscriminately receive brief and low‐intensity intervention at start of treatment and intensifying efforts in case of insufficient signs of recovery, may, however, prevent the accurate and timely selection of the best initial treatment. Although the stepped care approach is considered a resource efficient approach for patients who recover with minimal intervention (Meeuwissen, van der Feltz‐Cornelis, Christina M, van Marwijk, Rijnders, & Donker, 2008; Von Korff & Tiemens, 2000), the effectiveness of this approach is questionable in patients who need subsequent referral to highly specialized mental healthcare services. Secondary or even tertiary referral to highly specialized mental healthcare services delays the initiation of appropriate treatment, which, in turn, is associated with poor treatment outcomes in terms of relapse, recurrence, and chronicity (Hirschfeld et al., 1997; Keller, 1994; Meyers et al., 2002). An alternative to the stepped care approach is matched care. In this approach, patient management and initial treatment allocation is tailored to the individual patient needs (Gask & Khanna, 2011; Ridgway & Williams, 2011; Van Straten, Tiemens, Hakkaart, Nolen, & Donker, 2006). Successful application of this approach may reduce the number of treatment steps needed to achieve and sustain an adequate treatment response, benefit the quality of life of patients, and increase the cost‐effective use of resources.

A major problem in the application of matched care approach is the lack of clear individual patient indicators with which to match patients to the available treatment settings. In recent years, a wide array of individual patient factors has been examined to inform initial treatment selection in patients with MDD (Delgadillo, Moreea, & Lutz, 2016; Dunlop & Mayberg, 2014; Lener & Iosifescu, 2015; McGrath et al., 2013; Sotsky et al., 1991). Despite some progress, these have thus far not demonstrated their value in clinical practice and some approaches like neuroimaging are not feasible for use in daily clinical practice (Evans, Dougherty, Pollack, & Rauch, 2006). Information of individual patient factors collected as part of routine assessment procedures in the diagnostic phase after referral, however, has the potential to aid clinicians in the early identification of the patients with MDD in need of highly specialized care. The aim of this study, therefore, was to identify a range of clinical and nonclinical factors of patients with MDD in need of highly specialized care that could serve as input for the development of a decision support algorithm.

2. METHODS

Prior to the study period, a small working group was formed comprising 13 leading Dutch experts in the field of MDD from nine mental healthcare institutions. The workgroup included academically affiliated and community‐based practicing MDD specialists. This study progressed through two primary phases. First, a systematic review of the literature of the PubMed and the PsycINFO databases following PRISMA guidelines was conducted to serve as a scientific foundation. The aim of this systematic review was to comprehensively cover the factors associated with a broad range of unfavourable clinical outcomes in patients with MDD for which more intensive treatment is indicated. The systematic search of all databases yielded a total of 7,360 references, of which 16 were eligible for inclusion. Based on the included papers (n = 16), an initial list of 48 indicators of patients with a depression in need of highly specialized care was generated (see van Krugten et al., 2017 for details of this review). Subsequently, a structured conceptualization methodology known as concept mapping (Trochim, 1989) was employed to complement the initial list of indicators by clinical expertise and develop a consensus‐based conceptual framework. Concept mapping is a method that integrates a qualitative research design with quantitative analytic techniques to conceptualize a phenomenon of interest (Johnsen, Biegel, & Shafran, 2000; Trochim, 1989) and has been used in a wide variety of studies, including measurement development (Armstrong & Steffen, 2009; Conrad et al., 2011; Conrad, Iris, Ridings, Langley, & Anetzberger, 2011; Corcoran, 2005; Iris, DeBacker, Benner, Hammerman, & Ridings, 2012). In general, the concept mapping process involves the following five steps: (1) preparation, (2) brainstorming, (3) sorting and rating, (4) statistical analysis, and (5) interpretation (Kane & Trochim, 2007; Trochim, 1989). These steps are described below, along with details of how we implemented them in this study.

2.1. Step 1: Preparation

During the first step of concept mapping, a focal question was developed and relevant subject‐matter experts were selected. In collaboration with the small working group of experts, we developed a focal question for item elicitation. Our focal question was ‘'Which criteria distinguish depressive patients in need of a highly specialized mental healthcare treatment from patients in need of regular specialized mental healthcare treatment?'’. The focal question was developed to elicit a list of participants' ideas that were then analyzed for the study.

Working group members identified and selected subject‐matter experts from a broad range of disciplines. These experts were identified and selected based on their expertise in the assessment and/or treatment of patients with MDD or involvement in MDD research. In total, 184 national and international experts were invited to participate in the subsequent data collection activities. At the time of data collection, all participants were asked to sign an electronic consent form for participation and complete a brief demographic questionnaire. All data collection activities for this study were performed in English and Dutch in order to facilitate national and international subject‐matter expert participation.

2.2. Step 2: Brainstorming

In step 2, working group members distilled the original 48 indicators into a list with distinct statements by eliminating duplicate statements, editing statements for clarity, or combining similar statements. This process resulted in a list of 38 mutually exclusive indicators of patients with MDD in need of highly specialized care. Participants were then asked to individually review this list of indicators and engage in a brainstorming session to generate additional indicators. Brainstorming took place through a web‐based system specifically designed for concept mapping (Concept Systems® software, Incorporated, Ithaca, New York). The open‐ended focal question mentioned under Step 1 was used to elicit criteria from participants. In response to the focal question, participants were asked to generate as many criteria as possible and enter them into the system. The participants had 4 weeks to respond to our request. During this 4‐week period, they had the option of entering criteria in more than one session.

2.3. Step 3: Sorting and rating

Following procedures recommended by Trochim (Trochim, 1993), sorting and rating activities were performed as an individual activity via the aforementioned web‐based program. Participants were asked to sort the criteria into categories based on the principle of similarity, thereby building thematically related sets of items. Specifically, participants were instructed to group criteria in a way that ‘made sense to them’ and label their final groupings accordingly. The sole restrictions were as follows: (1) all criteria cannot be placed into a single category; (2) a criterion cannot be placed simultaneously into two separate categories; (3) categories named ‘Miscellaneous’ and ‘Other’ that group together dissimilar statements are not allowed; and (4) criteria cannot be sorted according to priority or value, such as ‘Important’, or ‘Hard To Do’.

After completing the sorting, participants were asked to rate each individual criterion on how important it was to distinguish between patients in need of highly specialized care from patients in need of specialized care. Responses were recorded on a 5‐point Likert scale ranging from 1 (not important at all) to 5 (extremely important).

2.4. Step 4: Statistical analysis

Concept Systems software was also used to analyze the data generated from the sorting and rating exercise. Three statistical procedures were sequentially performed. First, a nonmetric multidimensional scaling (MDS) analysis was carried out to plot the criteria and their cohesion on a two‐dimensional plane. The analysis yielded a so‐called “point map” on which the proximity of the points represents the frequency with which the criteria were sorted together by each of the individual participants. Points located closer to each other on the point map represent criteria sorted together most often, whereas points located further apart represent criteria sorted together less frequently. A stress value was calculated as part of the multidimensional scaling analysis to indicate how well the two‐dimensional configuration maps the original data. The stress value is an index of the goodness of fit of the MDS solution and ranges from 0 to 1, with lower values indicating a better fit. Subsequently, an agglomerative hierarchical cluster analysis using Ward's minimum variance method (Ward Jr, 1963) was carried out to partition the resulting MDS configuration into non‐overlapping clusters, thereby creating initial cluster maps. Mean importance ratings of the clusters were computed by averaging the average rating of each criterion in the clusters. Finally, paired t‐tests were carried out to compare the mean importance ratings of the various clusters. To adjust for multiple testing, a Bonferroni correction was used, dividing the conventional alpha of .05 by the number of independent tests.

2.5. Step 5: Interpretation

Since there is no objective standard or mathematical solution through which a final number of clusters can be selected (Shern, Trochim, & La Comb, 1995), working group members discussed the preliminary cluster solutions from the hierarchical cluster analyses to reach consensus on the optimal cluster number for answering the focal question. Following Kane & Trochim (Kane & Trochim, 2007), a range of cluster solutions was examined in a reverse stepwise cluster‐reduction process. In this process, two clusters merge (e.g., from 14 to 13 clusters) at each reverse step. Working group members worked backwards from 20 clusters and examined successively lower cluster solutions. At each level, a judgment was made about whether the merger made conceptual and interpretive sense until a cluster level was reached that yielded the fewest number of clusters but still retained the maximum amount of substantive information. In a digital survey, working group members were then asked to review the within‐cluster coherence of content and suggest criteria that could be moved from one cluster to another to increase conceptual clarity and assign cluster labels to the resulting clusters. Informed by the gathered working group input, the clusters were assigned final labels and some criteria were reallocated to a conceptually more appropriate cluster.

3. RESULTS

3.1. Expert participation

In total, 67 out of the 184 invited subject‐matter experts participated in one or more of the steps of the concept mapping process. The mean age of the experts was 50.42 years (SD = 10.93) and 41.54% (n = 27) were female. The mean years of work experience in the assessment and/or treatment of patients with MDD or involvement in MDD research was 23.31 years (SD = 11.30). The majority of the experts were psychiatrists (n = 44, 67.69%), followed by clinical psychologists (n = 12, 18.47%), clinical researchers (n = 4, 6.15%), psychotherapists (n = 4, 6.15%), and physicians (n = 1, 1.54%). There was equal representation of experts working in specialized mental healthcare settings and highly specialized mental healthcare settings (n = 34, 52.31% and n = 31, 47.69% respectively).

3.2. Concept mapping results

A total of 50 items were generated during the brainstorming stage and added to the initial list of 38 indicators derived from the systematic review, resulting in a list of 88 putatively relevant indicators of patients with MDD in need of highly specialized care. This list of 88 items was used in the subsequent sorting and rating steps of the concept mapping process.

Forty‐three (n = 43) experts sorted the 88 indicators into an average of 9.00 piles (SD = 3.04). The stress value of the MDS‐solution was .23, falling within the average range (.15–.35) of stress values typically attained in concept mapping studies (Trochim, 1993). The 10‐cluster concept map solution produced by the participant sorts and subsequent analysis is presented in Figure 1. This cluster solution provided the maximum number of interpretable clusters without losing distinctions between groups of indicators. The numbers on the map correspond to the indicators listed in Supporting Information Appendix A. Numbers closer together represent indicators that were more frequently sorted together than were indicators represented by points further apart. The more distance between numbers the less often they were sorted together (i.e., the less conceptually similar they were viewed by participants). Each cluster consists of indicators that were sorted together more frequently and contribute to an overarching conceptual domain. The shape and size of the clusters reflect the breadth or specificity of the clusters, with large clusters typically covering a broader, less well‐defined concept than smaller clusters. The 10 clusters were labeled as follows: (1) depression severity; (2) onset and (treatment) course; (3) comorbid personality disorder; (4) comorbid substance use disorder; (5) other psychiatric comorbidity; (6) somatic comorbidity; (7) maladaptive coping; (8) childhood trauma; (9) social factors; and (10) psychosocial dysfunction. The overarching conceptual domains, sample indicators per conceptual domain, mean cluster ratings, and cluster rankings are presented in Table 1. Mean importance ratings (i.e., ratings averaged across all indicators within a cluster) ranged between 2.53 and 4.42. On average, items in the depression severity cluster were rated most important to distinguish between patients in need of highly specialized care from patients in need of specialized care (M = 4.42), followed by items in the psychiatric comorbidity cluster (M = 4.18), and somatic comorbidity cluster (M = 3.95). No consistent significant differences were found between mean importance ratings of the clusters.

Figure 1.

Figure 1

Open in a new tab

Concept map of the main indicators of patients with MDD in need of highly specialized care (stress value = .23). Clusters represent the overarching conceptual domains of the 88 indicators of patients with MDD in need of highly specialized care. Labels were suggested by working group members and finalized by the project team. Numbers correspond to the indicators that were sorted into each category. Indicators that are closer together indicate higher degrees of similarity based on sorting

Table 1.

Conceptual domains, sample indicators, importance rating and ranking for the ten clusters

Importance
Cluster Sample indicatorsa Mean ratingb Ranking
1 Depression severity Greater depressive symptom severity
Psychotic symptoms
Current suicidal risk
Higher rates of melancholic features 		4.42 1
2 Onset and (treatment) course Younger age of onset
Longer duration of index depressive episode
More lifetime episodes
Lack of remission or partial remission after the previous depressive episode		 3.80 4
3 Comorbid personality disorder Higher axis II personality pathology score
Comorbid personality disorder		 3.68 6
4 Comorbid substance use disorder Alcohol abuse
Substance abuse		 2.86 9
5 Other psychiatric comorbidity A higher number of comorbid psychiatric disorders
Comorbidity with ADHD
Comorbidity with OCD
Comorbid (generalized) anxiety disorder		 4.18 2
6 Somatic comorbidity Greater levels of general medical comorbidity
Worse physical health function
A higher number of pain locations
Lower physical quality of life		 3.95 3
7 Maladaptive coping Disadaptive coping
High external locus of control
Less positive outcome expectancies		 2.96 8
8 Childhood trauma Higher prevalence of childhood trauma
Higher levels of trauma sequelae		 3.69 5
9 Social factors No social support
Lower monthly household income
Unemployment 		2.53 10
10 Psychosocial dysfunction Worse social functioning
Worse work function and social adjustment
More impaired daily function		 3.21 7
Open in a new tab

Notes. aIndicators for which there was the most consensus among participants regarding the categorization within the cluster.

bImportance was rated on a 5‐point Likert scale ranging from 1 to 5, with higher scores reflecting greater importance to distinguish between patients in need of highly specialized care and patients in need of specialized care.

4. DISCUSSION

The aim of this study was to identify patient‐related indicators that could facilitate the early identification of the subgroup of patients with MDD in need of highly specialized care. Drawing on clinical expertise and a literature review, a concept mapping approach was employed to develop a consensus‐based conceptual framework. Concept mapping is a mixed‐method participatory approach that facilitated the delineation of a shared understanding of clinical and nonclinical patient indicators that may justify referral to highly specialized mental healthcare programs. In total, 88 putatively relevant indicators of patients with MDD in need of highly specialized care were generated and categorized into the following 10 overarching conceptual domains: depression severity, onset and (treatment) course, comorbid personality disorder, comorbid substance use disorder, other psychiatric comorbidity, somatic comorbidity, maladaptive coping, childhood trauma, social factors, and psychosocial dysfunction.

To our knowledge, this is the first time that indicators of patients with MDD in need of highly specialized care emerging from the literature are appraised, refined, and complemented by clinical expertise. The resulting overarching conceptual domains of this concept mapping study repeat, to a certain extent, the main indicators of patients with MDD in need of highly specialized care found in the literature review (see van Krugten et al., 2017 for details of the review). Of the 88 putatively relevant indicators, 38 had been identified in the literature review but were made more detailed, worded more precisely, and complemented by clinical expertise before being used in the subsequent sorting and rating steps of the concept mapping process. As such, the use of clinical expertise in addition to evidence from the literature, allowed the summarization of patient indicators emerging from the literature in well‐defined overarching domains. These domains can serve as a starting point for the development of a selection algorithm, which, in turn, may contribute to systematic, evidence‐based treatment selection in patients with MDD.

At the domain level, importance ratings ranged from 2.53 to 4.42 on a 5‐point Likert scale ranging from 1 (not important at all) to 5 (extremely important). Domains with relatively low mean importance ratings appear to cover the nonclinical patient indicators, such as, treatment‐interfering maladaptive coping (domain 7) and social factors maintaining the depression (domain 9), whereas domains of relatively higher importance seem to describe the clinical patient indicators such as depression severity (domain 1), psychiatric and somatic comorbidity (domains 2, 6 and 3), and childhood trauma (domain 8). Although the high mean importance ratings of domains covering clinical patient indicators is consistent with findings indicating that most clinical decisions are largely based on ‘traditional’ clinical patient factors (Hajjaj, Salek, Basra, & Finlay, 2010), the impact of each domain on referral decisions in patients with MDD remains to be validated in an observational study. Future research should examine the relative importance and possible synergy of action between the domains.

This study has a number of strengths, including the systematic step‐by‐step procedure of the concept elicitation procedure, the relatively high number of participants, and the use of clinical expertise in addition to evidence from the literature. The present results should, however, also be viewed in the light of some limitations of this study. First, aiming for the early identification of patients with a highly specialized care need and the timely allocation of those patients to highly specialized mental healthcare settings, presupposes that there is something like a ‘right place’ and that getting there sooner is better than later. Although highly specialized care has been demonstrated to improve clinical outcomes in patients with complex and severe conditions in other areas of medicine (Pollack et al., 1991), the net benefit of highly specialized care in patients with MDD has, however, not yet been demonstrated. Future studies should therefore address the evaluation of the impact of highly specialized care on patient outcomes in this population. Second, in line with the inclusion criteria of the systematic review, the study results are restricted to patients aged 18 and over with a primary diagnosis of MDD treated in psychiatric specialized and highly specialized outpatient clinics. Hence, the findings of this study cannot be generalized to nonclinical samples, children and adolescents. Third, although the number of subject‐matter experts that participated in one or more of the steps of the concept mapping process falls within the average range (20–649) of participants in concept mapping research (Rosas & Kane, 2012), it is unclear whether the participants’ conceptualization is representative of the larger population. In addition, although effort was made to include subject‐matter experts from a broad range of disciplines and countries, the majority of the participants were psychiatrists and worked as treating clinicians and/or researchers in the Netherlands. A larger and more heterogeneous sample of the population might have resulted in a broader range of perspectives and enhanced the generalizability of the findings. Fourth, although involvement of experts is in accordance with evidence‐based medicine (Sackett, Rosenberg, Gray, Haynes, & Richardson, 1996), the patient indicators generated by the clinicians may be biased by preexisting perceptions, beliefs, or attitudes. Future research using a larger and more heterogeneous sample should explore to what extent the results are valid, stable, and generalizable.

Despite these limitations, the results of the present study provide a practical first step towards the early identification of patients with MDD in need of highly specialized care. The study findings highlight the need for a comprehensive assessment of patient indicators in determining the need for highly specialized care, and suggest that the treatment allocation of patients with MDD to highly specialized mental healthcare settings should be guided by the assessment of clinical and nonclinical patient indicators. The results of this study can serve as input for the development of a decision support algorithm to aid clinicians in the treatment allocation of patients with MDD in need of highly specialized care. Such an algorithm may be used to objectify clinical impressions and ultimately assist clinicians in selecting the most appropriate treatment strategy in a given clinical situation. As such, the results of this study have the potential to support and enhance personalized medicine, in which patient management and treatment is tailored to the individual patient needs (Ozomaro, Wahlestedt, & Nemeroff, 2013). Additional research is needed to evaluate the relative importance and possible synergy of action between the identified patient factors and the selection of an optimal decision threshold to distinguish patients with and without a need for highly specialized MDD care.

CONFLICT OF INTEREST

All authors declare that they have no conflict of interest. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Supporting information

SUPPORTING INFORMATION

Click here for additional data file. (72KB, docx)

ACKNOWLEDGMENTS

We gratefully acknowledge the contribution of the members of Decision Tool Unipolar Depression Consortium: A.J.L.M. van Balkom, C.L.H. Bockting, T.M. van den Boogaard, W.J. Broekema, A.E. Thijs van't Hoog, P.A.J.M. Huijs, M.A.J.M. Loo, P. van Oppen, F.P.M.L. Peeters, H.G. Ruhé, D.J.F. van Schaik, J. Spijker, and H.L. Van. This work was funded by the Dutch Network for Quality Development in mental health care (https://www.kwaliteitsontwikkelingggz.nl), project number PV140004.

van Krugten FCW, Goorden M, van Balkom AJLM, Spijker J, Brouwer WBF, Hakkaart‐van Roijen L. Indicators to facilitate the early identification of patients with major depressive disorder in need of highly specialized care: A concept mapping study. Depress Anxiety. 2018;35:346–352. 10.1002/da.22741

REFERENCES

  1. Armstrong, N. P. , & Steffen, J. J. (2009). The recovery promotion fidelity scale: Assessing the organizational promotion of recovery. Community Mental Health Journal, 45, 163–170. [DOI] [PubMed] [Google Scholar]
  2. Conrad, K. J. , Iris, M. , Ridings, J. W. , Rosen, A. , Fairman, K. P. , & Anetzberger, G. J. (2011). Conceptual model and map of psychological abuse of older adults. Journal of Elder Abuse & Neglect, 23, 147–168. [DOI] [PubMed] [Google Scholar]
  3. Conrad, K. J. , Iris, M. , Ridings, J. W. , Langley, K. , & Anetzberger, G. J. (2011). Self‐report measure of psychological abuse of older adults. The Gerontologist, 51, 354–366. [DOI] [PubMed] [Google Scholar]
  4. Corcoran, K. (2005). The Oregon mental health referral checklists: Concept mapping the mental health needs of youth in the juvenile justice system. Brief Treatment and Crisis Intervention, 5, 9–18. [Google Scholar]
  5. Delgadillo, J. , Moreea, O. , & Lutz, W. (2016). Different people respond differently to therapy: A demonstration using patient profiling and risk stratification. Behaviour Research and Therapy, 79, 15–22. [DOI] [PubMed] [Google Scholar]
  6. Dunlop, B. W. , & Mayberg, H. S. (2014). Neuroimaging‐based biomarkers for treatment selection in major depressive disorder. Dialogues in Clinical Neuroscience, 16, 479–490. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Evans, K. C. , Dougherty, D. D. , Pollack, M. H. , & Rauch, S. L. (2006). Using neuroimaging to predict treatment response in mood and anxiety disorders. Annals of Clinical Psychiatry, 18, 33–42. [DOI] [PubMed] [Google Scholar]
  8. Gask, L. , & Khanna, T. (2011). Ways of working at the interface between primary and specialist mental healthcare. The British Journal of Psychiatry, 198, 3–5. [DOI] [PubMed] [Google Scholar]
  9. Hajjaj, F. M. , Salek, M. S. , Basra, M. K. , & Finlay, A. Y. (2010). Non‐clinical influences on clinical decision‐making: A major challenge to evidence‐based practice. Journal of the Royal Society of Medicine, 103, 178–187. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Hirschfeld, R. M. , Keller, M. B. , Panico, S. , Arons, B. S. , Barlow, D. , Davidoff, F. , … Gorman, J. M. (1997). The national depressive and manic‐depressive association consensus statement on the undertreatment of depression. Journal of the American Medical Association, 277, 333–340. [PubMed] [Google Scholar]
  11. Iris, M. , De Backer, N. A. , Benner, R. , Hammerman, J. , & Ridings, J. (2012). Creating a quality of life assessment measure for residents in long term care. Journal of the American Medical Directors Association, 13, 438–447. [DOI] [PubMed] [Google Scholar]
  12. Johnsen, J. A. , Biegel, D. E. , & Shafran, R. (2000). Concept mapping in mental health: Uses and adaptations. Evaluation and Program Planning, 23, 67–75. [Google Scholar]
  13. Kane, M. , & Trochim, W. M. (2007). Concept mapping for planning and evaluation. Thousand Oaks: Sage. [Google Scholar]
  14. Keller, M. B. (1994). Depression: A long‐term illness. The British Journal of Psychiatry, 26, 9–15. [PubMed] [Google Scholar]
  15. Lener, M. S. , & Iosifescu, D. V. (2015). In pursuit of neuroimaging biomarkers to guide treatment selection in major depressive disorder: A review of the literature. Annals of the New York Academy of Sciences, 1344, 50–65. [DOI] [PubMed] [Google Scholar]
  16. McGrath, C. L. , Kelley, M. E. , Holtzheimer, P. E. , Dunlop, B. W. , Craighead, W. E. , Franco, A. R. , … Mayberg, H. S. (2013). Toward a neuroimaging treatment selection biomarker for major depressive disorder. JAMA Psychiatry, 70, 821–829. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Meeuwissen, J. A. , van der Feltz, Cornelis , Christina, M. , van Marwijk, H. W. , Rijnders, P. B. , & Donker, M. C. (2008). A stepped care programme for depression management: An uncontrolled pre‐post study in primary and secondary care in the Netherlands. International Journal of Integrated Care, 8, e05. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Meyers, B. S. , Sirey, J. A. , Bruce, M. , Hamilton, M. , Raue, P. , Friedman, S. J. , … Alexopoulos, G. (2002). Predictors of early recovery from major depression among persons admitted to community‐based clinics: An observational study. Archives of General Psychiatry, 59, 729–735. [DOI] [PubMed] [Google Scholar]
  19. Ozomaro, U. , Wahlestedt, C. , & Nemeroff, C. B. (2013). Personalized medicine in psychiatry: Problems and promises. BMC Medicine, 11, 132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Pollack, M. M. , Alexander, S. R. , Clarke, N. , Ruttimann, U. E. , Tesselaar, H. M. , & Bachulis, A. C. (1991). Improved outcomes from tertiary center pediatric intensive care: A statewide comparison of tertiary and nontertiary care facilities. Critical Care Medicine, 19, 150–159. [DOI] [PubMed] [Google Scholar]
  21. Ridgway, N. , & Williams, C. (2011). Cognitive behavioural therapy self‐help for depression: An overview. Journal of Mental Health, 20, 593–603. [DOI] [PubMed] [Google Scholar]
  22. Rosas, S. R. , & Kane, M. (2012). Quality and rigor of the concept mapping methodology: A pooled study analysis. Evaluation and Program Planning, 35, 236–245. [DOI] [PubMed] [Google Scholar]
  23. Sackett, D. L. , Rosenberg, W. M. , Gray, J. A. , Haynes, R. B. , & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn't. BMJ (Clinical Research Ed.), 312, 71–72. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Shern, D. L. , Trochim, W. M. , & La Comb, C. A. (1995). The use of concept mapping for assessing fidelity of model transfer: An example from psychiatric rehabilitation. Evaluation and Program Planning, 18, 143–153. [Google Scholar]
  25. Simon, G. E. , & Perlis, R. H. (2010). Personalized medicine for depression: Can we match patients with treatments? American Journal of Psychiatry, 167, 1445–1455. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Sotsky, S. M. , Glass, D. R. , Shea, M. T. , Pilkonis, P. A. , Collins, J. F. , Elkin, I. , … Moyer, J. (1991). Patient predictors of response to psychotherapy and pharmacotherapy: Findings in the NIMH treatment of depression collaborative research program. American Journal of Psychiatry, 148, 997–1008. [DOI] [PubMed] [Google Scholar]
  27. Trochim, W. (1993). The reliability of concept mapping. Annual Conference of the American Evaluation Association, Dallas, Texas. [Google Scholar]
  28. Trochim, W. M. (1989). An introduction to concept mapping for planning and evaluation. Evaluation and Program Planning, 12, 1–16. [Google Scholar]
  29. van Krugten, F. C. , Kaddouri, M. , Goorden, M. , van Balkom, A. J. , Bockting, C. L. , Peeters, F. P. … Decision Tool Unipolar Depression (DTUD) Consortium . (2017). Indicators of patients with major depressive disorder in need of highly specialized care: A systematic review. PloS One, 12(2), e0171659 https://doi.org/10.1371/journal.pone.0171659 [doi] [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. van Straten, A. , Tiemens, B. , Hakkaart, L. , Nolen, W. , & Donker, M. (2006). Stepped care vs. matched care for mood and anxiety disorders: A randomized trial in routine practice. Acta Psychiatrica Scandinavica, 113, 468–476. [DOI] [PubMed] [Google Scholar]
  31. Von Korff, M. , & Tiemens, B. (2000). Individualized stepped care of chronic illness. The Western Journal of Medicine, 172, 133–137. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Jr Ward, H. J. (1963). Hierarchical grouping to optimize an objective function. Journal of the American Statistical Association, 58, 236–244. [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

SUPPORTING INFORMATION

Click here for additional data file. (72KB, docx)

Articles from Depression and Anxiety are provided here courtesy of Wiley

RESOURCES