Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2016 Jan 1.
Published in final edited form as: Personal Disord. 2015 Jan;6(1):81–87. doi: 10.1037/per0000091

Generalizability of Pharmacological and Psychotherapy Clinical Trial Results for Borderline Personality Disorder to Community Samples

Nicolas Hoertel a,b,c,*, Saioa López a,d,*, Shuai Wang a, Ana González-Pinto b, Frédéric Limosin c,d, Carlos Blanco a
PMCID: PMC4372804  NIHMSID: NIHMS667559  PMID: 25580674

Abstract

Background

The present study sought to quantify the generalizability of clinical trial results in individuals with a DSM-IV diagnosis of borderline personality disorder (BPD) to a large representative community sample.

Method

Data were derived from the 2004–2005 National Epidemiologic Survey on Alcohol and Related Conditions (NESARC), a large nationally representative sample of 34,653 adults from the United States population. We applied a standard set of exclusion criteria representative of pharmacological and psychotherapy clinical trials to all adults with a DSM-IV diagnosis of BPD (n = 2,231). Our aim was to assess how many participants with BPD would not fulfill typical eligibility criteria.

Results

We found that more than seven out of ten respondents in a typical pharmacological efficacy trial and more than five out of ten participants in a typical psychotherapy efficacy trial would have been excluded by at least one criterion. Having a current history of alcohol or drug use disorder and a lifetime history of bipolar disorder explained a large proportion of ineligibility in both pharmacological and psychotherapy efficacy trials.

Discussion

Clinical trials should carefully consider the impact of exclusion criteria on the generalizability of their results. As required by CONSORT guidelines, reporting exclusion rate estimate and reasons of eligibility should be mandatory in both clinical trials and meta-analyses. As treatment trials of borderline personality disorder move from efficacy to effectiveness to better inform clinical practice, the eligibility rate must be increased by imposing less stringent eligibility criteria to allow for more generalizable results.

Keywords: borderline personality disorder, trials, treatment, psychotherapy, validity

Introduction

Restrictive eligibility criteria are generally used in randomized controlled trials (RCTs) in the hopes of ensuring the safety of vulnerable patients, reducing study costs, increasing study feasibility and/or interpretability, decreasing heterogeneity in response (and thereby increase statistical power) and complying with guidelines by regulatory agencies (Bucher, Guyatt, Cook, Holbrook, & McAlister, 1999; Hoertel, Le Strat, Limosin, Dubertret, & Gorwood, 2013; Humphreys, Harris, & Weingardt, 2008; Humphreys, Weingardt, Horst, Joshi, & Finney, 2005). However, growing evidence indicates that restrictive eligibility criteria are not always well justified (Van Spall, Toren, Kiss, & Fowler, 2007), result in diminished external validity (i.e., applicability of clinical trial results to routine clinical care) and in a sampling bias (Blanco, Olfson, Goodwin, et al., 2008; Blanco, Olfson, Okuda, et al., 2008), perpetuating thus the gap between research and clinical practice (Hoertel et al., 2014; Hoertel, de Maricourt, & Gorwood, 2013; Weisberg, Hayden, & Pontes, 2009). In addition, restrictive eligibility criteria may not even deliver their intended benefits: highly selective studies may increase the length of the recruitment period, raising study costs, and criteria intended to reduce heterogeneity in treatment response may have the reverse effect, thereby reducing rather than increasing statistical power (Hoertel et al., 2014; Humphreys et al., 2008). Over the last several years, concerns have emerged regarding whether results from tightly controlled trials generalize to typical patients in community settings, who may often present with complex clinical presentations (March et al., 2005). Because of the substantial burden of BPD on affected individuals, their families and social networks, and the larger society including health and social service systems, the need for an improved armamentarium of empirically-supported interventions and optimized methodology for clinical trials to evaluate these interventions remains critical. As a consequence, there has been a call to quantify the generalizability of RCTs results to the broader target population suffering from the disorder under study (Geddes, 2005; Wells, 1999).

As with other areas of psychiatric treatment research (Hoertel, Le Strat, De Maricourt, et al., 2013; Hoertel, Le Strat, Lavaud, et al., 2013; Licht, 2002; Rabinowitz, Bromet, & Davidson, 2003; Schneider, Olin, Lyness, & Chui, 1997; Westen & Morrison, 2001), pharmacological and psychotherapy treatment trials for borderline personality disorder (BPD) applied strict eligibility criteria that exclude patients with a variety of psychiatric and medical comorbidities (Stoffers et al., 2010; Stoffers et al., 2012). To our knowledge, no study has examined the representativeness of clinical trials for BPD. However, a substantial proportion of adults with BPD present with significant psychiatric comorbidities, including substance dependence, mood and anxiety disorders, other personality disorders, and general medical comorbidities (Frankenburg & Zanarini, 2004; Sansone & Sansone, 2011; Sjastad, Grawe, & Egeland, 2012; Zimmerman & Morgan, 2013). These findings support that clinical trial results for BPD might not be directly extrapolable to patients in clinical practice. Examining the application of exclusion criteria to a representative general population sample of individuals with BPD may help quantifying the impact of exclusion criteria on the generalizability of pharmacological and psychotherapy clinical trials results, as well as guide eligibility criteria operationalization for future clinical trials in BPD.

In the present report, we use the population generalizability estimator (PG estimator) method previously described by Blanco et al. (Blanco, Olfson, Goodwin, et al., 2008; Blanco, Olfson, Okuda, et al., 2008), which consists in applying exclusion criteria commonly used in pharmacological and psychotherapy clinical trials for BPD to a large, nationally representative sample to assess the generalizability of the criteria to community adults with BPD.

Material and methods

Data source

Data were drawn from the 2004–2005 National Epidemiologic Survey on Alcohol and Related Conditions (NESARC), the second wave that follows upon the Wave 1 NESARC, a nationally representative face-to-face survey of the U.S adult population, conducted in 2001–2002 (wave 1) and 2004–2005 (wave 2) by the National Institute on Alcoholism and Alcohol Abuse (NIAAA) (Grant, Dawson & Hasin, 2004; Grant, Stinson, et al., 2004). The target population included the civilian noninstitutionalized population, aged 18 years and older, residing in the United States. The cumulative response rate at Wave 2 was 70.2%, resulting in 34,653 Wave 2 interviews. The Wave 2 NESARC data were weighted to be representative of the U.S. civilian population based on the 2000 census (Grant et al., 2005). The research protocol, including written informed consent procedures, received full human subjects review and approval from the U.S. Census Bureau and the Office of Management and Budget (Grant, Dawson & Hasin, 2004; Grant, Stinson, et al., 2004).

Diagnoses were made according to the criteria of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) using the NIAAA Alcohol Use Disorder and Associated Disabilities Interview Schedule-IV (AUDADIS-IV), a fully structured diagnostic interview designed for experienced non-clinicians interviewers (Grant, Dawson & Hasin, 2004). All NESARC respondents were asked in the Wave 2 a series of BPD symptoms questions about how they felt or acted most of the time throughout their lives, regardless of the situation or whom they were with (Grant et al., 2008). They were instructed not to include symptoms occurring only when they were depressed, manic, anxious, drinking heavily, using medicines or drugs, experiencing withdrawal symptoms or physically ill (Grant et al., 2008). To receive a diagnosis of BPD, respondents had to endorse the requisite number of DSM-IV symptom items, at least 1 of which must have caused social or occupational dysfunction. Test-retest reliability of AUDADIS-IV BPD diagnosis was good (κ = 0.71, SE = 0.06) (Grant et al., 2008; Ruan et al., 2008). Intraclass test-retest reliability coefficient fell within the good range (95% ICC = [0.74–0.79], α = 0.83). Reliability (κ > 0.74) was good to excellent for substance use disorders. Reliability was fair to good for mood and anxiety disorders (κ = 0.40–0.60) and other personality disorders (κ = 0.40–0.67) (Grant et al., 2003; Grant, Stinson, et al., 2004; Grant et al., 2005).

Measures

Exclusion criteria commonly used in pharmacological and psychotherapy clinical trials for BPD were applied to a sample representative of the general population to evaluate the proportion of participants with a DSM-IV diagnosis of BPD that would be eligible for a typical pharmacological and psychotherapy clinical trial.

We used the exclusion criteria from clinical trials for BPD included in two recent systematic reviews in BPD, examining respectively the effects of pharmacological (Stoffers et al., 2010) and psychological (Stoffers et al., 2012) treatments on BPD. Of the 29 clinical trials included in the pharmacological review, the exclusion criteria analysis included 21 studies because 4 trials were published before 1985, 3 studies focused only on individuals aged less than 16 years and data from 1 study were not available. Of the 28 clinical trials included in the psychotherapy review, 27 studies were included in the present analysis because 1 study focused only on individuals aged less than 16 years. Two coders (SL and NH) independently collected all eligibility criteria from the 21 pharmacological trials and the 27 psychotherapy trials and selected criteria that occurred in >10% of the studies. Intercoder reliability was adequate (ICC = 0.85; 95% confidence interval [CI], 0.69–0.94). Disagreement was resolved by consensus. The median of the number of exclusion criteria used in the 21 pharmacological trials and in the 27 psychotherapy trials was respectively 8 and 5 (including also the criteria present in less than 10% of the studies). In order to reproduce a typical pharmacological and psychotherapy clinical trial with traditional exclusion criteria, we applied respectively the 8 and the 5 most commonly used criteria to all individuals with a DSM-IV diagnosis of BPD to the NESARC sample (Table 2).

Table 2.

Estimated percentage of adults with borderline personality disorder (BPD) in the NESARC excluded from typical clinical trials of psychotherapy treatments for BPD by traditional exclusion criteria.

Borderline personality disorder patients (n = 2,231)
Exclusion variable* % (N) 95% CI
Psychosis 5.22 (121) 4.11 – 6.60
Intellectual disability NA NA
Currently/past 12 months’ alcohol or drug abuse or dependence 29.87 (618) 27.48 – 32.37
Lifetime bipolar disorder 29.71 (663) 27.39 – 32.14
Currently receiving psychotherapy NA NA
Exclusion by at least one criterion 51.29 (1,106) 48.80 – 53.78
Open in a new tab

Percentages are weighted values.

*

Derived from the review of 27 psychotherapy clinical trials (method described in the paper).

Abbreviations: CI = confidence interval, NA = not available in NESARC, NESARC = National Epidemiologic Survey on Alcohol and Related Conditions.

The percentages of exclusion were estimated from data collected by the AUDADIS-IV. The criteria “current major depressive episode”, “current panic disorder”, “lifetime bipolar disorder” and “antisocial personality disorder” were diagnosed following the DSM-IV criteria. The criterion “currently/past 6 months’ drug or alcohol abuse or dependence” was defined as having a DSM-IV diagnosis of dependence or abuse on alcohol or drug within the past 12 months. The criterion “significant risk of suicide” was considered met if the person reported a suicide attempt during the past year, the time frame used by the AUDADIS-IV when assessing the presence of “current” symptoms. The criterion “pregnancy or breastfeeding” was assessed by the following question: “Were you pregnant during the past year?” The criterion “significant medical condition” was approximated by series of questions on 12-month angina pectoris, myocardial infarction or any other form of heart disease, stroke, cirrhosis or hepatic disease or diabetes mellitus and whether the diagnosis was confirmed by a physician. “Psychosis” was assessed by a single question: “Did a doctor or other health professional ever diagnose you with schizophrenia or psychotic illness or episode?” Participants who responded affirmatively were considered as having a psychosis. Information to approximate the criteria “currently taking any psychotropic medication” and “currently receiving psychotherapy” was not available in NESARC.

Analysis methods

We first defined the percentages (and their 95% confidence intervals (CIs)) of survey participants with a DSM-IV diagnosis of BPD (n = 2,231) who would have been excluded by individually applying each classical exclusion criterion in pharmacological and psychotherapy clinical trials for BPD. Because individuals would have been excluded by more than 1 criterion, we also calculated the overall percentage of subjects who would have been excluded by the simultaneous application of all criteria. Weighted prevalence estimates and 95% CIs were performed using SUDAAN 10.01 (RTI International, Research Triangle Park, North Carolina) to adjust for complex survey sampling effects including clustering data.

Results

Of the 2,231 participants who met DSM-IV criteria for BPD, 75.91% (95% CI = 73.60–78.09%) would have been excluded by at least one of the 8 most common and available criteria in pharmacological efficacy trials. The percentage of exclusion by at least one of the 5 most common criteria was substantially lower in psychotherapy efficacy trials, falling to 51.29% (95% CI = 48.80–53.78) (Tables 1 and 2).

Table 1.

Estimated percentage of adults with borderline personality disorder (BPD) in the NESARC excluded from typical clinical trials of pharmacological treatments for BPD by traditional exclusion criteria.

Borderline personality disorder patients (n = 2,231)
Exclusion variable* % (N) 95% CI
Psychosis 5.22 (121) 4.11 – 6.60
Currently/past 12 months’ alcohol or drug abuse or dependence 29.87 (618) 27.48 – 32.37
Lifetime bipolar disorder 29.71 (663) 27.39 – 32.14
Current major depressive episode 19.25 (440) 17.38 – 21.25
Significant suicidal risk 2.16 (37) 1.43 – 3.25
Significant medical condition 32.54 (749) 30.19 – 34.99
Pregnancy/breastfeeding 1.16 (24) 0.71 – 1.90
Currently taking any psychotropic NA NA
Exclusion by at least one criterion 75.91 (1,692) 73.60 – 78.09
Open in a new tab

Percentages are weighted values.

*

Derived from the review of 21 pharmacological efficacy trials (method described in the paper).

Abbreviations: CI = confidence interval, NA = not available in NESARC, NESARC = National Epidemiologic Survey on Alcohol and Related Conditions.

In pharmacological efficacy trials, the percentage of BPD participants excluded due to the application of a single criterion ranged from 1.16% (“pregnancy or breastfeeding”) to 32.54% (“significant medical condition”) (Table 1). In psychotherapy efficacy trials, this percentage ranged from 5.22% (“psychosis”) to 29.87% (“currently/past 6 months’ alcohol or drug abuse or dependence”) (Table 2).

In both pharmacological and psychotherapy efficacy trials, the criterion leading to the highest exclusion rate was respectively “significant medical condition” and “currently/past 6 months’ alcohol or drug abuse or dependence”. In pharmacology trials, having a current dependence on alcohol or drugs, a lifetime bipolar disorder or a current major depressive episode also excluded an important proportion of participants with BPD, whereas being pregnant was the criterion excluding the smallest percentage of individuals. In psychotherapy trials, having a lifetime bipolar disorder or a current major depressive episode also excluded many participants with BPD, whereas the criterion which excluded the lowest percentage of patients was “psychosis”.

Discussion

The present study examines the proportion of adults with borderline personality disorder in the general population who would not have been eligible for a typical pharmacological or psychotherapy efficacy trial for BPD. We found that more than 7 out of 10 participants with BPD in a typical pharmacological trial and more than 5 out of 10 in a typical psychotherapy trial would have been excluded from participation by one or more of the study criteria. These results suggest that traditional criteria used in pharmacological and psychotherapy trials for BPD tend to exclude the majority of individuals from participation. We found that having a current history of substance abuse or dependence and a lifetime history of bipolar disorder explained a large proportion of ineligibility in both pharmacological and psychotherapy trials for BPD. In pharmacological efficacy trials, having a significant general medical condition also excluded a substantial proportion of individuals with BPD.

Overall, the exclusion rates in pharmacological trials for BPD were comparable to those reported for major depressive disorder (Blanco, Olfson, Goodwin, et al., 2008), cannabis (Okuda et al., 2010), panic disorder (Hoertel, Le Strat, De Maricourt, et al., 2013), and generalized anxiety disorder (Hoertel, Le Strat, Blanco, et al., 2012) clinical trials (ranging approximately from 75% to 80%), whereas the proportion of participants who would have been excluded in psychotherapy trials for BPD appears to be within the same range with nicotine dependence (Le Strat et al., 2011), alcohol dependence (Blanco, Olfson, Okuda, et al., 2008), and bipolar disorder (Hoertel, Le Strat, Lavaud, et al., 2013) clinical trials (ranging from 50.5% to 65.9%). These results support the view that clinical trials examining the effects of pharmacological as well as psychological treatments on BPD suffer from restricted external validity since their results may not be readily generalizable to community samples and have implications for the design of clinical trials. As a consequence, clinical trials for BPD using traditional exclusion criteria tend to recruit “pure” rather than “typical” patients (Goldenberg et al., 1996). Furthermore, the presence of psychiatric comorbidities in BPD patients has been found to worsen the prognosis of the disorder (Swartz, Pilkonis, Frank, Proietti, & Scott, 2005; Tritt et al., 2005). In addition, a previous study (Wisniewski et al., 2009) showed that some exclusion criteria in clinical trials may be predictors of poor treatment outcomes, suggesting that increasing generalizability of results to practice could potentially reduce placebo response and remission rates (decreasing the risk of failed trials) but at the risk of some increase in adverse events.

Researchers often use exclusion criteria in response to concerns about patient safety (e.g., pregnancy, significant medical conditions), study feasibility or interpretability of results (e.g., potential impact of changes in patterns of substance abuse on outcomes during the trial). In other cases, the criteria may simply reflect a tradition that has evolved over time within a particular research area, resulting in progressive narrowing of the population of eligible patients (e.g., psychiatric comorbidity). While some of the exclusion criteria need to be implemented due to practical constraints outside the control of the investigators (e.g., pregnancy, significant medical condition), implementing others (e.g., having a lifetime history of bipolar disorder or psychosis or having a current history of major depressive episode or substance use disorder) may benefit from a more explicit rationale, since they can lead to the exclusion of a large proportion of patients and thereby negatively impact on external validity (Hoertel et al., 2014), limiting thus availability of evidence-based treatments for these patients. The exclusion of participants with substance use disorders or bipolar disorder may be particularly significant for both pharmacological and psychotherapy trials, since their lifetime prevalence rates in BPD patients are estimated to be 65.3% (Sansone & Sansone, 2011) for any substance use disorder and 14% for bipolar disorder (Zimmerman & Morgan, 2013). In pharmacological trials, a substantial proportion of participants are excluded because of significant medical conditions, which are common among BPD patients (Frankenburg & Zanarini, 2004). Therefore, whether the results of clinical trials apply in community setting is unclear. The designers of clinical trials should carefully consider the tradeoffs between the application of each exclusion criterion and its impact on generalizability (Blanco, Olfson, Okuda, et al., 2008). Specification a priori of the goals of the study and estimation of the proportion of individuals ineligible for the trial due to the individually application of each exclusion criterion to the target population would assist study design. For example, effectiveness studies could place a larger emphasis on the generalizability of their findings whereas efficacy studies may benefit from relatively stringent eligibility criteria to maximize detection of drug-placebo differences (Blanco, Olfson, Goodwin, et al., 2008; Cipriani et al., 2013; Sani et al., 2013).

Our study has several limitations.

First, we adopted specific conventions to translate clinical criteria to the NESARC sample. We followed a methodology described by Blanco et al. (Blanco, Olfson, Goodwin, et al., 2008; Blanco, Olfson, Okuda, et al., 2008) and considered exclusion criteria from 21 pharmacological and 27 psychotherapy clinical trials for BPD included in two recent reviews. Other conventions might have yielded different exclusion estimates. For example, the 12-month timeframe used by the AUDADIS-IV when assessing the presence of “current” symptoms could have led to an overestimation of the exclusion rates. The lack of information about lethality of method, likelihood of rescue, and intent to die may have also added noise to the estimation of significant suicidal risk based on a single item querying past-year suicide attempts, particularly in BDP participants with high levels of nonsuicidal and parasuicidal self-harm. In addition, three of the exclusion criteria (i.e., “currently taking any psychotropic medication”, “currently receiving psychotherapy”, and “intellectual disability”) could not be operationalized using the NESARC, which may have led to underestimate the proportion of participants excluded from clinical trials. While the percentage of people with intellectual disability and receiving psychotherapy in the general population could be considered as low (approximately 0.3% and 3.3%) (Lundvall, Rajaei, Erlandson, & Kyllerman, 2012; Hoertel, Limosin & Leleu, 2014; Olfson & Marcus, 2010), the percentage of adults taking a psychotropic medication is likely to be high (Verdoux & Begaud, 2004) and might potentially lead to an underestimation of the proportion of patients excluded in clinical trials. However, the percentage of excluded participants was high and consistent with those observed in previous research on the generalizability of clinical trial results for other psychiatric disorders to community samples. Future studies would benefit from examining the impact of these criteria and developing procedures to operationalize eligibility criteria selection, which will help refine estimates of the proportion of individuals ineligible for borderline personality disorder clinical trials.

Second, our approach focuses on the a priori eligibility of participants and was based on national epidemiological data (Blanco, Olfson, Goodwin, et al., 2008). It provides no information on individuals who actually enter those studies. In fact, a substantial proportion of eligible individuals may be unwilling to participate (Melberg & Humphreys, 2010). Furthermore, the likelihood of entering a trial may be influenced by several factors, including anxiety, extroversion, and performance measures (Mavissakalian & Guo, 2002). In this way, we estimate an upper bound of the generalizability of clinical trials.

Third, retrospective reporting of symptoms may be subject to mis-recall for the participants.

Fourth, data gathered from collaterals, that are important in BPD treatment trials and settings and may have help refine exclusion estimates, were not available in NESARC.

Last, the NESARC sample included only individuals aged 18 years or older. Information was unavailable for adolescents, who may have a lower rate of comorbidity and may then be more likely to be eligible for clinical trials (Le Strat et al., 2011). Future studies should examine the generalizability of clinical trials in children and adolescents.

Conclusion

Despite these concerns, the present study suggests that the current design of pharmacological and psychotherapy clinical trials for BPD suffers from restricted external validity, as also found in treatment trials of other common, debilitating psychiatric disorders. Our findings suggest that individuals with a current history of substance use disorder and a lifetime history of bipolar disorder are under-represented in clinical trials for BPD. Although several criteria are widely implemented, not all trials use all criteria. Conducting clinical trials with BPD patients is always complex, and the use of broader inclusion criteria is likely to further increase this complexity. However, as treatment trials of BPD move from efficacy to effectiveness to better inform clinical practice, the eligibility rate must be increased by imposing less stringent eligibility criteria that allow for more generalizable results. Future clinical trials would benefit from examining BPD treatment efficacy in this specific population, either by doing studies tailored to this population or relaxing exclusion criteria and raising sample sizes sufficiently to allow for careful examination of heterogeneity by affording improved statistical power (Geddes, 2005; Hoertel et al., 2014).

Acknowledgments

Funding/support: supported by NIH grants DA019606, MH076051 and MH082773 and the New York State Psychiatric Institute (Dr. Blanco) and a fellowship grant from Public Health Expertise (Dr. Hoertel).

Footnotes

Conflicts of interest: Dr. González-Pinto has received grants an served as consultant, advisor or CME speaker for the following entities: Almirall, AstraZeneca, Cephalon, Eli Lilly, Bristol-Myers Squibb, Glaxo-Smith-Kline, Janssen-Cilag, Jazz, Johnson & Johnson, Lundbeck, Merck, Otsuka, Pfizer, Servier, Sanofi-Aventis, Shering-Plough, Solvay, CIBERSAM, Carlos III Institute, Basque Government, Stanley Medical Research Institute, and Wyeth. Other authors report no conflicts of interest.

References

  1. Blanco C, Olfson M, Goodwin RD, Ogburn E, Liebowitz MR, Nunes EV, Hasin DS. Generalizability of clinical trial results for major depression to community samples: results from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2008;69(8):1276–1280. doi: 10.4088/jcp.v69n0810. [DOI] [PubMed] [Google Scholar]
  2. Blanco C, Olfson M, Okuda M, Nunes EV, Liu SM, Hasin DS. Generalizability of clinical trials for alcohol dependence to community samples. Drug Alcohol Depend. 2008;98(1–2):123–128. doi: 10.1016/j.drugalcdep.2008.05.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  3. Bucher HC, Guyatt GH, Cook DJ, Holbrook A, McAlister FA. Users’ guides to the medical literature: XIX. Applying clinical trial results. A. How to use an article measuring the effect of an intervention on surrogate end points. Evidence-Based Medicine Working Group. JAMA. 1999;282(8):771–778. doi: 10.1001/jama.282.8.771. [DOI] [PubMed] [Google Scholar]
  4. Cipriani A, Girlanda F, Agrimi E, Barichello A, Beneduce R, Bighelli I, Barbui C. Effectiveness of lithium in subjects with treatment-resistant depression and suicide risk: a protocol for a randomised, independent, pragmatic, multicentre, parallel-group, superiority clinical trial. BMC Psychiatry. 2013;13:212. doi: 10.1186/1471-244X-13-212. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Frankenburg FR, Zanarini MC. The association between borderline personality disorder and chronic medical illnesses, poor health-related lifestyle choices, and costly forms of health care utilization. J Clin Psychiatry. 2004;65(12):1660–1665. doi: 10.4088/jcp.v65n1211. [DOI] [PubMed] [Google Scholar]
  6. Geddes JR. Large simple trials in psychiatry: providing reliable answers to important clinical questions. Epidemiol Psichiatr Soc. 2005;14(3):122–126. doi: 10.1017/s1121189x00006357. [DOI] [PubMed] [Google Scholar]
  7. Goldenberg IM, White K, Yonkers K, Reich J, Warshaw MG, Goisman RM, Keller MB. The infrequency of “pure culture” diagnoses among the anxiety disorders. J Clin Psychiatry. 1996;57(11):528–533. doi: 10.4088/jcp.v57n1105. [DOI] [PubMed] [Google Scholar]
  8. Grant BF, Chou SP, Goldstein RB, Huang B, Stinson FS, Saha TD, Ruan WJ. Prevalence, correlates, disability, and comorbidity of DSM-IV borderline personality disorder: results from the Wave 2 National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2008;69(4):533–545. doi: 10.4088/jcp.v69n0404. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Grant BF, Dawson DA, Stinson FS, Chou PS, Kay W, Pickering R. The Alcohol Use Disorder and Associated Disabilities Interview Schedule-IV (AUDADIS-IV): reliability of alcohol consumption, tobacco use, family history of depression and psychiatric diagnostic modules in a general population sample. Drug Alcohol Depend. 2003;71(1):7–16. doi: 10.1016/s0376-8716(03)00070-x. [DOI] [PubMed] [Google Scholar]
  10. Grant BF, Dawson DA, Hasin DS. The Wave 2 National Epidemiologic Survey on Alcohol and Related Conditions Alcohol Use Disorder, and Associated Disabilities Interview Schedule- DSM-IV version. Bethesda, MD: National Institute on Alcohol Abuse and Alcoholism; 2004. [Google Scholar]
  11. Grant BF, Stinson FS, Dawson DA, Chou SP, Dufour MC, Compton W, Kaplan K. Prevalence and co-occurrence of substance use disorders and independent mood and anxiety disorders: results from the National Epidemiologic Survey on Alcohol and Related Conditions. Arch Gen Psychiatry. 2004;61(8):807–816. doi: 10.1001/archpsyc.61.8.807. [DOI] [PubMed] [Google Scholar]
  12. Grant BF, Stinson FS, Hasin DS, Dawson DA, Chou SP, Ruan WJ, Huang B. Prevalence, correlates, and comorbidity of bipolar I disorder and axis I and II disorders: results from the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2005;66(10):1205–1215. doi: 10.4088/jcp.v66n1001. [DOI] [PubMed] [Google Scholar]
  13. Hoertel N, de Maricourt P, Gorwood P. Novel routes to bipolar disorder drug discovery. Expert Opin Drug Discov. 2013;8(8):907–918. doi: 10.1517/17460441.2013.804057. [DOI] [PubMed] [Google Scholar]
  14. Hoertel N, Falissard B, Humphreys K, Gorwood P, Seigneurie AS, Limosin F. Do Clinical Trials of Treatment of Alcohol Dependence Adequately Enroll Participants with Co-Occurring Independent Mood and Anxiety Disorders? J Clin Psychiatry. 2014;75(3):231–237. doi: 10.4088/JCP.13m08424. [DOI] [PubMed] [Google Scholar]
  15. Hoertel N, Le Strat Y, Blanco C, Lavaud P, Dubertret C. Generalizability of clinical trial results for generalized anxiety disorder to community samples. Depress Anxiety. 2012;29(7):614–620. doi: 10.1002/da.21937. [DOI] [PubMed] [Google Scholar]
  16. Hoertel N, Le Strat Y, De Maricourt P, Limosin F, Dubertret C. Are subjects in treatment trials of panic disorder representative of patients in routine clinical practice? Results from a national sample. J Affect Disord. 2013;146(3):383–389. doi: 10.1016/j.jad.2012.09.023. [DOI] [PubMed] [Google Scholar]
  17. Hoertel N, Le Strat Y, Lavaud P, Dubertret C, Limosin F. Generalizability of clinical trial results for bipolar disorder to community samples: findings from the national epidemiologic survey on alcohol and related conditions. J Clin Psychiatry. 2013;74(3):265–270. doi: 10.4088/JCP.12m07935. [DOI] [PubMed] [Google Scholar]
  18. Hoertel N, Le Strat Y, Limosin F, Dubertret C, Gorwood P. Prevalence of subthreshold hypomania and impact on internal validity of RCTs for major depressive disorder: results from a national epidemiological sample. PLoS One. 2013;8(2):e55448. doi: 10.1371/journal.pone.0055448. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Hoertel N, Limosin F, Leleu H. Poor longitudinal continuity of care is associated with an increased mortality rate among patients with mental disorders: Results from the French National Health Insurance Reimbursement Database. Eur Psychiatry. 2014;29(6):358–364. doi: 10.1016/j.eurpsy.2013.12.001. [DOI] [PubMed] [Google Scholar]
  20. Humphreys K, Harris AH, Weingardt KR. Subject eligibility criteria can substantially influence the results of alcohol-treatment outcome research. J Stud Alcohol Drugs. 2008;69(5):757–764. doi: 10.15288/jsad.2008.69.757. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Humphreys K, Weingardt KR, Horst D, Joshi AA, Finney JW. Prevalence and predictors of research participant eligibility criteria in alcohol treatment outcome studies, 1970–98. Addiction. 2005;100(9):1249–1257. doi: 10.1111/j.1360-0443.2005.01175.x. [DOI] [PubMed] [Google Scholar]
  22. Le Strat Y, Rehm J, Le Foll B. How generalisable to community samples are clinical trial results for treatment of nicotine dependence: a comparison of common eligibility criteria with respondents of a large representative general population survey. Tob Control. 2011;20(5):338–343. doi: 10.1136/tc.2010.038703. [DOI] [PubMed] [Google Scholar]
  23. Licht RW. Limits of the applicability and generalizability of drug trials in mania. Bipolar Disord. 2002;4(Suppl 1):66–68. doi: 10.1034/j.1399-5618.4.s1.27.x. [DOI] [PubMed] [Google Scholar]
  24. Lundvall M, Rajaei S, Erlandson A, Kyllerman M. Aetiology of severe mental retardation and further genetic analysis by high-resolution microarray in a population-based series of 6- to 17-year-old children. Acta Paediatr. 2012;101(1):85–91. doi: 10.1111/j.1651-2227.2011.02417.x. [DOI] [PubMed] [Google Scholar]
  25. March JS, Silva SG, Compton S, Shapiro M, Califf R, Krishnan R. The case for practical clinical trials in psychiatry. Am J Psychiatry. 2005;162(5):836–846. doi: 10.1176/appi.ajp.162.5.836. [DOI] [PubMed] [Google Scholar]
  26. Mavissakalian MR, Guo S. Predictors of entering a long-term drug treatment study of panic disorder. Compr Psychiatry. 2002;43(2):88–94. doi: 10.1053/comp.2002.30803. [DOI] [PubMed] [Google Scholar]
  27. Melberg HO, Humphreys K. Ineligibility and refusal to participate in randomised trials of treatments for drug dependence. Drug Alcohol Rev. 2010;29(2):193–201. doi: 10.1111/j.1465-3362.2009.00096.x. [DOI] [PubMed] [Google Scholar]
  28. Okuda M, Hasin DS, Olfson M, Khan SS, Nunes EV, Montoya I, Blanco C. Generalizability of clinical trials for cannabis dependence to community samples. Drug Alcohol Depend. 2010;111(1–2):177–181. doi: 10.1016/j.drugalcdep.2010.04.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Olfson M, Marcus SC. National trends in outpatient psychotherapy. Am J Psychiatry. 2010;167(12):1456–1463. doi: 10.1176/appi.ajp.2010.10040570. [DOI] [PubMed] [Google Scholar]
  30. Rabinowitz J, Bromet EJ, Davidson M. Are patients enrolled in first episode psychosis drug trials representative of patients treated in routine clinical practice? Schizophrenia research. 2003;61(2):149–155. doi: 10.1016/s0920-9964(02)00322-5. [DOI] [PubMed] [Google Scholar]
  31. Ruan WJ, Goldstein RB, Chou SP, Smith SM, Saha TD, Pickering RP, Grant BF. The alcohol use disorder and associated disabilities interview schedule-IV (AUDADIS-IV): reliability of new psychiatric diagnostic modules and risk factors in a general population sample. Drug Alcohol Depend. 2008;92(1–3):27–36. doi: 10.1016/j.drugalcdep.2007.06.001. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Sani G, Kotzalidis GD, Vohringer P, Pucci D, Simonetti A, Manfredi G, Ghaemi SN. Effectiveness of short-term olanzapine in patients with bipolar I disorder, with or without comorbidity with substance use disorder. J Clin Psychopharmacol. 2013;33(2):231–235. doi: 10.1097/JCP.0b013e318287019c. [DOI] [PubMed] [Google Scholar]
  33. Sansone RA, Sansone LA. Substance use disorders and borderline personality: common bedfellows. Innov Clin Neurosci. 2011;8(9):10–13. [PMC free article] [PubMed] [Google Scholar]
  34. Schneider LS, Olin JT, Lyness SA, Chui HC. Eligibility of Alzheimer’s disease clinic patients for clinical trials. J Am Geriatr Soc. 1997;45(8):923–928. doi: 10.1111/j.1532-5415.1997.tb02960.x. [DOI] [PubMed] [Google Scholar]
  35. Sjastad HN, Grawe RW, Egeland J. Affective disorders among patients with borderline personality disorder. PLoS One. 2012;7(12):e50930. doi: 10.1371/journal.pone.0050930. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Stoffers J, Vollm BA, Rucker G, Timmer A, Huband N, Lieb K. Pharmacological interventions for borderline personality disorder. Cochrane Database Syst Rev. 2010;(6):CD005653. doi: 10.1002/14651858.CD005653.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Stoffers JM, Vollm BA, Rucker G, Timmer A, Huband N, Lieb K. Psychological therapies for people with borderline personality disorder. Cochrane Database Syst Rev. 2012;8:CD005652. doi: 10.1002/14651858.CD005652.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Swartz HA, Pilkonis PA, Frank E, Proietti JM, Scott J. Acute treatment outcomes in patients with bipolar I disorder and co-morbid borderline personality disorder receiving medication and psychotherapy. Bipolar Disord. 2005;7(2):192–197. doi: 10.1111/j.1399-5618.2005.00179.x. [DOI] [PubMed] [Google Scholar]
  39. Tritt K, Nickel C, Lahmann C, Leiberich PK, Rother WK, Loew TH, Nickel MK. Lamotrigine treatment of aggression in female borderline-patients: a randomized, double-blind, placebo-controlled study. J Psychopharmacol. 2005;19(3):287–291. doi: 10.1177/0269881105051540. [DOI] [PubMed] [Google Scholar]
  40. Van Spall HG, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA. 2007;297(11):1233–1240. doi: 10.1001/jama.297.11.1233. [DOI] [PubMed] [Google Scholar]
  41. Verdoux H, Begaud B. Pharmaco-epidemiology: what do (and don’t) we know about utilisation and impact of psychotropic medications in real-life conditions? Br J Psychiatry. 2004;185:93–94. doi: 10.1192/bjp.185.2.93. [DOI] [PubMed] [Google Scholar]
  42. Weisberg HI, Hayden VC, Pontes VP. Selection criteria and generalizability within the counterfactual framework: explaining the paradox of antidepressant-induced suicidality? Clin Trials. 2009;6(2):109–118. doi: 10.1177/1740774509102563. [DOI] [PubMed] [Google Scholar]
  43. Wells KB. Treatment research at the crossroads: the scientific interface of clinical trials and effectiveness research. Am J Psychiatry. 1999;156(1):5–10. doi: 10.1176/ajp.156.1.5. [DOI] [PubMed] [Google Scholar]
  44. Westen D, Morrison K. A multidimensional meta-analysis of treatments for depression, panic, and generalized anxiety disorder: an empirical examination of the status of empirically supported therapies. J Consult Clin Psychol. 2001;69(6):875–899. [PubMed] [Google Scholar]
  45. Wisniewski SR, Rush AJ, Nierenberg AA, Gaynes BN, Warden D, Luther JF, Trivedi MH. Can phase III trial results of antidepressant medications be generalized to clinical practice? A STAR*D report. Am J Psychiatry. 2009;166(5):599–607. doi: 10.1176/appi.ajp.2008.08071027. [DOI] [PubMed] [Google Scholar]
  46. Zimmerman M, Morgan TA. The relationship between borderline personality disorder and bipolar disorder. Dialogues Clin Neurosci. 2013;15(2):155. doi: 10.31887/DCNS.2013.15.2/mzimmerman. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES