Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Jul 1.
Published in final edited form as: Bioethics. 2017 May 15;31(6):443–453. doi: 10.1111/bioe.12349

The ethics of clinical trials research in severe mood disorders

Allison C Nugent, Franklin G Miller, Ioline D Henter, Carlos A Zarate
PMCID: PMC5469708  NIHMSID: NIHMS858439  PMID: 28503892

Abstract

Mood disorders, including major depressive disorder (MDD) and bipolar disorder (BD), are highly prevalent, frequently disabling, and sometimes deadly. Additional research and more effective medications are desperately needed, but clinical trials research in mood disorders is fraught with ethical issues. Although many authors have discussed these issues, most do so from a theoretical viewpoint. This manuscript uses available empirical data to inform a discussion of the primary ethical issues raised in mood disorders research. These include issues of consent and decision-making capacity, including patients’ motivations for participating in research. We also address drug withdrawals, placebo controls, and the overall safety of research. Finally, we examine the extant literature for studies discussing potential indirect benefits of clinical trials research to participants. Taken together, the evidence suggests that clinical trials research incorporating drug withdrawals and placebo controls can be conducted safely and ethically, even in patients with severe or treatment-resistant mood disorders. In fact, given the dearth of effective treatment options for this population, it is our opinion that a moral imperative exists to extend the offer of research participation to severely ill or treatment-resistant groups.

Keywords: mood disorders, depression, psychiatry, research, clinical trials, bipolar disorder

INTRODUCTION

Mood disorders, including major depressive disorder (MDD) and bipolar disorder (BD), are highly prevalent, frequently disabling, and sometimes deadly1. Despite the clear and urgent need for continued clinical trials research in mood disorders, such research is fraught with ethical dilemmas, particularly for severely ill populations. The goal of this manuscript is to demonstrate that clinical trials research, including drug withdrawals and placebo controls, can be conducted ethically in research subjects with severe and/or treatment-resistant mood disorders. In fact, we would go further and state that we believe that there is a moral imperative to study these patients, given the lack of treatment options for this group.

MDD is characterized by either depressed mood or anhedonia (inability to experience pleasure), along with other symptoms such as guilt, somatic symptoms, and suicidality. Symptoms must be present for at least two weeks, nearly every day, and result in impaired functionality2. BD involves periods of depression alternating with episodes of either mania (BD-I) or a milder version of mania called hypomania (BD-II). Mania is characterized by abnormally elevated or irritable mood as well as other symptoms such as grandiosity, racing thoughts, risk-taking behavior, and decreased need for sleep3. Several meta-analyses have demonstrated that traditional antidepressants are only marginally more effective than placebo for treating acute depressive episodes4. Furthermore, only one drug, lithium, has ever been developed specifically for BD—and that was in 1949.

Patients categorized as having “severe” mood disorders may fall into several categories, although all will meet criteria for an acute episode of a mood disorder. Some patients may be treatment-resistant, meaning that they did not achieve remission despite adequate trials of some or all available classes of treatment. Other subjects may have high scores on depression rating scales, poor quality of life, and impaired functionality (i.e. inability to work). Patients may also have other psychiatric co-morbidities, histories of physical, emotional, or sexual abuse, or problems with substance abuse or dependence. In practice, these groups frequently overlap. For the purpose of this manuscript, we address acutely ill subjects who meet some or all of these criteria.

Several facets of clinical trials in mood disorders have been decried as unethical due to concerns over decreased decision-making capacity and the risks of symptom exacerbation, including the inclusion of severely ill patients as well as the use of drug withdrawals and placebo controls5. However, despite years of debate, few empirical data actually exist to either support or challenge these concerns. In this context, a recent study by Nugent and colleagues (2016) reported data from 504 research subjects with either MDD or BD who had participated in a clinical trial for mood disorders at the National Institute of Mental Health (NIMH)6. Many of these participants had not responded to prior treatment with one or more approved agents; comorbidities, a history of abuse, and a history of suicidality were also prevalent. Here, we seek to reexamine some of the ethical arguments relating to this type of research in light of this study and other empirical data. While empirical data cannot substitute for principled ethical discourse, we believe it can be a vital adjunct in informing ethical judgment, bridging what is sometimes a large gap between theory and practical reality.

NON-CONTROVERSIAL ETHICAL ISSUES

We begin by enumerating issues related to the ethics of psychiatric research that are generally non-controversial7, as well as other issues that we do not intend to discuss further. First, placebo-controlled trials must have scientific justification for the necessity of the placebo control. Second, all trials, particularly those involving placebo controls, should have adequate safeguards in place to address the potential deterioration of a patient’s condition. We wish to make clear that the drug withdrawals addressed herein do not include withdrawal of effective and/or clinically indicated psychiatric medications. Finally, we do not intend to address research in populations where decision-making capacity is clearly compromised, such as those with psychosis or those who are involuntarily committed.

ISSUES OF CONSENT

Freely given informed consent is one of the ethical cornerstones of current research practice. Informed consent requires that decisional autonomy not be compromised by either external and potentially coercive forces, or by internal forces resulting from illness or impairment8. While researchers can certainly avoid coercive practices, it is more complicated to address issues related to the potential participant’s disorder. We would hypothesize that mood disorders may make potential research participants perceive non-coercive external forces (for instance, a doctor’s non-threatening recommendation) as influential in a way that is difficult to resist (due to, for example, loss of confidence in one’s own opinions if they differ from a trusted authority figure’s).

Decision-making capacity has been operationalized as the presence of four key abilities9. Prospective research participants must be able to both understand relevant information disclosed by investigators as well as appreciate the options in the context of their personal situation. Decision-making capacity also requires the ability to reason and the ability to communicate a choice. Other decision-making abilities specific to depression have been suggested. These include decisional authenticity (the ability to make decisions that would reflect the research participant’s desires in the absence of illness) as well as a minimal level of concern for one’s own wellbeing10. Another issue of potential concern is that patients with mood disorders may engage in catastrophic thinking—that is, believing that they will remain depressed no matter what actions they take—to a degree that compromises decision-making11.

It is generally accepted that all but the most impaired or catatonic participants with mood disorders have the ability to communicate a choice12. Scales such as the MacCAT-CR13—which assess decisional capacity as it relates to appreciation, understanding, and reasoning in a research setting—have been vital to empirical studies on decision-making capacity. A recent review examining studies of decision-making capacity in depression, either in the context of treatment or research, concluded that most subjects with depression have adequate decision-making capacity, although depression appears to have the greatest impact on an individual’s ability to appreciate available options14. Although one study reported that depressed inpatients showed significant impairments in the abilities needed for decision-making capacity compared to inpatients with non-psychiatric illnesses, this report included subjects who had been involuntarily committed and who had other features, such as psychosis15. While these factors would not necessarily disqualify these patients from research, it may be inherently problematic to attempt to voluntarily enroll a subject in research who is in a treatment facility involuntarily, and we do not intend to specifically address this group. Another study found that depressed patients were less likely to volunteer for research studies than non-depressed individuals, suggesting that a lack of appreciation could lead to less, rather than more, engagement in research16.

In this context, perhaps the most appropriate sample was described by Fisher and colleagues17, who evaluated decision-making capacity in a group of 31 patients with MDD or BD-II enrolled in a deep brain stimulation (DBS) trial. Most of these subjects demonstrated adequate decision-making capacity on the MacCAT-CR. Consistent with these findings, a study in bipolar mania found that while many patients had difficulty understanding the goals of research, and distinguishing research from clinical care, most could appreciate that they had an illness that needed medication and could discern the level of risk associated with various research interventions18. The difficulty in distinguishing research from clinical care, known as therapeutic misconception, may be prevalent in psychiatric research19, although this could potentially be due to optimism and misinterpretation of the questionnaires used to assess the phenomenon20.

Also relevant to the discussion of decision-making capacity and consent are the viewpoints of the patients themselves. Surveys of psychiatric patients21 have consistently found that they regard research into mental illness as ethically acceptable, even if patients are experiencing “a lot of emotional pain”22. Interestingly, a study of both clinical investigators and patients with psychiatric illness or serious medical illness found that researchers consistently misjudged important aspects of patients’ decision-making capacity23, and overestimated the degree to which a variety of negative states—such as stress, emotional pain, and despair—influence patient decisions. Patients also report that they are less easily pressured to participate in research than investigators believe24. Although research subjects who are enrolled in research in clinical inpatient units, dedicated research inpatient units, or outpatient clinics are likely to perceive research differently, we know of no studies addressing these differences.

Interestingly, research participants with mood disorders give a wide array of reasons for research participation, many of them altruistic in nature. For example, one study reported that 97.9% of patients with depression endorsed the notion that research would be “helpful for others (future patients)” as a reason for participation25; similarly, in another study, 94.1% of MDD patients reported that “to help others with depression in the future” was an important or very important reason for participation26. Findings in BD are similar27. It should be noted that even after the completion of research, participants with a current or past diagnosis of MDD reported that they would participate again28 and that their research participation was meaningful, even if they experienced intense or unexpected emotional reactions to participation29. Most interestingly, participants in active distress from an adult trauma were even more likely to state that they would participate in research again compared to those without such vulnerability, potentially indicating that open and detailed questioning of sensitive topics is not harmful and may even be beneficial30.

It has been hypothesized that depressed subjects may potentially lack decision-making capacity due to a basic lack of concern for their own well-being31 as well as to hopelessness, though surprisingly little has been written on the confluence of depressive symptoms and decision-making capacity in the context of research. The DSM-532, the diagnostic manual used for the official diagnosis of psychiatric illnesses, lists among the symptoms of depression “feelings of worthlessness or excessive or inappropriate guilt, recurrent thoughts of death, recurrent suicidal ideation without a specific plan, or a suicide attempt or a specific plan for committing suicide”. One could hypothesize that a subject with any degree of suicidality, by virtue of a nihilistic viewpoint, would be more willing to enroll in a study with an unfavorable risk/benefit ratio. However, this is not supported by the literature reviewed above, which indicates that depressed subjects are less likely to engage in research than their non-depressed counterparts33, although levels of suicidal ideation were not reported in that study. In addition, part of the function of any given institutional review board (IRB) is to ensure that research does not expose participants to risks that are excessive compared with potential benefits. Alternatively, hopeless patients may not be motivated to enroll in beneficial research, or may show apathy towards risks on the assumption that they will never get better; this issue has also been discussed in the literature34, though we know of no empirical evidence addressing this concern. Furthermore, extreme hopelessness and/or willingness to assume risk inappropriately should be relatively obvious in any properly conducted informed consent discussion, and could be deemed to invalidate consent. It should also be noted that because suicidal behavior is sometimes impulsive in nature35, at-risk patients may generally possess the necessary decision-making capacity and show only transient impairments associated with the wish to die. Because suicidal patients in the research clinic are actively and voluntarily seeking treatment, it is reasonable to assume that these patients would probably rather have their depression alleviated through successful treatment rather than death. Numerous publications have addressed theoretical ethical issues tied to research into suicidal behavior36, and all have concluded that such research can be done ethically. We know of no studies presenting data relevant to the safety of research in individuals with suicidal ideation, despite the urgent need for research in this area.

If substantial impairment in the abilities required for decision-making capacity were identified across populations with mood disorders, then it might be warranted to declare this a population needing special protections. According to the Common Rule37, which governs all research supported by the US federal government, “children, prisoners, pregnant women, mentally disabled persons, or economically or educationally disadvantaged persons” are vulnerable populations who may be especially susceptible “to coercion or undue influence”. The phrase “mentally disabled” has been narrowly interpreted to encompass those with intellectual disabilities, but has also been more broadly interpreted to include those with psychiatric disorders38. Because the designation of a “vulnerable population” is a regulatory rather than a philosophical distinction, it is somewhat difficult to operationalize. For the purpose of our discussion, we assume that a vulnerable population would have systemic impairments in decision-making capacity, or be in a specific situation that would make them subject to coercion or inappropriate influence39.

We conclude that the available evidence does not support declaring adult subjects with mood disorders to be a “vulnerable population” in this regulatory sense, and doing so may in fact be harmful. The literature reviewed above demonstrates that depressed subjects in general possess the required abilities for decision-making capacity; depressed subjects may also be less susceptible to coercion than their physicians assume, and may choose to participate in research for well-reasoned and frequently altruistic reasons. While we do not intend to suggest that all depressed subjects have sufficient decision-making capacity and freedom from coercion to participate in research, it would be inappropriate to declare the population vulnerable as a whole. Indeed, we would argue that formally classifying those with mood disorders as a vulnerable population could potentially contribute to stigma and devalue the societal contributions made by these individuals. In practice, other subject populations may present some of the same concerns. A patient with terminal cancer may not be clinically depressed but may still be willing to undergo risky phase I trials that have only a very small likelihood of direct benefit. The promise of receiving an unknown (and potentially therapeutic) drug may be enough to convince a dying patient to consent to considerable risk. Likewise, the suffering of those with depression is extremely intense; patients with depression have rated the amount of suffering they experience as on par with being blind, deaf, or dumb40, and severely ill patients rate the state of severe depression as equivalent to or worse than death41. In addition, MDD is associated with increased all-cause mortality, with a three-fold increased risk of death, even after adjusting for a variety of lifestyle risk factors42. Under these circumstances, participation in research trials carrying some risk may be perfectly reasonable, particularly for highly treatment-resistant subjects, even if there is only a small chance of symptomatic improvement.

Nevertheless, it may be desirable to implement particular safeguards to promote informed consent for research participants with mood disorders. For example, consent monitoring by an impartial observer may be useful. Consent monitors who function independently from the research team (for instance, the National Institutes of Health (NIH) use a Human Subjects Protection Unit (HSPU)) may observe study personnel during the informed consent process and intervene if they observe infringements on participant autonomy or impaired decision-making capacity. When indicated, consent monitors can perform formal capacity assessments or maintain oversight of subject participation throughout a research trial.

THE USE OF DRUG WITHDRAWALS

Perhaps the most contentious issue is that of withdrawing depressed subjects from current medications in order to enroll them in a study. We reiterate that we believe it is generally unethical to withdraw psychiatric medications from a patient in remission, or from a patient experiencing a substantial benefit from the drug, solely for the purposes of research43. Our discussion pertains only to the withdrawal of medications in acutely ill participants where the medications in question are, at best, of partial utility.

Although the issues involved in placebo-controlled research and drug withdrawals overlap, we discuss them separately here. Drug withdrawals, in contrast to placebo periods, typically produce no “placebo response” and may produce unpleasant withdrawal symptoms; moreover, subjects are fully informed of their treatment status. Drug withdrawals may be necessary in the context of psychiatric research for several reasons. Some authors, citing generalizability, have stated that clinical trials would be more relevant to clinical practice if new drugs were tested as add-ons to existing treatments44. While add-on treatment is certainly prevalent45, given the number of current drugs in use, and the potential interactions (both harmful and synergistic) between these agents, add-on trials would likely need to involve so many subjects that they would be essentially impossible to conduct. In addition, the absence of a pure placebo group in such trials could also render results essentially uninterpretable, as has been discussed in depth by other authors46. Second, mechanistic studies (conducted to determine an agent’s precise mechanism of action) require that only a single drug be used. Thus—and assuming that a drug-free state is scientifically justified—a discussion of the ethical acceptability of drug withdrawals depends on balancing potential harms and benefits to subjects and the value of the knowledge to be gained from the research.

In this context, the first question of interest is whether drug washouts are harmful. The recent report by Nugent and colleagues examined drug washouts in 225 subjects with MDD or BD who were tapered off at least one medication (some BD subjects had concomitant titration of a mood stabilizer or remained on mood stabilizer monotherapy)47. Overall, 9.9% of subjects entering a drug taper withdrew due to symptomatic worsening48 compared to 5.8% who withdrew due to symptomatic improvement. Of those subjects completing the taper, the average increase in symptoms compared to pre-taper was 4.2% (range: 67% increase in symptoms to 40% decrease in symptoms). Even in subjects tapering to no medications, the mean increase in symptoms was 5.1%.

We know of only one other peer-reviewed study examining drug withdrawals in depressed patients, which found that subjects remained stable or improved slightly on average during a drug withdrawal49. However, it should be noted that in the study by Nugent and colleagues, most drug tapers took place on an inpatient unit with a high staff to participant ratio in conjunction with careful monitoring; thus, the impact of the therapeutic milieu of the unit should not be overlooked. Drug tapers carried out under less intensive circumstances may not produce as favorable results.

The data reviewed above suggest that most research participants will not see a precipitous decline in symptoms in association with drug withdrawal, although there are indeed some individuals who will deteriorate significantly. While a minority of patients enrolled in drug-free trials may experience significantly more psychological pain than they would had they not enrolled in research, we would argue that as long as subjects are adequately informed of these risks, and appropriate safeguards are in place, individuals with mood disorders should be allowed to weigh the risks of drug-free periods against the potential benefit to themselves and to society and choose whether or not to participate. Nevertheless, there will likely be some cases where additional safeguards, such as decision-making capacity assessments, may be warranted given that drug withdrawals present a risk without guaranteeing a benefit. IRBs may also play a key role when evaluating risks by considering the suitability of the specific population and the existing safeguards in place.

Although research subjects may experience a further decline in mood following withdrawal of medication, this is quite different than precipitating relapse in a remitted patient. In addition, while ample research has explored the dangers of repeated relapse on subsequent course of illness50, there are essentially no empirical data investigating the long-term consequences of symptom exacerbation within an already ongoing acute episode.

Notably, the safety of drug-free research depends on the presence of adequate safeguards. If a subject seriously deteriorates, becomes acutely suicidal, or represents harm to others, it is morally imperative to remove them from the drug taper, potentially against their wishes. Procedures and cut-off scores requiring withdrawal from the study should be clearly outlined in research protocols and presented to the research subject during the consent process. In addition, particular care must be taken to ensure that research participants feel free to withdraw from research at any time. Even in the absence of external coercion, it is our experience that some particularly conscientious research participants may feel that they are “disappointing” researchers if they drop out of a trial early, or may be extremely motivated to remain in research in order to have access to the experimental therapy. In situations such as these, independent monitors who observe the clinical state of the patient throughout the research trial can ensure that autonomy is maintained over the entire duration of the study.

It should also be noted that psychiatrists have contended that there are sometimes indirect benefits to drug withdrawals that may be conferred on research participants51. Polypharmacy, the prescription of multiple drugs at the same time that usually entails adding drugs on top of other drugs that were only partially effective, is not well-studied. Although some studies of single drug augmentation do exist (most notably, the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) study52) it is unclear whether augmentation or washout and switching is a better strategy53. In addition, we know of no studies investigating more than two drugs administered concomitantly. Certainly, when a severely ill, treatment-resistant subject comes into a clinic taking six different medications with poorly understood interactions and side effects, it is likely that the patient could benefit from starting from a clean slate, building a new and more effective regimen from the ground up. However, research subjects are not chosen on the basis of whether a washout would be clinically useful to that subject; they are chosen for a drug washout simply because they are currently ill, taking medications, and wish to enter a trial that requires them to be unmedicated prior to randomization. In other words, in the research setting, the drug washout is included for scientific reasons and is not intended to produce clinical benefit. Thus, while drug washouts may constitute a benefit for some subjects, they will not necessarily benefit all subjects.

THE USE OF PLACEBO

A vast body of literature discusses the use of placebo in psychiatric trials54. Over the past few decades, a subset of authors have rejected the use of placebos in research when effective standard treatment exists, arguing that it is a violation of the Declaration of Helsinki55. This declaration states that investigational interventions must be tested against standard treatments, except where no proven treatment exists, or where placebo is methodologically required, and where subjects receiving placebo will not be subject to risk of “serious or irreversible harm”56. While testing new interventions against standard treatments may be a methodologically sound practice for many diseases/disorders, the situation in mood disorders is far less clear. First, placebo response is generally high in mood disorders57. Second, the use of placebo is frequently defended on scientific and methodological grounds; these issues have been discussed in depth elsewhere58, and so here we confine ourselves to a discussion of the ethical, rather than the scientific, concerns of placebo use, with one exception. In particular, studies criticizing placebo-controlled trials often fail to address the need for studies aimed at determining the mechanism of action of a drug through biological measures (often referred to as “mechanistic” studies). As noted above, placebos are generally required in mechanistic studies in order to control for the potential neurobiological effects of mood and in order to observe changes induced specifically by the drug. As mentioned previously, drugs currently used to treat depression are frequently ineffective, and our understanding of exactly how they work is poor. Studies of ketamine and the glutamatergic system59 have provided the first new target for antidepressant medications since the advent of Prozac in 1987, and additional new targets are needed. Without further elucidation of antidepressant mechanisms of action, new and more focused targets for drug intervention cannot be identified, and more efficacious treatments cannot be developed.

As mentioned above, most ethical arguments against placebo controls in mood disorders center around the fact that subjects in placebo arms do not receive standard, effective treatment. In the absence of treatment, critics have suggested that acute decompensation is not the only risk, and that worsening of symptoms can cause irreversible damage and result in a more refractory and difficult to treat disease60. While adequate safeguards can ensure that subjects who experience a decline in mood are promptly withdrawn from the trial, some argue that by this point, decompensation has occurred and the damage is already done61. It has also been suggested that severe symptom exacerbation can produce not only neurobiological harm but also, potentially, interpersonal and economic harm62. Because of these concerns, some critics have stated that the harms of placebo arms outweigh any potential benefits of the study, and that trials with an active control would serve to both provide patients with treatment and investigators with comparative information on efficacy and safety63.

We believe these arguments to be unconvincing in light of the available empirical data. With regard to the idea that research subjects forgo standard treatment during their acute depressive episode, we again note that available treatment options for acute depression, on average, are only modestly effective; only one-third of patients with depression will respond to their first antidepressant, and approximately two-thirds will respond only after trying several antidepressant types, leaving a full third of patients with suboptimal treatment options64. Even for those who do respond, a considerable lag of several weeks exists between the first dose and full response, and residual symptoms are extremely common65. If currently approved treatments were vastly more effective than placebo, asking research subjects to forsake standard treatment might indeed be unethical. In fact, however, approved drugs show aggregate improvement over placebo by only a few points on standard depression rating scales66 during the treatment of acute episodes, making this argument far less compelling. In addition, although some critics of placebo use contend that placebo exposes research participants to the possibility of symptom exacerbation and long-term harm, empirical data have in fact shown that, on average, patients receiving placebos often show significant improvement in their mood symptoms. Other authors have demonstrated that the risk of suicide is no greater in placebo arms, and that subjects receiving placebo showed improvement in their mood scores67. One meta-analysis of placebo effect in 75 studies published between 1981 and 2000 demonstrated that between 10% and more than 50% of people randomized to placebo showed a greater than 50% reduction in their depressive symptoms68. A more recent study examining 96 studies published between 1980 and 2005 found similar results, reporting a large placebo response69. Because there is evidence of systematic publication bias, these meta-analyses potentially underestimate the magnitude of placebo response70. In fact, most methodological issues with non-placebo-controlled studies arise precisely because so many depressed patients respond to placebo.

In the recent study by Nugent and colleagues, those who completed the placebo arm of a traditional parallel trial improved by 33.4%, nearly equivalent to the symptom improvement observed in those completing an active arm (31.5%, pooled across all trials)71. It is true, however, that more of the subjects receiving placebo withdrew due to worsening symptoms than those in active arms (26.7% vs. 14%). Although the number of subjects with data was quite small, the decline in mood experienced from screening to withdrawal did not significantly differ between those assigned to placebo (worsening by 18%, N=10) and those assigned to active treatment (worsening by 25%, N=6). Thus, while adverse outcomes may be more likely in the placebo group, the actual magnitude of the risk is likely the same regardless of randomization. Taken together, we believe that these data suggest that if autonomy is sufficiently respected such that research participants feel free to withdraw at any point, and adequate safeguards and automatic withdrawal criteria are well-defined, placebo arms do not carry significantly more risk than treatment arms. Combined with the evidence that most patients with mood disorders are fully capable of understanding research trials, as well as the fact that most psychiatric research participants consistently give altruistic reasons for study participation72, these trials do not seem to violate any ethical norms. Indeed, in our opinion, purposefully excluding a class of participants from research on the basis of misconceptions or suspicions not supported by the empirical evidence would be paternalistic and unjust. Furthermore, as many as a third of patients with depression may be refractory to standard treatment73. If a patient has exhausted all viable approved treatment options, the only remaining treatments are experimental. We would contend that we have an ethical obligation to offer research enrollment to treatment-resistant subjects, an idea discussed in greater detail below.

OVERALL SAFETY OF RESEARCH

Other than the recent study by Nugent and colleagues74, we know of no studies in mood disorders that have specifically examined the cumulative effects of research participation. That study examined every subject for whom the investigators had a rating score from the period of screening (before any drug taper or withdrawals) until their exit from research (either by withdrawal or completion), and found that overall, subjects’ mood scores improved by 26.4%; this included trials of drugs with only transient effects, such that patients would be expected to relapse by the conclusion of the trial.

Other findings in the literature similarly support the general safety of research. Notably, one large analysis of antidepressant clinical trial data submitted to the FDA reviewed data from 45 studies75 and found that 34 of 19,639 patients committed suicide during trials for a yearly rate of 0.76%, while 130 patients attempted suicide for a yearly rate of 2.9%. A follow-up analysis found 77 completed suicides in 48,277 depressed research participants, which equated to a rate of 0.59% per year76. The baseline rate of suicide in MDD is difficult to estimate; one observational study found that 36 out of 249 patients (14.4%) with MDD attempted suicide at least once over a five-year follow-up period77, with most attempts occurring during symptomatic phases, which was a relatively small fraction of the five-year interval. This would seem to indicate that in terms of suicide risk, research is relatively safe, likely due to the intensive monitoring and follow-up associated with clinical trials. Although there is a dearth of information available regarding the overall impact of research participation on mood, the extant data reviewed above suggest that research does not, in general, have deleterious effects.

PARTICIPANT-PERCEIVED BENEFITS

Several studies have attempted to assess the possible indirect benefits of research. One active-control trial of two antidepressants assessed reasons for participation and found that 56.5% of patients reported that the belief they would get better medical care was an important or very important reason for their participation, and 16.6% of respondents endorsed free prescription medication as an important or very important reason for participation78. In another study, acutely manic patients with BD agreed that free medication would motivate them to participate in a hypothetical trial, while asymptomatic patients agreed that participation might improve their care79. Another recent meta-analysis showed that access to services was a major theme supporting research participation, suggesting that trials were viewed as a “resource” by participants, particularly in areas where local resources were lacking, or where clinicians felt that they had nothing else to offer their patients80. Participation in a research trial, however, is not—and should never be—considered a substitute for medical care. Nevertheless, we note that although research is not clinical care, it would be naïve not to consider that access to mental health care is difficult in the United States for many due to the limited availability of providers, the affordability of health insurance, and the cost of drugs81. A 2003 cost-benefit analysis estimated that participating in a clinical trial for mood disorders saves uninsured patients a total of $408.10, excluding any payment for participating, at the cost of 24% efficacy compared to standard care82. Notably, this calculation only assumed eight weeks of treatment and four to six treatment appointments. The true cost-benefit ratio may be significantly more favorable, given that many clinical trials offer no- or low-cost standard care following trial participation. Research also frequently involves inpatient stays throughout the duration of the study, during which participants may have daily supportive contact with nursing staff and benefit from the therapeutic milieu of the unit, even in the absence of active treatment. The power of the therapeutic milieu is supported by the recent study by Nugent and colleagues83, which found a general improvement in mood during the course of the trial, regardless of treatment assignment. In addition, the standard clinical care provided post-research may also take place on an inpatient unit, with participants generally not discharged until symptoms are significantly improved. While the standard clinical care provided following research is not typically considered to be a benefit of research (because equivalent care is available in the community), this notion overlooks the fact that access to care in the community is not uniform, and that even suboptimal community care is far more costly for the patient. In reality, inpatient care for several months would likely be prohibitively expensive and non-reimbursable by insurance. For a potential subject with poor access to community mental health care, research may not only be a reasonable option, but a preferable one, even when subjects are fully cognizant of the differences between research and standard clinical care, and are fully aware that they may potentially spend a significant amount of time on a placebo treatment.

We do not in any way propose that these factors should be explicitly considered as benefits by IRBs in the context of prospective risk-benefit assessment for research protocols, or that the uninsured should be encouraged to participate in trials. We are also not the first authors to suggest that there are “indirect” benefits to research participation; others have noted that research participation may confer improved clinical care, expert assessments, and intensified clinical and psychosocial management available at low or no cost84. Instead, our motivation in discussing these indirect benefits is to emphasize that research participation may be a perfectly reasonable course of action by an ill patient, even for those without particularly altruistic motivations. Indeed, for patients who are profoundly treatment-resistant and/or severely ill, research participation may the most beneficial course of action. Although an extremely treatment-resistant subject would risk being randomized to a placebo group, this may be no worse for them than remaining on an ineffective medication regimen that may have undesirable side effects. Indeed, as we stated earlier, if a potential subject has not responded to treatment with approved agents, then it would seem that we have a moral imperative to allow these persons access to experimental agents available only in the context of research trials. There are obvious constraints that must be placed upon this obligation, such as ensuring that the subject is otherwise appropriate for research, maintaining the scientific integrity of the research, and not placing undue burdens upon the researcher.

CONCLUSIONS

This manuscript discusses the ethical issues involved in clinical trials research in mood disorders. In particular, we discuss the case for allowing, and even encouraging, research in severely ill and treatment-resistant participants. With regard to issues of consent and decision-making capacity, the literature seems to indicate that most depressed patients do have sufficient decision-making capacity, and that remaining concerns can be addressed with routine consent monitoring. Similarly, a review of the available literature on drug withdrawals and placebo controls indicates that while risks are present, they may be less than feared, and may also be minimized by imposing appropriate safeguards. In addition, the available data indicate that most participants will show improvement in their symptoms over the course of their research participation. In conjunction with both altruistic and self-care motives that may accompany research participation, the extant evidence suggests that research may be a reasonable option for those with severe mood disorders. In fact, the failure to extend research to severely ill and/or highly treatment resistant groups may be unwarranted and unjust.

Ethical research in patients with mood disorders requires a balance between the scientific integrity of research, potentially inducing patient discomfort, and protecting the safety and well-being of patients. As we progress toward the goal of improved treatment for these disorders, and complex ethical discussions continue, we hope that the knowledge gleaned from empirical research and from active discussion of these important issues can continue to enhance the moral integrity of psychiatric research.

Acknowledgments

Role of Funding Source

Funding for this work was supported by the Intramural Research Program at the National Institute of Mental Health, National Institutes of Health (IRP-NIMH-NIH; ZIA MH002927-05), by a NARSAD Independent Investigator Award to Dr. Zarate, and by a Brain and Behavior Mood Disorders Research Award to Dr. Zarate. Dr. Zarate is listed as a co-inventor on a patent for the use of (2R,6R)-hydroxynorketamine, (S)-dehydronorketamine, and other stereoisomeric dehydro and hydroxylated metabolites of (R,S)-ketamine metabolites in the treatment of depression and neuropathic pain. Dr. Zarate is listed as co-inventor on a patent application for the use of (2R,6R)-hydroxynorketamine and (2S,6S)-hydroxynorketamine in the treatment of depression, anxiety, anhedonia, suicidal ideation, and post-traumatic stress disorders; he has assigned his patent rights to the U.S. government but will share a percentage of any royalties that may be received by the government.

Footnotes

Disclosures: All other authors have no conflict of interest to disclose, financial or otherwise.

References

  • 1.Whiteford HA, et al. Global burden of disease attributable to mental and substance use disorders: Findings from the Global Burden of Disease Study 2010. Lancet. 2013;382:1575–1586. doi: 10.1016/S0140-6736(13)61611-6. [DOI] [PubMed] [Google Scholar]
  • 2.APA. Diagnostic and Statistical Manual of Mental Disorders. Fifth. Washington, DC: American Psychiatric Association; 2013. DSM-5. [Google Scholar]
  • 3.Ibid.
  • 4.Kirsch I, et al. Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. PLoS Med. 2008;5:e45. doi: 10.1371/journal.pmed.0050045. [DOI] [PMC free article] [PubMed] [Google Scholar]; Kirsch I, et al. The Emperor’s New Drugs: An Analysis of Antidepressant Medication Data Submitted to the U.S. Food and Drug Administration. Prevention and Treatment. 2002;5 [Google Scholar]
  • 5.Rothman KJ, Michels KB. The continuuing unethical use of placebo controls. N Engl J Med. 1994;331:394–398. doi: 10.1056/NEJM199408113310611. [DOI] [PubMed] [Google Scholar]; Friend WC, Weiler C. Re: CCNP position paper on the use of placebos in psychiatry. J Psychiatry Neurosci. 1996;21:354–359. [PMC free article] [PubMed] [Google Scholar]; Lehrman NS, Sharav VH. Ethical problems in psychiatric research. J Ment Health Adm. 1997;24:227–250. doi: 10.1007/BF02898516. [DOI] [PubMed] [Google Scholar]
  • 6.Nugent AC, et al. Safety of research into severe and treatment-resistant mood disorders: analysis of outcome data from 12 years of clinical trials at the US National Institute of Mental Health. Lancet Psychiatry. 2016 doi: 10.1016/S2215-0366(16)00006-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Charney DS, et al. National Depressive and Manic-Depressive Association consensus statement on the use of placebo in clinical trials of mood disorders. Arch Gen Psychiatry. 2002;59:262–270. doi: 10.1001/archpsyc.59.3.262. [DOI] [PubMed] [Google Scholar]; Pinals DA, Appelbaum PS. Ethical Aspects of Neuropsychiatric Research with Human Subjects. In: Davis KL, et al., editors. Neuropsychopharmacology: The Fifth Generation of Progress. American College of Neuropsychopharmacology; 2002. [Google Scholar]; Rosenstein DL, Miller FG. Ethical considerations in psychopharmacological research involving decisionally impaired subjects. Psychopharmacol Berl. 2003;171:92–97. doi: 10.1007/s00213-003-1503-1. [DOI] [PubMed] [Google Scholar]; DuVal G. Ethics in psychiatric research: study design issues. Can J Psychiatry. 2004;49:55–59. doi: 10.1177/070674370404900109. [DOI] [PubMed] [Google Scholar]; Edwards JG. Ethics of placebo. Hum Psychopharmacol Clin. 1989;4:235–236. [Google Scholar]; Miller FG. Placebo-controlled trials in psychiatric research: an ethical perspective. Biol Psychiatry. 2000;47:707–716. doi: 10.1016/s0006-3223(00)00833-7. [DOI] [PubMed] [Google Scholar]
  • 8.Beauchamp TL, Childress JF. Principles of Biomedical Ethics. 6. Oxford, UK: Oxford University Press; 2008. [Google Scholar]
  • 9.Grisso T, Appelbaum PS. The MacArthur Treatment Competence Study. III: Abilities of patients to consent to psychiatric and medical treatments. Law Hum Behav. 1995;19:149–174. doi: 10.1007/BF01499323. [DOI] [PubMed] [Google Scholar]
  • 10.Elliott C. Caring about risks. Are severely depressed patients competent to consent to research? Arch Gen Psychiatry. 1997;54:113–116. doi: 10.1001/archpsyc.1997.01830140021003. [DOI] [PubMed] [Google Scholar]
  • 11.Halpern J. When concretized emotion-belief complexes derail decision-making capacity. Bioethics. 2012;26:108–116. doi: 10.1111/j.1467-8519.2010.01817.x. [DOI] [PubMed] [Google Scholar]
  • 12.Grisso & Appelbaum.
  • 13.Appelbaum PS, et al. Competence of depressed patients for consent to research. Am J Psychiatry. 1999;156:1380–1384. doi: 10.1176/ajp.156.9.1380. [DOI] [PubMed] [Google Scholar]
  • 14.Hindmarch T, et al. Depression and decision-making capacity for treatment or research: a systematic review. BMC Med Ethics. 2013;14:54. doi: 10.1186/1472-6939-14-54. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Grisso & Appelbaum.
  • 16.Cohen BJ, et al. Willingness and competence of depressed and schizophrenic inpatients to consent to research. J Am Acad Psychiatry Law. 2004;32:134–143. [PubMed] [Google Scholar]
  • 17.Fisher CE, et al. The ethics of research on deep brain stimulation for depression: decisional capacity and therapeutic misconception. Ann N Y Acad Sci. 2012;1265:69–79. doi: 10.1111/j.1749-6632.2012.06596.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Misra S, et al. Appreciation of research information in patients with bipolar disorder. Bipolar Disord. 2008;10:635–646. doi: 10.1111/j.1399-5618.2008.00609.x. [DOI] [PubMed] [Google Scholar]
  • 19.Appelbaum PS, et al. The therapeutic misconception: informed consent in psychiatric research. Int J Law Psychiatry. 1982;5:319–329. doi: 10.1016/0160-2527(82)90026-7. Fisher, et al. [DOI] [PubMed] [Google Scholar]
  • 20.Kim SY, et al. Research participants’ “irrational” expectations: common or commonly mismeasured? IRB. 2013;35:1–9. [PubMed] [Google Scholar]; Kim SY, et al. Could the High Prevalence of Therapeutic Misconception Partly Be a Measurement Problem? 2015;37:11–18. Ibid. [PubMed] [Google Scholar]
  • 21.Edlinger M, et al. Attitudes of patients with schizophrenia and depression towards psychiatric research. Psychiatry Res. 2010;177:172–176. doi: 10.1016/j.psychres.2008.12.010. [DOI] [PubMed] [Google Scholar]; Roberts LW, Kim JP. Giving voice to study volunteers: comparing views of mentally ill, physically ill, and healthy protocol participants on ethical aspects of clinical research. J Psychiatr Res. 2014;56:90–97. doi: 10.1016/j.jpsychires.2014.05.007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Roberts & Kim.
  • 23.Roberts LW, Kim JP. Do investigators understand ethically-important perspectives of clinical research participants? A ‘piggy-back’ study of attunement and alignment in serious illness research. J Psychiatr Res. 2014;52:36–43. doi: 10.1016/j.jpsychires.2014.01.012. [DOI] [PubMed] [Google Scholar]
  • 24.Roberts & Kim. Giving voice to study volunteers: comparing views of mentally ill, physically ill, and healthy protocol participants on ethical aspects of clinical research.
  • 25.Edlinger, et al.
  • 26.Tallon D, et al. Involving patients with depression in research: survey of patients’ attitudes to participation. Br J Gen Pract. 2011;61:134–141. doi: 10.3399/bjgp11X567036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Misra, et al.
  • 28.Tallon, et al,; Widom CS, Czaja SJ. Reactions to research participation in vulnerable subgroups. Account Res. 2005;12:115–138. doi: 10.1080/08989620590957193. [DOI] [PubMed] [Google Scholar]
  • 29.Widom & Czaja.
  • 30.Ibid.
  • 31.Elliott.
  • 32.APA.
  • 33.Cohen, et al.
  • 34.Halpern.
  • 35.Simon TR, et al. Characteristics of impulsive suicide attempts and attempters. Suicide Life Threat Behav. 2001;32(Suppl 1):49–59. doi: 10.1521/suli.32.1.5.49.24212. [DOI] [PubMed] [Google Scholar]
  • 36.Pearson JL, et al. Intervention research with persons at high risk for suicidality: safety and ethical considerations. J Clin Psychiatry. 2001;62(Suppl 25):17–26. [PubMed] [Google Scholar]; Fisher CB, et al. Ethical issues in including suicidal individuals in clinical research. IRB. 2002;24:9–14. [PubMed] [Google Scholar]
  • 37.US Department of Health and Human Services. Code of Federal Regulations - Title 45 Public Welfare CFR 46. Washington, DC: 2009. [Google Scholar]
  • 38.For example, a commissioned report by the National Bioethics Advisory Commission seems to suggest that psychiatric patients, while not technically falling under the Common Rule vulnerable populations clause, may nonetheless be vulnerable and subject to special protections.; National Bioethics Advisory Commission. Research involving persons with mental disorders that may affect decisionmaking capacity: Report and recommendations of the National Bioethics Advisory Commission. Rockville, MD: National Bioethics Advisory Commission; 1998. [Google Scholar]
  • 39.Note that just as prisoners are a vulnerable population because their involuntary imprisonment makes them susceptible to undue or inappropriate influences, involuntarily committed psychiatric patients may also be uniquely vulnerable, although this should not be generalized to depressed patients as a whole.
  • 40.Kim SY. Benefits and burdens of placebos in psychiatric research. Psychopharmacol (Berl) 2003;171:13–18. doi: 10.1007/s00213-003-1458-2. [DOI] [PubMed] [Google Scholar]
  • 41.Revicki DA, Wood M. Patient-assigned health state utilities for depression-related outcomes: differences by depression severity and antidepressant medications. J Affect Dis. 1998;48:25–36. doi: 10.1016/s0165-0327(97)00117-1. [DOI] [PubMed] [Google Scholar]
  • 42.Lasserre AM, et al. Clinical and course characteristics of depression and all-cause mortality: A prospective population-based study. J Affect Disord. 2016;189:17–24. doi: 10.1016/j.jad.2015.09.010. [DOI] [PubMed] [Google Scholar]
  • 43.Note that it can be very difficult to determine whether a patient is receiving any benefit from a particular drug if they are still experiencing significant residual symptoms, given that mood disorder symptoms tend to wax and wane over time.
  • 44.Rush AJ, et al. Methodological issues in developing new acute treatments for patients with bipolar illness. Biol Psychiatry. 2000;48:615–624. doi: 10.1016/s0006-3223(00)00898-2. [DOI] [PubMed] [Google Scholar]
  • 45.Glezer A, et al. Polypharmacy prevalence rates in the treatment of unipolar depression in an outpatient clinic. J Affect Dis. 2009;117:18–23. doi: 10.1016/j.jad.2008.11.016. [DOI] [PubMed] [Google Scholar]
  • 46.Baldwin D, et al. Placebo-controlled studies in depression: necessary, ethical and feasible. Eur Arch Psychiatry Clin Neurosci. 2003;253:22–28. doi: 10.1007/s00406-003-0400-2. [DOI] [PubMed] [Google Scholar]
  • 47.Nugent, et al.
  • 48.Any reason for an early exit from the study pertaining to the intolerability of depressive symptoms (whether or not symptoms were objectively increased or were stable) was labeled as withdrawal due to symptomatic worsening.
  • 49.Grunebaum MF, et al. Clinical impact of a 2-week psychotropic medication washout in unipolar depressed inpatients. J Affect Disord. 2003;75:291–296. doi: 10.1016/s0165-0327(02)00168-4. [DOI] [PubMed] [Google Scholar]
  • 50.Kessing LV, et al. Course of illness in depressive and bipolar disorders. Naturalistic study, 1994–1999. Br J Psychiatr. 2004;185:372–377. doi: 10.1192/bjp.185.5.372. [DOI] [PubMed] [Google Scholar]
  • 51.Hoffman DA, et al. Polypharmacy or medication washout: an old tool revisited. Neuropsychiatr Dis Treat. 2011;7:639–648. doi: 10.2147/NDT.S24375. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 52.Rush AJ, et al. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: a STAR*D report. Am J Psychiatry. 2006;163:1905–1917. doi: 10.1176/ajp.2006.163.11.1905. [DOI] [PubMed] [Google Scholar]
  • 53.Gaynes BN, et al. Treating depression after initial treatment failure: directly comparing switch and augmenting strategies in STAR*D. J Clin Psychopharmacol. 2012;32:114–119. doi: 10.1097/JCP.0b013e31823f705d. [DOI] [PubMed] [Google Scholar]
  • 54.Charney, et al,; Khan A, et al. Symptom reduction and suicide risk in patients treated with placebo in antidepressant clinical trials: an analysis of the Food and Drug Administration database. 2000;57:311–317. doi: 10.1001/archpsyc.57.4.311. Ibid. [DOI] [PubMed] [Google Scholar]; Temple R, Ellenberg SS. Placebo-controlled trials and active-control trials in the evaluation of new treatments. Part 1: ethical and scientific issues. Ann Intern Med. 2000;133:455–463. doi: 10.7326/0003-4819-133-6-200009190-00014. [DOI] [PubMed] [Google Scholar]; Walsh BT, et al. Placebo response in studies of major depression: variable, substantial, and growing. JAMA. 2002;287:1840–1847. doi: 10.1001/jama.287.14.1840. [DOI] [PubMed] [Google Scholar]
  • 55.World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. 2013;310:2191–2194. doi: 10.1001/jama.2013.281053. Ibid. [DOI] [PubMed] [Google Scholar]
  • 56.Ibid.
  • 57.Walsh BT, et al. Placebo response in studies of major depression: variable, substantial, and growing. 2002;287:1840–1847. doi: 10.1001/jama.287.14.1840. Ibid. [DOI] [PubMed] [Google Scholar]
  • 58.For example, see Baldwin, et al.. Proponents of placebo argue that it is necessary to demonstrate the validity of the trial, given that trials with a placebo, an approved drug, and an experimental drug may fail to show that the approved drug is superior to placebo, potentially indicating a “failed trial.”
  • 59.Niciu MJ, et al. Glutamate receptor antagonists as fast-acting therapeutic alternatives for the treatment of depression: ketamine and other compounds. Annu Rev Pharmacol Toxicol. 2014;54:119–139. doi: 10.1146/annurev-pharmtox-011613-135950. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 60.Glass KC, Waring D. The physician/investigator’s obligation to patients participating in research: the case of placebo controlled trials. J Law Med Ethics. 2005;33:575–585. doi: 10.1111/j.1748-720x.2005.tb00520.x. [DOI] [PubMed] [Google Scholar]
  • 61.Ibid.
  • 62.Baldwin, et al.
  • 63.Vieta E, Cruz N. Head to head comparisons as an alternative to placebo-controlled trials. Eur Neuropsychopharmacol. 2012;22:800–803. doi: 10.1016/j.euroneuro.2011.11.011. [DOI] [PubMed] [Google Scholar]
  • 64.Trivedi MH, et al. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: implications for clinical practice. Am J Psychiatry. 2006;163:28–40. doi: 10.1176/appi.ajp.163.1.28. [DOI] [PubMed] [Google Scholar]
  • 65.Gartlehner G, et al. Second-Generation Antidepressants in the Pharmacologic Treatment of Adult Depression: An Update of the 2007 Comparative Effectiveness Review. Rockville (MD): 2011. [PubMed] [Google Scholar]; Paykel ES. Partial remission, residual symptoms, and relapse in depression. Dialogues Clin Neurosci. 2008;10:431–437. doi: 10.31887/DCNS.2008.10.4/espaykel. [DOI] [PMC free article] [PubMed] [Google Scholar]; Pigott HE, et al. Efficacy and effectiveness of antidepressants: current status of research. Psychother Psychosom. 2010;79:267–279. doi: 10.1159/000318293. [DOI] [PubMed] [Google Scholar]
  • 66.Kirsch, et al, Kirsch, et al.
  • 67.Khan, et al.
  • 68.Walsh, et al.
  • 69.Rief W, et al. Meta-analysis of the placebo response in antidepressant trials. J Affect Disord. 2009;118:1–8. doi: 10.1016/j.jad.2009.01.029. [DOI] [PubMed] [Google Scholar]
  • 70.Turner EH, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358:252–260. doi: 10.1056/NEJMsa065779. [DOI] [PubMed] [Google Scholar]
  • 71.Nugent, et al.
  • 72.Edlinger, et al, Tallon, et al.
  • 73.Trivedi, et al.
  • 74.Nugent, et al.
  • 75.Khan, et al,; Khan A, et al. Suicide rates in clinical trials of SSRIs, other antidepressants, and placebo: Analysis of FDA reports. Am J Psychiatry. 2003;160:790–792. doi: 10.1176/appi.ajp.160.4.790. [DOI] [PubMed] [Google Scholar]
  • 76.Khan, et al.
  • 77.Holma KM, et al. Incidence and predictors of suicide attempts in DSM-IV major depressive disorder: a five-year prospective study. 2010;167:801–808. doi: 10.1176/appi.ajp.2010.09050627. Ibid. [DOI] [PubMed] [Google Scholar]
  • 78.Tallon, et al.
  • 79.Misra, et al.
  • 80.Hughes-Morley A, et al. Factors affecting recruitment into depression trials: Systematic review, meta-synthesis and conceptual framework. J Affect Disord. 2015;172:274–290. doi: 10.1016/j.jad.2014.10.005. [DOI] [PubMed] [Google Scholar]
  • 81.Mojtabai R. Unmet need for treatment of major depression in the United States. Psychiatr Serv. 2009;60:297–305. doi: 10.1176/ps.2009.60.3.297. [DOI] [PubMed] [Google Scholar]
  • 82.Kim SY, Holloway RG. Burdens and benefits of placebos in antidepressant clinical trials: a decision and cost-effectiveness analysis. Am J Psychiatry. 2003;160:1272–1276. doi: 10.1176/appi.ajp.160.7.1272. [DOI] [PubMed] [Google Scholar]
  • 83.Nugent, et al.
  • 84.Roberts LW, et al. Placebos and paradoxes in psychiatric research: an ethics perspective. Biol Psychiatry. 2001;49:887–893. doi: 10.1016/s0006-3223(01)01111-8. ibid, ibid, ibid. [DOI] [PubMed] [Google Scholar]

RESOURCES