Threats to Benefits: Assessing Knowledge Production in Nonhuman Models of Human Neuropsychiatric Disorders

A key finding from the Work Group deliberations of the Hastings Center’s project “Actionable Ethics Oversight of Human-Nonhuman Chimeric Research” is that more analysis is needed on what it means for chimeric research to be scientifically justified and how to assess the benefits of such research. This finding is supported by a literature review of how benefits are typically understood in publications on harm-benefit analyses of experiments involving nonhuman animals.¹ The same literature review called for deeper analysis of benefits and how they are assessed.

This commentary responds to this gap as it applies to human-nonhuman chimeric research that seeks to model human neuropsychiatric disorders such as schizophrenia, obsessive-compulsive disorder, depression, or autism. This is not a merely futuristic or theoretical concern. The recent National Academies’ Report The Emerging Field of Human Neural Organoids, Transplants, and Chimeras specifically points out the promise of human neural transplants and chimeric models to ameliorate suffering due to autism spectrum disorders, depression, and schizophrenia.² Additionally, recent scientific papers report creation of chimeric organisms to model – and treat – neuropsychiatric disease.³ Papers that report chimeric models of Alzheimer’s disease indicate interest and progress among the neuroscientific community in chimeric models of brain disorders generally.⁴ The benefits to humans who suffer – now and in the future - from devastating neuropsychiatric disorders are supposed to justify the harms imposed upon nonhumans used in research and humans affected by that research. Benefits may include improved understanding of underlying pathology, preventative measures, and new treatments. But various factors affecting studies’ results and conclusions may threaten the production of benefits, and thus the underlying ethical justification for this research.⁵

After providing a general framework for classifying the benefits of biomedical research, I will focus on two factors that directly impact – and threaten – the production of knowledge in research that models neuropsychiatric disorders. Given that nearly 90% of behavioral neuroscience results fail to translate to humans,⁶ an understanding of these threats must be included in the ethical analysis of chimeric research. Joseph Garner, a scientist critical of current methods for developing nonhuman models of human disease, writes, “At the end of the day, the failure of animal results to translate is arguably the greatest laboratory animal welfare issue for our day.”⁷ For a project that is so centrally focused on animal welfare, the conceptual issues surrounding how to understand benefits and the practical problems with benefits’ assessment and production demand attention.

Classifying Benefits

The benefits of research are myriad. There’s the direct or primary benefit of performing this experiment, with the proximal outcome of knowledge production, and more distant hoped-for practical impacts on human health.⁸ Then there are indirect, sometimes called secondary, benefits that don’t have to do with knowledge production or the impact of knowledge production but rather capture other benefits of research, e.g. economic benefits to institutions from external funding and reputational benefits to investigators. (See Table 1)

Table 1.

Classifying benefits of research involving nonhuman animals

	Proximal	Distant
Direct/Primary	Knowledge gained in this experiment	Impact on human health; development of new therapeutics or interventions
Indirect/Secondary	External funding for research supports research institute	Professional reputation of investigator

There is consensus in the literature that indirect benefits should not alone qualify as sufficient reasons for conducting research involving nonhumans absent a compelling scientific rationale. In other words, these indirect benefits should not count in the harm-benefit analysis for a particular experiment.⁹ Indirect benefits may influence decision-makers, however, given the prospect of both institutional and professional advancement. Review committees tasked with approving and overseeing animal research comprise mostly employees of the institutions that stand to benefit from external funding for their research programs and close colleagues of investigators proposing to use nonhumans in their research.

Though a matter of some debate,¹⁰ both proximal and more distant direct benefits typically are invoked and included in the ethical justification of nonhuman animal use in general, and in the use of nonhuman animals in human-nonhuman chimeric research specifically. For example, de los Angeles et al. write in defense of modeling human neuropsychiatric disorders in nonhuman primates: “Currently available treatment options are often fruitless. Failure rates for experimental central nervous system drugs are higher compared with other classes of drugs. One possible roadblock is the inadequate quality of existing animal models, which impedes elucidation of disease mechanisms and development of new treatments.”¹¹ This rationale is taken as a key premise in justifying using chimeric organisms to develop improved nonhuman models. A chimeric study involving human-to-nonhuman neuronal cell transfer might be designed to (proximally) elucidate disease mechanisms, and yet hope (distantly) to give way to the development of new treatments in the future. Both are included here in the ethical justification of developing chimeric models of human neuropsychiatric disorders.

It’s important to note that in certain cases the distinction between proximal and distant direct benefits collapses. Indeed, in some cases, the knowledge produced in a particular experiment involving nonhumans applies directly to the safety and/or efficacy of a treatment in humans, as is the case is in some toxicology research. The production of “know-how” in an experiment, that is improved methods or new skills, can also have immediate applications for human (and nonhuman) welfare. Since most chimeric research falls under the umbrella of basic research into disease mechanisms, however, the distinction between proximal and distant benefits is apt. Scientists are actively pursuing research agendas involving chimeric studies that elucidate the fundamental genetic, biological, chemical, and electrical explanations of human neuropsychiatric disorders.¹²

Finally, sometimes experiments produce unanticipated direct and indirect benefits. An error might lead scientists to new discoveries. Reconfiguring a broken apparatus might lead to novel invention or method. Unanticipated benefits are sometimes cited in retrospective harm-benefit analyses of research with nonhumans, and then this is used to justify research with nonhumans writ large prospectively.¹³ Unanticipated benefits should not be included in the prospective analysis of the benefits of a particular experiment. Prospective consequentialist analyses of anticipated benefits or goods to come from a course of action must be limited to intended or otherwise anticipated consequences, otherwise there would be no bounds to what’s included in the consequences considered in ethical analysis.

Assessing Scientific Justification

Despite the difficulties of trying to understand and assess prospectively the benefits of developing novel nonhuman models of human neuropsychiatric disorders, it is undeniable that eliminating or minimizing the suffering caused by neuropsychiatric disorders would be a very good thing to do. There are various things we could do in response to real human suffering, now and in the future. One would be to invest in the amelioration of known social, political, and economic drivers of mental illness to adequately cover non-pharmaceutical based therapeutic interventions. Another would be supporting “hard science” research on the biological underpinnings of psychiatric disorders, as model creation seeks to do, with the goal of developing pharmaceuticals that modify or prevent pathologic processes. These approaches are neither exhaustive nor mutually exclusive, but sometimes budget restrictions render them so.

Even within a scientific research approach, choices abound about what species to study (including humans among the options, of course), what to model or to look for, and the unit of analysis. The point is: given options about how to proceed in the face of real human suffering, what’s worth our collective investment? Going the scientific route demands a justification, and the choices within that route also need to be justified. The scientific justification of any particular experiment is multi-pronged and multilayered.

But if we focus just on the direct, proximal benefits of chimeric research, namely, the production of new knowledge, the centrality of scientific justification in the assertion of benefit is apparent. While the distant benefits and practical implications for humans often motivate research, their manifestation is also highly uncertain.¹⁴ Despite uncertainty about distant benefits, every new proposed experiment or study should produce the proximal benefit of new knowledge (even if not testing a hypothesis). Moreover, this knowledge should fit into a trajectory of research that has some relevance to the distant benefits promised. Experiments that are not scientifically justified will be unlikely to produce the anticipated knowledge (i.e. because of deficiencies in the proposed methods or analyses) or may produce knowledge that is merely trivial and/or divorced from a research trajectory that promises to bring about distant benefits. Thus, at least two further questions may be asked in the assessment of scientific justification: (1) Is this experiment likely to produce the knowledge it expects to produce? And (2) Is this knowledge worth pursuing (versus merely trivial)?

Answering these questions requires understanding and investigating the different factors that affect both the likelihood of knowledge production and factors that influence whether a particular kernel of knowledge is worth pursuing. In the context of modeling neuropsychiatric disorders, at least two distinct factors require assessment: (1) research conduct and (2) challenges to model creation. The reliability of nonhuman models is directly relevant to the likelihood of achieving distant benefits.¹⁵ In what follows, I will say more about these two factors. Other factors that are not discussed may merit consideration as well.

Responsible Conduct of Research

Knowledge production within an experiment is predicated on the responsible conduct of research. With this term I mean to capture a wide range of practices that can affect research results and interpretation, including (but not limited to) methodological quality, statistical analysis, transparency in reporting experimental conditions and results, publication, and animal care, such as how nonhuman research subjects are housed, fed, transported, and generally treated. While sometimes serendipitous findings result from mistakes, irresponsible conduct, and in research with poorly treated nonhuman subjects, irresponsible research conduct should not be the norm for science. Rigorously described and deployed, reproducible, and replicable methods, and transparent reporting and publication, are key to scientific advancement.

Mounting evidence shows that preclinical research involving non-human animals overwhelmingly fails to produce knowledge that translates into the direct yet distant benefits for humans that underly its justification.¹⁶ More than 80% of potential therapeutics that have positive safety and efficacy signals in mice fail when tested in people.¹⁷ Numbers within the field of behavioral neuroscience are even worse: 90% of compounds that enter human trials will fail.¹⁸ Notwithstanding inherent uncertainty and windiness in scientific research, some attribute this failure to “sloppy science”: lack of random allocation or blinded outcome assessment, statistically under-powered experiments, failure to attend to biases, failure to validate assays or antibodies, construct invalidity — the list goes on.¹⁹ Statistical power, randomization, validation of assays, and controlling for bias all serve to produce generalizable, reliable, and reproducible results and are given in experiments involving human subjects. The same assumption cannot be extended to experiments with nonhuman research subjects. Irresponsible research conduct may produce false findings.²⁰

Additionally, reporting issue and publication bias affect what information from laboratories around the world is read by fellow scientists, so sometimes even excellent work goes unnoticed. For example, journals tend to publish results that confirm a hypothesis or present work with statistically significant results, rather than results that support a null hypothesis or experiments that didn’t work.²¹ As a result, neutral or negative findings are much less likely to be reported and taken up. This may lead to unnecessary – and so ethically unjustified – duplication of work. Furthermore, reporting and publication bias may support continuing down an unpromising line of research and endangering humans in clinical studies of compounds that “worked” in nonhumans, but where that result is based on faulty data or statistical analysis. In this case, what is discovered might be said to be merely trivial. Scientists discovered something for this population of laboratory raised and reared nonhumans, many of whom are genetically or otherwise modified, but that does not extrapolate to different nonhuman or human populations. The ethical question is: should the accumulation of trivial knowledge be considered a benefit or value-add? I think most people would probably say that the accumulation of trivial knowledge is not sufficient to justify the harms to nonhumans – and humans – that result.

To be fair, the nature of scientific inquiry means it can be difficult to discern prospectively whether a particular experiment will produce knowledge worth having or merely trivial knowledge. Over the past couple of decades, the scientific community and scientific oversight communities have recognized the pitfalls mentioned, and a variety of prominent national and international institutions have issued recommendations for improvement in the responsible conduct of research and publication of research.²² I refer readers and especially those tasked with ethical and merit review of experimental protocols to these sources. Being able to spot experiments that will produce false findings from experiments that will produce knowledge is a first task, and then being able to discern trivial knowledge from knowledge worth pursuing is a second one for review committees. Neither are easy tasks, conceptually or practically. But it is essential to discerning prospectively whether experiments are poised to the produce the benefits they promise.

Challenges to Neuropsychiatric Model Creation

While the previous section focused on conduct of research with nonhumans quite generally, this section will focus on some problems specific to creating nonhuman models of human neuropsychiatric disease. Nonhuman disease models display some or all the pathological processes or symptoms observed in the actual human disease and enable scientists to study a wide range of human diseases in vivo using experimental techniques that would be unethical to use in experiments with human research subjects. Disease models can be assessed for validity based on the extent to which they recapitulate the symptoms of human disease (face validity), instantiate the underlying pathological mechanisms or etiologic processes of human disease (construct or etiologic validity), and predict the likelihood that a treatment tested in nonhumans will work in humans (predictive or pharmacological validity).

There are limitations inherent in any nonhuman model of human disease,²³ but the limitations of nonhuman models are arguably greater with “disorders that often seem uniquely human.”²⁴ Rollin & Rollin write, “pathologies of the mind have characteristics that set them apart from other forms of illness, in ways that have deep relevance to both the validity and the ethical acceptability of these kinds of studies, particularly with respect to animal models.”²⁵ Nestler and Hyman note that, “Many of the symptoms used to establish psychiatric diagnoses in humans (for example, hallucinations, delusions, sadness and guilt) cannot be convincingly ascertained in animals. When there are reasonable correlates in animals (for example, abnormal social behavior, motivation, working memory, emotion and executive function), the correspondence may only be approximate.” As a result, “animal models are unlikely to mirror the full extent of a given human neuropsychiatric disorder.”²⁶ Further complicating the issue, the diagnosis of mental illness in humans wants for rigor and accuracy. Many of the symptoms and diagnostic criteria for human neuropsychiatric diseases are variable and overlapping. If clinicians cannot accurately distinguish and diagnose bipolar disorder, depression, or schizophrenia, how can researchers create a model of them?²⁷ Together, these observations about the nature of human neuropsychiatric disorders raise questions about the possibility of creating nonhuman models that achieve face, construct, or predictive validity.

Interestingly, the same justification for utilizing chimeric research to develop nonhuman models of neuropsychiatric disorders, namely that previous models have limited (if any) utility, is also cited by some as evidence that we should abandon model creation altogether, at least for now. The limited understanding of neuropsychiatric disorder in humans and lack of rigorous diagnostic criteria, especially at the biological, chemical, and genetic level, indicates that clinicians and scientists do not yet have a good enough grasp on what they are modeling to develop reliable and valid models.²⁸ This line of thinking would have the scientific community focus in the short term on research in humans and even in nonhumans who exhibit unusual behaviors “in the wild.”²⁹ The knowledge gained could then inform the creation of reliable and valid nonhuman models – models that obtain construct validity and predictive validity.³⁰ One team of researchers from Stanford suggest that the way forward for nonhuman models of disorders such as autism will involve validating models on the basis of biomarkers, rather than abnormal social behavior, memory, or motivation, as is the case currently. This skirts the problem noted above that many of the symptoms used to diagnose neuropsychiatric disorder in humans cannot be ascertained or modeled in nonhumans.³¹

It is well-beyond my own expertise to critique this – or any other – approach to model creation. Suffice it to say for now that there is criticism of the status quo in model creation and validation as well as concrete suggestions to improve it, coming from within the scientific community. These criticisms are not typically acknowledged in scientific and ethics papers that defend chimeric research on the basis that it will generate sorely needed models of neuropsychiatric disorders.

Those calling for more reliable, valid, chimeric nonhuman models of neuropsychiatric disorder, however, should be very concerned that future models produce knowledge worth having about the conditions under investigation and not merely trivial knowledge about a specific colony of chimeric animals. We are at risk of continuing to produce knowledge about (chimeric) nonhumans with strange behaviors and altered social practices – species atypical behavior that may indicate suffering or otherwise deleterious effects to well-being – but that tells us nothing about human disorder and suffering.

Conclusion

I do not conclude based on these remarks that the development of nonhuman models of neuropsychiatric disorder is - wholesale - scenically unjustified (as others do, on similar grounds³²). I am open to the possibility that certain instances or experiments are or will be designed, executed, and analyzed in ways that proximally produce knowledge worth having and more distantly contribute to the amelioration of human suffering. Yet the reality of research (mis)conduct and the past failures of nonhuman models to produce knowledge that translates to human neuropsychiatric disorders should lead to greater scrutiny of the scientific justification of this area of research. Very close scrutiny of the likelihood of any particular experiment or model to produce knowledge worth having is warranted. This could include the involvement of additional reviewers at the grant-making or ethical review stage, i.e. clinicians, philosophers of science, and scientists from a variety of disciplinary backgrounds. The assertion that making more nonhuman models of neuropsychiatric disorders will lead to groundbreaking discoveries should not be taken for granted by those writing about ethical issues in chimeric research, as has been shown to be the tendency in other bioethical defenses of new practices and technologies.³³

Much more attention should be paid to the theory and methods of model generation and forums dedicated to open dialogue and debate about nonhuman models. Forums like those suggested by the Work Group in this report’s main essay could be places for this interdisciplinary discussion. This should include discussion of the type and level of scrutiny needed to assess the scientific justification. Crucially, it must involve expertise from various fields – geneticists, stem cell scientists, ethologists, clinician-scientists who treat humans with neuropsychiatric disorders, philosophers of science, therioepistemologists,³⁴ among others. The traditional way in which models are conceived and developed, and well-documented evidence of research and publication practices that limit translatability of research, demand changes in the status quo. The ethical justification for using nonhumans in this type of research depends on these changes. Interdisciplinary collaboration and debate in prescribed forums for these conversations, however, may catalyze the changes to move research on neuropsychiatric disorders in a more fruitful and beneficial direction.

Biography

Carolyn P Neuhaus is a Research Scholar at The Hastings Center. She currently leads two projects investigating ethical issues in the community healthcare setting.