Using the concept of “deserved trust” to strengthen the value and integrity of biomedical research

Abstract

It is commonplace for science leaders and others to claim that the future of biomedical research rests in large part upon the public’s trust. If true, it behooves the biomedical research community to understand how it avoids taking chances with that trust. This commentary, which builds upon comments of noted trust scholar Russell Hardin about how best to enjoy trust, assumes that the key to being trusted is deserving to be trusted. Thus, it proposes using “deserved trust” to identify ways that the public’s trust in biomedical research could be better supported. Employing deserved trust to support the public’s trust leads us to consider what it is that the biomedical research community should be trusted to do, examine evidence about the effectiveness of current safeguards meant to assure that those things routinely get done, and identify new ways to equip individual researchers, research teams, and research institutions to assure that the public’s trust in their research is deserved rather than misplaced.

A. Introduction

Such matters as concerns over public receptivity to coronavirus disease 19 (COVID-19) vaccines,(Agiesta 2020) multiple retractions of recent COVID-19 studies,(Joseph 2020)(Abritis, Marcus, and Oransky 2020) and polling indicating, at least in some jurisdictions, that an erosion of public trust in research has already begun(Matthews 2020) remind us anew about the importance of the public’s trust in biomedical research. With past as prologue, we can expect public statements from research champions echoing past proclamations from luminaries such as Francis Collins, Director of the US National Institutes of Health (NIH). Following public backlash in 2010 about the financial ties between drug companies and academic scientists, he opined that “[t]he public trust in [biomedical research] is just essential, and we cannot afford to take any chances with the integrity of the research process.” (Wadman 2010)

Any similar attestations made in this time of COVID-19 will occur in a research ecology little changed from the one found in 2010, despite extensive work devoted to research integrity that has occurred in the interim. A recent commentary about research integrity, written by an interdisciplinary group of scholars with complimentary expertise in scientific research, research methodologies, bioethics, and research integrity, captures the scope of this work. The authors write that “there have been [multiple] declarations that outline the components of trustworthy research and the principles of research integrity. These include the Singapore Statement in 2010, the Montreal Statement in 2013, the Hong Kong Principles in 2019 and the European Code of Conduct for Research Integrity in 2011, revised in 2017, among others.” They further mention that “[m]any hundreds of articles have been written on the topic [of research integrity]: about threats to research quality from hyper-competitiveness and poor training; the unquestioning and inept reliance on metrics in evaluation [of researchers]; and systematic biases in peer review and publication.”(Mejlgaard et al. 2020) If this important body of reform-minded work is to have any chance of achieving its intended purpose of changing the research ecology in order to improve research, more researchers and research institutions need to be persuaded that the insights found in this substantial body of work can help to improve their research.

I want to put forth the proposition that we may be able to so persuade them if we leverage the widely shared belief in the research community, reflected by Director Collin’s and similar statements from other leading voices in science,(Smith 2014) (Cicerone 2010)(Nature 2010) that it is important to the future of the research community that it continues to enjoy the public’s trust. One way to leverage this belief is to follow the advice of noted trust scholar Russell Hardin, who claims that the key to enjoying trust is doing the things that support trustworthiness. (Hardin 2002, 30) In other words, if the research community wants to continue to enjoy the public’s trust it knows it will need, then it should make sure that it deserves their trust.

Knowing how to deserve the public’s trust in the context of biomedical research is far from straightforward, though. There is no consensus about what it is, exactly, that makes this research deserving of the public’s trust and thus no consensus either about what practices promote trustworthy biomedical research. This means that, currently, future public trust is being left as much to chance as it is design. (Yarborough 2014b) In what follows, I offer a proposal for how using the concept of “deserved trust” might change this.

Some may not see the need for such a strategy because they may disagree that we currently do leave the public’s trust too much to chance. Supporting their disagreement are current safeguards meant to protect research and its integrity. The safeguards include the numerous international statements and professional codes referenced above; professional norms, transmitted in science education; quality control mechanisms such as peer-review; research regulation activities like Research Ethics Committee (REC) review; and a growing global reliance on both research integrity programs and mandatory instruction in the responsible conduct of research. Hence, they may think that we are not leaving future public trust to chance because we are already heeding Hardin’s advice. In what follows, I will show why we need to do a much better job than we currently are of following Hardin’s advice. I will do this by offering a two-pronged process that will help us recognize, first, the extent to which, in fact, we are leaving the public’s trust to chance and, second, how we could stop doing so.

One prong consists in three general questions about the trustworthiness of research that relate to its reliability, social value, and ethical conduct that are relevant to biomedical research in general. I review ample published evidence below that helps answer those questions. The other prong consists in specific questions directed at individual research projects, teams, and/or institutions that researchers and research institutions themselves can answer about their own work. I provide a brief description below of how they could go about answering the questions. Though the two prongs are complimentary, they both have stand-alone value as well. We will see that the general questions reveal deficiencies in current safeguards and show the extent of work yet to be done if we are to heed Hardin’s advice. We will see how the specific questions can prompt the practices needed in specific research settings to ensure that any given research project does deserve the public’s trust.

B. Knowing the extent to which research currently deserves the public’s trust

The three questions in the first prong (see Table 1) about the reliability, social value, and ethical conduct of research all reflect characteristics of research that the public should rightly expect are routinely found in it.(Yarborough 2014a) Considering relevant evidence that speaks to the frequency with which these characteristics are present will shed light on both the effectiveness of current safeguards and whether we are “taking chances” with the trust that is “just essential” to the future of biomedical research.

Table 1:

Characteristics that make research deserving of the public’s trust

Characteristic	Explanation
Disseminated research results are reliable	The purpose of biomedical research, like all scientific research, is to advance understanding of the natural world.
Research creates increased social value	The desire for greater human well-being and improved health care steers public and other resources to the biomedical research community.
Research is conducted ethically	There are international norms and standards regarding the use and treatment of animals and humans that should be observed in the conduct of biomedical research.

B. 1. Are disseminated research results routinely reliable?

Though the reliability of published research reports is critical across the entire landscape of biomedical research, due to space constraints we will look at evidence about reliability in the critical area of preclinical research. This is the research that creates the diverse body of knowledge that advances basic understanding across the entire life sciences spectrum, including our understanding of disease and its mechanisms. When appropriately conducted, these studies, whether they produce positive or negative results, critically inform clinical translation efforts. This research frequently involves animal studies that, for example, can “identify new druggable targets and design or test specific therapeutic modalities, as well as detect diagnostic or therapeutic biomarkers. [It also] allow[s] the comparison of pathological phenotypes with the human disease, and [provides] insight into the underlying reasons for differences at varying levels of complexity, [thereby] allowing the identification of protective pathways in animals that could be enhanced in humans.”(Willmann et al. 2020) Preclinical studies both develop and test “novel therapeutics, including small molecules, biologics, gene modifiers and cell therapies”, as well as inform “optimal dosing regime[s] and route[s].”(Willmann et al. 2020) In short, these studies help determine which investigational treatment modalities migrate out of the laboratory into human trials.

To appreciate why the reliability of these preclinical studies matter, we need only consider patients with fatal neurodegenerative disorders like Amyotrophic Lateral Sclerosis who are asked to volunteer for Phase 1 clinical trials based upon specific preclinical studies. The individuals who do should be able to trust that the trial is worth conducting because there is a sound scientific rationale for it. Would such a level of trust be deserved? There is a substantial body of evidence that suggests that it frequently would not.

In saying that it would not, readers should not infer that this is due to the fact that new treatment modalities usually do not pan out. That would be an unfair and unrealistic standard of deserved trust, given the biological complexities of disease and its mechanisms that make successful translation exceptionally difficult. Rather, trust would be undeserved because, as one commentator has noted, “the real reason [behind failed translation is often due to the fact that] the preclinical experiments were not rigorously designed.”(Perrin 2014) This verdict echoes a similar sentiment expressed in the context of oncology studies, where commentators noted that “[u]nquestionably, a significant contributor to failure in oncology trials is the quality of published preclinical data.”(Begley and Ellis 2012) When these deficiencies occur, our ability to make reliable inferences about whether or not there is a reasonable ratio between the risks and benefits of conducting a Phase 1 trial on a new treatment modality is severely compromised.(Yarborough et al. 2018) This means that the trust of Phase 1 trial volunteers can be misplaced rather than deserved.

The above quotes about deficient preclinical studies is substantiated by more than a decade’s worth of research that focuses on preclinical studies.ⁱ(Ioannidis 2017)(Vogt et al. 2016)(Reichlin, Vogt, and Würbel 2016)(Egan et al. 2016)(Hartung 2013)(Hirst et al. 2014) (Macleod et al. 2015)(Howells, Sena, and Macleod 2014)(O’Connor and Sargeant 2014)(Lindner 2007)(Peers et al. 2014)(Garner et al. 2017) It reveals multiple, prevalent, and often interconnected design and reporting problems. To appreciate the extent the deficiencies can contribute to undeserved trust, consider, for example, the conclusion reached by the authors of a landmark study assessing excess significance bias in preclinical neurosciences studies. They concluded that, based upon an examination of more than 4,000 preclinical studies related to 160 human trials, only 8 of those clinical studies were sufficiently scientifically warranted.(Tsilidis et al. 2013) Equally disconcerting is a more recent analysis of failure rates in clinical trials of acute stroke.(Schmidt-Pogoda et al. 2019) that reaffirm worries raised in numerous prior analyses (Macleod et al. 2015)(Tsilidis et al. 2013)(Sena et al. 2010)(Crossley et al. 2008) Its authors estimate that a majority of the reports of positive findings from animal studies meant to inform clinical studies of acute stroke actually report what are likely to be false positive results.

Readers may question whether my concern about the trust implications of the reliability of published preclinical studies is based upon a handful of experiments that are being used to paint too negative a picture about the reliability of preclinical research. There is ample evidence showing that it is not. Consider, for example, the actions of prominent research journals and sponsors who are actively trying to instigate methodologic reforms in preclinical and other research in order to foster greater reliability. These include the more than one thousand journals and two dozen funding agencies(Enserink 2017) who endorse use of the ARRIVE Guidelines. First developed in 2010(Kilkenny et al. 2010) and just recently revised,(Sert et al. 2020) these guidelines are meant to improve research reports about animal research.

Other leading journals have adopted additional measures. For example, the journal Stroke employs reporting requirements that are more stringent than the ARRIVE guidelines. Its guidelines are specifically targeted to stroke-related research reports.(Minnerup, Dirnagl, and Schabitz 2020) The Nature family of research journals, on the other hand, in an effort to improve research reliability and reproducibility, requires authors “to make materials, data, code, and associated protocols promptly available to readers without undue qualifications.”(Reporting Standards and Availability of Data, Materials, Code and Protocols) Also worth noting is a series of Lancet articles published in 2014 that address ways that researchers and research institutions can increase value and reduce waste in research. Just in the first year of their publication, the articles in the series were downloaded more than 46,000 times.(Moher et al. 2016) As of early 2019, articles in the series had been cited over 900 times in PubMed Central registered articles.(Yarborough, Nadon, and Karlin 2019)

Finally, we must note that concerns about the reliability of research have not escaped the attention of major government research sponsors either. For example, both the NIH and the Deutsche Forschungsgemeinschaft have undertaken initiatives meant to increase research rigor and reproducibility in order to improve clinical translation.(Collins and Tabak 2014)(Deutsche Forschungsgemeinschaft) Such extensive reform efforts would not be in place if problems in research that erode the reliability of published research findings were occasional and not endemic. Collectively, this evidence paints a troubling picture about the extent to which the current preclinical research endeavor can be counted upon to serve as a reliable precursor for the Phase 1 trials that so many patients have no choice but to trust that they are worth volunteering for.

B. 2. Does research routinely create increased social value?

When we think about the countless patients with cancer or chronic disorders such as diabetes, heart disease, or high blood pressure who volunteer for Phase 3 studies, we find comparable concerns about potential misplaced trust due to the questionable social value of many of these trials. Due to space constraints, we note just three of several examples causing concern. They are of concern not because they are trials that do not result in new and improved treatments. That would be a counterproductive criterion of social value to impose on clinical trials. Just as we noted previously that preclinical studies that produce negative findings are valuable, so too are clinical trials that fail to demonstrate that a new drug is therapeutic. That is because, as Kimmelman and London have noted, the desired output of clinical research is actually not so much new contributions to the clinical arsenal as it is information instead “about the coordinated set of materials, practices, and constraints needed to safely unlock the therapeutic or preventive activities of drugs, biologics, and diagnostics.”(Kimmelman and London 2015) Accordingly, the clinical trials of questionable social value are those that are uninformative(Zarin, Goodman, and Kimmelman 2019) because steps that could be taken to prevent them from being uninformative are not taken. When this occurs, there is a “serious breach of trust …”(Zarin, Goodman, and Kimmelman 2019)

The first group of trials of questionable social value are redundant ones that launch in the absence of any effort to systematically determine what prior relevant clinical trials have been reported that, if consulted, would influence the design, conduct, and need of subsequent studies. (Robinson and Goodman 2011)(Lund et al. 2016) The second group are trials whose knowledge they are meant to generate never gets sufficiently distributed. Too many trial results are never published or, if they are, too often only partial, possibly misleading, results get disseminated. (Riveros et al. 2013)(Strand et al. 2017)(Hakala, Fergusson, and Kimmelman 2017)(Jefferson and Jorgensen 2018)(Goldacre et al. 2018) The third group is clinical trials that employ flawed designs that cause uncertainty about the drug’s ability to contribute to improved patient outcomes that nevertheless produce datasets that result in regulatory approval for either new drugs or new indications for previously approved ones. A recent study of anticancer drug trials illustrates this concern. It reports that fully two-thirds of newly approved anti-cancer drugs are based on clinical trials “with at least one of the following limitations: nonrandomized design, lack of demonstrated survival advantage, inappropriate use of crossover, or the use of suboptimal control arms.”(Hilal, Gonzalez-Velez, and Prasad 2020) This spectrum of evidence surely casts a pall of skepticism over claims that clinical research can routinely be trusted to produce social value sufficient to offset the moral, material, and financial resources it consumes.

B. 3. Is research routinely conducted ethically?

When one’s attention turns to evidence that speaks to whether the general public can trust that biomedical research can routinely be trusted to be ethically conducted, one might expect to see a more encouraging picture. After all, there is now decades long work by RECs that are meant to assure that certain benchmarks, such as an appropriate balance between the risks of conducting a trial and its anticipated benefits and the respectful treatment of research participants, are met. Despite this REC role, there are genuine worries that we cannot trust that such benchmarks are met with the regularity that they should be.(Yarborough 2020b) In the case of balancing potential risks and benefits in early trials, we all would presumably agree that research volunteers should be able to trust that no one would ever recruit them into a clinical trial in the absence of a favorable risk/benefit ratio. Evidence pertinent to Phase 1 trials yet again calls this presumption into question. It shows that too often RECs are unable to access study design information about preclinical efficacy studies that is needed to vet the reliability of inferences in those studies about potential efficacy.(Wieschowski et al. 2018) When a REC cannot adequately assess efficacy inferences, it cannot reliably assess the potential for benefit and thus determine whether the risk/benefit ratio in early phase trials based upon the efficacy inferences is reasonable. This matter is all the more acute when we combine the inability to vet efficacy inferences with what is known about the high prevalence of false positive findings in preclinical studies.

As for the respectful treatment of research participants, there is also ample reason to worry that informed consent processes are routinely deficient. For both early and late stage clinical trials, those processes regularly conceal rather than alert research candidates to evidence like that recounted above about both the scientific merit and social value of clinical trials that is germane to their deliberations about enrolling in clinical studies. The upshot is that the very regulatory regimes meant to assure that the rights of people to grant informed consent for research are honored routinely have the opposite effect instead.(Yarborough 2020a)

Considering this evidence relevant to the first prong of questions shows the imprudence of relying so heavily upon professional norms, peer-review, research regulation, research integrity programs, and mandatory instruction in the responsible conduct of research to show that the research community deserves the public’s trust. This means that research teams and institutions need to employ additional safeguards if they want to be truly confident that they deserve the trust that they solicit, which brings us back to a key challenge noted at the outset. How can we inspire more researchers and research institutions to be more reform-minded so that they will be motivated to put to good use the insights found in the previously noted substantial body of work about how to improve research and its integrity? The second prong of questions is offered in hopes that it might help motivate them to do so.

C. Knowing how individual researchers, teams, and institutions deserve the public’s trust

The second prong consists in three simple questions (see Table 2) that individual researchers, research teams, research institutions, and even research organizations and societies can all regularly ask and answer to help them demonstrate why they are deserving of the public’s trust. Answering the questions will identify what researchers are asking people to trust them with and why; what the accountability methods and practices are that are in place, as well as how well they work; and whether any additional steps to support trust should be taken. Here is a brief illustration of how the questions can work.

Table 2:

Three questions that can be used to demonstrate why a research project or institution deserves the public’s trust

Question	Justification
What is that we are asking people to trust us with, in order to accomplish what goal/purpose?	This question helps us remember the dependency of research on others and what the good is that we seek on their behalf.
What accountability/safety/precautionary methods and practices are in place that help assure that we can be trusted to do the things we want to do with what we are soliciting from others and how effective are they?	This question helps us recall that, just as a chain is only as strong as its weakest link, an endeavor like research is made vulnerable by its weakest accountability measure/practice.
What additional steps, if any, do we need to take to ensure that we have effective safeguards in place?	This question steers us away from complacency with current safeguards towards vigilance regarding matters pertaining to individual research projects.

A preclinical research team doing animal studies reflects on question 1 about what the team is asking people to trust them with and why. It notes it is asking taxpayers to trust it to use their tax dollars to support its research to, for example, identify “protective pathways in animals that could be enhanced in humans.”(Willmann et al. 2020) Since progress toward this goal will depend upon the rigor of its research and the accuracy and robustness of its research reports, by asking question 2 about the accountability methods and practices that are in place, as well as how well they work, the team realizes that, in order to assure the quality of its work, it cannot just wait to learn whether a journal will publish its results, given how weak a quality assurance mechanism peer review is. Thus, it is led to question 3 asking whether any additional steps should be taken and subsequently consider how it can buttress its work. By answering it the team can learn from the extensive research reform literature that there are multiple ways to do this.

The team members might decide, for example, upon at least three complimentary strategies. They will pre-register their protocol so that they can both receive feedback about their study design and establish for their eventual readers fealty between their research and what they eventually write about it.(Nosek et al. 2018)(Dirnagl 2020) Next, in order to ensure greater efficiency in the research community’s translation efforts, they will clearly state in their published reports whether theirs was a hypothesis generating or hypothesis testing study.(Kimmelman, Mogil, and Dirnagl 2014) Finally, they will share their data so that others can independently check their analyses and, should they desire since they will also have their protocol, attempt to replicate the study findings.(Institute of Medicine 2013) The same process of asking and answering the three questions could produce equally helpful insights, for example, to a team of clinical investigators or the leadership team of a research institution, since the questions are simple, intuitive, and highly adaptive to a range of settings. Again, they will be able to draw upon the extensive research reform literature to help them address any deficiencies the questions uncover.

D. Concluding remarks

The value of the two prongs of questions will obviously need to be established by research that studies their impact before we can know the extent to which they do in fact contribute to deserved trust. And even if they prove to be effective, we should not expect them to be panaceas that will ensure that the trust in research that is continuously solicited will always be repaid with research worthy of that trust. This would overlook far too many structural deficits in today’s research ecology, not the least of which is the frequent misalignment between success metrics in academic science and high quality research itself.(Barnett and Moher 2019)(Benedictus, Miedema, and Ferguson 2016)(Ware and Munafo 2014)

Nevertheless, their use could help to nudge the research community away from its heavy reliance on current safeguards the extensive evidence reviewed above has shown to be ineffective. There is reason to think this nudging could ensue because of the liberating effect of the questions. They permit researchers to temporarily imagine themselves and their work apart from the demands of, for example, peer reviewers, animal care committees, and RECs, to see what practices they could best employ that would cause them to conclude that they deserve the trust that they currently solicit and hope to continue to enjoy in the future. By employing those practices, they could also be more confident that they are not “taking chances” with the public’s trust that is “just essential” to the future of research.