Balancing External Validity and Concern for Psychosocial Harms in Translational Genetic Research

In this commentary we explore a challenge to translational genetic research: how to balance protecting research participants with ensuring the external validity of research so that research results are implemented in ways that protect future patients. Using our experience with a recently awarded National Institutes of Health (NIH) grant, we highlight the importance of external validity in ensuring that the results of translational studies are responsibly and ethically implemented in clinical practice and do not compromise the welfare of future patients whose care the study results are meant to inform. Future patients will be even more diverse and broad than study populations, and we urge researchers, and IRB and grant reviewers, to keep in mind the translational pathway that lies ahead of a study.

Translational research emphasizes the rapid movement of research findings from bench to bedside.¹ While early phase translational research (T1) is focused on basic science discoveries, late phase translational research (T2) is aimed at “translation of results from clinical studies into everyday clinical practice and health decision making”.¹ In this late phase, translational research aims to ensure “that new treatments and research knowledge actually reach the patients or populations for whom they are intended and are implemented correctly”.² T2 translational research requires an implementation mindset; that is, we must consider how interventions will be implemented in real world settings.³

Pragmatic trials play an important role in achieving the goals of late phase translational research. By piloting practices in real-world settings, pragmatic trial results can more realistically and quickly inform clinical and policy decision making about whether or how to implement a particular clinical practice.⁴ Pragmatic trials emphasize external validity—the extent to which study results will generalize outside the study context—by making the trial conditions as close as possible to the conditions in which the intervention would be implemented. To achieve this goal, they typically include broad study populations, participant-centered outcomes, and flexible protocols that accommodate natural variation among patients, providers, and administrators.⁵

In contrast, explanatory trials conducted during earlier phases of translational research seek to minimize outside factors that could affect clinical outcomes (e.g., treatment adherence, variation in clinical practice, heterogeneity in the study population) and are conducted under highly controlled conditions, including highly selected participants, intensive monitoring, and data collection.⁶ Explanatory trials prioritize internal validity, given the need to demonstrate efficacy or safety. Conducting a study in such “idealized settings” increases the chances that the findings will not reflect real-world settings or effects, and may not be feasible to implement.⁷ Therefore, insufficiently pragmatic trials can negatively impact late phase translational research. If findings from “idealized settings” suggest implementation but the intervention has not actually been tested in real-world settings, there is a risk that it will not have the same effect or be achievable in these contexts. Conversely, if findings from a highly controlled study indicate an intervention is ineffective, we cannot say the same holds true in a real-world setting, potentially depriving future individuals of a beneficial intervention.

We were recently awarded an NIH grant to investigate the psychosocial and health behavior effects of reporting specific adult-onset genetic results (i.e., pathogenic variants in the BRCA1/2 and Lynch syndrome genes) to adolescent biobank participants and their parents.⁸ Returning adult-onset genetic results to adolescent populations remains the subject of debate among relevant medical professionals and bioethicists, which has limited translation into clinical settings.⁹ The primary concerns regarding returning adult-onset genetic results to adolescents are psychosocial in nature—in particular, commentators have long speculated that disclosing this information will cause unnecessary harms like anxiety, depression, or a negative impact on the family or self-perception, especially since the results may not be clinically actionable until adulthood.¹⁰ These psychosocial concerns have dominated the discussion to date, while the potential benefits of this knowledge remain underexplored. The potential benefits include reducing anxiety and promoting psychological adjustment to the adult-onset genetic result among adolescents, and cascade testing of adult relatives with actionable genomic results, enabling early intervention and prevention through risk reduction procedures.¹¹ Recently some have argued that many of the concerns regarding the psychosocial harms of genetic information are outdated and need to be reconceptualized altogether.¹²

Although the prospect of psychosocial harms has dominated the debate, the existing evidence base suggests that disclosure does not cause major psychosocial harms in either pediatric or adult settings, and any psychosocial harm that does occur among adolescents is generally minor and self-limited.¹³ A systematic review of the psychosocial harms among children and adolescents concluded that the evidence base consistently signals a lack of psychosocial harms.¹⁴ A total of 13 studies, across 9 countries, covering 386 children or adolescents found no significant effects on anxiety, depression, or quality of life, yet clinical guidelines and perspectives still emphasize the psychosocial harms despite the lack of evidence.¹⁵

While the findings are consistent across these prior studies, many are limited in size or have returned pediatric onset condition results, with fewer studies exploring the impact of adult-onset results. The authors of the systematic review recommend further research to systematically explore the potential benefits and confirm the lack of long-term psychosocial harms to inform future clinical practice. We hypothesize, based on this existing evidence, that there will not be major psychosocial harms but our study is designed to provide rigorous, confirmatory evidence gathered in a real-world healthcare setting regarding the psychosocial effects, if any, of returning adult-onset genetic results to adolescents.

Our study, therefore, fits squarely on the translational genomics continuum beyond proof of concept and approaching late phase clinical translation, and is designed to inform future clinical practice and guidelines.¹⁶ While our study would not be considered fully pragmatic — and very few trials are in fact fully pragmatic — it is pragmatic on many dimensions of the commonly used PRagmatic Explanatory Continuum Indicator Summary tool (PRECIS).¹⁷ The grant was funded because of the study’s potential to help resolve the ongoing debate in the field, and assuming the study results support the return of adult-onset genetic results to adolescents, it will ultimately inform how to best implement this in future practice. Additionally, the study is being conducted within a learning healthcare system focused on late phase translational research with the dual purpose of producing generalizable knowledge and informing clinical practice. Evidence-based practice and guidelines should be developed based on data collected from real-world contexts. As a result, external validity and the ability to translate our findings into clinical practice are essential aspects of our study. Yet as we describe below our study also illustrates the challenges of translational science especially as research advances further on the translational continuum and existing evidence supports an intervention but is not yet robust enough for clinical implementation.¹⁸

Grant reviewers requested that we add a clinical psychologist to the disclosure session, alongside a genetic counselor, as an additional mechanism to protect participants from the potential psychosocial harms caused by returning genetic results. Our existing study design includes a robust psychological monitoring plan across many domains of functioning to assess the impact of disclosure. These psychological measures include, but are not limited to, anxiety, depression, family functioning, quality of life, body image, self-esteem, decisional regret, satisfaction, psychological flexibility, personal utility, perceived risk, adjustment, and empowerment. These measures are administered at baseline and at three time points in the 12–14 months following disclosure.¹⁹ Additional protections include having genetic counselors conduct disclosure sessions and making counseling from a licensed clinical psychologist available to those who wish. We will also conduct in-depth qualitative interviews to determine what, if any, psychosocial impacts arise that might not be captured with our quantitative measures.

In one sense, the grant reviewers’ additional requirement that a psychologist be present at disclosure is unsurprising, arguably even reasonable, because our study explores an area of ongoing controversy by proposing to deliver adult-onset genetic results to adolescents, who are considered a vulnerable population under the Common Rule.²⁰ Yet, the existing evidence consistently does not suggest major psychosocial harms as a result of this knowledge, and our study includes a rigorous psychosocial monitoring plan, raising questions regarding what is an appropriate level of psychosocial monitoring when there is an existing, if not yet sufficient, evidence base upon which a translational study is based.

Clinical guidelines for similar interventions advise discussing the potential psychological benefits and harms of the information during genetic counseling but do not require the presence of a psychologist or long-term psychosocial support.²¹ In fact, there is no mention of having clinical psychologists present during genetic disclosure in existing guidelines. It is also unlikely that the presence of a psychologist would be feasible to implement in clinical practice, due to costs for instance. Given the goal of our study is to translate findings into clinical practice, it should mirror the real world as much as possible in this context. Our proposed psychosocial monitoring plan attempts to strike a balance between protecting adolescents from harm while also ensuring our findings are relevant in broader clinical settings.

As a research team, we questioned whether the added presence of a psychologist during disclosure whose sole purpose is to assess for harms, coupled with ongoing and repeated psychosocial monitoring, might impact the study’s external validity especially since our primary study outcome is behavioral. Might repeated psychosocial monitoring in various forms — even though designed to protect participants — simultaneously signal that something more serious and highly concerning than they perceive is taking place, making disclosure appear, and perhaps become, more harmful than it otherwise would be? Alternatively, might this increased psychosocial monitoring mask or more thoroughly address anxiety and depression that participants would face in a more realistic setting (i.e., one in which constant monitoring is not feasible), thereby making the intervention appear safer than it would be under those real-world conditions? Regardless of the direction, this potential impact on external validity could threaten our study’s ability to contribute to clinical and policy decisions about whether and how returning results should be implemented in clinical practice.

It is important for grant reviewers, institutional review boards (IRBs), and researchers to consider: at what point along the translational science continuum might procedures or practices designed to protect participants from the psychosocial harms of genetic knowledge — such as requiring a clinical psychologist to be present during disclosure alongside a genetic counselor — become problematic for external validity and broader clinical implementation? Are we defaulting to intensive psychosocial monitoring in order to protect participants at the expense of external validity? We acknowledge the impact of psychosocial monitoring on external validity is unclear for our study. We cannot be certain of the impact of adding a clinical psychologist on external validity, although we think this could seriously impact the dynamic between genetic counselor and participants during disclosure.

The Common Rule makes clear that the risks to participants are typically justified in large part by the expected benefits of the study not for study participants narrowly, but for society (or future patients) more broadly in the form of knowledge production. For research to be approved, “risks to subjects [must be] reasonable in relation to anticipated benefits, if any, to subjects, and the importance of the knowledge that may reasonably be expected to result”²² (our emphasis). Knowledge production — in our case, externally valid knowledge — and protection of research participants are both essential to the ethical conduct of clinical trials. This is why IRBs, contrary to popular opinion, do have a legitimate role in attempting to ensure that a study is well-designed. If a non-therapeutic study is not properly designed, there can be no reasonable expectation of social benefit, and thus, under the Common Rule, no significant risks to participants can be justified.

One way we have addressed the uncertainty about the likely effects of various monitoring plans is by obtaining feedback from a participant group — the Precision Health Youth Advisory Council — which consists of representatives from our adolescent study population. Adolescent stakeholders have reviewed and provided input on the many psychosocial follow-up measures included in the study regarding length, burden, invasiveness, and whether new concerns might arise due to the measures themselves. This is not to suggest that such stakeholder input should replace grant or IRB review, or that it will justify less intensive monitoring plans in all cases, but this group provides a crucial perspective regarding the suitability and burden of monitoring plans in our study. Future researchers can form similar participant committees, and use information gained from relevant stakeholders to justify follow-up plans that might appear conservative in grant or IRB applications.

Our research team, which includes clinical psychologists, has decided that the presence of a clinical psychologist during disclosure, in addition to a genetic counselor, is not necessary in the context of our study’s existing psychosocial monitoring. Our IRB has approved this approach.

Another mechanism we are using is an Event Monitoring Committee (EMC) to provide independent oversight and monitor for psychosocial harms specific to our study.²³ EMCs are explicitly designed for social or behavioral intervention trials where the potential risks are primarily psychosocial rather than physical, and where traditional Data Safety Monitoring Boards (DSMBs) might not be applicable or suitable. EMCs have been used in studies with vulnerable populations and individuals deemed to be at high genetic risk. Our EMC will provide close monitoring of data as they are collected, including an early review of qualitative interview findings, to ensure we are aware of potential harms that might be occurring throughout the study and that might not be captured by quantitative measures or longer term follow up. Utilizing an EMC can help avoid defaulting to intense monitoring in research settings by enabling a real-time review of data as they are collected, including non-traditional forms of data like qualitative interviews, and to adjust psychosocial monitoring as needed. Future researchers, IRBs, and grant reviewers should consider EMCs as one way to review real-world data as they are collected rather than implementing the most conservative protections at the outset of a study.

The NIH is explicitly committed to translational science, but our experience highlights several challenges for translational research going forward, especially in areas of ongoing debate or controversy.²⁴ We urge well-meaning grant reviewers, IRB members, and researchers to keep in mind the translational pathway that lies ahead of a particular study and find ways of protecting research participants that do not compromise external validity and, hence, the welfare of future patients whom the results are meant to benefit. As clinical research moves along a spectrum and inches closer to implementation, we urge ethics reviewers to broaden the focus of their concern to include not only participants but also the future recipients of treatments, practices, and policies under study, who typically enjoy no analogous third-party oversight.²⁵ The failure to recognize this connection between pragmatic research methods — including participant protections — and policy risks exposing future patients to unintentionally harmful policy decisions, whether denial of access to a beneficial intervention or exposure to a harmful or inert and wasteful intervention. While the Common Rule provides some guidance on these matters, it does not explicitly address the complex new challenges raised by translational science, pragmatic trials, and other health services research when the goal is translation of evidence into clinical care but evidence is still being gathered in research settings.

Translational science, including pragmatic trials such as ours, challenges the boundaries between research and clinical care, especially as research advances on the translational continuum, and therefore also challenges traditional research ethics, which is premised on a sharp distinction between research and health care.²⁶ Future studies will need to find an appropriate balance between protecting participants from harms while ensuring data reflects real world settings, which may, in some cases, justify easing human subject protections in the service of external validity.