“The Biggest Struggle:” Navigating Trust and Uncertainty in Genetic Variant Interpretation

Abstract

Introduction:

As the utility of genomic sequencing increases, its use in healthcare will continue to expand beyond expert clinics towards non-specialist practices such as primary care. At the same time, discordance in genetic variant identification and classification between laboratories remains a concern for the field. This research assesses how clinicians with and without genetics expertise understand and trust genetic test results, underscoring how variation in the handling of genetic test results can have real impact on patient care.

Methods:

We conducted 40 interviews with genetics experts, including clinical geneticists and genetic counselors, and non-expert clinicians including primary care providers and cardiologists.

Results:

Clinical geneticists and genetic counselors reported spending significant time assessing the validity of results from genetic testing laboratories, conversing with laboratories about those results, and potentially reinterpreting results. Conversely, primary care providers and cardiologists without specific genetics expertise reported high levels of trust in lab accuracy and variant interpretation, and did not reassess results.

Conclusion:

We find significant variation in how genetics experts and non-experts understand the trustworthiness of genetic laboratory reports. This variation could lead to differences in patient care between clinical settings and requires additional guidance for clinicians regarding the handling of genetic test results.

Introduction

The price of genomic sequencing is quickly declining, increasing the accessibility of these tests and their promise to improve health outcomes. Recently, researchers found that genomic sequencing out-performed targeted gene sequencing tests in diagnosing infants with a suspected genetic disorder, identifying a molecular diagnostic variant nearly 50% of the time [1]. To improve clinical outcomes, professional societies like the American College of Medical Genetics and Genomics (ACMG) and the Association for Molecular Pathology (AMP) have sought to issue more standardized guidelines that clarify how these tests should be interpreted in patient care [2–4]. Yet despite these advances, discordance in genetic variant identification and classification between genetic testing laboratories continues to complicate the wide-scale use of this testing. The above study, for example, found that when two different laboratories detected the same variant, their classification of that variant differed 43% of the time [1]. Analyses of variant classifications in ClinVar suggest that of the 11% of variants with multiple submitters, classifications vary 17% of the time [5]. Indeed, as researchers from the Clinical Sequencing Evidence-Generating Research (CSER) Consortium have identified, continued discordance in genetic variant interpretation remains a critical place for improvement in clinical genomics [6].

In addition to ongoing concerns about how laboratories classify variants, a crucial question remains: how might this discordance have a real impact on the clinical management of patients? While some of these discrepancies may not impact clinical care, a study of variant interpretation discordance for Li-Fraumeni Syndrome suggested that 11% of families may receive discordant classifications leading to clinically important differences in medical management [7]. Additionally, clinicians may question or disagree with laboratory findings based on other data about patients that labs may not receive. Understanding how discordance between laboratories in variant identification and classification, and discordance between laboratories and clinicians, may impact patient care is particularly important as genomic sequencing becomes available to medical specialties outside of clinical genetics. Can clinicians without thorough knowledge of variant interpretation in laboratories integrate this kind of testing into their practice if discordance remains a serious concern? If clinicians are unaware of potential discordance issues, might this lead them to be overconfident in interpreting any given laboratory report?

To begin answering these questions, we draw upon 40 semi-structured interviews conducted with clinicians including clinical geneticists, genetic counselors, primary care providers, and cardiologists. Our approach focuses on genetic testing as a sociotechnical system, which suggests that the ways new technologies are designed and used cannot be understood without accounting for social context and relationships [8]. We build on a related concept of the “certainty trough,” first developed by sociologist Donald MacKenzie, which provides a theoretical lens to assess how expertise in a technological system like genomic sequencing impacts confidence and trust in others using and interpreting those systems [9]. One might expect that the people with the most intimate knowledge of a technology would have the highest trust and confidence in it, and those with the lowest levels of knowledge and expertise would report the least confidence and trust. Instead, MacKenzie finds the opposite, and we suggest that this theoretical insight can help explain discrepancies between the way clinicians with and without genetics expertise interpret and trust results obtained from genetics laboratories.

Methods

Our analysis draws from 40 semi-structured interviews with clinicians involved in the ordering and interpreting of genetic tests, including 20 clinical geneticists or genetic counselors with specific training in genetics and 20 cardiologists or primary care providers without specific genetics training. The data presented here are part of a larger study on uncertainty, evidence, and actionability in clinical genomics. We did not design the study to assess how clinicians view and respond to discrepancies in variant classification and interpretation—these insights instead emerged in the data collection and analysis process, as is common in qualitative research. Thus, we consider this research preliminary and hypothesis-generating, and we are unable to determine whether thematic saturation was reached in our sample on the specific topics of clinician trust in genetic laboratories or laboratory discordance. All participants in our sample were asked and answered questions about these topics. Participants were recruited by email through listservs or snowball sampling. Because this is a small qualitative study, we did not intend to enroll a fully representative sample but still sought demographic diversity. All participants were based in the United States, and most came from large academic medical centers. The study was determined to meet IRB exemption criteria by the IRB at the University of Pennsylvania in August 2020.

Our analysis explores similarities and differences between clinicians with particular genetics expertise versus clinicians without particular genetics expertise. We analyzed transcripts and field notes from interviews using the principles of situational analysis, a modified approach to grounded theory [10]. Two authors (KO and DMA) completed coding and thematic analysis using NVivo for Mac 1.7.1, and any coding discrepancies were resolved via discussion. Data analysis took place between January and September 2022. More comprehensive details about the methods for this study can be found in the Consolidated Criteria for Reporting Qualitative Studies (COREQ) checklist in the Supplementary Materials, which also includes our interview guide.

Results

Clinical Geneticists and Genetic Counselors

A main finding of our analysis is that clinicians with genetics expertise report significant frustration with and uncertainty about lab reports for whole genome or exome tests. As genetic tests get larger and are carried out by a greater number of labs, there are more opportunities for discordance to arise. Clinicians with specific expertise in genetics expressed concern that despite efforts to standardize reporting criteria across institutions, results can still vary:

“I struggle a little bit with the fact that all labs are using the ACMG criteria but yet we can still get different answers from labs even when they’re using the same criteria.”

- Clinical Geneticist

Faced with uncertainty over the accuracy of results, these clinicians draw on their own knowledge of genetics in treating patients. While clinicians do not dismiss lab results entirely, they nonetheless will sometimes provide alternative interpretations of findings to patients:

“I always tell [my patients] this literally: ‘I’m going to give you a second opinion on the variant’s interpretation. Trust my interpretation more than what’s written on the [lab] report.’”

– Clinical Geneticist

These clinicians reported frustration with multiple forms of discrepancies with laboratories—concerns about different laboratories’ ability to detect a variant (analytic validity), laboratories’ appropriate classification of that variant (classification accuracy), and clinician disagreements with lab classifications due to their additional knowledge about a patient’s clinical presentation and family history. All of these factors influenced their trust in laboratory reporting, and these concerns were often discussed together. For clinical geneticists and genetic counselors, this uncertainty has significant implications for clinical practice. One genetic counselor interviewed for this study described it as “probably the biggest struggle that we have currently in the field,” noting that:

“[Differences between genetic testing labs are] incredibly frustrating. It becomes much more [about] reaching out to other providers in the field and saying, ‘Which labs have you used in the past? Who do you like? Have I had good experiences sending this in the past? Am I confident in the results that have come back?’ But it usually only takes one bad experience, [like] a result that was missed…for you to completely change your mind…I don’t think it’s fair to anybody, but there’s not a good process right now.”

In general, clinicians with genetics expertise were more likely to raise doubts or second-guess the validity of results in comparison to clinicians without genetics expertise.

Clinicians Without Genetics Expertise

Given the declining price and expanding availability of genomic sequencing, clinical genetics is rapidly entering medical settings populated by non-experts. Therefore, this study also interviewed cardiologists and primary care providers who are involved in ordering genetic tests but who do not have specific genetics expertise. In contrast to clinical geneticists and genetic counselors, these providers expressed a higher level of trust in lab results. Rather than questioning the results reported by genetics labs, providers described them as “thorough”:

“We use [commercial testing company] fairly extensively. I think their reports are fairly thorough and I don’t have any…I don’t recall any reason to doubt the accuracy of what was reported to me or the context that they provide.”

– Cardiologist

While several factors could explain why these clinicians are more likely to trust lab results, one reason that they provided was lack of genetics knowledge:

“I don’t disagree with the genetic lab results. I wouldn’t know how to disagree with genetic lab results. The biggest problem I have is interpreting [results]…”

– Family Physician

Clinicians whose specialties are not dependent on specific knowledge of genetics and who have not spent significant time understanding genetic technology may not possess the necessary expertise to further analyze these results. Given that genetic counselors report time constraints as a deterrent to giving patients alternative explanations, in the absence of specific knowledge, non-experts are more likely to trust what labs report to them [11]. We did not identify any key differences between groups when broken down by demographic categories such as gender or race, although differences may be more apparent in a larger sample size.

Clinical Impact

The discrepancy between the level of trust invested in lab results by clinicians with and without genetics expertise is likely to have consequences for patient care. One interview participant described a situation in which questionable lab results led a patient to seek potentially unnecessary care elsewhere:

“[I saw] A patient who… had pretty typical [symptoms of a heart condition] but had never had genetic testing… She had the expected pathogenic variant, which was consistent with her [symptoms]. And she also had a variant in [another gene] and so we spent a lot of time trying to figure out whether that… was pathogenic or not. The genetic testing company called it pathogenic… because variants near there were pathogenic. These are complicated proteins with complicated structures and near doesn’t count. [We thought it was a variant of uncertain significance, or VUS]…We evaluated her sister, who is a carrier only of the VUS. Her sister’s [clinical presentation] was normal. Her sister’s three children were all variant carriers, but their [clinical presentations] were completely normal. There was no history of anything. We actually went to the trouble of doing in-vitro testing of that particular variant. It behaves completely normally in-vitro…”

– Clinical Geneticist/Cardiologist

The clinician’s training in both genetics and cardiology, combined with additional genetic analysis and evaluation of the patient’s family members, led them to the conclusion that there was insufficient evidence to support the need for a defibrillator for the patient’s sister. However, providers without genetics expertise often treat these results with less uncertainty, and in this case the patient wound up being treated by another care provider:

The awful bottom line is that the mother of these two girls demanded a defibrillator for her daughter [the sister] and we said we won’t do it. So she went somewhere else and got a defibrillator.”

– Clinical Geneticist/Cardiologist

In situations such as this, whether a clinician interprets lab results with certainty or not has significant consequences for patient care and clinical outcomes. Patients may also receive different diagnoses and treatments depending on access to specific providers.

Discussion

The concept of the “certainty trough” implies that, contrary to expectation, the people with the most knowledge about a technology, which in this case includes clinical geneticists and genetic counselors, end up reporting more uncertainty about the quality of the data and knowledge they are producing [9]. In comparison, people committed to genomics as an avenue for improving health outcomes, but who are users, rather than producers of knowledge, report the highest certainty in the quality of that data. In the case presented here, this is primary care providers and cardiologists—providers who order genetic tests but who do not necessarily have genetics expertise.

More specifically, our preliminary analysis suggests that clinicians with genetics expertise expressed frustration with many of their interactions with commercial genetics laboratories and spent significant time assessing lab quality regarding accuracy and validity, structure and content of lab reports, and billing requirements [11]. Even when clinicians found laboratories they liked, they often reassessed or reclassified variants based on their own research. This is despite efforts to standardize variant classification and reporting from organizations like the ACMG [2]. The impact of this effort from clinical geneticists and genetic counselors is unclear—while these clinicians may play an important role in resolving discrepancies between labs or understanding when labs may be missing important patient or family history details, the benefits or risks of clinicians making their own variant interpretations that differ from laboratory interpretations are unknown. In many cases, we would expect that laboratory geneticists are better positioned and have better training to interpret and classify variants, and clinician disagreements may introduce unnecessary or inaccurate complications.

As clinical genetics continues to move outside of expert spaces into places like primary care offices, the field faces the task of how to appropriately understand the content and context of lab results. Because participants in our sample with expertise in genetics tended to view lab results with much more scrutiny and skepticism than providers without that expertise, we expect that this will have an impact on the way patients understand their results, and their resulting care. Additional research would be necessary to confirm this finding. There are a number of ways the field of clinical genetics can address the problems that result from the “certainty trough:” First, genetics researchers and practitioners can try to further standardize how labs classify variants and report results, as is already occurring on ClinGen Variant Curation Expert Panels [12–14]. Despite existing guidelines, clinicians are still reporting significant differences in practice. Health systems could also standardize how clinicians are taught to understand these results. Or, due to differences in clinician confidence in genetic testing results, health systems could ensure that patients with more complicated genetic results get the expert care that they may need to truly make sense of their results. Given the complex sociotechnical nature of genetic testing, changes to all of these practices are likely required to provide more effective patient care.

Data Availability Statement

Due to the sensitive and potentially identifiable nature of the interview transcripts reported in this study, our research data is not publicly available. Redacted transcripts and other study materials may be made available upon request. Further enquiries can be directed to the corresponding author.