Improving recommendations for genomic medicine: building an evolutionary process from clinical practice advisory documents to guidelines

Abstract

Genomic sequencing and multigene panel tests are moving rapidly into clinical practice for a range of indications, but the evidence to guide appropriate use is currently limited. Well-crafted advice is needed to reduce unjustified practice variation, minimize risk of error and harm to patients, and encourage best practices. In the absence of definitive evidence, provisional advice can be helpful if it clarifies the potential benefits and risks of different courses of action and identifies the knowledge gaps most important to address in future research. This paper proposes an evolutionary process starting with clinical practice advisory documents (CPADs) and culminating in clinical practice guidelines (CPGs), using two case examples to illustrate the need for this process. When evidence is limited, CPADs can clarify current practice options and identify key knowledge gaps. Added evidence can then support updates to the CPADs over time. Ultimately CPADs can provide the foundation for definitive CPGs as the evidence base matures. This approach addresses an important challenge in genomics and may be applicable to other fields in which technology and practice are outpacing evidence generation.

INTRODUCTION

New genomic tests—including exome sequencing and large multigene panels—are rapidly entering clinical practice. These tests take advantage of the decreasing cost of gene sequencing, which offers the opportunity to conduct a comprehensive genetic analysis at a price similar to previous single-gene sequencing tests. Yet the technology is still evolving, and clinicians face a daunting array of testing options. Evidence on outcomes of different testing approaches is limited, and clinical laboratories may vary in their interpretation of the clinical significance of specific gene variants.^1,2

Clinical practice guidelines (CPGs)—defined as “statements that include recommendations intended to optimize patient care…informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options” (ref.³, p. 4)—offer a way to provide advice to clinicians and reduce unwarranted practice variation in genomic medicine. CPGs are widely used in clinical practice, and a number of existing CPGs address tests for specific genes or genetic conditions (e.g.,^4–6). However, the evidence available for tests that utilize massively parallel sequencing of multiples genes (“gene panels”), the exome, or genome is insufficient to support definitive CPGs.^7–10 As part of the LawSeq^SM project,¹¹ a multidisciplinary group considered the contribution of CPGs to the quality of genomic medicine. This group reports here on a proposal for an evolutionary process to address the challenge of limitations in evidence, starting with provisional advice in the form of a clinical practice advisory document (CPAD) as described below. We note that the process may be applicable to any field of medicine in which technology and practice outpace the production of definitive clinical evidence.

CRAFTING CLINICAL RECOMMENDATIONS

A committee of the Institute of Medicine (IOM, now the National Academy of Medicine) published a report in 2011 articulating criteria that define trustworthy CPGs.³ These criteria emphasize the avoidance of conflicts of interest by members of the guidelines group, systematic and transparent evaluation of evidence, stakeholder input, ratings of both the quality of evidence and the strength of recommendations, external review, and plans for updating. A related effort supported by the Canadian Institute for Health Research, the AGREE Enterprise, developed similar criteria.¹² These two initiatives provide a pathway for developing evidence-based CPGs. Both note the importance of linking the strength of recommendations to the quality of the supporting evidence.

The IOM report distinguished CPGs from “other forms of clinical guidance derived from widely disparate development processes (e.g., consensus statements, expert advice, and appropriate use criteria)” (ref.³, p. 5). The report acknowledged the potential value of such advisory statements, but deemed them beyond its scope. The report thus missed an opportunity to address the role of other forms of guidance in fastdeveloping domains of practice such as genomic medicine, and the potential evolutionary contribution of provisional advice to the eventual development of CPGs.

Our proposal addresses not only a gap in the IOM report, but also a struggle facing multiple professional societies. Societies such as the American College of Medical Genetics and Genomics (ACMG) and National Society of Genetic Counselors (NSGC) have generated guidance documents and other tools to support responsible practice in the fastdeveloping domain of genomic medicine, with advisory documents other than guidelines labeled as “policy statements,” “points to consider,” “clinical practice resources,” or similar terms.^13,14 This array reflects the importance of distinguishing between CPGs and other documents striving to guide practice and advance patient care prior to the availability of the kind of robust evidence base that allows formulation of CPGs. We propose a new form of practice advice, a CPAD, as a way to address this need. By proposing CPADs as a category of advice prior to CPGs and articulating standards for CPADs that support an evolutionary trajectory to CPGs, we seek to aid societies’ efforts to systematize the pathway to CPGs, reliably signal to practitioners the type of advice offered, and advance progress in genomic medicine.

What distinguishes a CPAD is that it addresses an emerging area of practice where both clinical experience and evidence are still limited but follows a rigorous process that provides a foundation for a future CPG. A comparison of CPGs and CPADs is provided in Table 1. CPADs rely primarily on judgments about potential benefits and risks of testing, because evidence about actual test outcomes is lacking. Although CPGs involve judgments as well, they are ideally based on substantial evidence about the benefits and harms of testing. Thus a CPAD produces a discussion of advantages and disadvantages of different test uses (similar to a points to consider approach), together with specific recommendations for the research that would be most helpful to clinicians, while a CPG produces recommendations supported by high quality evidence (Table 1). We argue that this distinction is important and that the use of the CPAD terminology helps to clarify the nature of the advice that can be offered when evidence is limited. The various advisory documents currently being created by ACMG,¹³ NSGC,¹⁴ and other professional organizations often contain some of the elements suggested for a CPAD and may offer a starting point for the development of this more transparent approach to provisional advice.

Table 1.

Comparison of CPADs and CPGs

Key feature	CPADs	CPGs
State of field and testing technology	Early in assessment of the field and testing technology	Later in assessment of the field and testing technology
State of the evidence	Insufficient evidence to evaluate benefits and harms of testing	Sufficient evidence to evaluate benefits and harms of testing
Product(s)	(1) Advice for clinicians based largely on expert judgment about the potential benefits and harms of testing, the uncertainties in current evidence, and the trade-offs for different approaches (2) Recommendations for research to improve clinical test use	Recommendations for clinical practice based on evaluation of the evidence, with strength of recommendations determined by quality of evidence
Updating	Annual review of relevant evidence with revision of CPAD or decision to proceed to CPG	Appropriate plan for updating, as determined after CPG development

CPAD clinical practice advisory document, CPG clinical practice guideline.

Evaluating evidence is a critical step in determining whether a CPAD is needed. The evidence required for different uses of genomic testing may vary, and the evidence available must be considered as a whole.^15,16 In general, a CPAD is likely to be the optimal approach when a systematic review of studies addressing test outcomes reveals insufficient evidence to evaluate the scope of benefits and harms from testing. Thus, as summarized in Table 2, lack of evidence about testing outcomes, or evidence derived from limited or poor quality studies, would argue for a CPAD rather than a CPG.

Table 2.

State of evidence suggesting need for CPAD

Analytic validity	• Sufficient evidence of test reliability to justify clinical use (requirement for both CPAD and CPG; evidence may be more mature in case of CPG)
Clinical validity	• Expert opinion regarding an association between gene variant(s) and health condition
	• Variable quality of data indicating association between gene variant(s) and health outcomes
Clinical utility	• Expert opinion regarding value of testing in health-care delivery
	• Significant deficits in evidence assessing:
	◦ Clinician, patient, and family views on value of testing; methods may include interviews, focus groups, surveys, and/or
	◦ Outcomes of testing such as diagnostic yield, rate of false positive/false negative results, outcomes of medical decisions based on test results, health improvement deriving from actions based on test results, and costs
Ethical, legal, and social implications	• Limited or absent evaluation of ethical, legal, and social implications of testing

CPAD clinical practice advisory document, CPG clinical practice guideline.

We use two case examples to illustrate the need for CPADs for genomic tests (Table 3). Each case represents an instance in which genomic testing is increasingly being used in clinical practice, yet the evidence base is limited for important aspects of test use.

Table 3.

Key questions related to genome-scale test examples

Case 1: exome testing in work-up of a child with neurocognitive and/or developmental problems
Appropriate candidates for testing	What clinical or laboratory evaluation should be undertaken prior to testing? What is the optimal age for testing?
Trio evaluation	What clinical presentations are best evaluated with trio testing?
Secondary findings	What are the pros and cons of seeking specified secondary findings?
Variants of uncertain significance (VUS) and pathogenic or likely pathogenic findings unrelated to clinical presentation	What pretest counseling is recommended related to the possibility of VUS and findings unrelated to clinical presentation? What laboratory and clinical follow-up is recommended for each?
Case 2: patient with a strong family history of breast cancer
Scope of gene panel	What are the relative benefits and harms of more inclusive vs. less inclusive panels?
Implications of patient characteristics	Should panels differ for patients with cancer vs. those being tested for a positive family history? How should family history or other patient characteristics influence the scope of the panel?
Moderate risk variants	What are the pros and cons of including genes associated with moderate risk?
Variants of uncertain significance (VUS) and pathogenic or likely pathogenic findings unrelated to clinical presentation	What pretest counseling is recommended on the possibility of VUS and findings unrelated to clinical presentation? What laboratory and clinical follow-up is recommended on each?
Clinical follow-up	What interventions should be informed by test results?

CASE EXAMPLES

Case 1: genomic work-up of a child with high suspicion of a neurodevelopmental genetic disorder but nonspecific presentation

Children presenting with a constellation of neurodevelopmental problems, congenital anomalies, and/or developmental abnormalities often have a genetic condition. If the presentation is consistent with a known syndrome, targeted genetic testing may provide confirmation. However, many children present with a nonspecific combination of findings. Recent studies document that an exome analysis may offer a higher diagnostic yield and more rapid diagnosis than sequential genetic testing.^17–21 Trio analysis (testing of an affected child and his/her parents) may further enhance diagnostic yield.^22–24 While a 2012 policy statement from the ACMG included this scenario among the indications for exome/genome testing,²⁵ and this testing approach has been described as an emerging standard,²⁶ unresolved questions remain.

One question is the optimal approach to identify candidates for exome-based diagnosis. Good quality phenotypic information helps to inform testing decisions and variant interpretation, underscoring the importance of defining the pretest clinical work-up, including the optimal age for evaluating the phenotype.¹⁷ In addition, there is a need to clarify when the more intensive process of trio analysis is preferred.

Another important question is whether—and if so, how far—to pursue secondary findings. Policy statements from ACMG recommend reporting pathogenic variants from 59 genes when exome (or other genomic) testing is done, irrespective of the age of the person tested.^27–29 However, other policy statements articulate arguments against this approach, based on the benefits to the child of avoiding predictive information that is not actionable in childhood and on parents’ rights to decline such findings.^29–32 In addition, exome analysis inevitably yields information about gene variants of uncertain clinical significance (VUS) and may also identify pathogenic (P) or likely pathogenic (LP) variants of unclear relevance to the patient’s clinical status.^17,26,33 Standards for addressing these findings, including when to offer them to parents and what laboratory and clinical follow-up should be pursued to resolve uncertainties, are evolving.

A CPAD could offer advice to clinicians on these questions, outlining the advantages and disadvantages of different approaches and clarifying the justifications and potential outcomes of exome diagnosis for different clinical presentations. Equally important, a CPAD could help guide the choice of research approaches most likely to resolve current uncertainties.

Case 2: genomic panel work-up of an individual suspected of, or at risk for, a specific genetic disorder with known causes (example: a woman with a strong personal and family history of breast cancer)

The number of genes known to be associated with hereditary breast cancer is small enough that exome analysis is not necessary; instead, a defined gene panel that tests genes implicated in inherited breast cancer risk can offer high sensitivity for identifying relevant gene variants. How large a panel to use, however, is an unresolved question. Many panels of different sizes are available from commercial laboratories, ranging from panels focused on a small number of genes commonly involved in inherited breast cancer risk to large panels that include genes for all known syndromes in which breast cancer occurs. Panels also differ in whether or not genes associated with low penetrance are included. The evidence for inclusion varies; the evidence base for some genes, such as BRCA1 and BRCA2, is strong but much less is known about other genes included on multigene panels.

A policy statement from the National Comprehensive Cancer Network notes that multigene testing to identify inherited cancer risk could increase efficiency and improve cost-effectiveness when multiple genes are in consideration; such testing might also be appropriate when initial testing for a particular inherited syndrome has produced a negative result.³⁴ Importantly, this policy statement identified several cautions about the use of multigene panels, including the variability in panels commercially available, the potential for inconsistent interpretation of gene variant pathogenicity among different laboratories, the fact that inclusion of genes associated with moderate risk may generate results of uncertain actionability, and the increased likelihood of finding one or more VUS with larger panels.

These cautions reflect important trade-offs. Smaller panels may have lower sensitivity but higher specificity than larger panels, and larger panels may generate more VUS and more findings of unclear clinical actionability. Larger panels may also increase the likelihood of findings unrelated to the patient’s clinical presentation, as a result of pleiotropy. The optimal panel scope may depend in part on the patient’s clinical presentation and family history. For example, the trade-offs for large panels may differ for patients with and without cancer, because the a priori likelihood of a positive finding is greater in those with cancer. Patient safety issues related to detecting a VUS in the absence of a pathogenic variant may also differ depending on the clinical status of the patient. For example, a VUS occurring in an unaffected woman may be presumed to be the cause of cancer susceptibility and lead to unwarranted prophylactic surgery.³⁵

A CPAD could offer guidance on these questions with respect to particular clinical presentations, identify key issues to address in pretest counseling, and identify follow-up options, including preventive and cancer management strategies that can be informed by test results. As with case 1, the CPAD would also seek to clarify the optimal research approaches to resolve uncertainties.

DEVELOPING CPADS

Professional organizations are likely to serve as leading conveners of committees to generate CPADs, as they are for CPGs, but advice may also come from multidisciplinary consensus groups assembled independently or groups convened by governmental agencies, health-care payers, patient advocacy organizations, or public–private partnerships. The ACMG has played an important role in developing guidance on technical aspects of genomic testing and interpretation, and both the ACMG and the NSGC have provided clinical practice resources for a range of genetic testing applications. Both organizations are positioned to play a key role in developing recommendations concerning appropriate uses of genomic testing.

When constituting groups to formulate specific CPADs, ensuring the appropriate range of expertise and perspectives is essential. The controversy surrounding the ACMG’s guideline on reporting secondary findings from genomic testing (e.g., ^36,37) underscores the importance of collecting input from a wide range of experts and stakeholders.^38,39 Although genomic expertise is essential, many uses of genomic tests require critical input from other specialties, such as the input of developmental pediatricians and neurologists for case 1 and oncologists for case 2. Input from patients and families must also be included, with individual testimony augmented where possible by evidence about typical clinical presentations and common patient experiences. Input on ethical and legal issues is crucial as well.

Differing assumptions and values are an important issue for CPADs addressing genomic tests and may not be easily resolved. On the one hand, strongly held views of leading experts hold potential for bias; on the other hand, specialized expertise may be particularly important when evidence is limited and the technology is rapidly evolving. This is arguably true for both our cases. Groups formulating CPADs thus need to include the views those of individuals who have developed or been early advocates of the testing being considered as well as individuals who are more skeptical. Advisory group members need to be open to conclusions that differ from, and cause a reassessment of, their prior expectations or practices. As called for in the IOM report on CPGs,³ transparency is also important, with disclosure of potential financial conflicts of interest by those involved. Some conflicts may be deemed unacceptable for a member of a group formulating the CPAD, such as participation by a person with a direct financial interest in test use. An advisory group may still wish to hear testimony from such interested parties as part of information gathering.

As in the case of other practice advisory documents,^30,40,41 joint efforts across different organizations may be optimal, to avoid conflicting advice that derives from a limited perspective or biased viewpoints,^8,39,42 and to ensure adequate resources for periodic updating. Collaborative Communities, defined by the Food and Drug Administration (FDA) as continuing forums in which private and public stakeholders work together to address problems “in an environment of trust and openness where participants feel safe and respected to communicate their concern,”⁴³ may offer a productive approach.

ACHIEVING SOUND PROVISIONAL ADVICE

A CPAD offers provisional advice based on limited evidence, and thus should describe the strengths and weaknesses of all acceptable testing approaches. To guide future research, a CPAD should also identify the evidence gaps that are most problematic for determining appropriate test use. As with a CPG, a CPAD should specify the clinical goal(s) of the testing it considers (Table 4).⁴⁴ For example, the primary goal of exome testing in case 1 is generally to make a definitive diagnosis, which may or may not inform treatment. In contrast, the goal for case 2 is improved health outcome, that is, reduction in cancer morbidity and mortality through the use of test results to guide preventive treatment or cancer management. clinical practice advisory document.

Table 4.

Topics to be addressed in a CPAD

Benefits	Based on expert evaluation of limited evidence, how might health care be improved by test results? What outcome benefits might the patient or family experience?
Harms	Based on available evidence and expert opinion, what is the potential for an erroneous, misleading, or misinterpreted result?
	What are the potential harms for the patient or family?
Uncertainties	What uncertainties in laboratory analysis, test interpretation, and outcomes of test use must be considered?
Issue of parsimony	How expansive should testing be? What is the potential for secondary or incidental findings from more expansive testing and what are the related harms and benefits?
Needed evidence	What additional studies are needed to address uncertainties in test use and develop further evidence?

CPAD clinical practice advisory document.

Specifying the individuals whose benefits should be considered is also important. In the case of cancer susceptibility testing, a major benefit of testing for some cancer patients may be to identify relatives who could benefit from preventive care (e.g., ⁴⁵). For other patients, the focus is on the implications of genetic risk information for their own cancer management. A CPAD needs to consider these different goals of testing, to clarify what evidence is relevant for different test uses. The advice provided by a CPAD should thus include the following elements (Table 4):

Specification of potential benefits and harms

An important component of CPAD development is specifying the expected benefits and potential harms for particular test uses based on available evidence and expert opinion (Table 4). In case 1, a definitive diagnosis provided by testing could potentially end a diagnostic odyssey, inform reproductive choices, and provide prognostic information to inform clinical management.³³ For case 2, the potential benefit is information to guide effective cancer prevention or management. For both cases, potential harms include failure to achieve an answer, the provision of uncertain or unsought information, and the costs and inconvenience of testing.

In estimating benefits and risks, the information provided by genomic testing must be understood in context. Interpretation of a cancer panel, for example, depends not only on the gene variants found but also on the patient’s family history. Clinicians thus need advice regarding the relevant factors to consider in ordering and interpreting testing in particular clinical contexts.

Finally, advice about test use needs to be transparent about underlying value judgments. For example, to maximize sensitivity, some clinicians might choose more inclusive breast cancer panels incorporating lower penetrance genes (such as ATM and CHEK2) and genes associated only infrequently with breast cancer, accepting that this testing approach involves both a higher rate of VUS and positive findings that span a wide range in risk prediction. If the CPAD is explicit about the trade-offs involved, those users who value panels with a higher positive predictive value and specificity can decide instead to use a more targeted panel focused on high penetrance genes.

Specification of uncertainties

As benefits and risks are estimated, it is helpful for CPADs to specify what is uncertain. For case 1 several studies have confirmed the increased diagnostic yield from exome testing,^17–21 but these studies have identified candidates for testing in somewhat different ways, and long-term evaluation of testing outcomes has not yet been reported. For case 2, data are needed on the cancer prevention and management outcomes that occur with and without test use, as well as clarification of the relative contribution of different candidate genes to diagnosis. The success of different approaches to VUS resolution is important for both cases.

Consideration of parsimony

The CPAD committee should consider the appropriate scope of genomic analysis to be undertaken, a controversial issue in genomic medicine. In other fields of medical practice, there is increasing concern with overutilization of medical tests,^46,47 and efforts such as the multispecialty Choosing Wisely program⁴⁸ seek to identify and reduce the use of low-value care. From this perspective, medical testing should be based on parsimony—that is, deliberately seeking only the information that is needed to address a clinical question or patient need, while avoiding extraneous information as much as possible—with the rationale that extraneous information can result in distracting or false positive findings, unnecessary work-up, and the potential for added costs and iatrogenic harm. The view of many advocates of genomic testing runs counter to this approach, endorsing instead an expansive approach in which genomic testing is accompanied by active efforts to ascertain secondary findings unrelated to the clinical question for which the patient sought care.^27,49 Arguably, this issue is one of the most challenging raised by genomic tests, because the technology readily supports production of a broad range of information beyond the clinical indication for testing.

The distinction between diagnostic and screening uses of testing is important in this context. When secondary findings are sought in exome analysis (as discussed in case 1), they represent a form of screening—that is, the evaluation of risk in the absence of medical indication—with a lower a priori likelihood of a true positive finding and a higher likelihood of false positive or ambiguous findings, as compared with diagnostic testing. Traditional screening criteria call for definitive evidence that screening can improve health outcomes⁵⁰—evidence that is lacking for most genes suggested for secondary analysis.²⁷ A major question in genomic testing is whether having additional data “in hand” (in the form of raw sequence) changes the calculus about what to analyze. A similar consideration arises with respect to the scope of gene panels. CPADs should therefore consider the potential benefits and limitations of a parsimonious versus a more expansive approach for the test scenarios they are addressing.

Specification of needed evidence

An important component of a CPAD is identification of the research necessary to resolve key uncertainties (Table 4). Specific recommendations for clinical trials, observational studies, or other qualitative or quantitative studies to address particular questions raised by the genomic test at issue can enhance the value of this component of the CPAD. In some instances, CPAD recommendations may include linking test use to evidence development. In this respect, the CPAD concept is aligned with an approach to guideline and evidence development in pediatric cardiology termed a Standardized Clinical Assessment and Management Plan (SCAMP).⁵¹ SCAMPs involve development of consensus recommendations for clinical care, with identification of key knowledge gaps. SCAMP implementation includes clinical data collection, including clinician explanations for any deviations from the recommendations. Based on these data, the guidance is periodically revised. Implementation of CPADs in genomic medicine could offer a similar opportunity to inform evidence development to address key uncertainties.

Moving from a CPAD to a CPG

Given the rapid pace of genomic research, the CPAD process should include an annual evaluation focused on the key uncertainties and evidence gaps identified in the original CPAD. If a systematic review reveals advances in evidence sufficient to resolve the uncertainties, transition to a CPG is appropriate (Table 1).

IMPLICATIONS FOR PAYERS

CPAD goals overlap, at least partially, with health payer decision-making. When new interventions such as genomic tests become available, health payers must determine coverage policies. Typically, this decision-making involves determining what payers consider to be medically necessary and appropriate, with criteria including the potential for a test to affect treatment and improve net health outcomes.⁸

Payers cite multiple sources of information in coverage decisions, including CPGs; clinical, cost-effectiveness, and budget impact studies; technology assessments; and systematic reviews.^9,52 Lack of evidence is a significant barrier to coverage.⁹ Evidence important to payers includes the clinical outcomes resulting from test use and the costs, including the cost of data interpretation and the potential downstream costs of data storage and reinterpretation as new evidence emerges.⁷ In this context, CPADs offer an opportunity to articulate reasoned arguments in support of particular uses of a genomic test when evidence is limited. Identification of evidence gaps may help payers to determine when a decision in favor of “coverage with evidence development” may be appropriate.^8,53 Payers may also value input from experts on study designs that can best address knowledge gaps.⁷

CONCLUSION

While recommendations based on robust evidence are the optimal result of guideline development, there is currently insufficient evidence to achieve this goal for genomic tests. However, provisional advice is possible and can serve to promote consistency in clinical decision-making and facilitate discussion among stakeholders about the most critical evidence needs. We propose the development of clinical practice advisory documents (CPADs) to address this need. Goals for CPADs include clarification of the strengths and weaknesses of different test uses and delineation of the research most needed to improve test use. CPAD development should include an explicit plan for periodic evaluation, so that as evidence and clinical experience accumulate, CPADs can provide increasingly informed advice. When the CPAD group deems the evidence sufficient to support evidence-based recommendations, the CPAD can be superseded by a CPG. CPADs can thus contribute to a dialogue among practitioners, patients, researchers, and policymakers, aimed at developing a common framework to support responsible use of a new technology, and have a crucial role to play in supporting the development over time of more definitive standards of practice.