A standardized, evidence-based protocol to assess clinical actionability of genetic disorders associated with genomic variation

Abstract

Purpose

Genome and exome sequencing can identify variants unrelated to the primary goal of sequencing. Detecting pathogenic variants associated with an increased risk of a medical disorder allows the possibility of clinical interventions to improve future health outcomes in patients and their at-risk relatives. The Clinical Genome Resource, or ClinGen, aims to assess clinical actionability of genes and associated disorders as part of a larger effort to build a central resource on the clinical relevance of genomic variation for use in precision medicine and research.

Methods

We developed a practical, standardized protocol to identify available evidence and generate qualitative summary reports of actionability for disorders and associated genes. We applied a semi-quantitative metric to score actionability.

Results

We generated summary reports and actionability scores for the 56 genes and associated disorders recommended by the American College of Medical Genetics and Genomics for return as secondary findings from clinical genome-scale sequencing. We also describe the challenges that arose during the development of the protocol which highlight important issues in characterizing actionability across a range of disorders.

Conclusion

The ClinGen framework for actionability assessment will assist research and clinical communities in making clear, efficient, and consistent determinations of actionability based on transparent criteria to guide analysis and reporting of findings from clinical genome-scale sequencing.

Introduction

Over the past decade, the cost of genome-scale sequencing (i.e., genome and exome sequencing) has dropped dramatically, which has led to its increased application in clinical and research settings.^1-3 The generation of genome-wide sequencing data allows the identification of findings in addition to the variants related to the primary reason for sequencing, such as variants in genes that are unanticipated (incidental findings) or purposefully sought (secondary findings).^{4, 5} Identification of a pathogenic variant associated with an increased risk of a significant, but preventable, medical outcome allows for the opportunity to implement interventions that prevent or mitigate future clinical manifestations in patients and their at-risk family members.^{6, 7} While some genes and associated disorders have a considerable evidence base for clinical actionability, the evidence supporting clinical actionability for most genetic disorders varies widely. As such, there is a need to develop and implement standardized, evidence-based methods to characterize the clinical actionability of genomic findings.⁸ These actionability profiles of genetic disorders could be used to facilitate decision-making surrounding which findings should be considered for return of results to improve future health outcomes.

The Clinical Genome Resource (ClinGen) is a National Institutes of Health-funded consortium of researchers and clinicians who are building an open-access and centralized resource to define the clinical relevance and actionability of genomic variants (http://clinicalgenome.org/).⁹ The ClinGen Actionability Working Group (AWG) is developing practical methods to identify genetic disorders with the greatest clinical utility when detected in previously undiagnosed adults. To this end, the ClinGen AWG developed a standardized protocol to generate evidence-based profiles of clinical actionability of genes and associated disorders in the form of summary reports and semi-quantitative metric (SQM) scores. While the methods to identify the evidence base are transparent and standardized, it is non-comprehensive to allow for practical and efficient consideration of numerous genes and disorders. Thus the summary reports generated by this protocol are focused and are intended to provide the initial guidance regarding the return of findings to patients based on clinical actionability and are not intended to provide guidance on clinical care that would occur downstream.

We applied the protocol generated by the ClinGen AWG to the list of 56 genes and associated disorders recommended by the American College of Medical Genetics and Genomics (ACMG) to be returned as secondary findings to patients when a pathogenic or likely pathogenic variant is detected during clinical genome-scale sequencing.^{10, 11} We describe the issues that arose during the process, many of which have led to improvements in the protocol, scoring metric, and ability to determine clinical actionability. These issues illustrate the challenges of the task and provide insight for future endeavors involving the characterization of clinical actionability. The ClinGen AWG framework provides a structure to enable research and clinical communities to make clear, streamlined, and consistent determinations of clinical actionability based upon transparent criteria to guide analysis and reporting of genomic variation.

Methods

The ClinGen AWG developed a protocol for the standardized synthesis of available evidence and the generation of consensus scores of clinical actionability of genes and associated disorders using reproducible and transparent methods (Figure 1). We adapted methods to identify relevant evidence and abstract information on clinical actionability into summary reports from the Evaluation in Genomic Applications in Practice and Prevention (EGAPP) working group's protocol to evaluate the evidence base for clinical actionability of incidental or secondary findings.¹² The SQM scoring methodology was based on a framework generated by the North Carolina Clinical Genomic Evaluation by Next-generation Exome Sequencing (NCGENES).¹³

Figure 1. ClinGen Actionability Working Group workflow.

AWG=Actionability Working Group; KST=Knowledge Synthesis Team

Clinical Scenario

The clinical scenario under consideration for this protocol was an adult patient who had undergone genome-scale sequencing as part of clinical care. This patient had an incidental or secondary finding in the form of a known or likely pathogenic variant causally associated with a genetic disorder unrelated to the original indication for sequencing, had not been previously diagnosed with this genetic disorder, but may have been symptomatic with associated clinical manifestations. For example, a pathogenic variant in the HFE gene may be discovered as a secondary finding in a patient known to have joint pain, might have undergone treatment for it, but had not been previously diagnosed with familial hemochromatosis.

Clinical Actionability Summary Reports

For the purposes of protocol development, clinical actionability was defined as clinically prescribed interventions specific to the genetic disorder under consideration which are effective for prevention or delay of clinical disease, lowered clinical burden, or improved clinical outcomes in a previously undiagnosed adult. Interventions included patient management (e.g., risk-reducing surgery), surveillance (e.g., colonoscopy), or specific circumstances the patient should avoid (e.g., certain types of anesthesia). Actionability included interventions to improve outcomes for at-risk family members (e.g., increased surveillance). Genetic testing recommendations for at-risk family members alone, however, were not considered sufficient to meet the criteria for actionability. In addition, for the purposes of this protocol, actionability did not include reproductive decision-making.

We assessed the clinical actionability for each topic (i.e., genes and associated disorders) in two stages. Stage I was a rapid rule-out step to eliminate topics from further consideration that did not meet the three criteria for minimal clinical actionability: (1) the genetic disorder is clinically actionable in an undiagnosed adult, (2) if penetrance or relative risk is known, there is at least one pathogenic variant with moderate penetrance (≥40%) or relative risk (≥2) in any population, and (3) the resulting genetic disorder is a significant health condition (Stage I dashboard of the Summary Report template is available in the Supplementary Materials). A single reference was sufficient to satisfy any of the Stage I criteria, so an exhaustive literature search was not necessary at this stage. The topic automatically proceeded to Stage II if all three Stage I criteria were met. If a topic did not meet Stage I criteria, however, the AWG could decide by consensus to make an exception and move the topic to Stage II. No further action was taken if a topic did not meet Stage I criteria and was not recommended for exception by the AWG.

The purpose of Stage II was to document and synthesize key evidence related to the clinical actionability of a pathogenic genomic finding (Table 1). The AWG Knowledge Synthesis Team identified the most current literature on clinical actionability for a given topic by using the standardized search methods outlined in Goddard et al.¹² Briefly, online resources (such as Online Mendelian Inheritance in Man (OMIM) (http://www.ncbi.nlm.nih.gov/omim), GeneReviews (http://www.ncbi.nlm.nih.gov/books/NBK1116/), OrphaNet (http://www.orpha.net), PubMed (http://www.ncbi.nlm.nih.gov/pubmed), and the National Guideline Clearinghouse (http://www.guideline.gov/)) were targeted and filtered to identify clinical guidelines and secondary literature, such as systematic reviews and meta-analyses. Primary literature and narrative reviews were not considered. This search method was transparent, reproducible, and appropriate for the current application, but was limited in scope for efficiency and practicality, and was thus not comprehensive. Each reference identified during the search was examined for relevance to actionability in the context of the clinical scenario. Any references deemed irrelevant were excluded.

Table 1. Knowledge Synthesis Team's Stage II methods to generate summary reports.

Systematic Identification of Sources
Clinical practice guidelines, systematic reviews, and meta-analyses OMIM, GeneReviews, and OrphaNet entries Clinical Utility Gene Cards Excluded: Narrative reviews and single-study research papers
Determination of Relevance
Relevant references provide information on actionability in an adult who has not been diagnosed with the genetic condition, though they may be symptomatic of the clinical manifestations associated with the genetic condition Actionability is defined as interventions or treatments that might lead to disease prevention or delayed onset, lowered clinical burden, or improved clinical outcomes
Tier Ratings of Relevant Sources
Tier 1Evidence from a systematic review, a meta-analysis clearly based on a systematic review, or a clinical practice guideline clearly based on a systematic review Tier 2Evidence from clinical practice guidelines or broad-based expert consensus with some level of evidence review, but using unclear methods or sources that were not systematically identified Tier 3Evidence from another source with non-systematic review of evidence (e.g., GeneReviews, OrphaNet, and Clinical Utility Gene Cards, or opinion of up to 4 experts that provides guidance) with primary literature cited Tier 4Evidence from another source with non-systematic review of evidence (e.g., GeneReviews, OrphaNet, and Clinical Utility Gene Cards, or opinion of up to 4 experts that provides guidance) with no citations to primary data sources Tier 5Evidence from a non-systematically identified source (e.g., nominated by an AWG member for inclusion)
Data Abstraction
Evidence with the highest tier is abstracted for 5 aspects of clinical actionability: What is the nature of the threat to health for an individual carrying a pathogenic allele of the given gene? How effective are interventions for preventing harm? What is the chance that this threat will materialize? How acceptable are the interventions in terms of the burdens or risks placed on the individual? Would the underlying risk or condition escape detection prior to harm in the setting of recommended care?

Once we identified all relevant references, each reference was assigned a quality tier based on predefined criteria (Table 1). Information from the highest-tiered source was abstracted and condensed into a summary report with standardized information documented for each topic (the Summary Report template is available in the Supplementary Material). The information abstracted addressed five of the most important aspects of clinical actionability: (1) the nature of the threat to health for an individual carrying a pathogenic allele of the given gene or genes, (2) the effectiveness of available interventions for preventing or mitigating harm, (3) the likelihood that the threat will materialize (i.e., the penetrance of that trait attributable to the variant), (4) the nature of the intervention in terms of the burdens or risks placed on the individual with the genomic finding, and (5) the chance that the underlying risk or disorder could escape detection prior to harm in the setting of standard care. Information for this last domain was abstracted only for the summary reports only and was not incorporated into the scores for clinical actionability.

AWG Review of Summary Reports

While the methods for identifying evidence on clinical actionability allow for the transparency and feasibility of addressing a large number of topics, these reports do not reflect a comprehensive overview of all primary literature and thus limit our ability to inform all aspects of clinical actionability. As such, discrepancies may emerge between the summary reports and primary study publications or widely accepted expert clinical practice regarding clinical actionability of a particular topic. To address this issue, AWG members reviewed preliminary summary reports and nominated additional sources for consideration, such as primary literature, when applicable (Tier 5, Table 1). This important contribution made the process more comprehensive and allowed all domains of clinical actionability to be scored.

Semi-quantitative Scoring Metric Application

Selection of Outcome–Intervention Sets for Scoring

Some genetic conditions are associated with a variety of distinct clinical outcomes that vary in terms of severity, actionability, or likelihood. Likewise, interventions vary with regard to effectiveness, risk, or burden. For example, hereditary breast and ovarian cancer (HBOC) is associated with breast and ovarian cancer, which are distinct in likelihood and actionability and the interventions for these outcomes vary in effectiveness, risk, and burden (e.g., surveillance versus risk-reducing surgery). Thus, assessing all possible outcomes and interventions simultaneously to generate a single clinical actionability score would be misleading. The AWG addressed this issue by considering specific outcome–intervention pairs within the defined clinical scenario and scoring each pair separately for clinical actionability. The outcomes selected for scoring referred to significant and likely actionable clinical manifestations of the genetic disorder, which were then paired with relevant interventions. HBOC, for example, had three outcome and intervention pairs identified for scoring: (1) breast cancer and surveillance, (2) breast cancer and risk-reducing surgery, and (3) ovarian cancer and risk-reducing surgery.

Scoring Method

Assuming a pathogenic variant in the gene or genes under consideration, the AWG members applied the SQM to the information in the corresponding summary report to score each topic for four domains of clinical actionability: (1) severity of the outcome, (2) likelihood of the disease, (3) effectiveness of the intervention, and (4) nature of the intervention in terms of burden and risk to the patient (Table 2). Each domain was scored from 0 to 3, resulting in a final “actionability score” which could range from 0 (least clinically actionable) to 12 (most clinically actionable). In addition, scorers rated the knowledge base regarding the likelihood of the disease and the effectiveness of the intervention; this scale ranged from substantial evidence (“A”) to poor evidence (“D”) as well as non-systematically identified sources (“E”).

Table 2. Semi-quantitative metric to score clinical actionability.

Domain	Scores
Severity: What is the nature of the threat to health to an individual carrying a clearly deleterious allele in this gene?	3 = Reasonable possibility of sudden death
	2 = Reasonable possibility of death or major morbidity
	1 = Modest morbidity
	0 = Minimal or no morbidity
Likelihood of disease: What is the chance that a serious outcome will materialize given a deleterious variant (akin to penetrance)?	3 = >40% chance
	2 = 5–39% chance
	1 = 1–4% chance
	0 = <1% chance
Effectiveness of specific interventions: How effective is the selected, specific intervention for preventing or significantly diminishing the risk of harm?	3 = Highly effective
	2 = Moderately effective
	1 = Minimally effective
	0 = Controversial or unknown effectiveness
	IN = Ineffective/No interventiona
Nature of intervention: How risky, medically burdensome, or intensive is a given intervention?	3 = Low risk, or medically acceptable and low-intensity interventions
	2 = Moderate risk, moderately acceptable or intensive interventions
	1 = Greater risk, less acceptable and substantial interventions
	0 = High risk, poorly acceptable or intensive interventions
State of the knowledge base: What is the level of evidence?	A = Substantial evidence, or evidence from a high tier (Tier 1)
	B = Moderate evidence, or evidence from a moderate tier (Tier 2)
	C = Minimal evidence, or evidence from a lower tier (Tier 3 or 4)
	D = Poor evidence, or evidence not provided in the report
	E = Evidence based on expert contributions (Tier 5)

^aDo not score the remaining categories.

Each topic was independently scored by multiple AWG members, then the scores were discussed among the larger AWG. We used consensus in assigning a single actionability score to each outcome–intervention pair.

Results

Clinical Actionability Summary Reports and Scores

The AWG applied the current protocol to the list of 56 genes recommended for return as secondary findings by ACMG.^{10, 11} Three topics did not pass Stage I of the assessment for clinical actionability based on evidence identified in the preliminary literature search. Two of the three that did not pass Stage I, retinoblastoma and WT1-related Wilms tumor, were not determined to be actionable in an undiagnosed adult, and the AWG did not make an exception for these topics to proceed to Stage II. The third disorder that did not pass Stage I based on evidence in the identified literature, Brugada syndrome, was determined to be actionable in an undiagnosed adult and a significant medical problem, but it did not meet the criteria for moderate penetrance or relative risk, as only 20-30% of people with this syndrome experience syncope and 8-12% have at least one cardiac arrest that could cause sudden death.¹⁴ However, the AWG made an exception for Brugada syndrome to proceed to Stage II due to the severity of the clinical outcomes. Thus, Brugada syndrome and the remaining topics from the 56 genes assessed by the AWG (ACMG 56) that passed Stage I have completed the AWG workflow and have been assigned consensus scores for clinical actionability (Table 3). These scores indicate mid to high levels of actionability and a range of available evidence to inform clinical actionability with regard to clinical interventions and the likelihood of clinical manifestations. The summary reports generated by the AWG are available on the ClinGen website (www.clinicalgenome.org).

Table 3. Clinical actionability scores for the ACMG 56^a.

Disorder	Gene(s)	Outcome/Intervention Pair	Severity	Likelihood	Effectiveness	Nature of Intervention	Total Score
Marfan syndrome	FBN1	Aortic dilation progression/Beta-blockers	3	3C	3A	3	12CA
		Clinically significant aortic aneurysm/ Surveillance	3	3C	3B	3	12CB
Loeys–Dietz syndrome	TGFBR1, TGFBR2, SMAD3	Clinically significant aortic aneurysm/ Surveillance	3	3C	3C	3	12CC
		Aortic dilation progression/Beta-blockers	3	3C	3C	3	12CC
Familial thoracic aortic aneurysms and dissections	FBN1, TGFBR1, TGFBR2, SMAD3, ACTA2, MYLK, MYH11	Clinically significant aortic aneurysm/ Surveillance	3	2Db	3Cb	3	11DC
		Aortic dilation progression/Beta-blockers	3	2D	3Cb	3	11DC
Familial hypercholesterolemia (heterozygous form)	LDLR, APOB, PCSK9	High cholesterol/Statins	2	3C	3A	3	11CA
Familial hypercholesterolemia (homozygous form)	LDLR, APOB, PCSK9	High cholesterol/Statins	2	3C	3A	3	11CA
		High cholesterol/Low-density lipoprotein apheresis	2	3C	3B	2	10CB
Von Hippel–Lindau syndrome	VHL	Pheochromocytoma/Surveillance	2	3C	2B	3	10CB
		Renal cell carcinoma/Surveillance	2	3C	3C	3	11CC
Dilated cardiomyopathy	TNNT2, LMNA	Sudden cardiac death/Surveillance and pharmacology	3	3C	2Cb	3	11CC
		Sudden cardiac death/Pharmacotherapy	3	3C	2B	2	10CB
		Heart failure/Surveillance and angiotensin-converting enzyme inhibitors	2	3C	2B	3	10CB
Multiple endocrine neoplasia type 2B	RET	Pheochromocytoma/Biochemical surveillance	2	3C	3B	3	11CB
		Medullary thyroid carcinoma/Risk-reducing surgery	2	3C	3A	1	9CA
Multiple endocrine neoplasia type 2A; familial medullary thyroid cancer	RET	Morbidity or mortality from pheochromocytoma/Biochemical surveillance	2	3C	3B	3	11CB
		Medullary thyroid cancer/Prophylactic thyroidectomy	2	3C	3A	1	9CA
		Morbidity or mortality from hyperparathyroidism/Biochemical surveillance	1	2C	3B	3	9CB
Peutz–Jeghers syndrome	STK11	Colorectal cancer/Surveillance	2	3A	3B	3	11AB
		Breast cancer/Surveillance	2	3C	2A	3	10CA
		Breast cancer/Chemoprevention	2	3C	2A	2	9CA
		Breast cancer/Risk-reducing surgery	2	3C	3B	1	9CB
		Ovarian cancer/Risk-reducing surgery	2	2A	3B	1	8AB
		Ovarian cancer/Surveillance	2	2A	0B	3	7AB
Neurofibromatosis, type 2	NF2	Vestibular schwannoma/Surveillance	2	3Bb	2B	3	10BB
Ehlers–Danlos syndrome, type 4	COL3A1	Vascular or organ rupture or perforation/Avoidance of invasive procedures	3	3C	2A	2	10CA
MYH-associated polyposis	MUTYH	Colorectal cancer/Surveillance and colectomy	2	3C	3B	2	10CB
Malignant hyperthermia susceptibility	RYR1, CACNA1S	Morbidity from malignant hyperthermia event/Avoidance of triggering anesthetics	2	2D	3B	3	10DB
Catecholaminergic polymorphic ventricular tachycardia	RYR2	Sudden cardiac death/Beta-blockers	3	2Eb	2B	3	10EB
Multiple endocrine neoplasia type 1	MEN1	Morbidity from other neuroendocrine tumors/Biochemical surveillance	2	3C	2B	3	10CB
		Morbidity from parathyroid tumors/Biochemical surveillance	1	3C	3B	3	10CB
		Morbidity from other neuroendocrine tumors/Imaging surveillance	2	3C	2B	2	9CB
PTEN hamartoma tumor syndrome	PTEN	Breast cancer/Surveillance	2	3C	2B	3	10CB
		Thyroid cancer/Thyroid ultrasonography	2	2C	2B	3	9CB
Hereditary paraganglioma-pheochromocytoma syndrome	SDHD	Paraganglioma development/Surveillance	2	3C	2C	3	10CC
	SDHB	Paraganglioma development/Surveillance	2	3C	2C	3	10CC
	SDHAF2	Paraganglioma development/Surveillance	2	2C	2C	3	9CC
	SDHC	Paraganglioma development/Surveillance	2	2C	2C	3	9CC
Hereditary breast and ovarian cancer	BRCA1	Breast cancer/Surveillance	2	3A	2A	3	10AA
		Breast cancer/Risk-reducing surgery	2	3A	3A	1	9AA
		Ovarian cancer/Risk-reducing surgery	2	2A	3A	1	8AA
	BRCA2	Breast cancer/Surveillance	2	3A	2A	3	10AA
		Breast cancer/Risk-reducing surgery	2	3A	3A	1	9AA
		Ovarian cancer/Risk-reducing surgery	2	2A	3A	1	8AA
Lynch syndrome	MLH1, MSH2, MSH6, PMS2	Colorectal cancer/Surveillance	2	3Ab	3A	2	10AA
		Endometrial cancer/Surveillance	2	3Ab	1A	2	8AA
		Endometrial cancer/Risk-reducing surgery	2	3Ab	3B	1	9AB
Romano–Ward long QT syndrome	KCNQ1, KCNH2	Sudden cardiac death/Beta-blockers	3	2C	2B	3	10CB
		Sudden cardiac death/ Implantable cardioverter defibrillator	3	2C	2A	2	9CA
	SCN5A	Sudden cardiac death/Beta-blockers	3	2C	0D	3	8CD
		Sudden cardiac death/ Implantable cardioverter defibrillator	3	2C	2A	2	9CA
Li–Fraumeni syndrome	TP53	Colorectal cancer/Surveillance	2	3C	3Cb	2	10CC
		Breast cancer/Surveillance	2	3C	2Cb	3	10CC
		Breast cancer/Avoidance of radiotherapy	2	3C	2Cb	3	10CC
		Ovarian cancer/Avoidance of radiotherapy	2	3C	2Cb	3	10CC
		Colorectal cancer/Avoidance of radiotherapy	2	3C	2Cb	3	10CC
		Sarcomas/Avoidance of radiotherapy	2	3C	2Cb	3	10CC
		Central nervous system tumors/Avoidance of radiotherapy	2	3C	2Cb	3	10CC
		Adrenocortical tumors/Avoidance of radiotherapy	2	3C	2Cb	3	10CC
		Sarcomas/Surveillance	2	3C	2E	3	10CE
		Adrenocortical tumors/Surveillance	2	3C	2E	3	10CE
		Breast cancer/Risk-reducing surgery	2	3C	3Cb	1	9CC
		Ovarian cancer/Risk-reducing surgery	2	3C	3Cb	1	9CC
		Colorectal cancer/Risk-reducing surgery	2	3C	3Cb	1	9CC
		Central nervous system tumors/Surveillance	2	3C	1Cb	3	9CC
		Ovarian cancer/Surveillance	2	3C	0Cb	3	8CC
Tuberous sclerosis complex	TSC1 TSC2	Morbidity and mortality from masses/Imaging	2	3E	2B	3	10EB
		Subependymal giant cell astrocytoma development/mTOR inhibitor treatment	2	2E	2B	2	8EB
		Pulmonary lymphangioleiomyomatosis development/mTOR inhibitor treatment	2	2E	2B	2	8EB
Brugada syndrome	SCN5A	Sudden cardiac death/ Implantable cardioverter defibrillator	3	2C	2B	2	9CB
Arrythmogenic right-ventricular cardiomyopathy	PKP2, DSP, DSC2, TMEM43, DSG2	Sudden cardiac death/ Implantable cardioverter defibrillator	3	2C	2A	2	9CA
		Sudden cardiac death/Anti-arrhythmic therapy	3	2C	1C	2	8CC
Hypertrophic cardiomyopathy	MYBPC3, MYH7, TNNT2, TNNI3, TPM1, MYL3, ACTC1, PRKAG2, MYL2	Sudden cardiac death/Implantable cardioverter defibrillator	3	2E	2B	2	9EB
Familial adenomatous polyposis	APC	Colorectal cancer/Surveillance and colectomy	2	3Ab	2B	1	8AB
Fabry disease	GLA	End-stage organ disease/Enzyme replacement therapy (classic males)	2	3E	1A	2	8EA
		End-stage organ disease/Enzyme replacement therapy (late-onset males)	2	3E	1A	2	8EA
		End-stage organ disease/Enzyme replacement therapy (females)	2	3E	1A	2	8EA
		Stroke prevention/Aspirin (classic males)	2	2E	0B	3	7EB
		Stroke prevention/Aspirin (late-onset males)	2	2E	0B	3	7EB
		Stroke prevention/Aspirin (females)	2	2E	0B	3	7EB

^aTable does not include two disorders from the original ACMG 56 list that did not pass Stage I: retinoblastoma and WT1-related Wilms tumor

^bScores where the score or level of evidence assigned by AWG differs from the summary report

AWG Challenges with Scoring Clinical Actionability

The design and application of the protocol to characterize clinical actionability has presented thought-provoking challenges. For example, defining the outcomes and interventions for scoring clinical actionability was straightforward for some topics (e.g., familial adenomatous polyposis, which is associated with colorectal cancer as its major, life-threatening manifestation that can largely be prevented with colectomy¹⁵) but difficult for others (e.g., Li–Fraumeni syndrome, a disorder associated with a vast array of neoplasms, including breast cancer, colorectal cancer, brain tumors, and osteosarcomas¹⁶). In the latter case, it was difficult to narrow down all potential outcomes and interventions for scoring. Some strategies the AWG considered included scoring only outcome–intervention sets for the most common outcomes or the most severe outcomes. In addition, interventions selected for scoring and their associated risk or burdens were often highly contextual and thus difficult to score. For example, for a woman with a pathogenic BRCA1 variant, the risk and burden of a recommended oophorectomy may be greater during childbearing years than in post-menopausal years. One potential option would be to generate scores across different contexts. In the example of BRCA1 and oophorectomy, this approach could entail generating scores by age (e.g., 25, 40, and 55 years); both the penetrance and burden of the intervention could vary significantly.

Assessment of the effectiveness of an intervention has been discussed extensively among the AWG. Some interventions overtly prevent an outcome, whereas others aim to minimize morbidity through early detection. For example, in women with BRCA1 and BRCA2 variants associated with hereditary breast and ovarian cancer, risk-reducing surgeries (i.e., prophylactic mastectomy and oophorectomy) are effective in reducing breast and ovarian cancer incidence while breast cancer surveillance, such as mammography and magnetic resonance imaging (MRI), are designed for early disease detection to reduce morbidity.¹⁷ Because the AWG pursues both types of interventions, they are scored separately for actionability and the surveillance option is judged by its effectiveness to ultimately prevent a poor clinical outcome rather than its effectiveness in detecting cancer. Similarly, patients with neurofibromatosis type 2, who are at a high risk of bilateral vestibular schwannomas, are recommended to undergo regular MRI surveillance.^{18, 19} MRI surveillance, of course, does not directly prevent vestibular schwannomas, but it is effective in triggering and directing distal and definitive clinical treatment (e.g., tumor excision). Therefore, when the AWG assessed clinical actionability, the effectiveness of both the proximal and distal effects of the intervention were considered. Thus, the concept of effectiveness with regards to our protocol could be fluid across outcomes and interventions and include both reduction in mortality and decreased or delayed morbidity.

The AWG also considered circumstances to avoid while evaluating the clinical actionability of a gene–disease pair. For example, patients with the vascular form of Ehlers–Danlos syndrome should avoid elective surgery and other invasive procedures (e.g., arteriograms) due to an increased risk of vascular rupture or organ perforation.²⁰ The difficulty in scoring the effectiveness of such an intervention lies in the fact that avoiding the circumstance does not improve the health of the patient per se but that not avoiding the circumstance can worsen it.

The protocol developed by the AWG relied heavily on published practice guidelines, with priority given to guidance based on evidence review. Paradoxically, many published guidelines do not provide evidence summaries for their recommendations. Thus, the actual effectiveness of a recommended intervention is sometimes unclear or unknown. For example, a recommendation for patients with Fabry disease is to take aspirin or other platelet-inhibiting agents to reduce their risk of stroke, although the effectiveness of this intervention was not described in the original source.²¹ AWG members were not comfortable scoring the effectiveness of this recommendation without evidence of stroke reduction for these patients, and the absence of supporting evidence limited the ability to measure effectiveness. This issue may eventually be addressed by new guideline criteria submitted to the National Guidelines Clearinghouse (http://www.guideline.gov/about/inclusion-criteria.aspx) as well as the Institute of Medicine's recommendations on developing trustworthy guidelines.²² These criteria would require guidelines to be transparent regarding the methods used, be based on a systematic review of evidence (where it exists), and provide an assessment of the benefits and harms of recommended care and alternative care options.

For rarer disorders, guidelines often base recommendations on evidence extrapolated from more common and related disorders. For example, evidence for the effectiveness of breast surveillance and risk-reducing surgery among women with BRCA1 and BRCA2 mutations has been used to recommend these interventions among women with rarer disorders that also increase the risk of breast cancer (e.g., Peutz–Jegher and Li–Fraumeni syndromes).¹⁷ To account for this extrapolation in these cases, the AWG adjusted the rating for the knowledge base when scoring actionability, such as scoring the knowledge base as a “C” even if the guideline meets criteria for a Tier 1 or Tier 2 reference (Table 2). Similarly, AWG members often gave the knowledge base a higher score than what the guideline tier warranted if their clinical opinion or experience indicated a discrepancy. For example, a Tier 3 source for the penetrance of colorectal cancer on familial adenomatous polyposis was identified, but the knowledge base for likelihood was scored as an “A” based on discussion by the AWG.

Discussion

The AWG applied a standardized protocol to assess clinical actionability to the list of 56 genes recommended by the ACMG for return as secondary findings.^{10, 11} As we expected, these disorders scored at moderate to high levels of actionability. However, the actionability scores for different outcome–intervention pairs within genetic disorders varied. For HBOC, for example, breast cancer and surveillance had a higher actionability score than did ovarian cancer and oophorectomy (10 and 8, respectively). These scores highlight the nuance of generating consensus scores on actionability that account for both the effectiveness and nature of the intervention. In the HBOC example, the intervention deemed more effective in eliminating the harm (i.e., risk-reducing surgery) received a lower actionability score than the intervention deemed less effective in eliminating the harm (i.e., breast cancer surveillance) that was also less burdensome and invasive. In addition, while some disorders have substantial evidence for clinical actionability (e.g., HBOC), others have limited evidence (e.g., hereditary paraganglioma-pheochromocytoma syndrome).

The SQM, which allowed us to balance qualitative and quantitative scoring, was useful because it provided a pre-defined range of scores rather than the presence or absence of potential clinical actionability. This approach also allowed for scorers' subjectivity, a necessary criterion for any metric that must deal with issues that, like actionability, are not entirely quantifiable and incorporate subjective perceptions. The fact that the AWG scoring framework and the Multi-Criteria Decision Analysis developed by EVIDEM²³ have many parallels and were developed independently reinforces the applicability of the scoring approach use by ClinGen.

The actionability scores generated by the AWG protocol should inform ongoing efforts by the ACMG and other groups to maintain and update lists to guide return of secondary or incidental findings. In addition, external stakeholders can apply the scores to prioritize genes for reporting incidental or secondary findings or for other purposes for which actionability is important. For example, the range of scores generated across a variety of genes and associated disorders could be used to identify a threshold indicative of a certain level of actionability to warrant the return of findings to patients. In addition, the end user could decide a priori whether a single outcome and the associated actionable intervention for a disorder and associated gene or genes are sufficient for return of the finding. Actionability scoring methods may also inform other contexts in which actionability is important, such as primary screening.

The limitations of the AWG protocol are worth noting. For example, the recommendations added to the summary reports were deliberately not comprehensive, as focusing on evidence- and expert-based clinical resources and secondary literature allowed generation of brief reports in a practical amount of time and in a consistent format. Thus, the protocol is not designed to identify all available evidence regarding actionability for a particular topic. However, this process identified gaps in synthesized knowledge and the need for future research and clinical translation to improve the knowledge base for clinical actionability of genetic disorders. It also highlights the need for evidence-based guidelines, as recommended by the National Academy of Medicine (formerly the Institute of Medicine).²² This would provide access to evidence summaries supporting clinical recommendations to better describe the effectiveness of clinical interventions, an important aspect of clinical actionability.

The current protocol was limited in scope and did not address certain patient-specific factors that are important to clinical care, such as personal utility, patient perspectives on the burden of interventions, or how the risks of outcomes and burdens of interventions vary with certain patient characteristics (e.g., age). In addition, the current protocol is limited to genetic disorders with clinical actionability among adults and does not address only childhood-onset disorders. Although the limited scope of the protocol was based on feasibility, it can be expanded in future iterations.

The SQM developed for this protocol allowed us to assign a range of scores, rather than the presence or absence of clinical actionability. It should be recognized that the scoring assignments for various aspects of actionability necessarily have an inherently subjective component (Table 2). For example, a disorder with the potential for sudden death (e.g., Brugada syndrome) was assigned a higher score for severity than a disorder that may lead to a more delayed mortality (e.g., Lynch syndrome). The rationale for this was that medical events that could lead to sudden death (e.g., aortic aneurysm) may have little to no clinical warning or window of time after onset to prevent death. Thus for these conditions, a higher score was assigned to account for the arguably greater need to identify patients early in order to implement preventative interventions. In addition, a more clinically invasive intervention (e.g., prophylactic mastectomy) was assigned a lower score for the nature of the intervention than a less invasive intervention (e.g., chemoprevention). Though some may argue that the invasive nature of the intervention may not make a condition more or less actionable, we pre-determined that our assessment of clinical actionability should account for the risk and burden to the patient. Overall, it is important to note that assessments of issues such as actionability are meant to provide a starting point for discussion.

Generation of the summary reports for clinical actionability of additional genetic disorders is ongoing. Advances in health care will provide evidence regarding clinical actionability, such as new or more effective interventions. In addition, our understanding of penetrance, phenotypes, and mutational prevalence of many genes and associated disorders is likely to change. Our knowledge is currently derived, for the most part, from populations accrued through biased ascertainment due to clinical manifestations or prior diagnosis. Study designs that can reduce this ascertainment bias will improve estimates of the prevalence and penetrance of genetic disease. Efforts to expand upon this protocol will likely include plans to ensure that summary reports stay up to date and remain a reliable resource.

In summary, the AWG protocol provides a structured, transparent framework for summary reviews of evidence regarding the clinical actionability of genomic variation which could be used in a variety of contexts. The ClinGen AWG framework provides needed support to the research and clinical communities in general for making clear, streamlined, and consistent determinations of clinical actionability based upon transparent criteria to guide the analysis and reporting of variation detected in genome-scale sequencing.