Evaluating Harms in the Assessment of Net Benefit: A Framework for Newborn Screening Condition Review

Abstract

Background

The Department of Health and Human Services (HHS) Advisory Committee on Heritable Disorders in Newborns and Children (“Advisory Committee”) makes recommendations to the HHS Secretary regarding addition of new conditions to the national Recommended Uniform Screening Panel for newborns. The Advisory Committee’s decision-making process includes assessing the net benefit of screening for nominated conditions, informed by systematic evidence reviews generated by an independent Condition Review Workgroup. The evidence base regarding harms associated with screening for specific conditions is often more limited than that for benefits.

Procedures

The process for defining potential harms from newborn screening reviewed the frameworks from other public health evidence-based review processes, adapted to newborn screening by experts in systematic review, newborn screening programs and bioethics, with input from and approval by the Advisory Committee.

Main findings

To support the Advisory Committee’s review of nominated conditions, the Workgroup has developed a standardized approach to evaluation of harms and relevant gaps in the evidence. Types of harms include the physical burden to infants; psychosocial and logistic burdens to families from screening or diagnostic evaluation; increased risk of medical treatment for infants diagnosed earlier than children with clinical presentation; delayed diagnosis from false negative results; psychosocial harm from false positive results; uncertainty of clinical diagnosis, age of onset or clinical spectrum; and disparities in access to diagnosis or therapy.

Conclusions

Estimating the numbers of children at risk, the magnitude, timing and likelihood of harms will be integrated into Workgroup reports to the Advisory Committee.

Introduction

In 2004 the United States Department of Health and Human Services (HHS) established the Advisory Committee on Heritable Disorders in Newborns and Children (Advisory Committee) to make policy and practice recommendations to the HHS Secretary [1]. One of the Advisory Committee’s key activities is to evaluate the existing data and make recommendations to the HHS Secretary about addition of new conditions to the national Recommended Uniform Screening Panel (RUSP) for newborn screening. Nominated conditions must meet specified criteria in order to be recommended for addition to the RUSP, including the certainty and magnitude of the net benefit of screening for the proposed condition [2, 3]. An independent Condition Review Workgroup synthesizes the data regarding these criteria for the Advisory Committee using findings from systematic reviews of literature, decision-analytic modeling [4], and stakeholder input [3], organized around the available “chain of evidence” [2] regarding newborn screening, follow-up diagnostic services and treatment.

Net benefit is determined by considering if the overall impact of potential benefits and harms of population-based newborn screening for a condition for early diagnosis and initiation of treatment exceeds the benefits derived from clinical diagnosis of affected children in the absence of screening. The certainty and magnitude of net benefit includes measures of likelihood of outcomes and numbers of individuals affected.

The goal of this report is to describe a framework for systematic assessment by the Advisory Committee of the actual or potential harms from newborn screening associated with conditions nominated for inclusion to the RUSP, either for the overall population screened or for specific sub-populations. An additional goal of the evidence review process is to identify research gaps related to harms [5], but not to find examples of previous harms from screening, if any. This framework has been integrated into the current evidence review process of examination of existing data [6, 7].

Methods

Other Established Public Health Assessments of Harm

Other well established deliberative groups have addressed the need to assess harms by creating methods to systematically categorize and assess the magnitude of potential harms related to preventative screening. For example, the U.S. Preventive Services Task Force (USPSTF) explicitly examines the existing data on harms of screening, diagnosis and treatment of each condition under consideration to make an overall determination of the “magnitude of net benefit” from screening for a specific condition or implementing a particular prevention strategy. Examination by the USPSTF includes an assessment of potential physical and psychological harms associated with both screening and early treatment, such as unnecessary diagnostic procedures derived from the false positives of a screening procedure [8]. The USPSTF generally searches broadly for publications of all harms, including study designs (e.g., case reports and case series) beyond those used to establish benefit of preventive services [8–10]. The U.S. Advisory Committee on Immunization Practices (ACIP) utilizes an explicit modeling approach to estimate the tradeoffs between benefits of vaccination (averted morbidity and mortality), potential harms of vaccine adverse events, and costs [11, 12].

The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) Working group is tasked by the Office of Public Health Genomics of the Centers for Disease Control and Prevention with evaluating evidence related to the analytic and clinical validity and clinical utility of genetic and genomic tests. Their process includes a modeling of estimates of clinical events, including potential benefits and harms (e.g. false positive results) of testing to patients, as well as a consideration of life extension, when appropriate. However, the EGAPP process is hampered by the limited data available on both the benefits and harms of many genetic tests, including uncertainty regarding clinical validity or utility [13]. Other established public health evidence review groups have faced similar challenges of insufficient evidence about harm [14], and at times have explicitly incorporated expert opinion into their decisions [15].

Harris et al. [16] recently proposed a taxonomy of the harms resulting from screening for specific medical conditions, identifying four major domains of harms: physical, psychological, financial, and opportunity costs. That assessment identified how each of these types of harms may occur within the stages of the screening process, including screening, diagnostic evaluation, treatment and monitoring. Methods have been proposed to quantify harms associated with screening that highlight the need to distinguish physical from psychosocial impacts [17] and to identify harms that might occur at the different stages [17, 18], including long-term monitoring for those not being actively treated. The experience of these deliberative groups informs the process of assessing harms of newborn screening for new conditions.

Findings

Defining Harm in the Context of Newborn Screening

In newborn screening, a harm is any adverse impact (i.e. event, risk, or burden) resulting from screening or related follow-up with respect to the well-being of a newborn or the psychosocial health of the family, and can occur at any point within the stages of screening. Potential harms must be considered in comparison to usual clinical care. For example, a child affected by a certain condition would typically be diagnosed some time after the development of symptoms. Therefore, the impact of screening-related harms on diagnosis and treatment would be incremental over the harms that typically accrue once the condition is diagnosed through a clinical evaluation. Other potential harms to newborns include findings that do not confer any clinical benefit, such as the identification of infants with clinically silent genetic variants.

For any given harm, several features should be considered when assessing its impact: number of children at risk, its severity, timing and likelihood. Additionally, because the actual risk and impact of identified harms may be largely unknown at the time of evidence review, consideration for “potential harms” is implicit in this evaluation.

Policy makers considering expanding NBS panels should also evaluate the opportunity costs for state public health programs due to the resources or trade-offs required for implementation [19]. While this type of harm is relevant to the broader context of NBS programs and the universal accessibility of its benefits, our focus is on the impacts of screening on newborns and their families. As part of the condition review process, the Condition Review Workgroup conducts a separate public health system impact assessment that includes an assessment of the resources required to expand newborn screening [3].

The Challenges of Assessing Harms in Newborn Screening

The evidence review process for each condition proposed for the RUSP focuses on identifying key elements for consideration by the Advisory Committee concerning the net benefits of screening and initiation of treatment for an affected child to optimize outcomes of the disorder [2]. The process for nominating conditions requires some documentation or probability of benefit to screening for that condition. In contrast, data on anticipated or demonstrated harms for the Advisory Committee’s consideration may be a lesser focus within the nomination process. Moreover, data on harms also may be less available or apparent. For example, the modest number of subjects often participating in clinical trials related to rare disorders or a lack of randomized trials or multiple open label studies may lead to an insufficient evidence base to delineate the range of potential harms or to estimate their likelihood [20]. Limitations of data about harms may also result from studies constrained by difficulties in subject recruitment, such as pediatric or neonatal intervention trials concerning life-threatening conditions or selectivity in subject enrollment for studies in which at-risk newborns were identified through affected older siblings rather than through population-based screening [21], or if populations previously screened are not representative of the heterogeneous population of the United States [22].

Challenges to detecting, measuring or reporting harms are not exclusive to NBS. Research on the implementation of new medical tests or interventions tend to primarily focus on medical benefit. Consequently, data on harms from clinical studies of tests may be underappreciated, especially if harms are unexpected, delayed, cumulative, non-medical (e.g. psychological) [7, 23–25], or if they accrue primarily in persons who do not have the targeted disease.

Assessing Harms Associated with Screening for a Specific Condition

The Condition Review Workgroup utilizes a set of key questions to explore relevant considerations for all aspects of screening for a particular condition as compared to usual care (i.e., clinical diagnosis) [2]. These questions are evaluated through a variety of approaches, including systematic review, surveys, or decision-analytic modeling. As part of the Workgroup’s consideration of net benefits, harms are assessed through three sets of key questions that relate to the potential harms of screening, diagnosis and treatment associated with the screening of newborns for a particular condition [2, 3].

Characterizing these harms (Table 1) will support their explicit assessment in condition evidence review, as well as to identify associated research gaps. Assessment of harms includes: (1) Description and characterization of the harm identified through the available data and expert review process: (2) Assessment of the incidence, likelihood and magnitude of each identified harm; (3) Determination of who may be harmed, whether distributed to the entire newborn population or to sub-populations, or to families.

Table 1.

Potential harms associated with newborn screening

Aspect of newborn screening	Type of potential harm
	Newborns	Parents/families
Screening (bloodspot or point-of-care)	Pain or other adverse impacts from screening False positives or false negatives of screening	Psychosocial harms associated with false positive laboratory results for unaffected infants
Diagnosis evaluation	Pain or other adverse impacts from diagnostic testing Missed or incorrect diagnosis Disparities in access to diagnostic testinga	Psychosocial harm from diagnostic or prognostic uncertainty in diagnosis, or degree or age of onset of disease manifestations
Treatment and long term follow-up	Pain or other adverse impacts of treatment Treatment with an uncertain impact of disease severity and/or the timing of manifestations Disparities in access to treatmenta	Psychosocial harm from uncertainty of outcomes Psychosocial, financial or other harms associated with long-term treatment Psychosocial harm from treatment decisions

^aSuch disparities could be considered a harm if disparities associated with screening were more pronounced than those encountered with clinical presentation

Harms Associated with Newborn Screening

Physical Pain or Medical Risk Associated with Screening

The dried-blood spot collection process involves a heel stick and is not a significant harm associated with adding another condition to screening panels because it is already a routine practice for other newborn screening tests. Beyond routine dried bloodspot collection, a harm may be identifiable as a painful acquisition of a sample if screening for a particular condition required additional sampling of blood or tissue sampling. For proposed point-of-care screening, pain or medical risk would arise if data collection would include any invasive practice previously not applied routinely to newborns [26], such as the use of an instrument that causes discomfort, requires sedation or is associated with another medical risk (e.g. radiation exposure from a medical scanner or a burn from a skin probe).

Missed Infants with True Diagnosis (Clinical “False Negatives”) Associated with Screening

The potential for delay in clinical diagnosis may result from false negative results from screening tests. Diagnostic delay from false negatives may arise by steering the diagnostic evaluation away from the true condition due to false reassurance of clinicians and families, especially if they are unaware of the potential for false negative results. While newborn screening overall attempts to minimize false negatives through the methodology for developing quality control of screening algorithms [27, 28], screening or diagnostic tests for some conditions, including point-of-care screening, may have relatively low sensitivity [29]. In addition to the fundamental challenges of using screening tests to discriminate populations with overlapping distributions of test characteristics, false negatives can arise from a number of clinical or laboratory practices. These practices include collecting the screening sample outside of the optimal time window, clinical intervention prior to sample collection (e.g. transfusion), sample collection or handling error, analytic insensitivity (e.g. cut-off values that differ by phenotype or ethnicity), incorrect interpretation of results or poor genotype-phenotype correlations in newborns [27, 30, 31].

Positive Laboratory Results Associated with Unaffected Infants (Clinical “False Positives”)

Positive screening results for an unaffected infant (“false positive”) or screening for condition with a long interval between a positive screening result and confirmatory diagnostic testing may induce considerable short-term parental anxiety [32–36]. Potential psychosocial or nonphysical harms include “the possibility for stigmatization, unnecessary anxiety, adverse impacts on parent and family relationships, and other ethical, legal, and social implications” [2]. Newborn hearing screening and other point-of-care testing paradigms may be especially prone to false positive results [37].

To date, falsely identifying unaffected infants due to positive screening results may have been the most commonly identified harm to families from newborn screening. The etiology of falsely identifying unaffected infants is inherent in the trade-off between laboratory cut-offs for screening sensitivity (needing to approach 100 %) and specificity. The impact of false identification unfolds with the need to verify laboratory results from newborns who are later determined to be trait carriers, born to affected mothers, or have clinical factors such as acute illness, prematurity, hyperalimentation, or transfusion. Although there is anecdotal evidence of parental concerns following positive screening results from unaffected infants, evidence of long-term harm is generally lacking [32]. Harms to the newborn screening system from frequent false identification may also arise, such as provider laxity about prompt re-testing and referral for infants, inclusive of those later determined to have true positive results [38], and burdens on public health and clinical staff [39].

Potential Harms Associated with Diagnostic or Prognostic Procedures

Physical Pain or Medical Risk Associated with Diagnostic Evaluation

Harm may arise from diagnostic processes that require invasive medical procedures (e.g. skin biopsies), or risky procedures such as anesthesia or radiation exposure due to diagnostic procedures (e.g. CT scans). Diagnostic processes that are performed on an affected infant or child prior to treatment would not generally be included as an incremental harm because the affected infant would be expected to be subjected to these (and other) tests with clinical presentation. The differential harm from screening would arise for infants with false positive screens who would not normally have undergone the testing, or for whom clinically indicated diagnostic testing would be at an older age when, in some cases, testing would be less onerous or risky.

Psychosocial Harm from Diagnostic or Prognostic Uncertainty

Newborn screening identifies infants at risk for a particular condition, with the expectation that diagnostic evaluation will be confirmed or ruled out. However, family anxiety may arise through diagnostic uncertainty when a standard clinical evaluation requires signs or symptoms typically observed in an infant or child who has had more disease progression (e.g. Krabbe disease) [40]. Uncertainty can arise due to broad phenotypic expression, poor genotype–phenotype correlation with a spectrum of disease manifestations, or delayed age of onset. Harms associated with uncertainty also include the possibility of abandoning the true diagnosis to follow other disorders, in addition to psychological harm to the child as well as to their family. Such harms could stem from requirement for monitoring for disease manifestations (e.g. Krabbe disease) [40], anxiety, reproductive issues for parents, and decisions pertaining to treatment, health care utilization, insurance, location, and infant–parent bonding [41–44]. Family anxiety may also arise from diagnostic or prognostic uncertainty when a decision needs to be made about particularly risky treatment (e.g. hematopoietic stem cell transplantation). These uncertainties may be short-lived or may persist and require long-term monitoring. A similar range of harms may arise from the medical futility of disorders for which no treatment currently exists if diagnosis were derived from incidental findings of screening [45].

Potential Harms Associated with Treatment

Physical Pain or Medical Risk Associated with Treatment

Standard care for an affected infant would not generally be included as differential harm associated with screening because treatment of an affected infant would be expected. Treatment of infants diagnosed by screening could occur earlier than for children diagnosed clinically. Although earlier treatment may be more effective, younger children may be more vulnerable to treatment-associated harms.

The potential adverse events or complications associated with a standard treatment should be included as a potential harm within our framework if the treatment is also likely to be offered to (1) test positive but clinically unaffected infants; (2) affected infants whose disease phenotype renders treatment as less effective or who might not require treatment; (3) younger children who may have greater vulnerability to adverse effects of treatment; or (4) children for whom risks from treatment would otherwise be substantially delayed due to later onset disease manifestation. In these scenarios, treatment harms could result if screening leads to medically risky treatment for children who would otherwise not have undergone those treatments, which is more likely for conditions with a broad spectrum of disease intensity, poor genotype–phenotype correlation, or variable age of onset. Substantial harms may occur if treatment is painful, medically risky or even life-threatening, such as hematopoietic stem cell transplantation or other major biologic therapy, or medications with narrow therapeutic windows or serious adverse effects.

Psychosocial Harm from Treatment Decisions

Harm may include family distress related to adverse disease outcomes if declining treatment (e.g., family refuses presymptomatic therapy because of their perceived risk of treatment), or if treatment is delayed or is not available despite disease progression (e.g., no matched donor for stem cell transplantation, expensive or complex treatment is not available in the local community).

Increased Health Disparities

Newborn screening has led to reduced health disparities through universal access to early diagnosis [46]. Nonetheless, differing access to treatment could exacerbate health disparities associated with screening. Disparities could result from differential access to or insurance coverage of diagnostic procedures or treatment centers if the impact is anticipated to be greater under newborn screening compared with clinical presentation. For example, a child may not have access to highly specialized diagnostic testing or treatment such as transplantation within their state, or if their insurance does not cover certain procedures when performed out-of-state. Additional disparities could also result if diagnostic testing or treatment being used were differentially effective depending on the ethnicity of the child. These disparities may also arise through the clinical diagnosis of a child affected by the condition.

Assessing the Impact of Harms

The potential impact of harms depends on several key considerations: (1) the number of children and/or families at risk of harm, (2) severity of those potential harms for each child or family, (3) certainty or likelihood that a particular harm would happen, and (4) the timing of when the harm would occur. Similar aspects pertaining to impact on families may be taken into account within the context of evidence review. Generating a precise estimate of impact of a particular harm based on these four considerations may be difficult given the lack of data available regarding the harms of screening for a particular condition. In most cases, estimating the impact of harms will require extrapolation from scant data. Despite considerable uncertainty, at a minimum the process of explicit data review and identification of data gaps is intended to enhance the transparency of assessing net benefit by identifying the gaps in evaluable data. State- or other population-based programs that pilot the screening of nominated conditions are expected to help address some of the research gaps, even as states may need to address variability in the context of their own geographic and/or population base. This framework may inform data collection from these and other studies pertaining to screenable conditions.

Integrating Harms into the Modeling Process to Assess Net Benefit

The process of identifying harms and assessing their impact as described here can be explicitly incorporated into the established system of decision analytic modeling that is used by the Condition Review Workgroup for determining the potential of net benefit [4]. This modeling process, adapted from those in use by the USPTF and the ACIP [8–12], aids in the estimation of net benefit by utilizing an approach to determine the upper and lower bounds of foreseen benefit on the population of newborns [4]. The refinement described here provides a framework to model estimates of potential harms, with estimates for each of the features that modulate the impact or magnitude of those harms when sufficient data are available.

In light of the expected paucity of completeness of data on harms, we acknowledge that formal quantitative addition of harms to the existing modeling paradigm may be more challenging for some nominated conditions, depending upon the nature and degree of data gaps. Like the USPSTF, the Condition Review Workgroup has recognized that it can be challenging to quantify with precision the magnitude of harms from any specific preventive measure. Nonetheless, that group has integrated the use of boundaries into their assessment of net benefits that rely on estimates of best and worst case scenarios regarding screening outcomes [8].

Conclusions

Developments in knowledge and technologies are facilitating opportunities to increase the benefit of newborn screening through the potential expansion of the RUSP. Nominated conditions are heterogeneous in risks associated with the screening test, diagnostic algorithm, disease manifestation, and response to treatment. Benefits to screening for early initiation of treatment are typically enjoyed by a small number of individuals for rare or unusual conditions.

Condition Review Workgroup reports to the Advisory Committee about conditions nominated to the RUSP require the systematic collection and synthesis of available data on the associated benefits and harms of implementation. Integrating the refined framework described here for assessing potential harms associated with nominated conditions into the current condition review process is intended to add to the published procedures used by the Advisory Committee in assessing the overall net benefits of screening for newborns and families through the identification and characterization of harms, modeling of their impact at the population level for those harms that can be adequately quantified, and the identification of research gaps.

Significance.

Through systematic evidence review, the Health and Human Services Advisory Committee on Heritable Disorders in Newborns and Children assesses the net benefit of each condition considered for addition to the national recommended uniform screening panel for newborn screening. We describe the standardized approach adopted by the Advisory Committee to evaluate harms and relevant gaps in the evidence for nominated conditions. This approach is based on consideration of other public health evidence-based review processes, expertise in newborn screening and bioethics.