Implementing Systematic Review in Toxicological Profiles: ATSDR and NIEHS/NTP Collaboration

The Agency for Toxic Substances and Disease Registry’s (ATSDR) Toxicological Profiles provide comprehensive qualitative and quantitative summations of potential adverse health effects from exposure to hazardous substances. This information is subsequently used to derive the Agency’s Minimal Risk Levels (MRLs). These Profiles and their attendant MRLs serve as the scientific basis for the Agency’s applied public health activities such as Site-Specific Health Assessments, Health Studies, Health Education, and Emergency Response. ATSDR’s Profile-development and MRL-derivation processes consist of a multi-tiered, critical evaluation and interpretation of the available scientific literature for a specific hazardous substance (ATSDR, 2003; ATSDR, 1996). Recently, ATSDR has been updating the approach used to develop Profiles by incorporating methods of systematic review (SR). Adoption of SR methods should provide for even more comprehensive, transparent, and organized examination and assessment of the information and conclusions presented in ATSDR’s Toxicological Profiles and Addenda.

Systematic review methods first gained traction in the area of health care interventions, prompting Congress, in 2008, to direct the Institute of Medicine to develop a set of standards for conducting SRs in order “…to assure objective, transparent, and scientifically valid systematic reviews…” of the effectiveness of medical and surgical interventions (IOM, 2011). Although originally intended to evaluate the strength of evidence used to develop guidelines for clinical practice and healthcare interventions (AHRQ, 2012), SR has become an increasingly important tool to search, analyze and summarize information used to make environmental health decisions (Silbergeld and Scherer, 2013; Woodruff and Sutton, 2011).

More recently, the National Toxicology Program’s (NTP) Office of Health Assessment and Translation (OHAT) implemented a program to utilize SR methodology to standardize and document their steps, process and decision-making when evaluating environmental health literature (Birnbaum et. al, 2013). Such information is used by OHAT to assess the state of the science on a topic or develop hazard identification conclusions. Another illustration of the applicability of SR methods to address environmental health issues was a recent recommendation by the National Academy of Science that the United States Environmental Protection Agency (USEPA) consider utilizing SR for evaluating its IRIS documents (NRC 2011).

Because of the demonstrated usefulness of SR and its expanding applications in the field of environmental health, ATSDR’s Division of Toxicology and Human Health Sciences (DTHHS) recently entered into a collaborative project with NTP OHAT to implement SR for updating and evaluating the scientific literature used to develop ATSDR Toxicological Profiles and Addenda. One new Toxicological Profile and three Addenda have been identified for initial development using SR methodology. Briefly, this process will utilize a steps-approach previously outlined by NTP OHAT for evaluating environmental studies used in developing its literature-based health assessments. (http://ntp.niehs.nih.gov/NTP/OHAT/EvaluationProcess/DraftOHATApproach_February2013.pdf). This approach provides a means to: (1) establish specific inclusion and exclusion criteria for individual studies; (2) distinguish between and utilize data from human, animal and in vitro studies; (3) evaluate the internal validity of studies (“risk of bias”); (4) rate confidence in the body of evidence for individual studies; and (5) translate and integrate evidence from animal, human, and in vitro studies to make conclusions regarding the potential hazard to humans. In addition, if the total body of evidence is sufficient, it may be possible to pool the extracted data to perform meta-analyses. Such pooled analyses generally result in increased accuracy and precision. Moreover, meta-analysis could be further restricted to utilize only those studies with a low or moderately low probability of risk. This would decrease the likelihood that effects detected in small studies were due to chance, and also decrease the likelihood that increased effects detected in large studies were due to chance (Egger et al., 1997a; Egger et al., 1997b). Such information would provide better insight into how direction and magnitude of response might change in response to changes in study design. In addition, such analyses would allow ATSDR to use summary information from multiple, validated studies in order to calculate a point of departure for MRL derivation, rather than relying on a single point estimate from just one study. Alternatively, meta-analyses, using SR-derived data, would enable identification of those studies with the lowest probability of risk bias, and thus, the best studies and data for deriving MRLs.

In conclusion, collaboration between NTP OHAT and ATSDR will provide both programs with the experience needed to continue to translate SR methods into the field of environmental health, a process that will inevitably include steps of methods development and refinement. Using SR methods will result in a more transparent and consistent science-based format for analyzing environmental health studies and for developing conclusions to guide ATSDR’s public health decisions and activities. Moreover, ATSDR’s 25+ years of experience with Site-Specific Health Assessments, Health Studies, Health Education, and Emergency Response can provide NTP OHAT with the practical knowledge and perspectives gained from actual, hands-on public health experience. Last, but not least, this collaborative effort will bring cross-Agency consistency to how environmental health information and knowledge is evaluated, interpreted, and applied.