A matrix for bridging the epidemiology and risk assessment gap

Abstract

Environmental epidemiologic research provides invaluable information for understanding the relationship between environmental exposures and health outcomes. Chemical risk assessment, a foundation of public health decision-making, benefits from having information from various disciplines including epidemiology. While epidemiology and risk assessment have common goals of understanding and reducing human health impacts associated with exposure to environmental chemicals, each discipline utilizes different terminologies and skill sets. This contributes to the challenges faced when seeking to use human data in risk assessment. For over twenty years, scientists have recognized that dialogue between risk assessors and epidemiologists is crucial, although to date no specific path forward has been developed for this purpose. This need for communication motivated the organization of a workshop to explore the question “What do risk assessors need in order to be able to improve the value of epidemiologic research for use in decision-making?” This paper describes the outcome of the workshop, specifically a Matrix designed as a communication tool. The Matrix includes a description of key elements that when included in epidemiology design and/or reporting enhance the use of epidemiology results for a risk assessment. The Matrix is not intended to supplant best practices for environmental epidemiology or existing frameworks on integrating multidisciplinary data. Rather, the goal of the Matrix is to improve understanding and communication between the disciplines. Bridging the gap between epidemiology and risk assessment will enrich both disciplines and enhance public health decision-making.

Introduction

The chemical risk assessment process provides an estimate of the likelihood that a specific chemical may contribute to adverse health outcome(s) in the general population and susceptible populations [1,2]. Risk assessors use information from a variety of fields including environmental epidemiology, toxicology, and exposure science to inform management and policy decisions (e.g., development of water quality criteria, soil clean-up targets, reference doses and cancer slope factors). Data derived from human research has certain advantages over animal or other experimental studies when assessing exposure-outcome associations. Obviously, the target species is directly relevant, obviating the need for inter-species extrapolations. Further, when exposure concentrations in the target human study population are known, risk assessors can avoid the use of high-to-low dose extrapolations required for most laboratory animal studies [3,4]. Human data are also valuable in cases where appropriate laboratory animal models are unavailable or of limited relevance [5]. The desire to minimize the use of animals in chemical testing intensifies the reliance on data from other sources, including human studies. Thus, human research can play an increasingly central role in risk assessment. In fact, data from epidemiologic research has been used as the basis for many chemical-specific health-based decisions (e.g., ionizing radiation, particulate matter air pollution, methyl mercury, arsenic, asbestos, lead) [6].

Various agencies have developed approaches for integrating human data into their risk assessments (e.g., [7]). Nevertheless, important challenges for using human data in risk assessment remain. Exposures can be difficult to fully characterize in observational epidemiologic studies [8,9]. Complete and consistent reporting of research methods and results are desirable for the process of risk assessment but can be curtailed due to limits on manuscript length [10]. Inter-study heterogeneity in terms of underlying study design, data analysis, exposure levels, population characteristics and reporting may preclude quantitative comparison(s) across studies. Overall, data with the most utility for risk assessment are derived from research that builds on past evidence and has well-characterized (or reported) exposure and outcome assessments [7,8,11–13].

The disciplines of epidemiology and risk assessment typically utilize different measures of risk, and different terminology and skill sets, but still have common goals. The need to improve links between epidemiology and risk assessment has been discussed for decades. Recommendations have been proffered but no specific path forward has been adopted to raise focus and awareness of epidemiology for public health applications [8,11,14–19]. A selection of evaluative tools regarding design, conduct, analysis, interpretation, and reporting of human studies is compiled on the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network website (https://www.equator-network.org/), but many of these are not directly applicable to environmental research. While guidance documents and frameworks related to observational epidemiology have been proposed and reviewed (e.g., [20–22]), these have been met with resistance for a variety of reasons including their eventual obsolescence, potential for dampening of creativity, and additional burden on researchers [23–26]. Importantly, none of these tools was designed to enhance communication and dialogue between the epidemiology and risk assessment disciplines.

With the goal of enhancing links between epidemiology and risk assessment, a workshopbrought together a group of international experts in epidemiology, exposure science, and risk assessment to explore the question “What do risk assessors need to be able to improve the value of epidemiologic research for use in decision-making?” Conversations and deliberations among this group of scientists occurred both at the Workshop as well as prior to and after the actual meeting. The outcome was a visually friendly tool – referred to as the Matrix - designed to help to bridge the communication gap between epidemiology and risk assessment. The purpose of the Matrix is to identify and describe key elements of epidemiology that when included in epidemiology design and/or reporting enhance its use for the risk assessment, decision-making and to facilitate communication between risk assessors and epidemiologists. In this paper, we describe the Matrix and discuss approaches for its use.

The risk assessment-epidemiology matrix

Briefly, the risk assessment process includes four main steps: 1) hazard identification, 2) dose-response assessment (recognizing that some use the term exposure-response to reflect internal versus external exposure approaches), 3) exposure assessment, and 4) risk characterization [1]. The Matrix is organized according to the first three risk assessment steps, and excludes risk characterization, which is a synthesis of critically evaluated data from the first three steps.

For each of the risk assessment steps included in the Matrix, three elements are identified as select “asks” of an epidemiology study (Table 1). It is recognized that the list of “asks” could be much longer; for example, the Matrix does not include considerations such as control for confounding and temporality, which are important for causal inference and risk assessment. However, the “asks” included in the Matrix are those that were identified by Workshop participants as increasing the utility of a study for risk assessment and yet may not be part of widespread practice for epidemiologic research.

Table 1.

Matrix of priority “asks” of epidemiologic studies for each risk assessment step.

Risk assessment step	Priority asks for risk assessment
Hazard ID	Confirm outcome.	Confirm exposure.	Report methods fully and transparently.
Dose response	Include information on shape of the curve.	Evaluate concordance with previous results.	Describe direction/magnitude of error.
Exposure assessment	Describe source-to-intake pathways.	Describe complete exposure data.	Describe direction/magnitude of error.

Within the Matrix, each cell includes one “ask” (Table 1) and more information on the “ask” is given in the associated text. Conceptually, the Matrix can be used as part of study design or as a guide in deciding what information from the study to include in a publication and associated supplementary materials. Some of the “asks” are related to analyses and reporting and are well suited for use of existing data sets. Journal editors, peer reviewers and research sponsors may also use the Matrix to predict the translational impact of a given study and to support increased funding opportunities for research focused on translation.

In general, the Matrix is:

• a communication tool to advance an understanding of risk assessment and to increase the impact of epidemiology data for risk assessment;

• designed to include elements that have a substantial impact on use of epidemiology data in risk assessment;

• not intended to supplant current best practices for environmental epidemiology or existing instruments;

• agnostic regarding exposure (i.e., the Matrix was not developed with any particular exposure in mind).

The following subsections describe each of the cells in Table 1. Each description begins with the key “ask” and is followed by explanatory information and/or examples.

Hazard identification

Hazard identification is the process of identifying the type of hazard (for example the target organ) posed by the specific exposure [27]. Three important elements from epidemiology studies used by risk assessors are: confirmation of the health outcome and the specific exposure and complete and transparent reporting of study methods (Table 1).

Confirm outcome:

• Risk assessors look for evidence that the methods used for the outcome ascertainment are valid. Studies that can accurately determine the presence or absence (or degree of severity) of the outcome are preferred.

High confidence is expected in outcome ascertainment that relies upon generally accepted or standardized approaches to provide confirmation and detailed information (e.g., medical records and disease registries). Some outcomes have associated quality frameworks that can assist in study design and study evaluation [28–31]. However, not all studies can provide well-defined outcomes that are sensitive and specific. Limitations of self-reported data are well-known, particularly when responding by proxy. Reliability and validation sub-studies can add confidence to the risk assessment. If the outcome is represented by a continuous measure that is expected to change over time (e.g., blood level of a biomarker of effect), repeated samples improve the sensitivity and specificity of the outcome or clinical diagnosis [7,32].

Confirm exposure:

• Identify how the exposure of interest is defined and measured. The more specific the characterization of the exposure, the more confidence risk assessors will have in using the data for hazard identification.

The means by which an exposure is defined, measured and/or estimated varies widely and depends largely on the study design and population. For example, occupational studies may rely on industrial hygiene measures from current and historical records to establish and quantify a specific exposure in time and space. In contrast, studies relying on non-specific exposures - for example by comparing “urban” and “rural” populations or using proximity to sources such as traffic or factories - may not define a specific exposure.

Report methods fully and transparently:

• A complete reporting of relevant methodologic information related to the assessment of the health outcome and chemical exposure increases the confidence in the use of a study in hazard identification.

While information on basic epidemiology study methods is required for most peer reviewed journals, comprehensive reporting is essential to consider the applicability of an individual study for risk assessment purposes, and when comparing evidence across studies. As such, risk assessors may seek more detailed information than is routinely provided. For example, risk assessors are interested in examining study methods to determine whether internally valid study results can be generalized to other populations.

Dose-response assessment

Dose-response assessment entails characterizing the relationship between the dose (or concentration or exposure or estimated level) of the exposure and the occurrence of an effect. Risk assessors look for information in epidemiologic studies on the shape of the exposure-disease curve, the degree to which study data are concordant with past studies, and the direction and magnitude of error (Table 1).

Include information on shape of dose-response curve:

• Epidemiologic studies that include a broad exposure range and multiple exposure levels and that use this information to assess the shape of the exposure-response relationship are more useful for risk assessment.

The Matrix “ask” is to include information beyond relative risks, regression coefficient values, or other similar point estimate results. In other words, can the investigator(s) go beyond the assumption of a linear relationship, and report the relationship between dose (exposure) and outcome over a range of exposures? Regression analysis in the absence of additional analyses to explore the shape of the curve does not inform the potential for biological differences or the potential for a threshold. For studies that include categorical estimates, risk assessors recommend reporting several exposure groups.

Evaluate concordance with previous studies:

• Analyses that consider and replicate the dose estimates of other studies enhance hypothesis testing.

Risk assessors review individual studies in the context of comparable information, both human and animal. Consistency of the dose estimates across studies allows the reviewer to interpret the biological relevance of the exposure. For example, while exposure categories within a given study of “low”, “medium” and “high” are common in epidemiologic studies, the lack of quantification limits their utility in the dose-response assessment. Specifically, the definition of a “low” exposure in one study may be defined as “high” in another population. Similarly, it is common practice to categorize exposure levels into quantiles (e.g., tertiles or quartiles) so that the sample sizes are equal and serve to increase the statistical power. Analyses that replicate the categorical definitions of previous studies foster comparisons across studies and extrapolation efforts.

Describe direction/magnitude of error:

• Quantifying the magnitude, direction and likelihood of bias on the risk estimate improves a study’s value to a risk assessment.

Regulators appreciate a formal evaluation of uncertainty due to systematic error before drawing conclusions or making policy decisions. The idea behind the Matrix is not to tell epidemiologists how to conduct assessments of error or to include guidance on which approach to use. Rather, the Matrix “ask” is a reminder that risk assessors seek information on the impact upon the effect estimate of such errors. Unlike random (sampling) error, quantified using p-values or confidence intervals, systematic error is rarely formally quantified [33]. For example, Lash et al. offered various options for bias analysis techniques and provided guidance for presenting and interpreting bias analysis results [34]. Detailed instructions for basic bias corrections and computer codes for more complex probabilistic and multidimensional models are now available through a variety of on-line sources (see for example: https://sites.google.com/site/biasanalysis/).

Exposure assessment

Exposure assessment provides a qualitative (e.g., low, high or confirmation of exposure) or quantitative assessment of the chemical(s) to which an identified population is exposed. Exposure assessments can consider all sources of exposure and the frequency and duration of exposure. Three elements of exposure assessment included as “asks” are: describe source-to-intake pathway; complete exposure data, and direction and magnitude of error (Table 1).

Describe source-to-intake pathways:

• Risk assessors seek information about the source(s) of exposure (including multiple routes of exposure, if applicable).

Risk assessors may evaluate multiple exposure pathways (e.g., air, soil, food) or sub-populations (e.g., workers, children) to provide guidance on which media will be the focus of an assessment and which pathways may drive risk [27]. Identification of determinants of exposure are informative for understanding risk-related pathways and routes. For example, by comparing environmental samples in air, soil and dust with personal samples of food and urine, investigators were able to identify inhalation as a source of exposure for one pesticide and diet as the source of another [35,36]. Inclusion of this kind of information in epidemiologic research is of high value for risk assessors. In risk assessment parlance, the various relevant sources and pathways of exposure are described in a graphic called a conceptual model.

Describe complete exposure data:

• Risk assessors benefit from having detailed information about the range and distributions of the exposures assessed in their study population as well as information on data quality.

A clear identification of the methods used to estimate exposure in an epidemiologic study will aid risk assessors in evaluating overall study quality. In addition, risk assessors need complete and detailed information about the range and distributions of the exposures in the study population for both relevance and extrapolation purposes. This goes beyond standard epidemiologic reporting practices such as ranges, means and standard deviations. For example, this might include describing the shape of the distribution, providing information on the exposure distribution’s tails, and reporting the proportion of measurements below the analytical limit of detection. For categorical exposure variables, risk assessors look for information on how (and why) the various categories are defined (e.g., are they based on the distribution of exposures in the population, on policy-relevant cut-points or biologically relevant cut-points)?

Detailed descriptions of methods and quality assurance data are also needed to allow appropriate interpretation of study results. Factors such as the timing of sampling, sample collection methods, field and laboratory instrument calibration data, adequacy of biomarker, geographical area, and number of samples are all critical to data interpretation in the risk assessment [15].

Describe direction/magnitude of error:

• An evaluation of the quantitative impact and direction of biases increases the utility for the risk assessor.

Epidemiologic studies often mention the potential for bias due to exposure misclassification, and/or exposure measurement error. However, a quantitative characterization of the different types and sources of errors associated with exposure assessment in an epidemiologic study can describe the potential impact of these errors on the risk assessment. Both Bayesian and non-Bayesian statistical techniques allow for correction of biased effect estimates resulting from errors associated with the exposure assessment (for a short summary of such techniques, see [14,37]). If quantification of uncertainty related to the exposure assessment is not possible, a thorough qualitative discussion of the type, direction, and likelihood of potential exposure measurement error and misclassification bias will provide useful context.

Admittedly exposure misclassification represents only one source of systematic error (bias) in epidemiologic research. Other well-recognized sources of bias include uncontrolled confounding and various selection biases such as differential loss to follow up and self-selection bias. The basic methods for accounting for the various types of bias were described many decades ago [38–40].

Discussion

The Matrix was developed with the goal of having an easy-to-use visual tool that bridges the communication gap between epidemiology and risk assessment. The Matrix is structured using risk assessment steps of hazard identification, dose-response and exposure assessment and incorporates specific elements related to design, analysis and reporting. Given the many advances in epidemiologic methodologies, incorporating one or more element into research practice(s) will likely be of great value. It is anticipated that the following communities will derive value from the use of the Matrix.

The epidemiologist: When designing an epidemiology study and preparing the study protocol, the investigative team can use the Matrix as a guide for including as many Matrix elements as feasible and desired to enhance the use of the ultimate study data for relevance for public health decisions. The value to the epidemiologist is an enhanced understanding of risk assessment needs and use of epidemiology data in public health decision-making.

The risk assessor: As noted above, regulatory bodies have processes, guidelines and frameworks in place to incorporate data from multiple disciplines into a risk assessment (for more information we direct the interested reader to: [1,41–43]). As a communication tool, the Matrix is not intended to indicate whether an epidemiologic study should or should not be used in an assessment. Rather, during the risk assessment process, the Matrix can guide requests for additional information from a study investigator. Selected publications and literature may be reviewed using the Matrix to systematically highlight the extent to which the Matrix “asks” were met and the role that epidemiology data did (or did not) contribute to the risk assessment.

The sponsor/stakeholder: Funding organizations and private stake-holders have a vested interest in the utility of the data generated by epidemiology research. The Matrix can be used to demonstrate the need for additional funds such that epidemiology studies can be designed with risk assessment in mind, and data reported to more fully meet the “asks” associated with the risk assessment process.

The educator: Epidemiology and environmental health are traditional public health core knowledge areas and are covered in required introductory courses for all public health degrees. In recent years, epidemiologists have expressed interest in the development of training and tools to help ensure environmental epidemiologic studies are more useful for human health risk assessments. The Matrix can assist in this goal. Collaborations with the Council on Education for Public Health [44,45] and the Association of Schools and Programs of Public Health would aid in the propagation of the Matrix to inform educators, schools and programs of public health. One of the recommended competencies for a doctoral degree in general epidemiology is to be able to apply epidemiologic evidence in “policy, practice and program developments” [46]. At a minimum, the Matrix will aid practitioners in better understanding how epidemiologic evidence is used in decision-making. Additionally, engaging epidemiologists, toxicologists and environmental health educators to use the Matrix as the foundation for a course or lecture on the use of epidemiology in risk assessment would prepare students for careers in policy-making.

The journal editor: Many journals already provide guidance for authors on journal expectations for transparency, reproducibility and scientific rigor in the conduct and reporting of epidemiologic studies. Additional recommendations, based on key elements in the Matrix, could be incorporated, particularly for journals with a risk assessment and/or epidemiology focus. Similarly, key items can be incorporated into the online manuscript submission software. Ideally, the author guidance should align closely with that provided to peer reviewers. Some journals and publishers offer online tutorials for both authors’ and reviewers’ papers with links to resources, reporting guidelines, and new tools; the Matrix could be added to these resources. Information on the Matrix could also be provided as part of editors’ short courses or workshops for authors and reviewers at society meetings and as part of university training programs.

Conclusions

The Matrix aims to facilitate communication and collaboration among practitioners of risk assessment and epidemiology, to advance their shared goals of improving environmental policy and related decision-making, and ultimately to improve public health. The Matrix – the result of a multidisciplinary workshop that facilitated a dialogue between scientists and policy makers in relevant fields - is designed to define the selected “asks” of risk assessment and to enhance the use of epidemiologic data in decision-making and understanding and cross-communication between the disciplines.

For over twenty years, scientists have recognized that dialogue between risk assessment and epidemiology is crucial in order to use epidemiologic data to support policy-making and to bridge the gap between the disciplines [19,47,48]. It is our hope that the Matrix can serve as a visual communication tool for this dialogue. The Matrix also provides a discussion point for course curricula, as well as grant and journal review processes. Of course, investigators may choose to address selected elements or none of the elements in the Matrix. However, bridging the gap between epidemiology and risk assessment will enrich both disciplines.