Measurement Error and Environmental Epidemiology: A Policy Perspective

Abstract

Purpose of review

Measurement error threatens public health by producing bias in estimates of the population impact of environmental exposures. Quantitative methods to account for measurement bias can improve public health decision making.

Recent findings

We summarize traditional and emerging methods to improve inference under a standard perspective, in which the investigator estimates an exposure response function, and a policy perspective, in which the investigator directly estimates population impact of a proposed intervention.

Summary

Under a policy perspective, the analysis must be sensitive to errors in measurement of factors that modify the effect of exposure on outcome, must consider whether policies operate on the true or measured exposures, and may increasingly need to account for potentially dependent measurement error of two or more exposures affected by the same policy or intervention. Incorporating approaches to account for measurement error into such a policy perspective will increase the impact of environmental epidemiology.

Introduction

Environmental epidemiologists work to inform public health decision making by estimating the health impacts of potentially toxic exposures. To estimate these impacts, we rely on measures of environmental exposures, health outcomes, and important covariates, which may be imperfect proxies for the actual quantities of interest. We refer to the difference between the quantity of interest and the measured value as measurement error. Exposure measurement error, in particular, is commonly known to bias estimates of exposure response in environmental epidemiology (1–5). While measurement error is common in this field, users of standard implementations of existing analytic approaches, typically regression models, must assume that measurement error is absent.

Measurement error can adversely affect population health by distorting results from epidemiologic studies. Results from environmental epidemiology studies are used to inform national and international standards on population exposures. For example, exposure response parameters from large air pollution studies have been important inputs into the US Environmental Protection Agency (EPA) standards on PM 2.5, which drive local and national regulation of pollution sources. An alternative to using exposure response parameters in this way is to estimate the health effects of policy changes directly in observational studies and to use these estimates as inputs into regulatory processes (6,7). While this approach has not yet become common in environmental epidemiology, direct estimates of the effects of hypothetical interventions have been used to shape international guidelines in other fields. For example, WHO guidelines on when to initiate antiretroviral therapy in HIV positive individuals were developed based on estimates of mortality under various treatment plans from observational studies (8–11). While approaches exist to account for measurement error in many regression settings, there has been less work on how to account for measurement error when estimating population intervention effects.

Here, we review the types of problems created by measurement error in environmental epidemiology and summarize some common methods to account for measurement error. We discuss these solutions in the context of estimating the public health impacts of environmental contaminants. We review the literature on the unique problems arising due to measurement error in population impact studies and comment on open issues regarding measurement error in estimating impacts of environmental policies.

What is mismeasured?

Epidemiology relies on measurements of physical quantities, any of which may be mismeasured (12). These quantities generally fall into three broad categories: exposures, outcomes, and covariates. Mismeasurement of any of these may result in bias. While the primary purpose of the current review is to address exposure measurement error, specifically as it may lead to bias of population impact estimates, outcome and covariate measurement error may also bias estimates of public health impact. We broadly define “exposure” in this setting as some external substance or hazard for which we are interested in learning about health impacts. We defer to existing literature to differentiate between environmental and non-environmental exposures (13,14), but generally we limit our discussion to external factors that vary over time and space and could be targets of regulation. We acknowledge that the meaning of “environmental exposure” may change according to context, and our discussion applies across many of such contexts.

Types of measurement

In general, environmental exposures are measured on a quantitative, or continuous, scale. Occasionally we may quantify exposure measures using two or more discrete levels. When an exposure measurement is binary, mismeasurement can be referred to as “misclassification” and can be quantified using sensitivity and specificity, which describe the probability of being correctly classified. In such cases, sensitivity and specificity (or predictive values) can be used to account for misclassification in tabular and regression approaches (2,15). Because most environmental chemicals under regulation are quantified on a continuous scale, we focus our review on measurement error of continuous exposures.

Types of error

The types of measurement error and methods to account for measurement error have been described in seminal papers punctuating the history of environmental epidemiology (3–5,16–19). We briefly review these concepts here, and, in the next section, we connect them to contemporary issues in environmental epidemiology.

Measurement error can arise either though deterministic or random mechanisms. Both types may be important, and they can be usefully described as separate entities. Deterministic measurement error refers to measurements that are incorrect by some set, predictable quantity. One common source of deterministic measurement error is “batch effects”, where laboratory measurements of exposure vary systematically by the order of analysis due to lab contaminants or other factors that vary over time (20,21). Deterministic measurement error is generally approached by “de-trending” the exposure measures, which roughly amounts to subtracting the batch specific mean from each exposure measurement. This type of analysis may be warranted if the analyst has knowledge of contamination at the lab (20) or other outside information regarding trends in the exposure measurement. For example, such analyses may be warranted in the case of estimating health effects of long term vitamin D serum levels from a single measurement, which are known to vary cyclically throughout the year (22). This type of analysis can be generalized using flexible approaches (20,22,23).

The random component of measurement error is the sum of the unpredictable forces that result in mismeasurement. The magnitude and direction of bias as well as the approaches to account for such bias are informed by the structure of the measurement error. The structure can be displayed using graphical approaches, such as those outlined by Richardson and Gilks (24), Hernán and Cole (25), and Vanderweele and Hernán (26). These graphical approaches can be used to distinguish between classical and Berkson measurement error (3) as well as differential and nondifferential, and dependent and independent error structures. Under special circumstances, the bias from each of these sources of error can be either negligible or predictable. Rather than focus on these special cases, we take a more general view that each of these error structures is a potential source of bias and that analytic choices should depend on the magnitude, instead of the direction of this bias. As shown by Richardson and Gilks (24), the error structure can inform how we correct for measurement error bias in general settings where either Berkson error, classical measurement error, or a combination of the two may be of concern.

Under classical measurement error, the measured exposure is assumed to be randomly distributed around the true exposure value such that the expected value of the measured exposure is the true exposure (16). Conversely, under Berkson error, the true exposure is assumed to be randomly distributed around the measured exposure value, such that the expected value of the true exposure is the measured exposure (1,27). Berkson error may be particularly important in air pollution studies where individual measurements are derived from an estimated exposure surface (28) or in occupational studies where individual measurements are derived from a job-exposure matrix (16). Often, it is useful to think of both Berkson and classical components operating simultaneously.

Exposure measurement error can be differential or nondifferential with respect to other variables. Usually we use the term “differential exposure measurement error” to refer to the scenario where the error in the measured exposure is associated with the outcome. In some settings, exposure measurement may be differential with respect to confounders or modifiers. Conversely, “nondifferential exposure measurement error” refers to the scenario where the error in the measured exposure is independent of the outcome or covariates.

Finally, exposure measurement can also be dependent or independent (26). Dependent measurement error occurs when errors in the measurements of exposure and outcome are correlated. This can occur when exposure and outcome are measured using a common instrument, such as a questionnaire. Independent measurement error occurs when errors in measurement of exposure and outcome are independent of each other.

Accounting for measurement error

Ideally, one would eliminate (or reduce) bias due to measurement error by improving the measurements taken during the study. However, environmental epidemiologists are often tasked with providing estimates to inform policy decisions using existing data. In such settings, environmental epidemiologists approaches to account for measurement error may be affected by their approaches to informing public health decision making.

We illustrate two potential approaches for how epidemiologists may inform policy decisions using the example of radon exposure and lung cancer. As a rough approximation, we consider that policy decisions are informed by two important agents: the “epidemiologist” and the “regulator.” Panel A of Figure 1 describes a contemporary (“standard”) approach used by environmental and occupational epidemiologists, for example that which was used by the Committee on the Biological Effects of Ionizing Radiation (BEIR) to estimate population lung cancer risk after exposure to ionizing radiation (29). Consider an example in which we wish to estimate the population risk of death due to lung cancer after exposure to radon gas. In this setting, the primary goal of the epidemiologist is to use data from a study (indexed by j) to estimate the study-specific dose-response parameter β_j for the increase in cancer risk (or rate) after occupational exposure to radon gas among underground uranium miners (30–33). The exposure response parameter β_j is a function of the exposure distribution in the study X_j, the study-specific covariate distribution Z_j, information about measurement error π, and some statistical model that may be unique to the study. Each β_j can be assumed to be internally valid for the study population j, after correcting for measurement error using the information in π.

Figure 1.

Diagrams showing how results from environmental epidemiology inform policy under A) the standard perspective, and B) a policy perspective. Legend: j indexes individual studies of environmental exposures. X_j represents the exposure distribution in study j, Z_j represents the covariate distribution in study j, and β_j is the estimated exposure response function from study j. g indexes specific target populations where interventions or policies may be applied. X_g is the distribution of exposure in the population g, Z_g is the distribution of covariates in population g. π represents assumptions and information about the measurement error. ψ̄_gC, is the estimated population impact under the standard framework presented in panel A, which is estimated using β̄ averaged across all studies. ψ̄_gM is the estimated population impact under the policy approach in panel B, based on the average ψ_jg estimated across all studies.

In the standard approach, the regulator then meta-analytically combines the β_j to calculate the average parameter value across populations β̄. This average parameter is used to estimate the effect of a specific policy at the population level (ψ_gC), typically using a life-table analysis of a standard target population defined by a covariate (e.g. age, sex) distribution, given by Z_g under the population exposure under some policy, given by X_g. For example, the BEIR committee estimates the change in the population risk of lung cancer mortality in the US population of men and women of all ages (Z_g), if radon exposure in the United States (X_g) could be eliminated.

Under what we term a policy or population health perspective (34), the division of labor between the epidemiologist and the regulator changes, as shown in Panel B of Figure 1. Using modern epidemiologic approaches, one can combine study population information (X_j, Z_j) with measurement error information (π), information about the target population (X_g, Z_g), and information about the potential intervention in a single model. The output from this type of analysis is a study-specific estimate of the policy target, ψ_gj, or the change in health outcomes we would expect to see in the target population if the intervention were implemented, based on results from study j. The regulator can then combine these study specific estimates into an overall estimate of population risk under a policy, given by ψ_gM. For example, if we wish to estimate the effects of various interventions on radon exposure on lung cancer mortality, we can estimate the change in risk under potential interventions directly (6,7,35). We note that the numerical value for ψ_gM need not equal ψ_gC if β varies across populations.

The policy perspective is appealing simply for the reduction in the number of steps required for an epidemiologic study to inform policy. Such an approach has been conceptualized as casting observational data into the framework of randomized trials (36). Furthermore, using a policy perspective to analyze observational environmental studies as though they were randomized trials of proposed policies reduces bias in some settings, such as when exposure may vary over time (37).

The perspective of the investigator (“standard” vs “policy”) and the parameter of interest (i.e., β_j vs ψ_gj) affect the issues to be considered when accounting for mismeasured variables. However, under both perspectives, all approaches to estimate an exposure response parameter rely on additional information about the measurement process. Information needed to estimate an exposure response parameter in the presence of measurement error can be encoded in a validation study, a reliability study, parametric assumptions, or the investigator’s prior knowledge of the measurement process. The data alone cannot identify any exposure response parameter, but must be accompanied by assumptions regarding (among other factors) the extent and nature of measurement error (38).

Quantitative methods to account for measurement error rely on additional information about the measurement process to act as a lever when accounting for measurement bias. In a validation study, a “gold standard” version of the exposure is measured on a group of subjects in addition to the routinely measured exposure. Validation studies can be internal (i.e., conducted among a subset of study participants) or external (i.e., conducted among a group of people outside the study). In a reliability study, the routinely measured exposure is measured multiple times for a subset of study participants (39). If no validation data are available, approaches to account for measurement error can be based on prior information about the measurement process. For example, in a study of occupational radon exposure, Stram et al. used prior estimates of the coefficient of variation in radon measurements to help correct for radon-measurement error in a cohort of uranium miners (40). Alternatively, one can make parametric assumptions about the relationship between the effect estimate and amount of error in measured variables, as we discuss below in an example of simulation-based methods in another cohort of miners (41).

Accounting for measurement error under the standard perspective

Under the paradigm of Panel A of Figure 1, characterized by a focus on regression coefficients and here termed the “standard perspective,” mismeasurement of exposure X_j or covariates Z_j may bias estimated regression coefficients. In the example scenario where we wish to estimate the relationship between radon exposure and lung cancer mortality, exposure measurement error can arise from two sources: 1) assignment of exposure values based on an area-level average exposure (such as a job exposure matrix), resulting in Berkson-type error and 2) error in the radon monitoring station measurements, resulting in classical error (42). If we are interested in estimating the parameters of the excess relative rate model for the relationship between radon exposure, and lung cancer mortality, estimated regression coefficients and resulting inference may be biased (40,41,43–46). The degree of bias resulting from measurement error arising from use of a job exposure matrix was recently explored by Greenland et al (47).

Under the standard perspective, environmental epidemiologists have made significant progress in accounting for measurement error to yield a valid estimate of β_j. Here, we briefly describe these approaches and point interested readers to papers describing recent applications. The choice of methods used to account for exposure measurement error is driven by the error structure and information available about the measurement process.

As noted above, a data analyst can sometimes account for exposure measurement error using parametric assumptions or prior knowledge. For example, the simulation extrapolation (SIMEX) approach (48) relies on simulating estimates of the parameter of interest β under a scenarios in which additional measurement error is added to exposure or covariates and then extrapolating results back to a scenario with no measurement error. SIMEX requires specifying a parametric form for the relationship between the amount of measurement error added and the estimated regression coefficient. SIMEX is appealing in many settings because it does not require validation data. In addition, under each measurement error scenario, β may be estimated with standard models and statistical software (49). In a simulation study based on historical radon exposures in miners, Allodji et al showed that SIMEX compared favorably with other likelihood based approaches to estimate a relative rate parameter (45). The authors used the approach to correct for both Berkson and classical error, which they suggest may have resulted in large biases in previously estimated exposure response parameters in occupational studies of radon exposure (41). However, the authors showed that SIMEX may be unreliable when parametric assumptions about measurement error are inaccurate.

Alternatively, one could encode knowledge or beliefs about the measurement error process using likelihood-based approaches. These approaches can be Bayesian, in which a prior distribution is specified for parameters governing the measurement process (50,51), or non-Bayesian, in which these parameters are assumed to be fixed. The non-Bayesian approaches (described by Carrol (1)) have seen little use in the literature, in part due to the relatively simple implementation in the Bayesian framework. Fearn et al showed that regression calibration provided a reasonable approximation to a full-likelihood implementation in the context of domestic radon exposures and lung cancer (52) an approach that was motivated by the bias that results from using likelihood approaches with many parameters (53).

If data from a validation or reliability study are available, a host of other methods may be used. Regression calibration (17) can be used to account for measurement error in point and interval estimates from regression models by adjusting these estimates using the estimated association between gold standard and observed exposure from a validation or reliability study. Regression calibration approaches have been valuable in environmental and nutritional epidemiology (18,54), but may have limited utility when misclassification is differential with respect to the outcome (55). Multiple imputation and full likelihood approaches offer a way forward in this setting. Multiple imputation leverages information from a validation study to adjust individual-level exposures prior to estimating exposure effects (56), while full and pseudo-likelihood approaches simultaneously estimate the parameters of interest in the outcome model with the parameters determining the measurement process (57).

Measurement error may also arise when measurements are subject to limits of detection (LOD), in which the exposure measure is known only to be below some threshold, or limits of quantification (LOQ), in which the exposure is known to be present (i.e., nonzero) but cannot be given an exact value below some threshold (58,59). In these settings, the quality of the measurement of exposures subject to LOD and LOQ are subject to the restrictions imposed by technology used to quantify exposure. Exposure values that are below the LOD or LOQ threshold are generally considered to be missing data, which can be imputed in a variety of ways. Importantly, if these imputations are based on the distribution of measured exposure for values above the LOD/LOQ, measurement error of these values can result in poorer performance of such imputations (60). Thus, accounting for measurement error of exposure above the LOD/LOQ threshold and handling measurements below the LOD/LOQ threshold are issues that could be usefully considered simultaneously. We focus the remainder of the review on problems in which LOD/LOQ is not considered.

Recent examples from the standard perspective

To assess the recent use of these approaches in environmental epidemiology, we performed a brief narrative review of literature on measurement error in environmental epidemiology from 2015–2016. Several themes emerged. First, with improvements in GIS technology, there is increasing focus on accounting for measurement error arising spatial misalignment (61). As in settings where exposure measurement is based on a job exposure matrix, measurement error in of spatial exposures often has both Berkson-like and Classical-like components (28,62). For example, air pollution studies rely on estimates of individual exposure based on measurements from individual air monitors, which may be subject to classical error, and a spatial model assigning exposure to individuals located at specific points in space, which may be subject to Berkson error. Sheppard et al present an overview of the history of accounting for measurement error in spatially misaligned data (63). More recently, Huque et al demonstrated that the expected amount of bias depends on the correlation between exposure and the random error from the regression model, and proposed parametric (64) and semiparametric (65) approaches to address this bias. Other recent work has addressed such spatial misalignment between measurements of exposures, covariates, and health outcomes using a variety of methods, including corrections in two-stage exposure models (66), SIMEX (67), and likelihood-based approaches (68).

Even outside spatial epidemiology, investigators are recognizing classical and Berkson sources of bias in their studies and wishing to account for both simultaneously. Such approaches are increasingly common in occupational studies, where both types of error may be present, with several examples mentioned above (40,41). In another example, Masiuk et al compare several methods (“new” regression calibration, efficient SIMEX, and a novel modified score equation) to account for measurement error characterized by additive classical error and multiplicative Berkson error in estimates of radiation risk on thyroid cancer (69). Finally, study questions involving exposure mixtures and multiple pollutants have mandated approaches to account for simultaneous, possibly dependent, mismeasurement of 2 or more quantities (5,70,71).

Accounting for measurement error under a policy perspective

Under a policy perspective, measurement error has been less frequently addressed. While approaches to estimate the effects of interventions on exposures, rather than exposure response parameters, have been applied (6,7,35,72–76), few have quantitatively accounted for measurement error. Panel B of Figure 1 clarifies additional considerations when using epidemiologic results to inform policy.

First, we may be interested in how measurement error of all of the features that lead to a population impact estimate, which include confounders, multiple exposures, and effect modifiers, affect the results, rather than only measurement error of exposures and confounders. For example, when using results from studies of underground uranium miners to inform guidelines on residential radon exposure, we would take into account error in the measurement of any effect modifiers that differed between the mining population and the general population (77–79).

Policies to change levels of environmental exposures will often affect multiple pollutants (80), each of which is subject to mismeasurement. Because these exposures share a common source or arise due to similar activities, they are often correlated. In the radon example, dictating that a miner stops accessing the mine after his radon exposure reached a certain threshold would simultaneously reduce his exposure to other health hazards found in the mine, such as diesel exhaust, silica, or arsenic (81). Because multiple mismeasured variables may be present, accounting for measurement error will require knowledge not only about single exposure-disease relationships, but also about the covariance structure. Approaches such as multiple imputation and Bayesian methods, which rely on simulating values of the true exposure from a hypothesized distribution, may be relatively straightforward to implement in this framework.

Finally, a policy-oriented approach must assess whether the intervention or policy would target the error-prone measures of exposure or the true exposure values. In the radon example, a policy that forced a miner to leave the mine after reaching a certain exposure threshold would be based on the measured, error-prone exposure, rather than the true exposure.

In principle, a policy perspective approach can account for each of these issues explicitly. Methods to account for measurement error in the tools typically used to estimate population intervention effects exist, and have been applied to estimate more traditional parameters with these tools (82–84). In addition, the sophisticated tools developed to account for measurement error in traditional regression models can be integrated into approaches to estimate population impact. In the simplest approach (e.g., in settings with a time-fixed exposure), one could specify a regression model for the outcome of interest, account for measurement error using existing or novel tools to account for measurement error in regression models, and simulate the predicted health outcomes under various interventions on the exposure distribution. This is a basic form of g-computation (85), which has been used in more complex settings to estimate health effects of workplace policy changes (6,7), changes to social norms (72), and the effects of multiple, simultaneous interventions on lifestyle factors (76,86,87).

We have recently demonstrated a Bayesian approach to estimating the effect of new limits on radon exposure on lung cancer mortality in a cohort of American Indian miners from the Colorado plateau (ψ_gj) while accounting for Berkson error (88). This preliminary work, which builds on a general Bayesian approach to estimating intervention effects (89), suggested that there was a small, downward bias from measurement error in the estimation of the impacts of occupational policies on radon exposure.

When using estimates of parameters from epidemiologic studies to make decisions about public health policies, the uncertainty in the parameter estimate is important (90). An analysis that acknowledges substantial potential for measurement error in the text of a manuscript but fails to account for measurement error quantitatively in the analysis or assumes that the measurement error parameters are known will underestimate the uncertainty in the final estimate of population impact. This overconfidence could lead to suboptimal decision making (91). Bayesian approaches (50,51,89,92), which naturally incorporate uncertainty in the parameters determining the measurement process and propagate this uncertainty through to the final posterior distribution of the parameter of interest, offer a formal approach to quantify uncertainty about such nonidentified bias sources.

Conclusions

We close with the proposition that measurement error itself can be harmful to human health. One of the central tasks of public health research is the generation of scientific results that inform policy and decision making processes. Policy decisions regarding regulation of the myriad of exposures that make up our external environment, many of which are at odds with a healthy state of humans’ internal homeostasis, are threatened by measurement error. Such error may bias estimates of population impact and obscure or amplify uncertainty in the results.

Much of the current literature addressing measurement error is focused on estimating unbiased regression coefficients. This approach captures one aim of epidemiology: being the basic science of public health. Further refining our results to increase their utility for the end-products of epidemiology, namely policy decisions, will lead us towards “an epidemiology of consequence” (93,94). Under such a policy approach, many of the issues arising from measurement error in regression models remain, and new considerations emerge. However, accounting for measurement error in estimates of population impact that feed directly into regulatory decisions will increase the relevance and importance of results from environmental epidemiology.