Survey of practices of handling exposure measurement errors in modern epidemiology: are the best practices in statistics being adopted by epidemiologists?

Abstract

Background

Measurement errors in epidemiological studies can impact the validity and reliability of findings. Without proper context, inferences (causal or otherwise) based on these findings may be compromised. The consequences of measurement error are well known, but in practice commonly ignored when interpreting findings in epidemiological research.

Methods

We examined papers published in 2022 in three leading epidemiology journals (International Journal of Epidemiology, Epidemiology, and American Journal of Epidemiology) to assess the occurrence and handling of exposure measurement error (EME). We randomly sampled 64 papers that assessed exposure-outcome relationships. Two authors independently reviewed the selected papers and searched for (a) explicit definition of the exposure in question and how it was measured, (b) an acknowledgment of the possibility of exposure measurement error and (c) statistical investigation of the expected impact or adjustment for EME.

Results

Our review of recent epidemiological studies reveals encouraging progress on the interpretation and adjustment of EME; however, room of improvement still exists. Among our sample of 64 articles, 2 (3.1%) articles reported exposures for which measurement error did not exist, 3 (4.7%) articles lacked a well-defined research question which precluded proper classification, 8 (12.5%) articles ignored EME, 24 (37.5%) reported on EME or discussed EME as a limitation but treated it as “negligible” without investigating further, 14 (21.9%) articles conducted sensitivity analyses to describe the potential effect EME may have on the studies and 8 (12.5%) articles attempted to quantitatively estimate the impact of EME on the reported risk estimate. Further, 8 (12.5%) articles erroneously claimed that EME would bias risk estimates towards the null (2 of which were also included above).

Conclusions

Modern epidemiological research shows improved handling and interpretation of EME, while some concerns persist. For instance, the epidemiological literature indicates that it is still resistant to adoption of state-of-the-art methods for managing measurement errors. We recommend that the practice of qualitatively discussing the impact of measurement error in exposure in epidemiology be replaced with modern developments in statistics and comprehensively accounted for.

Background

Measurement errors in epidemiological studies can impact the validity and reliability of findings. Without proper context, inferences (causal or otherwise) based on these findings may be compromised. Measurement error can introduce bias, leading to spurious associations or even mask true relationships. Errors can occur in exposures, covariates (measures of confounding), effect modifiers, and outcomes, and may arise from errors in questionnaire responses, answers to investigator interviews, medical records, incorrect diagnoses, laboratory errors, and intrinsic within-subject variability [91]. There is a vast statistical literature devoted to this problem, with several textbooks [16, 35] specifically attempting to improve application of this statistical lore by epidemiologists.

Measurement errors can be classified as random or systematic. Random error can include sampling variability, where differences in measurements from a sample arise when compared to the population of interest and instrumental variation which can lead to inconsistent results. While random error is due to chance, systematic error results in consistently affected measurements all in the same direction due to flaws in study design, data collection, or analysis. Systematic error can also include differential misclassification of either the exposure or outcome, especially when a particular demographic may be more likely to report inaccurate exposure or outcome information. Exposure Measurement Error (EME) can be categorized in several ways depending on the nature and implications of errors within the study itself. Random (non-differential with respect to the outcome) EME may be expected to lead to the attenuation of estimated associations [29], but it is impossible to predict the actual bias in any one given analysis [48, 80, 86]. EME may also be systematic, differential with respect to the outcome, in which case any observed association may be expected to be biased away (in either direction) from the null. Other complications may arise, in the form of correlated measurement errors among several covariates considered in the analysis, even if these are independent of the outcome, as often occurs when covariates are derived from the same interview or error-prone source of data [51, 52, 53]. The consequences of measurement error are well known, but in practice commonly ignored when interpreting findings in epidemiological research [26, 27, 91]. EME can lead to bias in estimates of exposure-disease associations, and it is commonly taught that this leads to attenuating risk ratios towards null if the errors are non-differential with respect to the outcome [29]. Therefore, understanding and evaluating EME is a critical concern within epidemiological research as it can distort the associations between exposure, risk factors, and health outcomes.

Further, EME may also be qualitative or quantitative depending on how exposure is categorized, measured, and analyzed. Qualitative EME can arise when exposure is misclassified without regard for its intensity or level (i.e., a smoker incorrectly categorized as a non-smoker). In contrast quantitative EME involves incorrect assessment of the dose or level of exposure which may significantly over or under-represent actual exposure levels (as in self-reported smoking status). When a phenomenon that exists on a continuous scale, such as cumulative inhalation of smoke from cigarettes, and is subject to uncertainty, when it is categorized, the errors in the resulting classification can be dependent on the outcome, i.e., the issue lies with continuous exposure being associated with the outcome. This occurs even when measurement error on the continuous scale is not dependent on the outcome, so long as there is an association with the outcome. This curious feature of categorization under uncertainty has long been recognized by some epidemiologists [31], illustrated with formal mathematic analyses [36] and made more accessible in applied examples [77]. Because epidemiologists typically assume that the exposure they study may cause the outcome that they seek to observe, and that all exposures they categorize are subject to uncertainty, it follows that it is more reasonable to assume exposure misclassification in epidemiology may be differential rather than non-differential with respect to the outcome. In practice, it is straight-forward to simulate how far misclassification deviates from non-differential if some assumptions can be made (as they must be made) about the uncertainty about exposure level and the plausible strength of studied exposure-outcome associations, e.g. see [39].

In a previous survey Jurek, Maldonado and Greenland assessed error in measuring exposures across a selection of epidemiological investigations [49]. They randomly selected articles appearing in The American Journal of Epidemiology, Epidemiology, and The International journal of Epidemiology for the 2001 publication year. In total the authors reviewed fifty-seven articles and concluded that 39% did not mention exposure measurement error. Thirty-five studies did at least mention exposure measurement error, however only one study quantified the impact of EME on study results [50]. The authors sought to bring attention to measurement error and its effects within epidemiological literature. A similar survey of the approach to this measurement error problem in occupational epidemiology likewise revealed a lack of widespread adoption of quantitative approaches [13]. This is a serious failing of the field, because there is evidence that risk assessors who use epidemiology find the unsophisticated treatment of measurement error in exposure in published reports to be a barrier to utilizing epidemiology in regulatory risk assessment [67].

Our aim was to update the previous work with a survey of the treatment of the exposure measurement error problem across three general epidemiological journals. We aimed to highlight the ways in which measurement error and misclassification of exposure are accounted or not accounted for in modern epidemiology, and thus gauge to what extent advances in statistics inference in the presence of measurement error have been adopted by epidemiologists.

Methods

A sample of the entire 2022 publication year of epidemiological publications appearing across three leading epidemiological journals (The International Journal of Epidemiology (IJE), Epidemiology (EPI), and the American Journal of Epidemiology (AJE) were selected. Briefly, papers were sampled based on abstract and title; commentaries, letters to the editor, and corrections were excluded. An initial sample of eighty publications was obtained via random sampling in R (version 4.2.2) and re-examined against the inclusion criteria. Similar to [49]and [50] we only wanted to include articles that investigated exposure - outcome associations (injury or disease); sixteen papers were excluded as they did not meet this criterion, leaving 64 articles.

Each article was independently screened and read in-depth by MKH and AR. Papers were critically analyzed and searched for (a) explicit definition of the exposure in question and how it was measured, (b) an acknowledgment of the possibility of exposure measurement error (including search for citation of papers on reliability or validity of exposure measure that was used), and (c) statistical investigation of the expected impact or adjustment for EME. We also investigated whether information that could have been used to adjust for measurement errors appeared to exist in the academic literature prior to the publication of reviewed papers and/or was cited in selected papers but not used for this purpose.

Results

64 articles were included in our analysis. Two studies (3.1%) included in our sample had an exposure for which measurement error in exposure was not a concern. Of the remaining studies, 8 (12.5%) articles did not mention EME, even though it is reasonable to expect that the exposure of interest could have been measured with error (Table 1). Of the remaining 54 articles, 3 (4.7%) lacked a well-defined research question and were not further classified, and the rest either reported on the degree of exposure measurement error but did not adjust for EME (Table 2), attempted to adjust for EME, or explicitly mentioned EME in the discussions. Specifically, among our sample of articles, 24 (37.5%) reported on EME or discussed EME as a limitation but treated it as “negligible” without investigating further (Table 2). 14 (21.9%) articles conducted sensitivity analyses to describe the potential effect EME may have on the studies and 8 (12.5%) articles attempted to quantitatively estimate the impact of EME on the reported risk estimate. Further, 8 (12.5%) articles erroneously claimed EME would bias risk estimates towards the null, two of which were previously included in Table 2. None of the reviewed papers appeared to employ state-of-the-art statistical tools to mitigate bias from EME.

Table 1.

Surveyed articles that ignored exposure measurement error and some reasons for why this may not have been the best practice

Author	Estimated Exposure	True exposure	Reasoning for presence of exposure measurement error	Examples of unused source of information on validation or reliability of exposure
De La Paz et al. [68]	Diagnosis of toxic oil syndrome and inclusion into patient registry	Ingestion of denatured rapeseed oil with 2% aniline	Investigators assumed the same level of consumption of the oil and uniform levels of contamination with aniline.	Peterson PJ. Assessment of exposure to chemical contaminants in water and food. Sci Total Environ. 1995 Jun 16;168(2):123-9. doi: 10.1016/0048-9697(95)00461-h. PMID: 7481730. Note: Articles on accuracy of exposure assessment in the investigation that assembled the cohort based on high exposure to the contaminated oil in Spain were not identified.
Batty et al. [8]	Ethnic identity	Ethnic origin	Ethnicity was self-classified based on UK census questionnaires. Self-reported measurements of exposure are at higher risk for misclassification.	Ponterotto, J. G. (2001). Handbook of Multicultural Counseling. Hungary: SAGE Publications. Ludi Simpson, Stephen Jivraj, James Warren, The Stability of Ethnic Identity in England and Wales 2001–2011, Journal of the Royal Statistical Society Series A: Statistics in Society, Volume 179, Issue 4, October 2016, Pages 1025–1049
He et al. [38]	Self-reported maternal infections during pregnancy	Maternal infections during pregnancy	Maternal infections during pregnancy were self-reported based on questionaries (ALSPAC, JPS, MoBa, TIHS), telephone interviews (DNBC) or medical records and self-reported interviews (CPP). Self-reported measurements of exposure are at higher risk for misclassification.	Dietz P, Bombard J, Mulready-Ward C, Gauthier J, Sackoff J, Brozicevic P, Gambatese M, Nyland-Funke M, England L, Harrison L, Taylor A. Validation of self-reported maternal and infant health indicators in the Pregnancy Risk Assessment Monitoring System. Matern Child Health J. 2014 Dec;18(10):2489-98. doi: 10.1007/s10995-014-1487-y. PMID: 24770954; PMCID: PMC4560102.
Von Holle et al. [85]	Infant growth characteristics (weight, length, weight-to-length)	Infants’ growth trajectory	Classification of infants into grown trajectories was done using statistical models but errors associated with these models were not considered as a source of exposure measurement error.	None identified that are specific to errors in models developed by the authors.
Duffy et al. [25]	Report of providing care to an Ebola case	Duration of contact with person capable of transmitting Ebola virus	Authors reported that, “[p]rovision of care was self-reported by study participants, and a comprehensive set of activities which constituted provision of care was not delineated when household members were asked if they provided care. Thus, there may have been some misclassification bias, and the possibility that persons not residing in the household provided some care also cannot be excluded (p. 8).”	None identified
Alegre et al. [4]	BMI	Overweight and Obesity	Baseline BMI with cox regression were used to investigate associations with COVID-19 mortality, with some measurements predating COVID-19 by years. Variations in BMI year to year were not investigated as potential exposure misclassification.	Adab P, Pallan M, Whincup PH. Is BMI the best measure of obesity? BMJ. 2018 Mar 29;360:k1274. doi: 10.1136/bmj.k1274. Erratum in: BMJ. 2018 May 23;361:k2293. doi: 10.1136/bmj.k2293. PMID: 29599212
Antonio-Villa et al. [5]	Multiple social inequalities	Impact of social inequalities on individual health	Authors used ecological indices of social inequalities and did not mention EME, it is reasonable to believe EME exists in ecological studies	Summers K, Accominotti F, Burchardt T, Hecht K, Mann E, Mijs J. Deliberating Inequality: A Blueprint for Studying the Social Formation of Beliefs about Economic Inequality. Soc Justice Res. 2022;35(4):379-400. doi: 10.1007/s11211-022-00389-0. Epub 2022 Apr 1. PMID: 35382060; PMCID: PMC8972749.
Mutevedzi et al. [63]	No defined exposure of interest	N/A	Authors conducted a purely descriptive study with no causal hypothesis question; however, it is reasonable for EME to be present when attributing a case to a geographic area (such as if migration between areas occurs). Further it’s possible for various variables to be misclassified such as smoking, drinking, self-reported obesity, etc.	None identified– Authors did not pose a causal question

Table 2.

Surveyed articles that reported on the extent of exposure measurement error but either asserted or implied that it was negligible in the sense that it was not deemed to be a threat to biasing results enough to alter stated conclusions of the articles

Authors	Estimated Exposure	True exposure	Description of validity or reliability of exposure provided by the authors	Some resources that may have been useful in assessing the bias from exposure measurement error in more details
Chechet et al. [19]	Seropositivity	Past SARS-CoV-2 infection	Biopanda COVID-19 IgM/IgG Rapid Test Kits do not measure seropositivity with perfect accuracy. There is a delay between infection and seropositivity, which then wanes. Authors specified that, “[t]he CE-marked Biopanda COVID-19 IgM/IgG Rapid Test Kit [sensitivity >99% (95% CI: 93.3%–100%), specificity 98.6% (95% CI: 94.9%–99.8%)] was used… The presence of IgM antibodies indicates a recent infection (7–28 days prior to sampling); IgG antibodies typically appear approximately 14 days after infection and endure for at least several months,…" (p.1364).	Ciotti M, Benedetti F, Zella D, Angeletti S, Ciccozzi M, Bernardini S. SARS-CoV-2 Infection and the COVID-19 Pandemic Emergency: The Importance of Diagnostic Methods. Chemotherapy. 2021;66(1-2):17-23. doi: 10.1159/000515343. Epub 2021 Mar 19. PMID: 33744904; PMCID: PMC8089410. Goldstein ND, Quick H, Burstyn I. Effect of Adjustment for Case Misclassification, and Infection Date Uncertainty on Estimates of COVID-19 Effective Reproduction Number. Epidemiology. 2021 Nov 1;32(6):800-806. doi: 10.1097/EDE.0000000000001402. PMID: 34310444; PMCID: PMC8478862.
Leclair et al. [59]	Level of American football, self-reported	Repetitive head impacts (RHI	Level of American football was used as a surrogate for RHI sustained while playing football. But it was noted by the authors that “[the] study was also limited in that highest level of American football playing served as a proxy measure for RHI. However, we were unable to consider other measures of exposure, such as frequency of RHI, or even duration of play as Mez et al. did (2020), because the methods employed rely on having information on the exposure for the target population" (p. 1441).	Clara, K. Profiling Brain Trauma in Professional American-style Football and the Implications to Developing Neurological Injury. University of Ottawa. 2019; Mez J, Daneshvar DH, Abdolmohammadi B, et al. Duration of American football play and chronic traumatic encephalopathy. Ann Neurol. 2020;87(1):116–131.
Nguyen et al. [65]	Parental self-reported vaccine hesitancy: “not at all hesitant, not that hesitant, somewhat hesitant, or very hesitant?" (p. 1627).	Parental vaccine hesitancy	The validity of self-report of hesitancy was evaluated mostly qualitatively and judged to be consistent (reliable), however, comparison with another question from the same survey aimed to elicit similar information on belief in benefits of vaccines did not show a perfect agreement with the one selected by the authors.	Opel DJ, Taylor JA, Mangione-Smith R, Solomon C, Zhao C, Catz S, Martin D. Validity, and reliability of a survey to identify vaccine-hesitant parents. Vaccine. 2011 Sep 2;29(38):6598-605. doi: 10.1016/j.vaccine.2011.06.115. Epub 2011 Jul 16. PMID: 21763384. National Center for Health Statistics. The Cognitive Evaluation of Survey Items Related to Vaccine Hesitance and Confidence for Inclusion on a Series of Short Question Sets. Hyattsville, MD: National Center for Health Statistics, 2020. P. 100-102.
Chazelas et al. [18]	Nitrite and nitrate intakes from natural sources and food additives based on self-report of 24-hour food intake linked to database on content of food and intake	Nitrite and nitrate intakes	Authors cited literature that could have been used to adjust for measurement error in exposure in a sophisticated manner, using tools developed specifically for nutritional epidemiology.	Touvier M, Kesse-Guyot E, Mejean C et al. Comparison between an interactive web-based self-administered 24 h dietary record and an interview by a dietitian for large-scale epidemiological studies. Br J Nutr 2011;105:1055–64. Lassale C, Castetbon K, Laporte F et al. Validation of a Web-based, self-administered, non-consecutive-day dietary record tool against urinary biomarkers. Br J Nutr 2015;113:953–62. Lassale C, Castetbon K, Laporte F et al. Correlations between fruit, vegetables, fish, vitamins, and fatty acids are estimated by web- based nonconsecutive dietary records and respective biomarkers of nutritional status. J Acad Nutr Diet 2016;116:427–38.
He et al. [37]	Single nucleotide polymorphisms (SNPs) for lung function that were independent of outcome and exposure, can only affect outcome via exposure and strongly associated with it (i.e., instrumental variable, IV).	Maternal lung function	The first stage F-statistic of >10 was used to select SNPs. It is derived from regression analysis and that regression could indicate the degree of error that use of a given SNP introduced into analysis.	None identified but methods to adjust for measurement error in IV exist Paul Gustafson, Measurement Error Modelling with an Approximate Instrumental Variable, Journal of the Royal Statistical Society Series B: Statistical Methodology, Volume 69, Issue 5, November 2007, Pages 797–815 Vansteelandt S, Babanezhad M, Goetghebeur E. Correcting Instrumental Variables Estimators for Systematic Measurement Error. Stat Sin. 2009 Jan 1; 19:1223-1246. PMID: 20046952; PMCID: PMC2743431.
Huntley et al. [42]	Enforceable vs. unenforceable stay-at-home orders	Change in be behavior due to enforceable vs. unenforceable stay-at-home orders	“The identified executive orders in Web Table 1 were cross validated with other available databases and were found to be in agreement, suggesting that the orders reviewed were directly those specifying SAH policy and no other related emergency public health measures" (p. 2). If the rating of orders was not perfect and did not agree with perception of these orders, exposure may have been misclassified. Compliance with SAH orders was not accessed.	J Raifman, K Nocka, D Jones, et al. COVID-19 US state policy response. https://statepolicies.com/data/graphs/stay-at-home-order/. Updated July 27, 2020. Accessed October 14, 2020.”
Aker et al. [2]	Two physician visits or one hospitalization due to asthma	Confirmed asthma diagnosis	The authors used “a validated ICES algorithm for identifying asthma" (p.966), and note that information on accuracy is imperfect but do not speculate on the impact of potential inaccuracies.	Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T. Identifying patients with physician-diagnosed asthma in health administrative databases. Can Respir J 2009; 16:183–88
Ako et al. [3]	Self-reported gallbladder disease	Medical diagnosis of gallbladder disease	The authors state that “[t]he primary exposure was gallbladder disease. Data on gallbladder disease was collected at baseline. Participants were asked, “Did a doctor ever say that you had gallbladder disease or gallstones?” “Did you ever have a procedure to remove gallstones?” and “Did you have your gallbladder removed?” During follow-up, incident physician-diagnosed gallbladder disease continued to be reported by participants until study close-out in 2005. Data on cholecystectomy was collected at baseline but not during follow-up" (p.1375).	Not provided by authors
Bo et al. [10]	Annual PM2.5 concentration	Air pollution constituents	“The model was validated by comparing the modelled annual average PM2.5 concentration with the annual monitored ground-level PM2.5-monitoring data from >70 monitoring stations across Taiwan, with corresponding correlation coefficients ranging from 0.72 to 0.83" (p.227).	Krall, J. R., Chang, H. H., Sarnat, S. E., Peng, R. D., & Waller, L. A. (2015). Current Methods and Challenges for Epidemiological Studies of the Associations Between Chemical Constituents of Particulate Matter and Health. Current environmental health reports, 2(4), 388–398. https://doi.org/10.1007/s40572-015-0071-y
Bodnar et al. [11]	Diet rich in fruits and vegetables	Fruit and vegetable consumption	"The FFQ has acceptable validity relative to other self-reported assessment tools in many samples of pregnant women" (p.26–31). “FFQ data have been shown to be subject to a greater degree of systematic measurement error than other self-reporting methods, at least for absolute intakes" (p.67, 68). "This error can have unexpected effects, but attenuation of associations and reduced precision are common" (p.69).	Block G, Hartman AM, Dresser CM, et al. A data-based approach to diet questionnaire design and testing. Am JEpidemiol. 1986;124(3):453–469. Block G, Woods M, Potosky A, et al. Validation of a self-administered diet history questionnaire using multiple. diet records. J Clin Epidemiol. 1990;43(12):1327–1335. Johnson BA, Herring AH, Ibrahim JG, et al. Structured Measuring error in nutritional epidemiology: applications in Pregnancy, Infection, and Nutrition (PIN) Study. J AmStat Assoc. 2007;102(479):856–866. Mares-Perlman JA, Klein BE, Klein R, et al. A diet history questionnaire ranks nutrient intakes in middle-aged and older. men and women similarly to multiple food records. J Nutr. 1993;123(3):489–501. Boucher B, Cotterchio M, Kreiger N, et al. Validity and reliability of the Block98 food-frequency questionnaire in a sample of Canadian women. Public Health Nutr. 2006;9(1): 84–93. Block G, Coyle LM, Hartman AM, et al. Revision of dietary. analysis software for Health Habits and History Questionnaire. Am J Epidemiol. 1994;139(12):1190–1196.
Wang C-R et al. [88]	Dietary diabetes risk-reduction score	Dietary nutrient intake	Authors state that “it has been suggested that the approaches using baseline diet data only in general yield a weaker association than do these using the cumulative averages" (p.483).	Hu FB, Stampfer MJ, Rimm E, et al. Dietary fat and coronary heart disease: a comparison of approaches for adjusting for total energy intake and modeling repeated dietary measurements. Am J Epidemiol. 1999;149(6):531–540.]
Wang C. et al.[87]	Ratio of recent vs. Usual physical activity	Physical activity, caloric burning	Authors do not provide validity or reliability on exposure.	Andrade R, Wik EH, Rebelo-Marques A, et al. Is the acute: chronic workload ratio (ACWR) associated with risk of time-loss injury in professional team sports? A systematic review of methodology, variables, and injury risk in practical situations. Sports Med. 2020;50(9):1613–1635.
Chen et al. [20]	Air pollution from stationary measurements and spatial models	Air pollution constituents	“There may be exposure misclassification in this study because we used exposure data from a spatial model rather than at the personal level. Additionally, women’s behaviors with respect to exposure might change after conception. They are prone to taking measures, such as wearing a mask in seriously air-polluted days when going out or using an air purifier at home, to reduce levels of air-pollution exposure during pregnancy. However, information on the behavior changes was unavailable in this study" (p.209).	Krall, J. R., Chang, H. H., Sarnat, S. E., Peng, R. D., & Waller, L. A. (2015). Current Methods and Challenges for Epidemiological Studies of the Associations Between Chemical Constituents of Particulate Matter and Health. Current environmental health reports, 2(4), 388–398. https://doi.org/10.1007/s40572-015-0071-y
Cote et al. [21]	Fruit and vegetable pesticide residue status	Oral Pesticide consumption	The exposure assessment used in this study has been previously validated, with correlation between PRBS assessment and presence of urinary biomarkers of pesticide exposure in 2 separate cohorts.	Hu Y, Chiu YH, Hauser R, et al. Overall and class-specific scores of pesticide residues from fruits and vegetables as a tool to rank intake of pesticide residues in United States: a validation study. Environ Int. 2016;92-93:294–300. Chiu YH, Williams PL, Mínguez-Alarcón L, et al. Comparison of questionnaire-based estimation of pesticide residue intake from fruits and vegetables with urinary concentrations of pesticide biomarkers. J Expo Sci EnvironEpidemiol. 2018;28(1):31–39.
Johnson et al. [44]	PM2.5 and NO2 residential location	Air pollution constituents	While the authors did not explicitly provide validity or reliability of exposure, they did impute missing exposure information, conduct sensitivity analyses, and use Bayesian methods for analyses.	Krall, J. R., Chang, H. H., Sarnat, S. E., Peng, R. D., & Waller, L. A. (2015). Current Methods and Challenges for Epidemiological Studies of the Associations Between Chemical Constituents of Particulate Matter and Health. Current environmental health reports, 2(4), 388–398. https://doi.org/10.1007/s40572-015-0071-y
Julian-Serrano et al. [46]	Dietary assessment via FFQ	Nutrient consumption	Validation of the FFQ was performed within a subset of the NIH-AARP Diet and Health Study, using 2 24-hour dietary recalls.	Thompson FE, Kipnis V, Midthune D, et al. Performance of a food-frequency questionnaire in the US NIH-AARP (National Institutes of Health–American Association of Retired Persons) Diet and Health Study. Public Health Nutr. 2008;11(2):183–195.]
Kendzia et al. [54]	Exposure to welding fumes	Specific inhalation exposure to welding fume constituents	The authors used a welding-exposure matrix (WEM) based on quantitative measurements. WEM was combined with work histories and the authors note that this approach has been coupled in the past with sensitivity analyses to assumptions in the exposure assessment.	de Vocht, F., Burstyn, I., Ferro, G., Olsson, A., Hashibe, M., Kromhout, H., & Boffetta, P. (2009). Sensitivity of the association between increased lung cancer risk and bitumen fume exposure to the assumptions in the assessment of exposure. International archives of occupational and environmental health, 82(6), 723–733. https://doi.org/10.1007/s00420-008-0373-6
Lergenmuller et al. [60]	Self-reported use of sunscreen and SPF	Dermal exposure to benzene and other sunscreen constituents	"Exposure misclassification, inevitable in epidemiologic studies, is likely nondifferential in cohort studies, although differential misclassification can occur when forming categories" (p.47). [Flegal KM, Keyl PM, Nieto FJ. Differential misclassification arising from nondifferential errors in exposure measurement. Am J Epidemiol. 1991;134(10):1233–1244.]	Petersen B, Wulf HC. Application of sunscreen—theory and reality. Photodermatol Photoimmunol Photomed. 2014; 30(2–3):96–101
Shitole et al. [76]	Lab method used to measure biomarkers of NEFA exposure	Exposure to NEFA	“One-time measurements of NEFA are known to be unreliable owing to the high variability NEFA flux and NEFA’s short half-life (3). Repeated measures would help address this variability, but this was not available for postload NEFA. Thus, our single measurements of NEFA may be underestimating the associations with longer-term events" (p.1245).	Karpe F, Dickmann JR, Frayn KN. Fatty acids, obesity, and insulin resistance: time for a reevaluation. Diabetes. 2011; 60(10):2441–2449.
Soares et al. [79]	Childhood tramua screener for childhood maltreatment	Childhood trauma	“Although there is poor agreement between retrospective and prospective measures of childhood adversities, retrospective measures have been shown to be valid in population studies, thought they might underestimate the association with objectively measured outcomes, such as CVD" (p.563).	Grabe HJ, Schulz A, Schmidt CO et al. A brief instrument for the assessment of childhood abuse and neglect: the Childhood Trauma Screener (CTS). Psychiat Prax 2012; 39:109–15.
Van Gennip et al. [83]	Caregiver report of provider diagnosis of Asthma	Confirmed Asthma diagnosis	Studies examining the validity of questionnaire-based asthma diagnosis in children, using questions similar to those used in the current study, have reported a specificity higher than 95% compared with health claims as the reference standard and a specificity higher than 87% compared with a clinical assessment as the standard. Furthermore, the use of a stricter definition of asthma to reduce potential misclassification yielded similar results.	Boal AH, Smith DJ, McCallum L, et al. Monotherapy with major antihypertensive drug classes and risk of hospital admissions for mood disorders. Hypertension. 2016;68(5): 1132–1138. Mehta R, Hodakowski A, Cai X, et al. Serum phosphate and retinal microvascular changes: the Multi-Ethnic Study of Atherosclerosis and the Beaver Dam Eye Study. OphthalmicEpidemiol. 2017;24(6):371–380.
Wei et al. [89]	FFQ assessment of dietary copper intake	Oral copper consumption	“The intake of copper, zinc, and iron measured with Willett has been validated with 7-day dietary records (energy-adjusted intraclass correlation coefficients = 0.60, 0.49, and 0.54, respectively)" (p.1203).	Yuan C, Spiegelman D, Rimm EB, et al. Validity of a dietary questionnaire assessed by comparison with multiple weighed dietary records or 24-hour recalls. Am J Epidemiol. 2017; 185(7):570–584
Wesselink et al. [90]	Self-reported vaccination and infection of COVID-19	Confirmed COVID-19 infection	“In validation studies of influenza vaccination in the past year, 97% agreement was found between vaccination status based on self-report and medical records" (p.1392).	King JP, McLean HQ, Belongia EA. Validation of self-reported influenza vaccination in the current and prior season. Influenza Other Respi Viruses. 2018;12(6): 808–813
Zhong et al. [95]	PM2.5 exposure based on spatial maps and models	Air pollution constituents	“A recent study by McIsaac et al. (29) demonstrated the potential to misclassify exposure as spatial resolution decreased" (p.1537).	McIsaac MA, Sanders E, Kuester T, et al. The impact of image resolution on power, bias, and confounding: a simulation study of ambient light at night exposure. Environ Epidemiol. 2021;5(2): e145

No measurement error in exposure

There are two examples in our sample for which measurement error in exposure did not exist. Wootton et al., [92] studied the trend in features of attention deficit hyperactivity disorder with age (age in years) that was measured precisely, with ignorable error. Thus, the issue of EME was not even discussed by the authors, although it does contain an assumption that features of ADHD that were measured are invariant within a given year, which is a problem of measurement error in the outcome. Boone et al., [12] wanted to see how well anthropometric measures predicted the level of visceral adipose tissue (VAT). In such a study, predictors are treated as measured, i.e., one is not interested in the true association of anthropometric measures with VAT, only the strength of association with measured values. The “true” exposure is the one that was measured, as one would use to make a prediction of VAT in practice. No etiological or causal claims are made, which is appropriate, because Boone et al., [12] was essentially an instrument calibration investigation.

Studies which lacked a well-defined research question

Several studies which lacked a well-defined research question were not classified into either Table 1 or Table 2 despite inadequate treatment of EME. These included [45, 66] and [69]. Both Nordholm et al., [66] and Rautava et al., [69] were socioeconomic epidemiological studies which did not clearly articulate what exactly was being measured in relation to socio-economic status which made critique of the authors’ methodology difficult. Similarly, Joyce et al., [45] attempted to report on the impact of parental education on epigenetic aging but did not provide a well-defined research question and did not attempt to answer a causal question, which did not allow for critique of the authors’ methods.

Unsupported claims of “biased towards the null.”

Notably, most studies included in our sample did not utilize the common and erroneous trope that any measurement error would bias effect estimates towards the null. However, this phrase was present within 8 of the 64 studies [40, 41, 75, 76, 81, 82, 88, 94]. In all these studies, authors acknowledged that measurement error in exposure was present and yet they elected to declare that it was unimportant to their conclusions. No evidence of why non-differential misclassification would be expected to occur or was in fact non-differential was presented in any of the following studies that acknowledged misclassification of exposures. On the contrary, dichotomization of unknown level of exposure [75, 82, 94] or dose [81] is more likely to induce differential rather than non-differential misclassification of exposure [36].

Toyib et al., [82] assessed the long-term exposure of PM_2.5and assumed a stable distribution of pollutants over time, which led them to acknowledge the presence of EME. Estimates derived from statistical models of air concentrations (not personal measurements) were used as both continuous variables and dichotomized at medians. The authors postulated non-differential error that would “tend to diminish observed effect”, incorrectly citing Armstrong [6] to support their claim: the cited work is about mathematic expectation of bias due to EME, not actual bias in any given study. In another study of air pollution, Sheridan et al., [75] dichotomized exposure to wildfire smoke, estimated from remote-sensing data, based on subject’s residence within ZIP codes. They acknowledged EME due to the use of such a binary exposure variable but stated “that the misclassification… would be nondifferential and lead to more conservative effect estimates,” presenting this as a “hypothesis” that remains untested [75].

Sperling et al. [81] conducted a nested case-control study among Danish women for the use of antidepressants and endometrial-cancer risk. The authors used different classes of antidepressants as the exposure of interest defined as having two or more current filled prescriptions. The authors acknowledged the potential for EME due to non-compliance. The authors stated that “non-compliance is likely to bias the result towards the null; however, in order to reduce this potential bias, we defined drug use as two or more filled prescriptions” [81]. Curiously, dichotomization is presented by Sperling et al. [81] as an effort to minimize bias from measurement error, which reflects a heuristic often taught to epidemiologists. While misclassification due to this procedure may be reduced if one is certain about assessed exposure to be above or below a chosen threshold, it does create its own set of problems, such that it is hard to be sure whether net bias from EME is reduced [36].

Zhang et al. [94] was concerned with risks of consumption of ultra-process food. The authors measured ultra-process food consumption, assessed based on responses to a food frequency questionnaire at one point in time. Reliability and validity measures of the questionnaire are cited in text, enabling authors to adjust for EME, if they choose to do so, e.g. see [51, 52]. Error-prone exposure estimates were categorized, making differential exposure misclassification impossible to rule out [36]. The authors noted that “misclassification bias cannot be ruled out. However, this potential non-differential misclassification might bias toward the null hypothesis” [94]. Similarly, in a cohort study assessing visual acuity and cycling accidents, with visual acuity measured in adolescence, the authors stated “in terms of misclassification of visual acuity, it is most likely nondifferential because the change in vision would unlikely relate to the outcome, future cycling injury. Assuming no other source of bias, nondifferential misclassification tends to result in conservative estimate, although there are exceptions” [41], p. 252.

Heck et al. [40] conducted a cohort study of pediatric cancer risk due to parents with viral hepatitis, identified by diagnostic codes in the database of the national health insurance scheme and noted that there was the potential for EME due to the lack of universal screening in the population. The authors stated that “misclassification of… diagnoses would be expected to have biased results to the null” [40]. However, one can legitimately be concerned that persons who were not voluntarily screened for viral hepatitis would also have longer time to diagnosis of their children due to suboptimal access to care, thereby creating a correlation of survival time and missingness of (error in) exposure. The remaining two studies Shitole et al. [76] and Wang CR et al. [88] are described in more detail in Table 2 below.

In contrast to the 8 studies described above, one study outlined in Table 2 [54] did acknowledge the unexpected consequences that EME may have on study results. Specifically, the authors stated “[t]hese limitations may have led to errors in the estimation of lifetime welding-fume exposure. Misclassification in the exposure variable may lead to under-or overestimation of risk ratios, especially if continuous data are grouped into exposure categories” [54]. The authors should receive credit for not falling into the common pitfall of the non-differential biased towards the null fallacy, and accurately identifying the complexity EME can have on results.

Studies that ignored exposure measurement error

Table 1 describes 8 papers that appear to have ignored measurement error even though what they estimated can be credibly argued to be a proxy for true exposure that can have causal relationship with the outcomes. For exposures in four of these papers we were able to identify how measurement error in exposure could have been quantified, either from materials present with author’s own analysis [85] or external references [4, 8, 68]. In the case of Duffy et al., [25] we were not able to identify a source that can help us estimate misclassification of reports of providing care for Ebola cases or intensity of contact with such a patient, but we also are not experts in this area. He et al., [37] stated that misclassification of exposure was a limitation but said nothing about its extent or structure that would help a reader to interpret the results considering the stated limitation.

Studies that reported on the extent of measurement error but treated it as negligible

Table 2 summarizes 24 papers in which authors reported the degree of measurement error and exposure and yet did not apply this knowledge to quantitatively adjust for this source of bias. Some of these papers show overconfidence in citing other sources to bolster the claim exposure measures were “valid,” even though examination of the citations reveals that measurement error in exposure may not have been ignorable [18, 59, 65]. To expand on this point further, consider one of the nutritional epidemiology articles, Chazelas et al [18] investigated the relationship between nitrate and nitrite intakes and cancer risk among 101,056 adults from the French NutriNet-Sante cohort. Dietary exposure to nitrites and/or nitrates was assessed using 24-h dietary records and linked to a comprehensive composition database which accounted for commercial names and brands of industrial products. Participants were invited to complete three web-based 24-h dietary records which were validated by a trained dietitian and against blood and urinary biomarkers where participants reported all foods and beverages consumed during the 24-hour period. Nitrite and nitrate intake was estimated from the combined intake from food, food additives, and drinking water. The authors cite at least four articles that quantify EME in various steps of their exposure assessment and yet they gave no indication why these errors are too small to worry about it, while nutritional epidemiology as a whole is replete with concerns about EME and means to adjust for it, e.g. [22, 51, 52, 53].

Three studies briefly justified the reliability of exposure measurements with reference to the sensitivity and specificity of the test instrument [19], cross validation with online resources [42] or used multivariable Mendelian randomization to obtain “unconfounded” estimates of the effect [38]. To expand further, Huntley et al., [42] compared COVID-19 outcomes in states with law-enforceable stay-at-home orders versus those with unenforceable or no stay-at-home orders. Two investigators reviewed language from each state’s policy response to the corona-virus pandemic extruded from official governmental websites. The authors did not investigate the effect of stay-at-home order misclassification even though there is a reason to believe that the concept of “enforceable” was not interpreted in the same way by residents of the states affected by the orders as they were by the investigators. Additionally, He et al., [38] conducted a two-sample Mendelian randomization study to investigate maternal respiratory health markers and birthweight. The authors noted that genetically predicted asthma and lung function may not correspond to lung function and asthma during pregnancy, however no further explanation was given or explored, and the correlation of SNPs (instrumental variable) with the latent construct (lung function in pregnancy) was not reported, even though it was estimated and used to select SNPs. As shown in Table2., while authors may have confidence that their exposure measurements were measured accurately, there do exist statistical methods in the wider body of literature that would indicate that an investigation into the effect of exposure measurement error was warranted.

Studies that quantitively investigated the impact of or adjusted for exposure measurement error

Eight studies quantitatively investigated the impact of or adjusted for EME [9, 24, 28, 32, 34, 41, 73, 78]. These studies used various methods to account for EME avoiding the common assumption that misclassification always biases results toward the null. While the efforts cited below are not formally recognized as adjustment for measurement error in exposure, they are certainly in the spirit of understanding how choices in exposure assessment affect the results of exposure-response analysis. Formalization of such intuitive sensitivity analyses within existing statistical methods should prove beneficial in terms of better appraising total uncertainty in estimated effects, and adoption of modern statistical methods to address these biases. Freedman et al., [32] conducted a population-based cohort study on patients diagnosed with diabetes to investigate the relationship between metformin and prostate cancer risk. Metformin exposure was defined as the purchase of prescription metformin or metformin combination pill. The metformin dose was taken from the purchasing data and estimated based on the defined daily dose suggested by practitioners. The defined daily dose was an “assumed average maintenance dose per day for a medication used for its main indication in adults” [32], p. 627. The authors noted that their analysis was based on “the assumption that the amount purchased equaled the amount consumed” [32], p. 627. Because Freedman et al., [32] “anticipated that the relationship of metformin to prostate cancer risk would differ depending on both duration of use and recency” they developed exposure metrics using different time windows, addressing both measurement error and reverse causation. This amounted to a sensitivity analysis of measurement error in the exposure metric. Consideration of errors at the level of exposure metric, such as time window over which exposure is accumulated should be seen as distinct from consideration of measurement error in daily dose per se. Bias in exposure-response analysis is most related to errors in exposure metric and the choice of Freedman et al., [32] to focus on it among other sources of error is appropriate.

In a more formal approach to measurement error, Ben-Hassan et al., [9] treated three of the predictors of dementia as manifest variables of a latent measure in a latent-process mixed model; the uncertainty in this calibration did not appear to have been carried through to predictive modeling. It must be noted that because the aims of Ben-Hassan et al., [9] were to make predictions, the usual concerns about bias in etiologically-relevant effect estimates do not arise: unbiased predictions can be made using error-prone observations, which is the whole basis of regression calibration approaches to measurement error [70].

Sharma et al., [73] assessed cancer risk in autoimmune hepatitis patients, accounting for EME through sensitivity tests, including exclusions for chronic liver disease, prevalent cancer, and validating exposure in a random sample of 100 medical records, deriving a positive predictive value. However, it is unfortunate that the validation study was not used to adjust for misclassification of exposure and that the validation study was not designed to determine if misclassification errors were related to the health outcome.

Two studies used quantitative bias analyses to explore the robustness of exposure outcome associations [28, 78]. Esposito et al., [28] investigated the association of placental disease and preterm delivery with opioid use during pregnancy among a cohort of pregnant women between 2000 and 2014. The authors quantitatively adjusted for EME through bias analyses which showed “that potentially differential exposure misclassification could impart considerable uncertainty in the magnitude of association, with an expected bias away from the null” [28] p. 762. Similarly, Sivaraman et al., [78] used quantitative bias analyses to investigate the association between depression and suicide in the presence of a confounder, such as anxiety [78].

Ding et al., [24] analyzed data on 1,120 premenopausal women using causal mediation analysis to quantify the role of PFAS on menopause and the mediating effect of sex hormones. The authors evaluated the interaction between exposure (PFAS) and mediator (sex hormone) using regression models, setting the mediator at a fixed value while changing PFAS exposures. This was one of the more sophisticated modeling approaches used to investigate the effect of EME in our sampled studies.

In a cohort study of visual acuity and cycling injuries, [41] partially evaluated exposure measurement error quantitively by performing analyses over different years of follow-up to account for improvement or worsening of visual acuity with age [41].

Finally, Gueltzow et al., [34] assessed various health behaviors and their contribution to risk of depression. Various inherently continuous exposures that were not known precisely were categorized. The authors conducted a limited sensitivity analysis for the definition of smoking. Furthermore, they performed probabilistic bias analysis to address social desirability bias in self-reported health behaviors using the lowest reported estimated positive and negative predictive values (eAppendix, Sect. 3 of [34]). Misclassification parameters were treated as constants and the bootstrap procedure adopted by the authors only accounted for systematic errors. Accounting for uncertainty in misclassification parameters and integrating all sources of error would have been preferable to the approach of [34], with guidance available in textbooks and teaching articles [56–58]. However, in our sample of studies,[34] treated misclassification of exposures in the most sophisticated manner compared to the rest.

Studies that conducted sensitivity analysis to investigate the impact of exposure measurement error

A total of 14 studies in our sample conducted sensitivity analyses with varying definitions of exposure to determine how robust their findings were to various choices of codifying exposure [1, 7, 17, 23, 33, 34, 47, 61, 62, 64, 71–73, 93]. Sears et al., [72] conducted a case-cohort analysis among never-smokers in Denmark to measure urinary cadmium and incident heart failure. The authors employed multiple methods to account for potential EME in urinary cadmium measurements due to urine dilution. Cartus et al., [17] explored different definitions of their exposure, sever maternal morbidity, in sensitivity analyses. They correctly state that “misclassification may be differential or nondifferential.” Likewise, Grandi et al., [33] investigated the long-term mortality among women due to early pregnancy loss by using different definitions of pregnancy loss. The authors provided a nuanced discussion about how exposure misclassification may have occurred, which can help develop a formal method to adjust for measurement error in their situation. Rotem et al., [71] assessed maternal clinical thyroid conditions in women that were recorded in electronic medical records and diagnosis of ADHD in their offspring. Exposure in this case was the disruption of homeostasis of thyroid hormones during pregnancy and the authors acknowledged that their various measures of this exposure were error prone. They addressed this concern by considering multiple definitions of thyroid conditions and their treatments. Michikawa et al., [62] assessed maternal exposure to air pollution during the first trimester of pregnancy and umbilical cord insertion abnormalities. The authors acknowledged EME due to use of stationary exposure monitors, rather than personal exposure measurements from the breathing zone, and assumptions regarding the special homogeneity of PM_2.5 concentrations. Consequently, they conducted a sensitivity analysis to address exposure measurement error by restricting the sample to women with exposure estimates that were argued to be more representative of the monitoring sites of particles, namely within 5 and 10 km. Justification for such restriction on distance was not given. Junna et al., [47] investigated the impact on mental health due to unemployment, as defined by state statistics. Realizing that there are many ways to conceptualize and measure unemployment, the authors varied definitions of exposure to unemployment to see how it impacted the observed associations. Newsome et al., [64] assessed ivacaftor (cystic fibrosis treatment) based on the period when it was available and the genetic eligibility of patients; given that the actual dose may have depended on genetic eligibility for treatment, the authors varied this definition, and conducted sensitivity analyses on the assumptions in exposure assessment.

Lu et al., [61] assessed incretin-based drugs and incidence of prostate cancer among patients with type 2 diabetes, exposure was defined as an “intention-to-treat” approach where patients were considered exposed to the study drugs until the end of follow-up. In sensitivity analyses the detection bias between exposure groups was assessed in various hazard ratio analyses by balancing exposure groups with PSA screenings. A second sensitivity analysis was conducted by repeating the authors initial analysis but with an inverse probability of censoring to account for potential differential morality between exposure groups.

Dimitris et al., [23] used causal mediation analyses to investigate the effect of twin versus singleton pregnancy on gestational diabetes modified by BMI, in a sensitivity analysis the defined gestational age and glucose tolerance tests were both broadened and restricted. The authors noted that “the total effect moved towards the null as exposure and outcome definitions… were broadened… and away from the null as exposure and outcome definitions were restricted” yet the inferences on natural effects remained similar [23], p. 283.

Auger et al., [7] conducted a cohort study of over 2 million pregnancies to investigate the association of anesthesia delivered during the first trimester with offspring congenital heart defects. Sensitivity analyses were restricted to women without heart defects, without viral infections, and to singleton pregnancies. The authors also computed E-values to assess unmeasured confounders [7].

Yim et al., [93] used the Nurses’ Health Study II (NHS II) to investigate the association between grandparents smoking during pregnancy and the risk of ADHD in grandchildren. Similar to [7] E-values were calculated to assess unmeasured confounding on exposure and outcome, exposure was also modified by removing mothers who smoked during pregnancy to isolate the effect of grandmothers who smoked during pregnancy [93]. It should be noted that not all the sensitivity analyses described above explicitly adjust for EME, however the authors should be given credit for exploring the robustness of the studied associations.

Adams et al., [1] studied the role of staff vs. resident designations in transmission of SARS-CoV-2 in long-term care facilities. While many variables in their study may be subject to uncertainty, there seems to be little reason to suspect that classification of subjects into roles in the healthcare facility is in doubt. The authors used surveillance data on COVID-19 cases from the state and county health departments in Georgia. Cases were linked to other case characteristics including vaccination status and resident or staff status to determine which group (if any) played a larger role in COVID-19 transmission. The authors note several limitations including that the role of resident or staff was inferred based on age potentially resulting in misclassification. Depending on how cases of COVID-19 were defined, the resulting counts are subject to uncertainty. The authors do note that results were similar after conducting a sensitivity analysis where long term care facilities role was determined by different criteria, however the results of this analysis were not shown in detail and exposure measurement error is not further mentioned in the study.

Discussion

While disciplines of epidemiology have made considerable progress in addressing measurement error in exposure in a more sophisticated manner in the last two decades, as a whole it is still resistant to adoption of state-of-the-art methods [55, 74] for managing this source of bias. For instance, Keogh et al., [55] provides a detailed overview of the various effects of measurement error and misclassification and their effect on standard statistical procedures such as linear and multiple regression analyses. The authors discuss in detail different ancillary studies which may elucidate the nature and magnitude of measurement error including validation studies [55]. The authors also discuss several reference instruments for common exposures studied in epidemiology such as FFQs, smoking, and physical activities. In nutritional epidemiology and studies of outcomes for which smoking may be a causal factor, the authors suggest validating exposures with recovery biomarkers, calibration studies and biochemical measures of various metabolites [55]. Shaw et al., [74] continues the authors review of measurement error and provides different statistical methods for adjustment including Bayesian methods, likelihood methods, moment reconstruction, and imputation [74]. Our analysis showed that about 12.5% of recently published papers in top journals in the field failed to acknowledge and explore the potential effect of measurement error in exposure on their results. This is an improvement from the seminal study from 20 years ago, where almost 40% of surveyed studies did not mention measurement error in exposure as a potential source of bias [49]. Our analysis of published articles did not reveal a single example of state-of-the-art adjustment for measurement error in exposure even though such methods have been codified in at least two textbooks published over the past 20 years and specifically aimed at epidemiologists [15, 35]. Even sensitivity analyses that were performed by some of the authors of papers we reviewed fell short of the best practices in quantitative bias analysis, which is again a subject of textbooks [56, 58] and teaching articles [57]. Nonetheless, the demand for quantitative bias analysis of exposure misclassification appears to be on the increase, as reflected in the recent publication of a monograph by the International Agency for Research on Cancer aimed to teach and popularize these methods [43].

We were concerned that some of the authors who did not address measurement error in exposure had no data on quality of exposure estimates that would guide both acknowledgement and quantitative treatment of the problem. However, our own search for reports on the validity and reliability of exposure measures used in survey studies that ignored measurement error in exposure revealed that in most cases information on the plausible extent of measurement error was easily obtained from a simple literature search of validation studies of the exposure measurement tool (such as validation studies of FFQs in nutritional epidemiology). Of course, nothing would replace a validation or reliability study catered to the peculiarities of every given study. In the minority of studies that performed such validation studies [73], the results were only partially used to address bias from measurement error. It is amazing to us that in epidemiology it is acceptable to collect data using methods that are not known to be valid or even reliable! We are not sure what other branches of science would find this cavalier approach acceptable. There is likely an opportunity to improve both training of epidemiologists and evaluation of their work by journal editors in remedying this fundamental concern: knowing what one measures.

Although our study was a random sample of 64 articles across three epidemiological journals, our sample did have several nutritional epidemiological studies and air pollution studies. In total, our sample included eight nutritional epidemiological studies [11, 18, 21, 46, 88, 89, 94] which utilized FFQs in various ways to assess dietary intake. In general, as is common in the field of nutritional epidemiology, FFQs are often subject to recall and reporting bias. It was common for the authors to acknowledge these limitations but not attempt to adjust for EME despite resources to do so (Table 2). Similarly, our sample included 7 air pollution studies [10, 20, 44, 62, 75, 82, 95] with varying measures of exposure. In general, these authors utilized ecological assessments of air pollution based on residential location and occasionally from stationary measurements and spatial models. Like the nutritional epidemiology studies in our sample, the authors acknowledged EME but did little to account for EME despite methodologies within air pollution epidemiology to account for EME (Table2).

Reasons for the continued (yet dwindling) disregard for measurement error in exposure may stem from several common misconceptions regarding the phenomena [84]. These include misconceptions on universality, a common belief that measurement error in exposure only affects studies with subjective measures, such as [8, 25, 42, 47, 65], erroneous belief that non-differential exposure misclassification can be assured by study design, confusion about expected effect of random error in exposure (attenuation in effect estimate) versus its realization in any given analysis [80], and possibly a concern that adjustment for misclassified exposure creates bias [14]. However, in practice even objective measurements can be error prone due to instrument calibration issues, sampling errors and environmental factors. This may have been the case within studies with objective exposure measures such as PCR tests for past COVID-19 infection, BMI, receiving medical intervention [4, 19, 64]and SNPs as measures of asthma and maternal lung function [38].

Bias towards the null was frequently postulated in several of the studies reviewed [40, 75, 76, 81, 82, 88, 94] due to non-differential misclassification of exposure. As noted elsewhere, the default hypothesis of non-differential misclassification contributing to bias towards null may not be supported and therefore cannot be claimed unless specific assumptions are met [48]. Validity and reliability studies that seek to estimate measurement error by outcome status would help to address this issue: testing whether errors in exposure depend on the outcome. Furthermore, validity and reliability studies can then be applied to performing calculations that determine the extent and direction of bias in effect estimates.

Our work suffers from several limitations. We surveyed a rather small number of papers from top journals in the field. It is reasonable to suppose that the treatment of measurement errors in exposure is worse in lower tier journals. We also likely did not capture recently published studies in top epidemiology journals that treated measurement error in a sophisticated manner, but this would speak to their rarity. Therefore, it may be instructive to also highlight work in the future that has done exemplary work addressing realistic and complex measurement errors in exposure. Our attention was on measurement errors in the main exposure of interest to the authors of the published papers. However, measurement errors in other variables, e.g. adjusted for in effort to control for confounding, leads to similar concerns of biases in unexpected directions, especially if errors in estimates are correlated (as is often the case when variables are derived from the same sources of primary data) [30, 35, 52, 53].

We conclude that much remains to be done to bring epidemiology practices in line with the state-of-the-art methods developed for adjustment for measurement error in exposure. The practice of qualitatively discussing the impact of measurement error in exposure in epidemiology should be consigned to the past considering developments in statistics that render such discussions obsolete [55, 74] and unhelpful in aiding users of epidemiologic data to interpret it in quantitative risk assessments [67].

Authors’ contributions

A.J.R, M.H., and I.B wrote the original manuscript text, determined the methodology, and conducted the overall analysis. A.J.R and M.H prepared Table 1, and 2. I.B. and G.M reviewed and edited the manuscript for content.

Funding

This research has been supported in part by The Center for Truth in Science (CTIS).

Data availability

No datasets were generated or analysed during the current study.

Declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Not applicable.