Table 4.

Reliability of other assessment methods in estimating past occupational exposures in case–control studies in the population.

Authors, year	Exposure	Assessment method	Comparison method	Reliability test	Results
Pronk et al., 2012	Diesel engine exhaust	Use of expert-derived algorithms to assess exposure probability, intensity, and frequency based on occupational histories with specific task information	Case-by-case assessment by an occupational hygienist	Weighted κ for ordinal exposure measures; Spearman correlation for continuous exposure measures	Weighted κ = 0.68– 0.81 for ordinal exposure probability, frequency, and intensity; Spearman ρ = 0.70– 0.72 for continuous exposure frequency and intensity
Bourgkard et al., 2013	Asbestos and PAHs	Algorithmic assessment based on task-based questionnaire data	Reference case-by- case assessment by two experts by consensus based on full interview data; population-based asbestos JEM (Févotte et al., 2011)	Weighted κ for ordinal exposure levels; OR for lung cancer and asbestos exposure	κ = 0.61 for asbestos and 0.36 for PAHs against referent expert assessment; κ = 0.26 against asbestos JEM; lung cancer OR = 1.18 (95% CI 1.06–1.31) based on algorithm- derived exposures and 1.02 (95% CI 0.91– 1.16) based on JEM- assessed exposures
Friesen et al., 2013	Diesel engine exhaust	Algorithm-based assessment (Pronk et al., 2012) to assess exposure probability, intensity, and frequency based on questionnaire responses	Case-by-case assessment by three experts individually and by aggregate	Weighted κ for exposure probability, intensity, and frequency	κ = 0.58–0.81 (median = 0.70) between individual expert rating and algorithmic assessment; κ = 0.82 for aggregated expert assessment versus algorithmic assessment
Wheeler et al., 2013	Diesel engine exhaust	Use of tree-based statistical learning models to predict exposure probability, frequency, and intensity using previous expert assessments as training data	Case-by-case assessment by an occupational hygienist	Percent agreement for presence of exposure, and ordinal exposure probability, frequency, and intensity	Percent agreement = 92–94 for presence of exposure; percent agreement = 7–90 for ordinal exposure probability, frequency, and intensity
Peters et al., 2014	Diesel engine exhaust, pesticides, and solvents	Expert-derived algorithms were used to assess presence/ absence of exposure from information obtained from questionnaires	Case-by-case assessment by an occupational hygienist	κ agreement on presence of exposure	κ = 0.51–0.84 (median 0.73)
Friesen et al., 2014	TCE	A systematic process was developed to extract free-text responses in occupational histories by identifying keywords and phrases associated with exposure	Case-by-case expert assessment	Percent agreement on presence of exposure	Percent agreement = 98.7
Friesen et al., 2015b	Diesel engine exhaust	Hierarchical clustering model grouped jobs with similar exposures based on questionnaire responses	Algorithmic assessment of exposure probability, intensity, and frequency (Pronk et al., 2012)	ICCs within job title clusters	ICC > 80% for exposure probability with >500 clusters w in model; ICC > 70% for exposure frequency and intensity with > 200 model clusters
Wheeler et al., 2015	Diesel engine exhaust	Use of ordinal and nominal classification tree models to predict exposure probability, frequency, and intensity using expert assessment information	Case-by-case assessment by an occupational hygienist	Somer’s d for nominal and ordinal exposure metrics (probability, frequency, and intensity)	Somer’s d = 0.61–0.66
Friesen et al., 2016b	Diesel engine exhaust	Application of classification tree models (Wheeler et al., 2013)	Case-by-case assessment by two experts independently	Weighted κ for ordinal measures of exposure probability, intensity, and frequency	Weighted κ = 0.09– 0.91; model performance was better for unexposed and highly exposed jobs, and for predicting exposure probability and intensity

CI = confidence interval; ICC = intraclass correlation coefficient; OR = odds ratio; TCE = trichloroethylene.