Abstract
Objectives
There is great interest in evaluating gene-environment interactions with chemical exposures, but exposure assessment poses a unique challenge in case-control studies. Expert assessment of detailed work history data is usually considered the best approach, but it is a laborious and time-consuming process. We set out to determine if a less intensive method of exposure assessment (a job exposure matrix [JEM]) would produce similar results to a previous analysis that found evidence of effect modification between expert assessed-lead exposure and risk of brain tumors by a single nucleotide polymorphism in the ALAD gene (rs1800435).
Methods
We used data from a study of 355 patients with glioma, 151 patients with meningioma and 505 controls. Logistic regression models were used to examine associations between brain tumor risk and lead exposure and effect modification by genotype. We evaluated Cohen’s kappa, sensitivity and specificity for the JEM compared to the expert-assessed exposure metrics.
Results
Although effect estimates were imprecise and driven by a small number of cases, we found evidence of effect modification between lead exposure and ALAD genotype when using expert- but not JEM-derived lead exposure estimates. Kappa values indicated only modest agreement (< 0.5) for the exposure metrics, with the JEM indicating high specificity (~0.9) but poor sensitivity (~0.5). Disagreement between the two methods was generally due to having additional information in the detailed work history.
Conclusion
These results provide preliminary evidence suggesting that high quality exposure data are likely to improve the ability to detect genetic effect modification.
Introduction
Case-control studies are often the most feasible study design for the investigation of occupational exposures and rare diseases. However, occupational exposure assessment can be quite challenging in this setting, given the low prevalence of exposure to many specific agents in the study population and the inability to obtain workplace records to reconstruct exposures accurately.
Job exposure matrices (JEM) and expert assessment of self-reported job histories are two occupational exposure assessment methods that have been commonly utilized in case-control studies 1. There is some overlap between these procedures since both are typically developed by industrial hygienists who based their decisions on literature reviews and/or knowledge of industrial processes and occupational tasks. A JEM is a database that uses coded job and industry titles to assign exposures to a specific agent 1,2, while expert assessment involves the evaluation of detailed self-reported work history information from structured questionnaires by industrial hygienists to assign exposures 1,3,4. Thus, there may be a gain in accuracy over a JEM because experts can account for within-job variability of exposures. Expert assessment of self-reported job histories has some limitations, including its dependence on subject recall of detailed work information and the experience and subjectivity of the expert. Nonetheless, expert assessment of subject specific job history information currently is considered by many to be the best exposure assessment method for case-control studies 5. However, expert assessment is also a costly and time-consuming process1; as such, there is great interest in determining if less labour-intensive methods can produce similar results.
The issue of accurate exposure assessment in case-control studies is assuming even greater attention than in the past with the growing interest in studies of gene-environment interactions. It has been shown that even small errors in the assessment of environmental factors can result in biased estimates of interaction parameters and substantial decreases in study power 6,7. Also, unless we use the most accurate exposure assessment methods in studies evaluating genotypes and environmental exposures, there is a greater probability of detecting associations with genotypes rather than environmental exposures, since genotyping typically has a lower degree of classification error 8.
While there have been comparisons of occupational exposure assessment methods in previous case-control studies 9–15, none, to our knowledge, have compared exposure assessment methods in the context of gene-environment interactions. We evaluated occupational exposure assessment for lead by a JEM in a case-control study of adult brain tumors that previously reported evidence of effect modification of expert assessed lead exposure by the G177C polymorphism (rs1800435) of the δ-aminolevulinic acid dehydratase (ALAD) gene (G = ALAD1 and C = ALAD2) 16. We determined whether risk estimates produced by the JEM-based exposure assessment were similar to those produced by the expert assessment. We also present the agreement, sensitivity and specificity of the JEM exposure estimates in comparison to the expert estimates.
Materials and Methods
Study Population
The study population has been described in detail previously 16,17. Adult patients diagnosed with a primary glioma (489), meningioma (197) or acoustic neuroma (96) during 1994–1998 at one of three hospitals were enrolled into the study (92% participation). Controls were 799 patients admitted to the same hospitals for various nonneoplastic conditions that were frequency matched (1:1 ratio) to cases based on categories of sex, age, race, hospital and residential proximity to the hospital (86%).
Blood samples were obtained from 382 patients with glioma (78%), 158 patients with meningioma (68%), and 540 controls (62%). Given the small number of patients with acoustic neuroma, they were excluded from this analysis. The study protocol was approved by the institutional review board of each participating institution, and written informed consent was obtained from each patient or proxy.
Genotyping
Genotyping of the rs1800435 polymorphism was conducted by the Core Genotyping Facility of the National Cancer Institute using a medium-throughput Taqman assay 18. There was a 90% concordance rate for quality control duplicate samples, and 94% of samples were successfully genotyped (355 glioma cases, 151 meningioma cases, 505 controls). There was no statistically significant departure from Hardy-Weinberg equilibrium among controls.
Data collection
A trained research nurse administered an in-person interview to each patient or proxy (8%) 16. For all patients, a life-time occupational history (job title, type of industry, dates, activities and materials and chemicals used) was obtained along with information on other potential risk factors for brain tumors. In addition to providing a standard work history, participants answered one of 63 questionnaires on specific jobs with detailed questions developed by an industrial hygienist regarding specific tasks and workplace conditions within each job 19.
Lead exposure assessment
A single expert, blinded to the case/control status of the participants, carried out the assessment over an 18 month period. For the expert assessment, exposure guidelines were constructed based on a comprehensive literature review of peer-reviewed articles and technical reports linking job titles and industries with air level and/or blood level measurements of lead by decade 16. These guidelines were used in conjunction with detailed questions on tasks (welding, soldering, painting, sanding etc.) and workplace conditions (ventilation, use of personal protective equipment etc.) provided by the subject to assign an intensity [estimated airborne concentration (0, 5–9, 10–29, 30–49, 50–249 or ≥250 μg/m3)], a frequency [time exposed to lead (<1, 1–9, 10–29 or 30–40 hours/week)] and a probability [likelihood of lead exposure actually occurring (0, 1–9, 10–49, 50–89 or ≥90%)] for each job reported by a study participant.
The JEM, which had been developed by different investigators for a different study of brain tumors, assigned intensity and probability of lead exposure (none = 0, low = 1, medium = 2, high = 3) independently for each three-digit occupation and industry code from the 1980 Census list of occupations and industries 20. A separate metric for frequency of lead exposure was not developed, but was incorporated into the metric for intensity. The intensities and probabilities were based on published literature, computerized exposure databases, technical reports and expert experience. Each final intensity and each final probability score was obtained by multiplying the occupation and industry scores for each job to create a 7 point scale (0, 1, 2, 3, 4, 6, 9). If either the occupation or industry code was missing, the final intensity and probability score was obtained by squaring the individual occupation or industry score. For example, if the industry intensity score was missing, but the occupational intensity score was 2, the occupational score would be squared to give a final intensity score of 4.
Participants were considered “ever exposed” if they had at least one lead exposed job at any exposure intensity, but probability of ≥10% (≥2 for the JEM). These probability levels were chosen to provide reasonable sensitivity and high specificity of the exposure assessment methods, minimizing the attenuation of effect estimates by exposure misclassification in population-based case-control studies 21. Ever exposure was set to missing for individuals with incomplete data for any job in their work history, unless exposure to lead was indicated for at least one job with complete data.
For the expert assessment method, lifetime cumulative lead exposure was calculated by multiplying the number of years in each lead exposed job by the midpoint of the estimated airborne concentration range and the estimated frequency of exposure and then summing the cumulative exposures across jobs. For the JEM, the product of the number of years in each job and the estimated intensity score for each job was summed across jobs to derive a cumulative exposure metric. Cumulative lead exposure was set to missing for individuals with incomplete information for any job in their lifetime job history. In order to facilitate comparison between the two methods, the cumulative expert and JEM exposure metrics were categorized into four groups based on percentile distributions among the controls [unexposed, ≤80th percentile, > 80th – 95th percentile, > 95th percentile, unknown]. Unexposed individuals comprised 60 to 70% of the distribution of controls for the cumulative expert and JEM exposure metrics, respectively.
For a subset of subjects whose characterization as ever versus never exposed differed according to the exposure assessment method, an industrial hygienist (PAS), who supervised the expert assessment in the original study, reviewed the subjects’ work history information and JEM assessment to identify reasons contributing to the differences.
Statistical analyses
Unconditional logistic regression was used to evaluate the associations of glioma and meningioma with exposure to lead (ever exposure and categories of cumulative lead exposure). Effect modification was evaluated with likelihood ratio tests comparing nested unconditional logistic regression models that did and did not include cross-products terms for lead exposure and ALAD genotype. Odds ratios (OR) and 95% confidence intervals (CI) for the association between lead exposure and glioma and meningioma were calculated for ALAD1 homozygotes and ALAD2 carriers; given the small number of ALAD2 homozygotes (n = 1 glioma case; n = 3 meningioma cases; n = 6 controls) they were combined with the heterozygous participants in the analyses. The matching variables age, sex, race, hospital and residential proximity to the hospital were included in all analyses.
Akaike’s information criterion22 (AIC) and Bayesian information criterion23 (BIC), were calculated to compare the goodness of fit between models using the expert-based exposure data to models using the JEM-based exposure data. Smaller values of these statistics indicate models that better fit the data.
Compared to the expert exposure assessment metrics, sensitivity, specificity percent agreement and the Cohen’s kappa statistic (which assesses agreement beyond chance) were calculated for the similarly derived JEM estimates for meningioma cases and controls. For cumulative lead exposure, JEM sensitivity and specificity were assessed for successful classification of subjects as “highly” exposed (i.e. in either of the two highest categories of exposure). Weighted Cohen’s kappa statistics were also calculated for cumulative exposure where a single category difference in agreement was given a weight of 0.66 and a two category difference in agreement was given a weight of 0.33. All statistical analyses were completed in STATA (Version 10, College Station, Texas).
Results
The distribution of matching factors among cases and controls has been previously reported16 and is not discussed here. As seen in Table 1, the JEM, for both the ever and cumulative exposure metrics, classified a greater proportion of participants as unexposed compared to the expert-derived estimates. For both metrics, the JEM and expert assessments classified different numbers of individuals as “unknown”. In some instances, exposure values could not be assigned based on the JEM because of missing occupation and industry codes, but the expert was able to assign exposure values using the detailed work history information. In other instances, where the JEM assigned exposure values if one of the two codes (occupation or industry) was missing, the expert determined that exposure could not be assigned based on the occupation or industry title alone, and in the absence of detailed work history information, set the exposure values to missing.
Table 1.
Risk of glioma and meningioma with exposure to lead determined by an expert and a job exposure matrix by ALAD G177C (rs1800435) genotype in the NCI Brain Tumor Study
| Overall | ALAD1* | ALAD2* | P† | |||||||
|---|---|---|---|---|---|---|---|---|---|---|
| GLIOMA | ||||||||||
| Ever exposed to lead (Expert) | Cases (%) | Controls (%) | OR (95% CI) | Cases (%) | Controls (%) | OR (95% CI) | Cases (%) | Controls (%) | OR (95% CI) | |
| No | 196 (55) | 288 (57) | 1.0 | 169 (56) | 236 (56) | 1.0 | 27 (50) | 52 (62) | 1.0 | 0.2 |
| Yes | 157 (44) | 216 (43) | 0.8 (0.5,1.1) | 130 (43) | 183 (44) | 0.7 (0.5,1.0) | 27 (50) | 33 (39) | 1.1 (0.5,2.3) | |
| Unknown | 2 (1) | 1 (<1) | - | 2 (1) | 1 (<1) | - | 0 (0) | 0 (0) | - | |
| Ever exposed to lead (JEM) | ||||||||||
| No | 253 (71) | 365 (72) | 1.0 | 214 (71) | 305 (73) | 1.0 | 39 (72) | 60 (71) | 1.0 | |
| Yes | 94 (26) | 136 (27) | 0.8 (0.6,1.1) | 80 (27) | 113 (27) | 0.9 (0.6,1.4) | 14 (26) | 23 (27) | 0.7 (0.3,1.6) | 0.8 |
| Unknown | 8 (2) | 4 (1) | - | 7 (2) | 2 (<1) | - | 1 (2) | 2 (2) | - | |
| Cumulative Lead Exposure (Expert) | ||||||||||
| Unexposed | 196 (55) | 288 (57) | 1.0 | 169 (56) | 236 (56) | 1.0 | 27 (50) | 52 (61) | 1.0 | 0.8 |
| ≤80th percentile | 77 (22) | 110 (22) | 0.8 (0.5,1.1) | 64 (21) | 93 (22) | 0.7 (0.5,1.1) | 13 (24) | 17 (20) | 1.1 (0.4,2.6) | |
| > 80th to 95th percentile | 48 (14) | 83 (16) | 0.6 (0.4,0.9) | 40 (13) | 70 (17) | 0.5 (0.3,0.9) | 8 (15) | 13 (15) | 0.7 (0.3,2.1) | |
| > 95th percentile | 21 (6) | 21 (4) | 1.0 (0.5,2.0) | 17 (6) | 18 (4) | 0.9 (0.4,1.9) | 4 (7) | 3 (4) | 1.8 (0.3,8.9) | |
| Unknown | 13 (4) | 3 (1) | - | 11 (4) | 3 (1) | - | 2 (4) | 0 (0) | - | |
| P-trend‡ | 0.1 | 0.09 | 0.9 | |||||||
| Cumulative Lead Exposure (JEM) | ||||||||||
| Unexposed | 253 (71) | 365 (72) | 1.0 | 214 (71) | 305 (73) | 1.0 | 39 (72) | 60 (71) | 1.0 | 0.8 |
| ≤80th percentile | 15 (4) | 31 (6) | 0.6 (0.3,1.2) | 14 (5) | 28 (7) | 0.6 (0.3,1.2) | 1 (2) | 3 (4) | 0.4 (0.04,4.6) | |
| > 80th to 95th percentile | 53 (15) | 78 (15) | 0.8 (0.5,1.2) | 45 (15) | 62 (15) | 0.9 (0.5,1.4) | 8 (15) | 16 (19) | 0.6 (0.2,1.7) | |
| > 95th percentile | 22 (6) | 26 (5) | 0.9 (0.5,1.7) | 18 (6) | 22 (5) | 0.9 (0.4,1.7) | 4 (7) | 4 (5) | 1.1 (0.3,4.9) | |
| Unknown | 12 (3) | 5 (1) | - | 10 (3) | 3 (1) | - | 2 (4) | 2 (2) | - | |
| P-trend‡ | 0.4 | 0.4 | 0.6 | |||||||
| MENINGIOMA | ||||||||||
| Ever exposed to lead (Expert) | ||||||||||
| No | 108 (72) | 288 (57) | 1.0 | 86 (74) | 236 (56) | 1.0 | 22 (63) | 52 (61) | 1.0 | 0.09 |
| Yes | 42 (28) | 216 (43) | 0.9 (0.5,1.5) | 29 (25) | 183 (44) | 0.8 (0.4,1.3) | 13 (37) | 33 (39) | 1.8 (0.7,4.8) | |
| Unknown | 1 (<1) | 1 (<1) | - | 1 (1) | 1 (<1) | - | 0 (0) | 0 (0) | - | |
| Ever exposed to lead (JEM) | ||||||||||
| No | 123 (81) | 365 (72) | 1.0 | 95 (82) | 305 (73) | 1.0 | 28 (80) | 60 (71) | 1.0 | 0.9 |
| Yes | 26 (17) | 136 (27) | 0.9 (0.5,1.7) | 20 (17) | 113 (27) | 0.9 (0.5,1.7) | 6 (17) | 23 (27) | 1.0 (0.3,3.1) | |
| Unknown | 2 (1) | 4 (1) | - | 1 (1) | 2 (<1) | - | 1 (3) | 2 (2) | - | |
| Cumulative Lead Exposure (Expert) | ||||||||||
| Unexposed | 108 (72) | 288 (57) | 1.0 | 86 (74) | 236 (56) | 1.0 | 22 (63) | 52 (62) | 1.0 | 0.04 |
| ≤80th percentile | 17 (11) | 110 (22) | 0.7 (0.4,1.3) | 15 (13) | 93 (22) | 0.7 (0.4,1.4) | 2 (6) | 17 (20) | 0.5 (0.09,2.5) | |
| > 80th to 95th percentile | 15 (10) | 83 (16) | 1.0 (0.5,2.1) | 9 (8) | 70 (17) | 0.7 (0.3,1.8) | 6 (17) | 13 (15) | 2.4 (0.7,8.8) | |
| > 95th percentile | 8 (5) | 21 (4) | 2.7 (1.0,7.8) | 3 (3) | 18 (4) | 1.2 (0.3,4.8) | 5 (14) | 3 (4) | 13.2 (2.4,72.9) | |
| Unknown | 3 (2) | 3 (<1) | - | 3 (3) | 3 (1) | - | 0 (0) | 0 (0) | - | |
| P-trend‡ | 0.4 | 0.6 | 0.007 | |||||||
| Cumulative Lead Exposure (JEM) | ||||||||||
| Unexposed | 123 (81) | 365 (72) | 1.0 | 95 (82) | 305 (73) | 1.0 | 28 (80) | 60 (71) | 1.0 | 0.9 |
| ≤80th percentile | 5 (3) | 31 (6) | 0.6 (0.2,1.8) | 4 (3) | 28 (7) | 0.6 (0.2,2.0) | 1 (3) | 3 (4) | 1.1 (0.09,12.5) | |
| > 80th to 95th percentile | 16 (11) | 78 (15) | 1.1 (0.5,2.1) | 13 (11) | 62 (15) | 1.2 (0.6,2.5) | 3 (9) | 16 (19) | 0.7 (0.2,3.0) | |
| >95th percentile | 4 (3) | 26 (5) | 0.9 (0.3,2.8) | 3 (3) | 22 (5) | 0.8 (0.2,3.1) | 1 (3) | 4 (5) | 1.1 (0.1,12.0) | |
| Unknown | 3 (2) | 5 (1) | - | 1 (1) | 3 (1) | - | 2 (6) | 2 (2) | - | |
| P-trend‡ | 0.9 | 0.9 | 0.8 | |||||||
ALAD1 homozygotes: n(%) controls = 420 (83), n(%) glioma cases = 301 (85), n (%) meningioma cases = 116 (77); ALAD2 carriers: n(%) controls = 85 (17), n(%) glioma cases = 54 (15), n (%) meningioma cases = 35 (23)
P-value for effect modification of lead exposure by ALAD genotype
Test for trend excluded Unknown category
There was no evidence of an overall association between lead exposure and glioma or of effect modification of the relationship between lead and glioma by ALAD genotype using either method of exposure assessment (Table 1). There was evidence of an association between lead exposure and meningioma among individuals with the highest category of expert assessed cumulative lead exposure [OR = 2.7 (95% CI: 1.0, 7.8)], but no similar evidence was found when examining the JEM cumulative exposure metric [OR = 0.9 (95% CI: 0.3, 2.8)]. Also for meningioma, the metric based on expert-assessment indicated borderline evidence of effect modification by ALAD genotype with ever exposure to occupational lead (p = 0.09) and statistically significant evidence of effect modification with cumulative lead exposure (p = 0.04). Neither metric derived from the JEM showed any evidence of effect modification of meningioma risk with ALAD genotype.
For glioma, model fit statistics were very similar when comparing logistic regression models using the expert-assessed exposure data to models using the JEM-assessed exposure data (results not shown). While for meningioma, fit was similar for the overall expert and JEM models (results not shown), AIC and BIC were lower for the expert models that included SNP effect modification terms compared to the JEM models with these terms (expert ever exposure: AIC = 654, BIC =743; JEM ever exposure: AIC = 670, BIC = 769; expert cumulative exposure: AIC = 640, BIC = 747; JEM cumulative exposure: AIC = 650, BIC = 757), indicating a better fit of the models to the expert-assessed exposure data than to the JEM-assessed exposure data.
Tables 2 and 3 show the cross-classifications of meningioma cases and controls with respect to the expert- and JEM-derived ever versus never and cumulative exposure metrics, respectively. In general, the expert assessment tended to classify more subjects as exposed or into higher categories of exposure when compared to the JEM. For example, among all meningioma cases and controls, the expert assessment classified 40% of subjects as ever-exposed, while the JEM classified 25% as ever-exposed.
Table 2.
Comparison of ever lead exposure determined by expert assessment with exposure determined by a job exposure matrix for meningioma cases and controls in the NCI Brain Tumor Study
| Overall | ALAD1 | ALAD2 | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
| ||||||||||||
| Expert | Expert | Expert | ||||||||||
| JEM | 0 | 1 | U | JEM | 0 | 1 | U | JEM | 0 | 1 | U | |
| 0* | 364 | 122 | 2 | 0 | 298 | 100 | 2 | 0 | 66 | 22 | 0 | |
| 1* | 28 | 134 | 0 | 1 | 22 | 111 | 0 | 1 | 6 | 23 | 0 | |
| U* | 4 | 2 | 0 | U | 2 | 1 | 0 | U | 2 | 1 | 0 | |
| Sensitivity† | 0.52 | 0.53 | 0.51 | |||||||||
| Specificity† | 0.93 | 0.93 | 0.92 | |||||||||
| % agreement | 76 | 76 | 74 | |||||||||
| Kappa | 0.47 | 0.48 | 0.43 | |||||||||
0 = never, 1 = ever, U = unknown
Restricted to non-missing data
Table 3.
Comparison of cumulative lead exposure determined by expert assessment with exposure determined by a job exposure matrix for meningioma cases and controls in the NCI Brain Tumor Study
| Overall | ALAD1 | ALAD2 | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
| ||||||||||||||||||
| Expert | Expert | Expert | ||||||||||||||||
| JEM | 0 | 1 | 2 | 3 | U | JEM | 0 | 1 | 2 | 3 | U | JEM | 0 | 1 | 2 | 3 | U | |
| 0* | 364 | 70 | 42 | 8 | 4 | 0 | 298 | 58 | 34 | 6 | 4 | 0 | 66 | 12 | 8 | 2 | 0 | |
| 1* | 10 | 12 | 13 | 1 | 0 | 1 | 9 | 12 | 11 | 0 | 0 | 1 | 1 | 0 | 2 | 1 | 0 | |
| 2* | 18 | 37 | 29 | 9 | 1 | 2 | 13 | 32 | 23 | 6 | 1 | 2 | 5 | 5 | 6 | 3 | 0 | |
| 3* | 0 | 7 | 13 | 10 | 0 | 3 | 0 | 5 | 11 | 9 | 0 | 3 | 0 | 2 | 2 | 1 | 0 | |
| U* | 4 | 1 | 1 | 1 | 1 | U | 2 | 1 | 0 | 0 | 1 | U | 2 | 0 | 1 | 1 | 0 | |
| Sensitivity† | 0.49 | 0.49 | 0.48 | |||||||||||||||
| Specificity† | 0.88 | 0.88 | 0.87 | |||||||||||||||
| % agreement | 63 | 64 | 61 | |||||||||||||||
| Kappa | 0.29 | 0.30 | 0.24 | |||||||||||||||
| w-Kappa‡ | 0.42 | 0.43 | 0.37 | |||||||||||||||
0 = no exposure, 1 = ≤to 80th percentile, 2 = >80th to 95th percentile, 3 = >95th percentile, U = unknown
Classified as 2 or 3; restricted to non-missing data
Weighted Cohen’s Kappa (single category difference in agreement given weight of 0.66 and two category difference in agreement given weight of 0.33); restricted to non-missing data
When examining all subjects, ALAD1 homozygotes and ALAD2 carriers, the sensitivity and specificity for the JEM ever/never exposure metric was approximately 0.5 and 0.9, respectively, for each of the three groups. The percentage of subjects showing exact agreement of exposure classification among these groups was approximately 75% and the corresponding Kappa values was approximately 0.4 for all three groups.
When considering cumulative exposure, the sensitivity (specificity) of the JEM for successful classification of “high” exposure (>80th percentile, i.e. categories 2 and 3) was approximately 0.5 (0.9) among the three groups (all subjects, ALAD1 homozygotes and ALAD2 carriers). The percentage of subjects showing exact agreement of exposure classification among these groups was approximately 60%. Kappa (weighted-kappa) values for these three groups were approximately 0.3 (0.4).
An expert industrial hygienist (PAS) examined the JEM and work history information for the 31 ALAD2 carriers in Table 2 for which the JEM and expert exposure assessments did not agree with respect to having ever been exposed to lead. For these 31 individuals, the JEM and expert assessed exposures differed for 43 of the 169 (25%) reported jobs because of differences in the assignment of probability or intensity of exposure between the two exposure assessment methods. For 37 of these 43 jobs, which varied widely with respect to occupation and industry titles, the expert assessment indicated exposure where the JEM did not. For most of these jobs (27) this was because work history data used in the expert assessment indicated the occurrence of lead exposure that would not be expected based on examining occupation and industry titles alone. For example, the JEM indicated zero exposure to lead for a job as an editor in the newspaper industry. However, the job history indicated that the individual spent part of his time in the production area during a time period when molten lead was still being used for typesetting. In only one instance did the work history data indicate a lack of lead exposure for a job which otherwise would have been expected based on the occupation and industry titles alone. For nine jobs, differences were attributed to the expert’s knowledge of technical details for those particular jobs, and for six jobs the potential for lead exposure was uncertain and could have been argued either way.
A similar examination of sources of discrepancies between the cumulative exposure metrics was difficult given differences in the intensity scales and incorporation of frequency of exposure data between the two exposure assessment methods. However, patterns were similar to those observed for the ever versus never exposure analysis.
Discussion
An association between lead and meningioma and its modification by the ALAD rs1800435 polymorphism was observed based on expert assessment of exposure but not when using a JEM exposure matrix. Based on kappa statistics, there was fair to moderate agreement between the exposure metrics derived from the JEM and expert exposure assessment methods 24. While the JEM displayed reasonable specificity compared to the expert assessment, its sensitivity was only modest. As expected, the kappa statistics, sensitivity and specificity values did not vary appreciably when subjects were stratified by genotype. Although neither the expert nor the JEM approach is perfect, we believe that in this case, the expert approach is likely to be more accurate because, unlike the JEM, expert assessment has the ability to account for within-job variability by using detailed questionnaire-based work history information (e.g. specific tasks, control measures etc.) specific to the study at hand.
The exposure prevalence for lead, based on expert assessment, was approximately 40% among controls (Table 1) which may seem high. We believe the prevalence is realistic because of the calendar time of the study, and because we considered all exposures (including low exposures). The mean blood lead level in the US population circa 1970 was 12.8 ug/dL 25. Though we estimated occupational exposure prevalence in our study, this figure does indicate fairly ubiquitous exposure to lead.
The risk estimates and corresponding 95% CI and p-values observed for the expert-assessed lead data differ slightly from those previously reported16. This is because the previous analyses considered only those jobs with an exposure intensity of greater than or equal to 10 μg/m3 to be exposed to lead. To facilitate comparison between the expert and JEM assessment methods, we did not impose this restriction in the current analysis.
Expert- and JEM-based exposure assessments have been compared in previous case-control studies 9–14. In our study, we observed slightly higher levels of agreement between expert- and JEM-based exposure assessments than observed in other studies examining various exposures 9–11. For example, in a case-control study of glioma, Benke et al (2001) calculated a kappa of 0.33 for ever exposure to lead 9 while we calculated a kappa of 0.5 for ever exposure to lead among meningioma cases and controls. Even though we observed a higher kappa value, 0.5 only represents a moderate level of agreement 24. As with our study, previous evaluations of various exposures in case-control studies observed poor sensitivity, yet high specificity, for JEMs compared to expert assessments 12–14. Rybicki et al (1997), for example, observed a sensitivity of 0 and a specificity of 0.93 when comparing lead exposure estimates derived from a JEM versus expert assessment 12.
Although in this paper we consider expert assessment as the more accurate method, it is also imperfect. The quality of the assessment depends on the experience of the expert 5, and there may be differences in exposure assignment as the study progresses, although the latter issue can be somewhat mitigated with detailed and standardized rules 1,26. While the ability to account for within-job variability is a strength of expert assessment because of the potential gain in accuracy, this gain may be offset by limitations in the ability of participants to recall detailed work information. The use of self-reported job histories also raises issues of response bias (i.e. cases indicate greater exposures to lead because of their disease status), but this is not likely to be a problem in our study given that our questionnaire was designed and administered in such a way as to to assess the potential for exposure to a wide variety of agents without prior knowledge of what exposures would be of most interest. Thus, any resulting misclassification of exposure would likely be non-differential, and the risk estimates would most typically be biased towards the null.
Use of a biomarker for cumulative lead exposure such as bone lead measurements rather than questionnaires would have been ideal. However, evaluation of the association between lead exposure and brain tumors was not the primary objective of this study when it was initiated, and, as such, biomarker data for lead exposure were not collected. In a previous comparison of exposure assessment methods including biomarker data, Tielemans et al (1999) found that assessment of individuals as exposed versus unexposed to chromium by job specific questionnaires compared better to urinary measurements than when using a JEM to assess exposure 15. Although urinary chromium concentrations were clearly increased in subjects classified as exposed by the job specific questionnaire, the exposed group from the job specific questionnaire was restricted to those individuals that were determined to be highly exposed, and kappa statistics indicated only poor to moderate agreement. While in the absence of actual measurement data, expert assessment is considered the best approach to date for assessing past exposures in population-based case-control studies 5, resources should be directed towards developing better methods that address the limitations of expert assessments.
Expert assessment has been reported to provide greater statistical power than other methods (including JEM-based exposure assessment) for detecting associations between exposure and disease 27. In the analysis of gene-environment effect modification, statistical power becomes an even greater issue as studies typically require large sample sizes to detect effect modification 28. It has been demonstrated that even small errors in the assessment of environmental factors can result in biased interaction parameters and substantially increased sample size requirements for the detection of effect modification 6,7. In our analyses, misclassification of exposure by the JEM as compared to the expert assessment resulted in smaller odds ratios and less likelihood of detecting an effect. These results indicate that investigators would benefit from using the most accurate method of exposure assessment available, since the attenuating effects of exposure misclassification would result in increased sample size requirements to detect effect modification 29 that would offset any savings from using a less costly method of exposure assessment 1.
As genome-wide association studies identify genetic polymorphisms associated with disease, there is increasing interest and need for evaluating interaction with environmental factors. Although we recognize the need for replication of the effect modification results given the small sample size of variant carriers exposed to lead, preliminary findings suggest that high quality exposure data are likely to improve the ability to detect genetic effect modification.
What this paper adds
Expert assessment of occupational exposures in case-control studies is considered the best approach but is a laborious process.
We examined if a less intensive method of exposure assessment, the job exposure matrix, would produce similar results in the context of gene-environment interactions, which has not been previously evaluated.
A gene-environment interaction that was previously detected using expert assessment was not found when using a job exposure matrix
The results suggest that high quality exposure data are needed to detect gene-environment interactions.
Acknowledgments
Funding
This study was funded under contract N01-CO-12400 from the Intramural Research Program of National Cancer Institute, National Institutes of Health, Department of Health and Human Services.
Footnotes
Competing Interests
None.
Ethics approval
The study was approved by the Human Subjects Review Board of the National Cancer Institute.
Exclusive License
The Corresponding Author has the right to grant on behalf of all authors and does grant on behalf of all authors, an exclusive licence (or non-exclusive for government employees) on a worldwide basis to the BMJ Publishing Group Ltd and its Licensees to permit this article (if accepted) to be published in Occupational and Environmental Medicine and any other BMJPGL products to exploit all subsidiary rights, as set out in our licence.
References
- 1.McGuire V, Nelson LM, Koepsell TD, Checkoway H, Longstreth WT., Jr Assessment of occupational exposures in community-based case-control studies. Annu Rev Public Health. 1998;19:35–53. doi: 10.1146/annurev.publhealth.19.1.35. [DOI] [PubMed] [Google Scholar]
- 2.Hoar SK, Morrison AS, Cole P, Silverman DT. An occupation and exposure linkage system for the study of occupational carcinogenesis. J Occup Med. 1980;22:722–6. [PubMed] [Google Scholar]
- 3.Stewart WF, Stewart PA. Occupational case-control studies: I. Collecting information on work histories and work-related exposures. Am J Ind Med. 1994;26:297–312. doi: 10.1002/ajim.4700260304. [DOI] [PubMed] [Google Scholar]
- 4.Gerin M, Siemiatycki J, Kemper H, Begin D. Obtaining occupational exposure histories in epidemiologic case-control studies. J Occup Med. 1985;27:420–6. [PubMed] [Google Scholar]
- 5.Teschke K, Olshan AF, Daniels JL, et al. Occupational exposure assessment in case-control studies: opportunities for improvement. Occup Environ Med. 2002;59:575–93. doi: 10.1136/oem.59.9.575. discussion 594. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Burstyn I, Kim HM, Yasui Y, Cherry NM. The virtues of a deliberately mis-specified disease model in demonstrating a gene-environment interaction. Occup Environ Med. 2009;66:374–80. doi: 10.1136/oem.2008.039081. [DOI] [PubMed] [Google Scholar]
- 7.Garcia-Closas M, Rothman N, Lubin J. Misclassification in case-control studies of gene-environment interactions: assessment of bias and sample size. Cancer Epidemiol Biomarkers Prev. 1999;8:1043–50. [PubMed] [Google Scholar]
- 8.Vineis P. A self-fulfilling prophecy: are we underestimating the role of the environment in gene-environment interaction research? Int J Epidemiol. 2004;33:945–6. doi: 10.1093/ije/dyh277. [DOI] [PubMed] [Google Scholar]
- 9.Benke G, Sim M, Fritschi L, Aldred G, Forbes A, Kauppinen T. Comparison of occupational exposure using three different methods: hygiene panel, job exposure matrix (JEM), and self reports. Appl Occup Environ Hyg. 2001;16:84–91. doi: 10.1080/104732201456168. [DOI] [PubMed] [Google Scholar]
- 10.Luce D, Gerin M, Berrino F, Pisani P, Leclerc A. Sources of discrepancies between a job exposure matrix and a case by case expert assessment for occupational exposure to formaldehyde and wood-dust. Int J Epidemiol. 1993;22 (Suppl 2):S113–20. doi: 10.1093/ije/22.supplement_2.s113. [DOI] [PubMed] [Google Scholar]
- 11.Nam JM, Rice C, Gail MH. Comparison of asbestos exposure assessments by next-of-kin respondents, by an occupational hygienist, and by a job-exposure matrix from the National Occupational Hazard Survey. Am J Ind Med. 2005;47:443–50. doi: 10.1002/ajim.20168. [DOI] [PubMed] [Google Scholar]
- 12.Rybicki BA, Johnson CC, Peterson EL, Kortsha GX, Gorell JM. Comparability of different methods of retrospective exposure assessment of metals in manufacturing industries. Am J Ind Med. 1997;31:36–43. doi: 10.1002/(sici)1097-0274(199701)31:1<36::aid-ajim6>3.0.co;2-v. [DOI] [PubMed] [Google Scholar]
- 13.Stengel B, Pisani P, Limasset JC, Bouyer J, Berrino F, Hemon D. Retrospective evaluation of occupational exposure to organic solvents: questionnaire and job exposure matrix. Int J Epidemiol. 1993;22 (Suppl 2):S72–82. doi: 10.1093/ije/22.supplement_2.s72. [DOI] [PubMed] [Google Scholar]
- 14.Stucker I, Bouyer J, Mandereau L, Hemon D. Retrospective evaluation of the exposure to polycyclic aromatic hydrocarbons: comparative assessments with a job exposure matrix and by experts in industrial hygiene. Int J Epidemiol. 1993;22 (Suppl 2):S106–12. doi: 10.1093/ije/22.supplement_2.s106. [DOI] [PubMed] [Google Scholar]
- 15.Tielemans E, Heederik D, Burdorf A, et al. Assessment of occupational exposures in a general population: comparison of different methods. Occup Environ Med. 1999;56:145–51. doi: 10.1136/oem.56.3.145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Rajaraman P, Stewart PA, Samet JM, et al. Lead, genetic susceptibility, and risk of adult brain tumors. Cancer Epidemiol Biomarkers Prev. 2006;15:2514–20. doi: 10.1158/1055-9965.EPI-06-0482. [DOI] [PubMed] [Google Scholar]
- 17.Inskip PD, Tarone RE, Hatch EE, et al. Cellular-telephone use and brain tumors. N Engl J Med. 2001;344:79–86. doi: 10.1056/NEJM200101113440201. [DOI] [PubMed] [Google Scholar]
- 18.Rajaraman P, Schwartz BS, Rothman N, et al. Delta-aminolevulinic acid dehydratase polymorphism and risk of brain tumors in adults. Environ Health Perspect. 2005;113:1209–11. doi: 10.1289/ehp.7986. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Stewart PA, Stewart WF, Heineman EF, Dosemeci M, Linet M, Inskip PD. A novel approach to data collection in a case-control study of cancer and occupational exposures. Int J Epidemiol. 1996;25:744–52. doi: 10.1093/ije/25.4.744. [DOI] [PubMed] [Google Scholar]
- 20.Cocco P, Dosemeci M, Heineman EF. Brain cancer and occupational exposure to lead. J Occup Environ Med. 1998;40:937–42. doi: 10.1097/00043764-199811000-00001. [DOI] [PubMed] [Google Scholar]
- 21.Dosemeci M, Stewart PA. Recommendations for reducing the effects of missclassification on relative risk estimates. Occupational Hygiene. 1996;3:169–76. [Google Scholar]
- 22.Akaike H. International Symposium on Information Theory. Budapest, Hungary: Akademiai Kiado; 1973. [Google Scholar]
- 23.Schwartz GE. Estimating the dimension of a model. Annals of Statistics. 1978;6:461–464. [Google Scholar]
- 24.Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33:159–74. [PubMed] [Google Scholar]
- 25.Pirkle JL, Brody DJ, Gunter EW, et al. The decline in blood lead levels in the United States. The National Health and Nutrition Examination Surveys (NHANES) JAMA. 1994;272:284–91. [PubMed] [Google Scholar]
- 26.Stewart PA, Stewart WF. Occupational case-control studies: II. Recommendations for exposure assessment. Am J Ind Med. 1994;26:313–26. doi: 10.1002/ajim.4700260305. [DOI] [PubMed] [Google Scholar]
- 27.Siemiatycki J, Dewar R, Richardson L. Costs and statistical power associated with five methods of collecting occupation exposure information for population-based case-control studies. Am J Epidemiol. 1989;130:1236–46. doi: 10.1093/oxfordjournals.aje.a115452. [DOI] [PubMed] [Google Scholar]
- 28.Hwang SJ, Beaty TH, Liang KY, Coresh J, Khoury MJ. Minimum sample size estimation to detect gene-environment interaction in case-control designs. Am J Epidemiol. 1994;140:1029–37. doi: 10.1093/oxfordjournals.aje.a117193. [DOI] [PubMed] [Google Scholar]
- 29.Rothman N, Garcia-Closas M, Stewart WF, Lubin J. Metabolic polymorphisms and susceptibility to cancer. Lyon; Oxford: International Agency for Research on Cancer; distributed by Oxford University Press; 1999. [Google Scholar]