Performance of self-reported occupational exposure compared to a job exposure matrix approach in asthma and chronic rhinitis

Abstract

Objectives

Self-reported exposure to vapors, gas, dust, or fumes (VGDF) has been widely used as an occupational exposure metric in epidemiologic studies of chronic lung diseases. Our objective was to characterize the performance of VGDF for repeatability, systematic misclassification, and sensitivity and specificity against exposure likelihood by a job exposure matrix (JEM).

Methods

We analyzed data from two interviews, 24 months apart, among adults with asthma and chronic rhinitis. Using distinct job as the unit of analysis, we tested a single response item (exposure to VGDF) against assignment using a job exposure matrix (JEM). We further analyzed VGDF and JEM among a subset of 199 subjects who reported the same job at both interviews, using logistic regression analysis to test factors associated with VGDF inconsistency and discordance with JEM.

Results

For 436 distinct jobs held by 348 subjects studied, VGDF was reported for 193 (44%); moderate to high exposure likelihood by JEM was assigned to 120 (28%). The sensitivity and specificity of VGDF against JEM was 71% and 66%, respectively. Among 199 subjects with the same job at both interviews, 32% had discordant VGDF status (kappa= 0.35). Those with chronic rhinitis without concomitant asthma compared to asthma alone were more likely to have a VGDF report that was discordant with the JEM (OR 3.6 [95% CI 1.4–9.0]; p=0.01). Rhinitis was also associated with reported VGDF in a job classified by JEM as low exposure (OR 3.9 [95% CI 1.6–9.4]; p=0.003).

Conclusion

The VGDF item is moderately sensitive measured against JEM as a benchmark. The measure is a useful assessment method for epidemiological studies of occupational exposure risk.

Occupational factors are important contributors to adult asthma and rhinitis.[1,2] Nonetheless, there is no single, standardized approach used to systematically assess relevant work-related exposures in survey-based research. This presents an important methodological challenge for researchers who wish to analyze the work-related burden of respiratory disease. Various approaches have been used in exposure classification. These include: stratification by broad exposure groups (e.g., blue collar versus white collar job titles);[3] use of a single expert or expert panels;[4–6] use of a job exposure matrix (JEM) based on the likelihood of exposure within certain job/industry categories;[7–10] and self-reported exposure, typically based on structured interview responses.

For self-reported exposure, one of the most common strategies has been to query respondents using a single, dichotomous item ascertaining any work-related exposure to “vapors, gas, dust or fumes” (VGDF). This is often asked in reference to a long working period or even an entire career.[11–15] Because self-reported exposure to VGDF is a very widely used measure, delineating its performance characteristics in survey research applications is critical. Test-retest consistency is one important performance characteristic. Assessing the validity of the VGDF tool is less straightforward, given that direct industrial hygiene measures of workplace exposures are unlikely to be available, especially in studies spanning multiple work sites over many years of employment. It is possible, however, to assess the specific exposures that may be understood by respondents as subsumed under the broad VGDF rubric, for example, through survey administration of both the single broad item followed by a specific exposure checklist. Moreover, assessing the sensitivity, specificity, and sources of misclassification of the VGDF item against a JEM approach based on occupation (sometimes treated as the de facto survey research “gold standard” in assigning exposure risk), provides multiple measures of the criterion validity for VGDF.

We focused on these methodological questions using longitudinal data that allowed the study of both JEM and VGDF measures of exposure. The analysis presented here includes responses from surveys administered to the same cohort of adults with asthma and chronic rhinitis interviewed twice over an approximately 24-month period. Using these data, we were able to compare reported VGDF exposure to a checklist of specific exposures; evaluate the sensitivity, and specificity against exposure likelihood based on JEM assignment; compare test-retest responses to the VGDF item among persons who maintained the same jobs over the study period; and test the association between selected demographic, job, and health condition variables against inconsistency in VGDF self-report and discordance between such self-report and JEM assignment.

METHODS

Survey Recruitment

The data used in this analysis are derived from questionnaire responses to a survey administered during two waves of interviews carried out as part of an ongoing longitudinal study of adult asthma and allergic rhinitis. The survey methodology, including initial subject recruitment, has been reported in detail previously.[16–21] In brief, the study cohort consists of adults with asthma and/or allergic rhinitis, recruited in several phases, between the ages of 18 to 55 at the time of recruitment, living in northern California at the time of recruitment, and having a physician’s diagnosis of either asthma, rhinitis, or both. The first recruitment (1993–1994) was carried out through a random sampling of pulmonary and allergy sub-specialists. This was later supplemented (1996–1997) with a similar recruitment through a sample of family medicine specialist physicians. A third group of participants was added through random digit dialing that identified persons reporting a physician’s diagnosis of asthma and/or rhinitis (1999).

Cohort Follow-up

All groups were merged into a single cohort and were all surveyed together in 2000–2001 and then at approximately 2 year intervals thereafter. For the current analysis, we used data from two study waves: interviews that took place in 2000–2001 and in 2003–2004. In 2000–2001, the baseline interview wave for this analysis, 548 subjects successfully completed the structured telephone survey using computer-assisted telephone interview (CATI) software. Re-interviews approximately 2 years later (mean time elapsed, 22.3±2.7 months) were successfully completed for 416 (76%) of the 548 subjects interviewed 2 years previously.

Employment Status

We assessed employment status based on subjects’ responses to standard labor market items. Of the 548 subjects interviewed initially, 408 (74%) were employed at the time of that survey, but only 287 of these were included among the 416 re-interviewed subjects. In total there were 314 (75%) currently employed at the time of the second survey, which included 27 not employed at baseline. We limited this analysis to the 348 subjects who were employed at the time of at least one of the two interviews.

Other Survey Data

Subject interviews also included items covering demographics, cigarette smoking, socioeconomic status and health conditions. Dichotomous variables were created for race (white, non-Hispanic v. all others), education (high school graduate or less v. some college or more), and cigarette smoking status (never v. former or current smoker). The subjects were divided into groups by health condition, categorized as asthma and concomitant rhinitis, asthma alone without rhinitis, and chronic rhinitis alone without asthma.

Definitions of Exposure

At both interviews, subjects were asked about current job, type of business, occupation, and their usual work activities. These items were asked as open-ended questions with responses noted in text format that could be reviewed later for coding purposes. We used the 2000 Census codes to assign codes for each subject’s occupation and industry. [22] The initial assignment of codes was done by one of the study’s staff and was later reviewed by the project’s industrial hygienist (PJQ), based on open-ended job descriptions and reported duties as stated above. This expert review resulted in changes to the originally assigned job codes when indicated.

In addition to occupation, subjects were also asked a general question regarding exposure to VGDF specifically worded as follows: “At your current job, do you come in regular contact with any dusts, fumes, gases, vapors, chemical or pharmaceutical products?” Only those responding positively to the general exposure question were asked to respond to a checklist of 19 specific exposures. Subjects who denied exposure to VGDF in response to the global item were not questioned about these 19 specific substances, a strategy designed to minimize respondent burden. For those who held the same job at both interviews we used the checklist responses reported at baseline for the analysis. This checklist was developed for this study, with exposures categorized for analysis such that exposure to one or more constitutes exposure to that group. The specific exposures, grouped by type, are: plant dust or pollen; natural rubber or latex; cotton or saw dust; wheat flour or grain dust; hair dander or fur; enzymes (biological materials); paints, dyes or inks; pharmaceuticals; formaldehyde; sealant or shellac; urethane or polyurethane; epoxy or glues; photo chemicals (synthetic organic chemicals); and cleaners or bleach; irritant chemicals; combustion products; metal dusts or fumes; refrigerants; and flux or solder (irritant gases or fumes). This selection was based, in part, on a checklist approach used to elicit exposures in a study of chronic obstructive pulmonary disease (COPD).[13,23,25] Seven of those items were adapted for this checklist, contributing five separate items of the 19; the remaining 12 were items new to this study. The original checklist had featured dusts and fumes with known or suspected COPD risk, whereas the checklist used in the current study was comprised of substances known to cause or exacerbate asthma or rhinitis, based on the general literature in this field and not targeted for any specific occupations or industries as reported by the cohort at hand.

A total of 150 different 2000 Census occupational codes were assigned to 436 distinct jobs held by the 348 subjects in the analysis. Census code numerical grouping differentiated among major categories of occupations, allowing dichotomization to professional, managerial, administrative and sales occupations (by the most frequent as a group) as opposed to technicians, service, manufacturing, construction and agriculture. We constructed the JEM by assigning each of the 150 jobs codes present in the data set a likelihood ranking of exposure, modifying the initial assignment if indicated after review of the any narrative job details altering the default assignment. The JEM was adapted from a job-exposure classification system originally used in the European Community Respiratory Survey and then further adapted for use in studies of airway diseases.[12] All of the subjects’ jobs were coded as having low, moderate or high likelihood of exposure to sensitizing or irritant agents known to cause or exacerbate asthma.

The initial assignment of exposure likelihood based on job code was then followed by review of the narrative (open-ended) job description elicited in the interviews consistent with the methodology of Kennedy.[9] Using this methodology, in cases where the job descriptions did not match the initially assigned classification based on job code alone, the exposure likelihood was modified. For example, the occupational code “22” (supervisor) would typically be classified as low exposure likelihood; but if an individual indicated specific exposures in their job description narrative the exposure JEM was re-assigned as “moderate” (e.g., working as a supervisor onsite in construction areas; a construction trade worker would automatically have been assigned a JEM ranking of moderate exposure likelihood). Another example is the code “290” (audio-video engineer), who would also be classified as low exposure likelihood barring additional information; however, in a case in which the narrative described the direct use of chemicals in repairing equipment, the exposure likelihood was also raised to moderate. Such narrative reviews resulted in changes to 61 (14%) of the 431 JEM assignments in this analysis. One key aspect of our JEM approach is that the assignment was not modified by subjects’ responses to the global VGDF item or to any of the specific exposure in the 19 item checklist. For the current analysis, we took a dichotomous approach, combining the “moderate” and “high” exposure likelihood assignments into one category and compared that with the jobs where the assignment was rated as “low” likelihood.

Data Analysis

We employed two principal strategies in the data analysis. In the first approach, the unit of analysis was the reported job; in the second, the unit of analysis was the study subject. In the first strategy, each subject could contribute up to two observations, one each for the job reported at baseline and at the follow-up interview, if the jobs differed between interviews. There were 436 such distinct jobs among the 348 subjects included in this analysis. Thus, exposure frequencies with a total n=436 could be calculated for JEM status, each exposure to VGDF, and, among the VGDF exposures, the prevalence of each of the 19 specific materials and their parent groups (biological materials, synthetic organic chemicals, and irritant gases and fumes). In the second analytic strategy, as noted above, the interview respondent was the unit of analysis. Although we ascertained job status and demographics for all 348 subjects included in the study, the subject-based analysis focused on the 199 subjects interviewed at both baseline and follow-up who reported no job change or substantive change in duties (in narrative text) over the interval.

The baseline demographic and health characteristics of the 348 subjects with or without VGDF exposure at the time of the baseline of the two interviews were compared. If subjects were not working at the time of the first interview, they were considered part of the non-VGDF group. The Chi-squared test was used to test for differences between these categorical variables. For all 436 jobs, simple Kappa coefficients and their 95% confidence intervals (CIs) were calculated for concordance between self-reported VGDF and JEM classification. We also calculated the sensitivity and specificity of VGDF self-report against JEM assignment, treating the later as the “gold standard.” We repeated the Kappa and sensitivity/specificity calculations testing each of three groups of specific exposures (biological materials, synthetic organic chemicals, and irritant gases and fumes) against the JEM assignment. The simple Kappa coefficient and 95% CIs for test-retest concordance between baseline and second interview were calculated for the 199 subjects with the same job at both interviews. We used multiple logistic regression analysis to estimate the associations between demographics, health condition and occupational grouping as independent variables and three dependent variables: VGDF discordance in self-report comparing the two interview waves; VGDF reported, but JEM assignment categorized the job as being of “low” exposure likelihood (over-report of VGDF); and VGDF not reported, but JEM assignment categorized the job as being of “moderate” or “high” exposure likelihood (under-report of VGDF). We limited the latter three analyses to subjects interviewed at both waves who were employed in the same job at both time points (n=199). For the analysis of VGDF over-report we excluded subjects who were classified as under-reporting exposure (n=17); for under-report, we excluded those classified as over-reporting (n=49). Consequently, the referent outcome for those models was comprised of subjects both reporting VGDF and assigned a moderate-high exposure likelihood JEM. Analyses used SAS 9.1 (SAS Institute, Cary, SC).

RESULTS

The 348 study subjects included 226 (65%) with asthma and concomitant rhinitis, 56 (16%) with asthma alone, and 66 (19%) with rhinitis alone. Study subjects were mainly female (68%), white, non-Hispanic (75%), and well educated (89% some college or more), with a mean age of 43.9±8.6 years. There were 188 (34%) who were former or current cigarette smokers. Comparing the participants with reported baseline VGDF exposure (n=136) with those reporting no exposure to VGDF (n=212), the non-VGDF group was more likely report a higher education level (p=0.04) and less likely to report rhinitis alone (p=0.04). Although the non-VDGF group was also less likely to be male, this difference is not statistically significant (p=0.06). Age, race-ethnicity, and smoking status were similar by VGDF status.

Exposure by JEM and VGDF

When analyzed at the level of jobs held (n=436), as shown in Table 1, four in five held a professional, managerial, administrative or sales occupation. Consistent with this, most (72%) were assigned a low likelihood of exposure JEM. Of 426 jobs, 193 (44%) were reported by the respondents to involve regular exposure to VGDF. For each of these jobs, a 19-item checklist of specific exposures was elicited (see Methods).

Table 1.

Occupational Group and Exposure Characteristics for 436 Distinct Jobs Reported by 348 Subjects Over Two Survey Interviews Two Years Apart.

Characteristic	Frequency
	N	%
Occupation Group
Professional, managerial, administrative, sales	350	80%
Technician, service, trades, agriculture	86	20%
Exposure likelihood by JEM
Low likelihood of exposure	316	72%
Moderate likelihood of exposure	104	24%
High likelihood of exposure	16	4%
Self-reported exposure to VGDF	193	44%
No self-reported specific exposure to any of 19 specific     materials*	10	5%
Self-reported specific exposure to any of 19 specific materials*	183	95%
Exposure to Irritant Gases or Fumes (6 items)*	122	67%
Exposure to Biological Materials (6 items)*	162	87%
Exposure to Synthetic Organic Chemicals (7 items)*	99	54%

If the same job is reported at both interviews, self-reported exposure is based on the first interview survey response.

VGDF = vapors, gas, dust, or fumes; JEM = job-exposure matrix

^*Percent of n from preceding header.

Irritant Gases or Fumes: cleaners, bleach; irritant chemicals; smoke or combustion byproducts; metal fumes or dust; refrigerants; flux or solder

Biological Materials: plant dust, pollen; natural latex; cotton dust; wood dust; grain dusts or flours; animal hairs or dander; enzymes

Synthetic Organic Chemicals: paints, dyes, or inks; pharmaceuticals; formaldehyde; sealant or shellac; urethanes; epoxies or other glues; photographic chemicals

For the checklist responses, the most common group of exposures were from the biological materials; the least common, but still reported for over half of the occupations with VGDF exposure, was synthetic organics (Table 1). There were 10 jobs (5%) for which, despite the self-report of VGDF, none of the19 checklist items elicited a positive response.

Concordances among Exposure Measures

The performance of self-reported exposure to VGDF (the single questionnaire item) against JEM assignment is shown in Table 2. Self-report of VGDF had a sensitivity = 71% and a specificity = 66%, when compared with moderate to high exposure likelihood by JEM assignment. Based on the specific item checklist, the subset of biological materials (sensitivity 64%; specificity 73%) and irritant gases (sensitivity 56%, specificity 83%) performed similarly. Reported exposure to at least one of seven synthetic organics, however, manifested the poorest sensitivity (44%), with a corresponding increase in specificity (85%).

Table 2.

Agreement, Sensitivity, and Specificity of Self-Reported Exposure to Vapors, Gas, Dust or Fumes against Job Exposure Matrix Classifications for 436 Distinct Jobs

Exposure Status		Job Exposure Matrix (JEM) Classification
	Moderate – High Exposure N=120	Low Exposure Likelihood N=316	Kappa (95% CI)
VGDF	85 (71%)	108 (34%)	0.31 (0.22–0.39)
Exposure Not Reported	35 (29%)	208 (66%)
Irritant Gases or Fumes*	67 (56%)	55 (17%)	0.38 (0.29–0.48)
Exposure Not Reported	53 (44%)	261 (83%)
Biological Materials*	77 (64%)	85 (27%)	0.34 (0.25–0.43)
Exposure NotReported	43 (36%)	231 (73%)
Synthetic Organics*	53 (44%)	46 (15%)	0.31 (0.21–0.41)
Exposure Not Reported	67 (56%)	270 (85%)

VGDF = vapors, gas, dust, or fumes; JEM = job-exposure matrix

Values in Bold = Sensitivity; Values in Bold Italics=Specificity

^*Based on multi-item check list (see Figure 1)

Although there were 287 subjects who were employed at the time of both interviews, there were only 199 who held the same job and job duties at baseline and follow-up. There were 29 subjects who reported VGDF at baseline and not at follow-up, while 35 reported such exposure at follow-up but not baseline. Altogether, this represents 64 (32%) discordant responses among 199 repeat interviews and a corresponding kappa statistic of 0.35 (95% CI 0.22–0.48). When we re-estimated reporting of VGDF across interview waves stratified by condition, the kappa was higher and similar for those with asthma and rhinitis (0.49; 95% CI 0.26 –0.56) and for those with asthma alone (0.43; 95% CI 0.11 to 0.74), but was poorer for those with rhinitis alone (0.12; 95% CI -0.14 to 0.38).

Among the same 199 subjects without job change, we studied predictors of test-retest discordance as well as predictors of VGDF-JEM mismatch (the latter based on VGDF report in the first interview, see Methods). In the multivariate analysis (Table 3), the factors associated with discordance in self-report between waves were increasing age, OR 2.2 (95% CI 1.4–3.6; p=0.01); female sex, OR 2.5 (95% CI 1.2–5.5; p=0.02); and disease status of rhinitis only, OR 3.6 (95% CI 1.4–9.0; P=0.01). The only diagnostic group that was associated with VGDF-JEM mismatch (over-reporting exposure to VGDF) was the rhinitis only group with an OR of 3.9 (95% CI 1.6–9.4; P=0.003). Underreporting of exposure was associated with selected trades (technician, service, manufacturing, construction, agriculture), with an OR of 4.0 (95% CI 1.2–13.5; P=0.02).

Table 3.

Factors Associated with Self-reported VGDF Discordance, Over-Reporting and Under-Reporting in 199 Subjects with the Same Job in Both Waves

	Discordant VGDF		Over-Report VGDF *		Under-Report VDGF **
	n = 199		N = 49/182		n = 17/150
Factors	OR	95% CI	OR	95% CI	OR	95% CI
Age, per 10 years	2.2	1.4, 3.6	1.1	0.7, 1.8	0.8	0.4, 1.5
Female	2.5	1.2, 5.5	0.9	0.4, 1.9	2.4	0.7, 8.8
White, non-Hispanic	0.6	0.3, 1.4	0.9	0.4, 2.0	1.0	0.3, 3.2
High School Graduate or less	0.6	0.2, 2.0	0.6	0.1, 2.4	0.3	0.0, 2.6
Never Smoked v. Ever Smoked	0.8	0.4, 1.5	0.7	0.3, 1.5	2.2	0.6, 8.8
Selected trades† v. all others	1.1	0.4, 2.6	0.3	0.1, 1.1	4.0	1.2, 13.5
Disease Status
Asthma with Rhinitis (referent)	1.0	--	1.0	--	1.0	--
Asthma Only	0.8	0.3, 2.0	0.7	0.2, 2.1	1.2	0.3, 4.7
Rhinitis Only	3.6	1.4, 9.0	3.9	1.6, 9.4	0.6	0.1, 5.3

^*Over-report = Self-reported VGDF, JEM classification low. Excludes 17 subjects who under-reported VGDF.

^**Under-report = No self-reported VGDF, JEM classification is moderate/high. Excludes 49 subjects who over reported VGDF exposure.

^†Selected trades = Technician, service, manufacturing, construction, agriculture.

Discussion

This analysis focused on central methodological questions relating to the performance of self-reported VGDF as a measure of occupational exposure, particularly relative to a JEM approach. As with any other survey approach, the performance characteristics of the VGDF method indicate that it does have limitations that should be borne in mind. It is moderately sensitive against JEM assignment when the latter uses an expert review approach as recommended by Kennedy.[9] Its specificity, based on that standard, is less and the kappa was low. Moreover, inconsistency (test retest discordance) and inaccuracy (presuming JEM classification to be correct) differed systematically by diagnostic group. Those with chronic rhinitis alone were more likely to report VGDF status differently despite a lack of job change and to report VGDF (based on their baseline interview) in a job for which the JEM assignment was low exposure likelihood, consistent with a confounding effect. Future studies should take this into account with appropriate methods, including stratified analyses where possible.

Self-reported exposure, whether based on a single item, group of items, or a more detailed questionnaire, has limitations. Potential misclassification, due to either under- or over-reporting exposure, presents a serious challenge. Some subjects may be unaware of the substances they have worked with or, conversely, if they have a disease that they associate with workplace factors, may over-report exposures. In our study, this morbidity-driven bias may have been a relatively greater factor among subjects with chronic rhinitis alone, leading them to over-report exposure even though rhinitis is typically a less severe condition than asthma. Differential self-reported exposure misclassification linked to health status can lead to over- or underestimation of effects, as opposed to non-differential misclassification which would tend to bias toward the null.[24] Although we do not have an obvious explanation for the differential rhinitis effect, we have previously reported that those with rhinitis alone, compared to asthma, have decreased work effectiveness on the job, even though persons with asthma have lower work force participation. [18]

Another limitation of our study, as well as most community-based studies, is the lack of independent quantification of exposures. This is a typical problem, given that industrial hygiene measurements or biological monitoring data are rarely available. Other potential limitations of this particular study should be acknowledged as well. Because all the subjects had airway disease, these findings may not be applicable to survey methods of exposure assessment in other conditions, such as cancer, or among those with no chronic health problems. Studying those currently working may have introduced selection effects if former workers would have systematically differed in performance measures. Our data collection method (telephone interviews) may have introduced measurement error not present with other methods, for example, face to face interviews with flash cards. Another source of error may have been subjects who considered VGDF as including relatively minor exposures such as perfumes or the equivalent of house dust on surfaces. This may account for the low percentage reporting VGDF but denying exposure to any of the items on the checklist and is consistent with what we observed in a comparable survey in a different cohort.[25] Finally, treating as “discordant” a change in VGDF report over two interviews, when there was no change in occupation or industry or a narrative indication of change, could have overestimated inconsistency, if working conditions did indeed vary.

In this analysis, we used a JEM modified by expert review as the “gold standard” against which to measure the performance of a VGDF item. We did not compare this to performance of the JEM prior to such review, although 14% of the assignments did change as a result of this. Furthermore, we did not study a hybrid JEM-self report methodology, for example, using selected data from the 19 item checklist to modify the JEM classifications as a supplement to the expert review step. The data shown in Table 2 suggest that, for subgroups of checklist items (in particular, irritant gases and fumes), specificity and overall agreement with the JEM are higher than for the single VGDF item. Of interest, a recent case-control study of occupational risk for COPD employed just such a hybrid method of expert review-modified JEM based not on open-ended text as used in our study, but rather on a multi-tem check-list. [26]

The JEM approach has been used extensively since the 1980s as an objective method to assess occupational exposures. The use of JEMs based on both European and U.S. occupational coding systems (often called generic JEMs) has been central to a number of case-control studies, primarily in analyses of occupationally-related cancer. In an extensive review of occupational exposure assessment for case control studies, the sensitivity of a generic JEM was low when compared to self-report or expert assessment. [27]

A major problem with the generic JEM approach is that it fails to take into consideration the variability of exposures within the same job classification. Expert review can theoretically reduce this problem by modifying the JEM assignment based on additional data on actual job duties and/or exposures reported by the individual subjects.[9,28] It is also important to acknowledge that the JEM approach is probabilistic, thus a “low” likelihood of exposure is not the same a “no” exposure.

Expert assessment of exposure and generic and modified JEM assignment have been used as the de facto “gold standard” to assess self-reported VGDF, as shown in Table 4. In this tabular presentation, there is range of sensitivity and specificity for self-reported exposure tested against expert interview or JEM or for agreement between self-reported exposure and JEM assessed with the kappa statistic. In analyses other than the current study, among persons with airway disease, sensitivity estimates have ranged from 48% to 65% and specificity from 80%–83%; among general samples or among those without lung disease, the sensitivity has ranged from 42% to 64% and the specificity from 74% to 91%. Kappa values have ranged between 0.30 and 0.71. The values from the current study fall within the range of these other reports, with the exception of the very low kappa (0.12) for self-report against JEM assignment among those with rhinitis alone.

Table 4.

Comparative performance of self-reported exposure to vapors gas dust and fume (VGDF) and job exposure matrix (JEM) in multiple studies

Author (ref)	Study Population	Comparison	Findings
Bakke et al (11)	Cohort study, subjects with and without asthma.	VGDF (gas and dust only) vs. expert interview (no JEM used)	Asthma vs. all others: sensitivity 65% vs. 64%; specificity 80 vs. 91%.
de Vocht et al (14)	Cohort study, subjects with and without asthma.	VGDF (single item) vs. JEM (expert review)	Asthma vs. all others: sensitivity 48% vs. 42%; specificity 83% vs. 87%; kappa 0.31 vs. 0.32
Le Moual et al (15)	Several cohorts, general population samples.	VGDF (single item or modified checklist) vs. JEM (based proportion of jobs with self-reported VGDF; no expert review step)	Agreement between VGDF and JEM: kappa range 0.30 to 0.58 by cohort for single item; kappa 0.71 for modified checklist (one cohort)
Blanc et al (25)	Cohort study, subjects with and without respiratory disorders.	VGDF (single item) vs. JEM (no expert review step)	Sensitivity 64%, specificity 74% overall; COPD or asthma vs. all others: kappa 0.40 vs. 0.36.
Current study	Cohort study, all subjects with asthma or rhinitis.	VGDF (single item) vs. JEM (expert review)	Sensitivity 71%, specificity 66% overall; asthma and rhinitis v. rhinitis alone: kappa 0.49 vs. 0.12.

In multivariate analysis, rhinitis was indeed a risk factor for over-report of exposure against the JEM, which would contribute to a poor kappa. Indeed it was only factor in that analysis that did manifest a statistically significant association with over-reporting. Moreover, in multivariate analysis rhinitis was also associated with increased odds of a discordant self-reported exposure on repeated survey, as were increased age and female gender. The test-retest performance characteristics of self-reported occupational exposure has not been well studied, although self-assessed generic “dustiness” has been reported to have better test-retest performance that querying a more specific exposure (asbestos). [29]

Main messages

• Self-reported occupational exposure to vapors, gas, dust and fumes, ascertained as a single survey item, demonstrates performance characteristics that make it applicable to epidemiological research, bearing in mind that reporting may vary in relation to health status.

• When possible, using multiple occupational exposure assessment measures is a preferable analytic strategy to relying on a single metric alone.

Policy implications

• Epidemiological approaches to exposure assessment in occupational studies can provide data applicable to analyzing health outcomes when direct worksite industrial hygiene assessments are not available.

• Population-based analyses across multiple occupations and industries are feasible and can provide valuable public health information.

Our study is uniquely placed to address the methodological uncertainties of survey-based exposure assessment in airway disease in that it used systematic survey data including multiple exposure assessment measures and also afforded longitudinal re-assessment. In summary, based on this analysis, we found that the VGDF single-item approach does have limitations but that, overall, it performed in a manner indicating that it can be a useful exposure assessment method. Attention should always be given to potential biases; our analysis suggests that rhinitis is a health condition that could lead to differential misclassification. No single method is likely to meet all needs. Whenever multiple approaches to exposure assessment can be applied within the same analysis, this is likely to provide greater confidence in interpreting its results.

Figure 1. Frequency of reports for 19 specific exposures, grouped by type, among 199 subjects with the same job at both interviews.

Figure 1 provides a graphic representation of the frequencies for self-reported exposures in response to the 19-exposure checklist grouped in the three exposure categories.