Agreement between an expert-rated mini job exposure matrix of occupational biomechanical exposures to the lower body and technical measurements or observation: a method comparison study

Abstract

Objectives

Investigating the agreement between an expert-rated mini job exposure matrix (JEM) of lower body exposures and technical measurements of worktime spent standing/walking and observation-based estimates of time spent kneeling/squatting and total load lifted per workday.

Methods

We chose 16 job titles from the 121 job groups in the lower body JEM and included them in the mini JEM. New expert ratings for the mini JEM were performed by the same five occupational physicians who performed the ratings for the lower body JEM. For each job title and type of exposure, the exposure estimates were a mean of the five independent ratings. Technical measurements of standing/walking for all 16 job titles, and for 8 job titles workplace observations were performed of kneeling/squatting and total load lifted per workday. Data were collected from September to December 2015 and supplemented by data from the NOMAD and DPhacto studies collected between 2011 and 2013. All data were collected in Denmark. Agreement between expert-based and measured/observed lower body exposures by job titles was evaluated using Spearman’s rank correlation, Bland-Altman plots evaluated systematic deviations and limits of agreement (LoA).

Results

Standing/walking showed a rank correlation of 0.55, kneeling/squatting 0.83 and total load lifted per workday 0.71. The mini JEM estimates did not systematically deviate from the technical measurements/observations for time spent standing/walking (mean difference 0.20 hours/workday, LoA −1.63, 2.03 hours/workday) and kneeling/squatting (mean difference −0.35 hours/workday, LoA −1.21, 0.51 hours/workday). For total load lifted per workday, the mini JEM systematically overestimated the exposures compared with the observations (mean difference −909 kg/workday, LoA −3000, 1147 kg/workday).

Conclusions

There was moderate to very high agreement between an expert-rated mini JEM of standing/walking, kneeling/squatting, and lifting exposures and corresponding technical measurements/observations. This method comparison study supports the use of the expert-based lower body JEM in large-scale occupational epidemiological studies.

Strengths and limitations of this study

• The same five occupational physicians who performed the ratings for job groups in the lower body job exposure matrix (JEM) performed new ratings to create a job title specific mini JEM.

• The expert ratings for the mini JEM were supported by vignettes.

• The included job titles represented a wide range of exposures.

• The participants were included based on convenient accessibility rather than systematic sampling.

• The number of included job titles was limited.

Introduction

Job exposure matrices (JEMs), which assign occupational exposure estimates to individuals according to their job title, have proved useful in large-scale epidemiological studies, while individual-based methods for assessment of occupational exposures (technical measurements, observation, self-report, and case-by-case expert assessment) may be less feasible due to the high cost (purchase of monitors and work hours) and risk of selection bias in participation. JEMs are cost efficient and can provide estimates of both present and past exposures independent of symptom/disease status.^{1 2} Moreover, exposure assessment using JEMs is group-based and therefore less subjected to bias towards the null due to random error compared with individual-based exposure assessment.^{3 4}

Within the past two decades, several general population JEMs have been developed to assess occupational exposures to the lower body, either based on self-reports,^{2 5–11} expert ratings,^{4 12 13} a hybrid of these methods,^{2 14 15} or expert ratings based on observations.¹⁶ Some of these JEMs have shown cumulative lifting estimates to strongly correlate with self-reported physical work demands throughout working life.¹⁷ Additionally, various estimates of occupational biomechanical exposures based on JEMs have shown good predictive validity in cohort studies of inguinal hernia repair,^18–20 total hip replacement,⁴ surgery for varicose veins,²¹ sickness absence and permanent work disability.²²

A nationwide Danish Occupational Cohort with eXposure data (DOC*X) has recently been established as an open resource to facilitate register studies of disease and disability associated with occupational exposures.²³ The lower body JEM, which was originally constructed to study hip and knee osteoarthritis,⁴ is now available as one of a series of general population JEMs in the DOC*X. However, few studies have investigated the validity of JEM-based estimates of exposures to the lower body, where some have compared JEM-based exposures between countries^{2 24} or against self-reported exposures.6 10 11 25 Yet, we are not aware of studies, which investigated the agreement between lower body exposure estimates based on JEMs and technical measurements/observations.

Technical measurements of biomechanical exposures and physical activity have been shown to be more valid than self-report, for example, by use of questionnaires,²⁶ due to recall bias²⁷ or differential misclassification.^{28 29} In addition, technical measurements allow detailed investigations of temporal patterns and transitions between postures/activities.³⁰ Technical measurements of occupational lifting are not yet feasible to the same extent as measurements of, for example, postures and movements. The use of pressure measurement insoles seems a promising method,³¹ but for the time being, we think that direct observation is the optimum choice.

Correct ranking of JEM-based exposure estimates is important for research on exposure–response relationships. If the ranking of the exposure estimates is correct, inaccuracies may be rectified by calibration against measurements/observations.³² However, an investigation of the agreement between expert ratings of the job groups in the lower body JEM and technical measurements/observations was out of reach for us due to resource constraints. Instead, we chose a number of job titles, where it would be feasible to perform measurements/observations. As the lower body JEM contains exposure estimates for job groups, and not job titles, we created a job title specific mini JEM based on new ratings made by the same five specialists in occupational medicine, who made the ratings for the lower body JEM (see below). The estimates in the mini JEM could then be compared with measurements/observations. The participants in the present study were included based on convenient accessibility (see below) and therefore could not be expected to be representative for all Danish employees with the same job title. For this reason, we added evocative descriptions (vignettes) to indicate to the experts what kind of job tasks the participants performed during the measurements/observations.

The aim of this study was to investigate the agreement between exposures to the lower body according to a job title specific mini JEM and technical measurements of time spent standing/walking and observation-based estimates of time spent kneeling/squatting and total load lifted per workday.

Methods

Design

Optimally, participation of≥10 representative workers from each of the 122 job groups in the lower body JEM would be required to investigate the agreement between the JEM-based exposure estimates and measurements/observations.^{33 34} To construct the mini JEM, we selected 16 job titles, which represented a wide range of exposures and both sexes. The expert ratings were performed specifically for the job titles selected for the mini-JEM. The five experts were specialists in occupational medicine with profound knowledge about work place exposures and substantial experience in exposure assessment based on clinical interviews with patients. The experts, who provided the original exposure estimates for the lower body JEM,⁴ independently assessed time spent standing/walking, time spent kneeling/squatting, and total load lifted per workday for all 16 job titles. Based on this information, we constructed an expert-rated mini JEM for the purpose of the present study. For eight of the job titles, we performed new technical measurements and observations, and for two of these titles, we added previous technical measurement data to the data collected for this study. For the remaining eight job titles, we only had access to previous technical measurement data on time spent standing/walking during work. All previous data were obtained from the NOMAD³⁵ and DPhacto studies,^{36 37} where the data collection took place between 2011 and 2013. New and previous technical measurements were performed with identical methods, and all data were collected in Denmark. For descriptive purposes, we added codes according to the Danish version of the International Standard Classification of Occupations from 1988 (DISCO-88). These codes were not provided to the experts. Online supplemental table S1 shows the expert-rated exposure estimates from the lower body JEM for the job groups that contain the job titles selected for the mini JEM.

Construction of the expert-rated mini JEM

We chose to evaluate the experts’ ability to rate exposures to the lower body for 16 job titles covered by job groups in the lower body JEM. To do so, we developed a mini JEM for this study. The lower body JEM concerns biomechanical work exposures to the lower body and provides quantitative exposure estimates in terms of time spent standing/walking (hours/day), time spent kneeling/squatting (hours/day), and total load lifted (kg/day). To construct the lower body JEM, 5 experts individually rated a number of occupational mechanical exposures for 122 job groups.⁴ When establishing the lower body JEM,⁴ the experts were asked to rate the typical exposures of each job group based on their clinical experience. For construction of the mini JEM, the same experts were asked to re-rate the exposures for each of the 16 job titles based on vignettes,³⁸ without looking up any exposures in the original lower body JEM. The vignettes were made by the project group that performed the measurements/observations. The project group included being a mixture of occupational physicians, and medical laboratory technologist, physiotherapists and researchers experienced in exposure to occupational physical activity.

The ratings for the job titles in the mini JEM differed from the ratings for the job groups in the lower body JEM as described in the introduction. Therefore, the exposure estimates from the lower body JEM could not be used for the present comparison of methods. Apart from the use of vignettes, the rating process took place as described previously.⁴ In brief, the experts independently rated the number of hours/day spent standing/walking (in half-hour intervals), the number of hours/day spent kneeling/squatting (in half-hour intervals), and the total load lifted during work (kg/day). They rated the exposures blinded for the results from the measurements/observations. To ensure that the time estimates added up to a full workday of 8 hours, the experts also assessed the number of hours/day spent sitting.

Enrolment of participants

The intention was to recruit ≥10 participants for each job title. For logistic reasons, we limited our recruitment to the Copenhagen area. We contacted relevant companies by telephone, and if a company was willing to participate, employees with the selected job titles were informed of the aim of the study and asked for written consent to participate. The Danish Data Protection Agency accepted the data handling and storage (journal number 2015-54-0995) and The Danish National Ethics Committee approved the study (journal number H-2-2012-011).

Patient and public involvement

The companies and participants were not involved in the development of the research questions, design of the study, choice of outcome measures, or recruitment to the study, due to the methodological aim of the study.

Data collection

The data collection took place from September to December 2015. For each participant, observations and technical measurements were carried out simultaneously during one to four whole workdays in a row. Work periods were considered valid if they comprised at least 4 continuous hours (in case of split duties, 75% of the participants’ working time).³⁹ All exposure data were extrapolated to an 8-hour workday.

The technical measurements were performed with five GT3X+accelerometers (3-Axis Logging Accelerometer; ActiGraph, Florida, USA) as described previously.³⁹ The accelerometers were mounted on the right thigh (frontal, midway between the iliac crest and patella), the thorax (either at the back (medial, at the T1/T2 level) or at the front (manubrium of sternum)), and both calves (posterior, just below the insertion of the gastrocnemius muscle). Registrations were made with a sampling frequency of 30 Hz. The Acti4 software (The National Research Centre for the Working Environment, Copenhagen, Denmark and Federal Institute for Occupational Safety and Health, Berlin Germany), validated for estimation of time spent in various body postures and physical activities,^{34 35 40} was used to determine time spent standing/walking.

Trained physiologists and physiotherapists, using a modified Track Recording and Analysis on Computer (TRAC)/PEO approach,^{37 41} performed the workplace observations. In short, kneeling/squatting and lifting were continuously observed and recorded using a handheld computer (Samsung model GT-P3100 or SM-T280, Samsung.com) with the Pocket Observer software (Pocket Observer V.3.1, Noldus.com). Kneeling was defined as ‘at least one knee in contact with the ground’ and squatting was defined as ‘having the knee bent >90° without seated support’.³⁹ Start and stop times of kneeling/squatting were recorded and the durations were summed up for each participant.

We recorded all manual lifting, but only included lifting not alleviated by tools or collegial assistance. The first time a new object was lifted, the observer asked the participant about the weight of the object; the following order of priority was applied: (1) a written weight on the object, (2) a listed weight of the object, (3) weighing the object and (4) participant-reported weight of the object. Each lifting event was classified according to the weight lifted; 5–9.9, 10–14.9, 15–19.9, 20–29.9 and ≥30 kg. For each participant, the total load lifted (kg/day) was calculated as ∑mean kg in each weight category * the number of lifting events in each category.

Statistical analyses

To investigate the agreement between the mini JEM and the measures/observations by job title, we visually assessed the degree of symmetry between the expert-rated and measured/observed job exposures³³ and performed corresponding Spearman’s rank correlation analyses; rank one was given to the lowest exposure value. The Spearman’s rank correlations were interpreted as follows: 0.00–0.29 poor, 0.30–0.39 moderate, 0.40–0.69 strong and ≥0.70 very strong agreement.⁴² Finally, we assessed systematic deviations and limits of agreement (LoA) between the job exposure estimates based on expert ratings and measurements/observations using Bland-Altman plots with 95% LoA³⁸ at job title level. We used SPSS for all statistical analysis (IBM SPSS Statistics; V.24).

Results

This study included data for 16 job titles, presented in table 1. Due to recruitment difficulties, we did not succeed in enrolling a minimum of 10 participants for each of the job titles that had not been measured or observed earlier. Measurements/observations were performed for 766 participants, between 5 and 375 participants for each job title. The characteristics of the participants and the total number of measurements/observations included for each exposure are shown in table 2. The mean age was 45.9 years (SD 10.0). The average total duration of measurements/observations per participant and job title was 21.3 hours for standing/walking, 9.8 hours for kneeling/squatting, and 6.9 hours for total load lifted per workday.

Table 1.

Job titles and DISCO-88 codes (DISCO-88=Danish version of the International Standard Classification of Occupations from 1988), sex distribution of the participants, descriptive texts (vignettes), and number of participants with technical measurements/observations, N=766

Job title, (DISCO-88 code), and sex distribution	Vignette	Technical measurements of standing/walking, during workhours* n	Observations of kneeling/squatting, during workhours n	Observations of total load lifted per workday n
Carpenter (7124), women: n=0, men: n=8	Workers engaged in renovations/restoration of a large apartment complex, replacing windows, doors, roofs, balconies, and so on.	8	8	8
Cleaning assistant (9132), women n=100, men n=13	Workers performing cleaning tasks at hospitals, airports and schools	113	0	0
Construction labourer (8332), women: n=0, men: n=10	Workers in a large construction company engaged in digging trenches, installing/maintaining electrical wiring and cables, and backfilling holes	10	0	0
Cook, kitchen assistant, matron (5122), women: n=10, men: n=0	Workers in staff canteens and nursing home kitchens preparing hot and cold dishes	10	10	10
Docker (9330), women: n=0, men: n=15	Harbour workers engaged in docking ships and ship maintenance	15	0	0
Garbage collector (9161), women: n=1, men: n=12	Workers performing garbage collection in a large garbage company	13	0	0
House painter (7141), women: n=2, men: n=3	Workers performing interior painting	5	5	5
Machinist (7223), women: n=0, men: n=6	Workers manufacturing pumps for offshore industry	5	5	6
Nursing assistant (5132), institution-/home-based care worker, domestic helper/cleaner (5133), women: n=360, men: n=15	Workers in nursing homes (day and evening shifts)	375	0	0
Office worker (4190), women: n=53, men: n=73	Workers in manufacturing companies performing administrative work	126	0	0
Packing assistant, hand packer (9320), women: n=38, men: n=0	Workers engaged in surveillance and manual packing in a wholesale food company and in a pharmaceutical company	38	8	8
Paviour (7122), women: n=0, men: n=8	Workers in a large construction company engaged in paving	8	0	0
Plumber (7136), women: n=0, men: n=7	Workers engaged in replacement of the heating system in a large apartment block	6	6	7
Shop assistant, shop sales person (5220), women: n=1, men: n=5	Workers in three large discount grocery stores, mainly stocking shelves, but also sitting at the cashier	5	5	6
Smith (7221), women: n=0, men: n=8	Workers engaged in repair and maintenance of machines and equipment in a wholesale food company	8	0	0
Storage worker, warehouse assistant (9330), women: n=9, men: n=9	Warehouse worker in a wholesale food company engaged in manual packing of pallets	18	11	11

*The number of participants with technical measurements exceeds the number with observations of kneeling/squatting and lifting. This is because data on technical measurements from earlier studies were included.

Table 2.

Characteristics of the participants, N=766

Characteristic	Mean	SD	%	N
Age (years)	45.9	10.0		766
Sex (% women)			74.9	766
Seniority in current job (years)	15.3	10.8		766
Total duration per participant of measurement of standing/walking during work (hours)	21.3	9.7		763
Total duration per participant of observation of kneeling/squatting during work (hours)	9.8	3.9		58
Total duration per participant of observation of total load lifted during a workday (hours)	6.9	1.1		61
Mean duration per participant per workday of measurement of standing/walking during work (hours)	2.9	1.3		763
Mean duration per participant per workday of observation of kneeling/squatting during work (hours)	1.3	0.5		58
Mean duration per participant per workday of observation of total load lifted during a workday (hours)	0.9	0.2		61

Figure 1 shows symmetry plots of the exposure estimates from the mini JEM and the corresponding estimates based on measurements/observations according to job title. The plot for standing/walking pictures quite symmetric exposure estimates across all job titles, which means that the estimates from the mini JEM often agreed well with the measurements/observations with regard to the ranking of the exposures. For shop assistants, the technical measurements estimated approximately 2.5 more hours of standing/walking per workday than the mini JEM. For kneeling/squatting, the mini JEM in general estimated at least 100% higher exposures than observed, for example, for carpenters, the duration of kneeling/squatting was 2 hours/day according to the mini JEM, while only 1 hour/day was registered by observation. An exception from this pattern was seen for house painters where the mini JEM underestimated this exposure. For total load lifted per day, the mini JEM without exception, estimated a higher exposure than observed.

Figure 1.

Symmetry plots of the expert-rated and measured/observed work exposures according to job title.

Table 3 presents the Spearman’s rank correlations. The rank correlations were moderate (0.55) for standing/walking, and very strong for kneeling/squatting and total load lifted per day (0.83 and 0.71, respectively).

Table 3.

Spearman’s rank correlation coefficients between expert-based ratings of job titles in the mini JEM and the corresponding technical measurements/observations for: standing/walking, kneeling/squatting and total load lifted during a workday

Type of work exposure	Number of job titles in the mini JEM	Correlation coefficient	P value*
Standing/walking (hours/day)	16	0.55	0.027
Kneeling/squatting (hours/day)	8	0.83	0.011
Total load lifted (kg/day)	8	0.71	0.047

*The p values show that the probability of no correlation is <0.05.

JEM, job exposure matrix;

Figure 2 shows Bland Altman plots which illustrate systematic deviations and 95% LoA (grey markings) between the expert-rated estimates in the mini JEM and the measured/observed exposures. For standing/walking, the mean difference in absolute terms (0.20 hours/workday) was within the 95% LoA (−1.63 to 2.03 hours/workday), except for ‘shop assistant, shop sales person’ where the mini JEM significantly underestimated the exposure. However, in relative terms as percentage of an 8-hour workday, this mean difference corresponds to a difference of 2.5% (95% LoA −20.4 to 25.4%). For kneeling/squatting, the mean difference in absolute terms was −0.35 hours/day (95% LoA −1.21 to 0.51 hours/day). In relative terms, this mean difference corresponds to a difference of 4.4% of an 8-hour workday (95% LoA −15.1 to 6.3%). For total load lifted, the mean difference was −909 kg/workday (95% LoA −3000 to 1147 kg/workday), and the estimates for ‘storage worker, warehouse assistant’ were outside the LoA.

Figure 2.

Bland-Altman plots of standing/walking (A), kneeling/squatting (B), and total load lifted per workday (C). Differences were calculated as measured/observed exposure minus expert-rated exposure (ie, positive values mean that the experts underestimated the exposures, while negative values mean that the experts overestimated the exposures).

Discussion

This study compared the ranking of occupational biomechanical exposures to the lower body based on expert ratings and measurements/observations. The rank correlations between expert ratings and technical/observational methods for standing/walking and kneeling/squatting were moderate to very strong, and highest for kneeling/squatting. No systematic deviations between expert ratings and technical/observational methods for standing/walking and kneeling/squatting were seen. However, the relative LoA indicated differences up to 25% of an 8-hour workday for time spent standing/walking, and up to 15% for kneeling/squatting. For total load lifted per workday, the experts systematically rated the exposures to be almost 1 ton higher than observed. Thus, the specific exposures for the job titles need to be interpreted with caution. This also applies for the use of the JEM estimates to calculate cumulative exposures.³⁷

Strengths and limitations

A strength of this study was that the same five occupational physicians, who performed the ratings for the job groups in the lower body JEM, performed new ratings supported by vignettes to create a job title specific mini JEM. In this way, the comparisons of the exposure estimates from the mini JEM with the exposure measurements/observations for each job title should reflect the validity of the lower body JEM. The study also benefited from inclusion of job titles, which represented a wide range of exposures.^{1 39} The main limitation of this study was that the representativeness can be limited because the participants were included based on convenient accessibility and because we did not succeed in recruiting 10 workers for several of the job titles due to limited resources.

The quality of the vignettes presented to the experts varied. For example, the vignette for ‘shop assistant, shop sales person’ (‘workers in three large discount grocery stores, mainly stocking shelves, but also sitting at the cashier’) signalled that at least some of the working time was spent sitting (in Denmark, cashiers usually sit down while working), which might explain why the experts underestimated the time spent standing/walking of this job title. Only office workers spent little time standing/walking, and none of the included jobs had observed lifting exposures>2000 kg/day. This means that we were unable to evaluate the agreement for low exposures to standing/walking and high exposures to lifting.

In future studies, calibration of the lower body JEM by technical/observational measurements would enhance the quality of exposure estimates—also for calculation of cumulative exposures.

Conclusion

This method comparison study found moderate to very high agreement between the ranking of standing/walking, kneeling/squatting, and total load lifted in the expert-rated mini JEM and technical/observational measurements of these exposures. The mini JEM overestimated the total load lifted per workday, but still, our study lends support to the use of the expert-rated lower body JEM in large-scale occupational epidemiological studies.

Data availability statement

Data can be used by application to the DOC*X steering group.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

The Danish Data Protection Agency accepted the data handling and storage of data (journal number 2015-54-0995) and The Danish National Ethics Committee approved the study protocol (journal number H-2-2012-011).