Summary of body of prevalence studies reviewed
This systematic review included a large number (65) of prevalence studies across a moderate number (28) of countries in all six WHO regions (Africa, Americas, Eastern Mediterranean, Europe, South-East Asia, and Western Pacific). There were four studies of general populations of workers, and 61 studies of worker populations in high-exposure industrial sectors (e.g., construction) or occupations (e.g., construction workers). The occupational risk factor’s definition is perhaps relatively straightforward: any (high) occupational exposure to noise (≥85dBA). The assessors rated the level of expected heterogeneity as “High”.
Step 1 of QoE-SPEO: judging the level of expected heterogeneity
The assessors said the “details and examples of health studies of workers” were helpful additions to this step. They felt that “consideration of both within- and between-worker variations in exposure prevalence is an important stipulation”.
They reported the approach would benefit from further explanations of epidemiological terms, “so that researchers from more distant areas of public health can use and understand the instrument”. They said that assessing expected heterogeneity can be complex, especially for a risk factor like noise, for which the entire working population could be exposed: “considering heterogeneity across sectors and job titles may be challenging, when no prior knowledge exists on the population-level”. Rating heterogeneity across the entire body of evidence was challenging (i.e., evidence from both population-based samples comprising various job titles and from specific industrial sectors or occupations with more narrowly defined job characteristics).
Assessors suggested that an explanation be included to illustrate when “expected or real heterogeneity can occur between studies with low and high prevalence”. They also suggested that the instructions can be adapted “for cases when the prevalence of a novel or less studied risk factor in the entire working population is of interest”. Further, they proposed that it may be appropriate to assess heterogeneity separately for evidence from population-based studies (where heterogeneity is expected, at least between workers, and sometimes also within workers) and evidence from studies in specific industry sectors (where less such heterogeneity is expected); or an approach to average across different study designs could be developed.
Step 2 of QoE-SPEO: assessing downgrade domains
The assessors felt the instructions and examples for Step 2 were clear and aided understanding. However, they raised several limitations of this step, relating to the following domains:
Domain of Risk of bias
The assessors thought that specific consideration should be given to whether the evidence comes from population-based studies, if the risk factor of interest is prevalent in the entire working population: “evidence from those studies should be attributed higher weight” (at least when the target prevalence is that of all workers in the population). They raised the concern that mixing evidence from high-prevalence industry samples and population-based samples could lead to unrealistic overestimation of all-of-population prevalence.
Domain of Imprecision
Assessors requested a tentative definition of what can be considered a “narrow” 95% confidence interval. Additionally, they felt the domain would benefit from “suggestions as to how to reconcile epidemiological heterogeneity rated in Step 1 with observed statistical heterogeneity in the meta-analysis of prevalence model”.
Domain of Publication bias
Assessors highlighted that, in studies that report both prevalence and effect of a risk factor, the size and statistical significance of the effect are more likely to be “driving publication bias, rather than the prevalence of the risk factor”. In studies of effect, which are commonly used to extract prevalence data from, greater exposure contrast between exposed and unexposed groups of workers is more likely to reveal significant risk of disease due to higher statistical power, and therefore, “studies where the prevalence of a risk factor is not high may be more likely to be published (because of significant findings), rather than studies in which the prevalence of that risk factor is high and the majority of workers are exposed”. They suggested that assessors should be prompted to consider the primary aims of the included studies.
Another caveat in the publication bias domain is that the suggested funnel plot and Egger’s test for asymmetry as means of detecting publication bias are discouraged in meta-analyses of prevalence (Hunter et al. 2014). Moreover, the Eger test in general is not advised for dichotomous outcomes (Page et al. 2021).
Step 3 of QoE-SPEO: reaching a final decision (rating) on the quality of evidence
Assessors said the instructions provided for this step were clear and detailed, and recognized that rating the overall quality of evidence is an iterative and transactional process between assessors. Provision of a completed example could help assessors.