Exploring Study Designs for Evaluation of Interventions Aimed to Reduce Occupational Diseases and Injuries

Abstract

Effective interventions to reduce work-related exposures are available for many types of work-related diseases or injuries. However, knowledge of the impact of these interventions on injury or disease outcomes is scarce due to practical and methodological reasons. Study designs are considered for the evaluation of occupational health interventions on occupational disease or injury. Latency and frequency of occurrence of the health outcomes are two important features when designing an evaluation study with occupational disease or occupational injury as an outcome measure. Controlled evaluation studies—giving strong indications for an intervention effect—seem more suitable for more frequently occurring injuries or diseases. Uncontrolled evaluation time or case series studies are an option for evaluating less frequently occurring injuries or diseases. Interrupted time series offer alternatives to experimental randomized controlled trials to give an insight into the effectiveness of preventive actions in the work setting to decision and policy makers.

1. Introduction

The worldwide burden of occupational diseases(ODs) and injuries is high [1]; although, there are large variations in and between countries in ODs incidences [2]. For instance, in The Netherlands the annual incidence of occupational diseases varies between 0.3% (physician reported) and 5.7% (workers reported) [3]. Global estimates of ODs economic costs vary between 1.8% and 6.0% of gross domestic product [4]. In principle, ODs can be prevented by means of control measures at worksites. However, the majority of control measures are not evaluated in terms of reductions in ODs [2]. Barriers against controlled trials of interventions are methodological (e.g., the infrequent occurrence of ODs), practical (e.g., too difficult to perform in practice and costly to intervene across a large enough workers' population), or organizational (workplace restructurings during interventions) [5].

To overcome some of these barriers, the choice for an optimal study design should take into account the setting and context of the workers' population (job, sector of industry), application of the intervention (worker, company, sector, national), and outcome measure (expected frequency of disease or injury given a certain time frame). The use of workers' health surveillance data or existing databases like disease or injury registries can provide opportunities to evaluate interventions at company, sectorial, or national level. Registries of ODs are often maintained for regulatory or compensational reasons, but also offer the possibility of evaluating the impact of interventions on a wider scale than company level. However, the feasibility of existing databases depends strongly on the type of intervention and type of disease or injury outcome and requires careful study design.

In this short communication we focus on the choice of potential study designs to evaluate the effectiveness of interventions on ODs. Ideally, interventions aimed to reduce ODs should have a proven impact on exposures to hazardous agents and work demands (e.g., based on efficacy studies) and be attuned to the exposed worker’s population and work setting (e.g., through participatory approaches or qualitative research) first before performing studies to evaluate the effectiveness on diseases or injuries. The aim of this paper is: (1) to explore study designs that are potentially useful to evaluate preventive interventions with ODs as an outcome measure; and (2) to provide purposively selected examples of the application of these study designs.

2. Materials and methods

Latency and frequency of occurrence of occupational injuries and diseases were considered to be important aspects to take into account in study designs for the evaluation of ODs interventions [6].

In this short communication, shorter latency was, arbitrarily, defined as an interval of ≤ 12 months between exposure to risk factors and occupational diseases and zero for occupational injuries; longer latency was defined in terms of > 12 months for occupational diseases. The frequency with which a disease or injury occurs in a population over a time period was, arbitrarily, defined as lower at ≤ 5% and consequently higher with > 5%. Consequently, four quadrants of ODs outcomes were explored: higher frequency–shorter latency, higher frequency–longer latency, lower frequency–shorter latency, and lower frequency–longer latency.

Alongside the four ODs quadrants for possible study designs, a number of applicable Hill's [7] viewpoints were used for prioritizing the different study designs as a means to assess the capability of establishing an ODs intervention effect, i.e., experimental, association, biological gradient, temporality, analogy, specificity, plausibility, consistency, and coherence. The ranking for detecting possible intervention effects alongside the four quadrants of study designs were labeled as: offering strongest–moderate–low–weakest possibilities for detecting intervention effects.

In addition, purposely selected study examples and discussion of alternative study designs in the evaluation of ODs interventions using workers' health surveillance data was undertaken. The idea and approach of discussing different study designs for the evaluation of ODs interventions and providing study examples was based on Bonell et al [8], who considered alternatives to randomization in the evaluation of public health interventions.

3. Results

Four quadrants for study designs aimed to evaluate interventions on occupational injuries and diseases are summarized in Table 1. In all quadrants, at least five out of nine of Hill's [7] viewpoints can be considered alongside the proposed study designs, i.e., analogy, specificity, plausibility, consistency, and coherence, for establishing indications of intervention effects.

Table 1.

Study designs for evaluating interventions on occupational diseases or injuries

Outcome measure	Shorter latency∗ ≤12 mo	Longer latency >12 mo
Higher frequency† > 5%	Randomized controlled trials Indication for effect: strongest Example 1	Controlled studies Indication for effect: moderate Example 2
Lower frequency ≤ 5%	Interrupted time series Indication for effect: low Example 3	Time series, case series Indication for effect: weakest Example 4

^∗Latency: time interval between exposure to hazardous agents or work demands and occurrence of diseases or injuries.

^†Frequency: number of diseases or injuries in a population over a time period.

All controlled evaluation studies in the two highest quadrants—giving the best possibilities for establishing intervention effects—seem more suitable for more frequently occurring events [like needle stick injuries (Example 1) or occupational asthma (Example 2); Table 2]. Randomized experimental studies—offering the possibilities for strongest indication of an intervention effect with all nine applicable Hill's [7] viewpoints—seem more suitable for events with a shorter latency time. Nonrandomized controlled studies, lacking the experimental feature, are offering moderate indications for an intervention effect.

Table 2.

Examples of the four study designs

1. Safety needles & workshop on needle stick injuries [9] Intervention: technical device on safety needles & interactive workshop in health care workers. Outcome characteristics: frequent & shorter latency needle stick injuries. Evaluation design: cluster randomized controlled trial. Results: 66% reduction in needle stick injuries [odds ratio: 0.34; 95% confidence interval (CI): 0.13–0.91).

2. Regulation, market, & education on occupational asthma [10], [11] Interventions: national legislation, market forces, education, & regulatory activity. Outcome characteristics: frequent & longer latency occupational asthma. Evaluation design: controlled before & after study. Results: positive impact of legislation & changes in the supply chain (e.g., latex & glutaraldehyde) but less evidence of impact of education and regulatory activity (e.g., flour).

3. Regulation on occupational injuries [12] Intervention: regulation on safety measures in construction workers. Outcome characteristics: nonfrequent & shorter latency fatal & nonfatal injuries. Evaluation design: meta-analysis on interrupted time series in systematic review. Results: initial & sustained increase in fatal (effect size of 0.79; 95% CI: 0.00–1.58) & nonfatal injuries (effect size 0.23; 95% CI 0.03–0.43).

4. Screening program & regulation on solvent induced encephalopathy [13] Intervention: ban on indoor use of solvent-based paints & workers health surveillance in painters. Outcome characteristics: nonfrequent, longer latency chronic solvent-induced encephalopathy. Evaluation design: case series (yearly) on screening. Results: downwards trend year prevalence of newly diagnosed chronic solvent-induced encephalopathy from max 102 cases to 1 case.

Uncontrolled evaluation time or case series studies in the two lower quadrants are an option for less frequently occurring events such as fatal injuries (Example 3) or chronic solvent-induced encephalitis (Example 4), giving possibilities for establishing indications for intervention effect. Interrupted time series—giving possibilities for establishing low indications for intervention effects—seem the best option for nonfrequently occurring events with shorter latency time, but lacking the possibilities for establishing association and biological gradient. Case (series) studies seem a possibility for nonfrequent events with long latency time to start the research chain of establishing an intervention effect in the longer term.

4. Discussion

Controlled before and after studies or interrupted time series methods applied to injury or disease data offers alternatives to experimental randomized controlled trials in providing insight into the effectiveness of preventive actions in the work setting to aid decision and policy makers.

These alternative designs suggested here, and lacking randomization and also sometimes control groups, are more susceptible to bias and, therefore, should be transparent by analyses and reporting confounding factors and commenting on the plausibility of an effect [8], [14]. Therefore, investigators must report their methods thoroughly and be conscious and critical of the assumptions they must make whenever they adopt these designs [14]. Evidence is more convincing when confounders are well understood, measured, and controlled; there is evidence for possible causal pathways linking intervention and outcomes and/or against other pathways explaining outcomes [8].

In 1965, Hill [7] defined plausibility in terms of biological plausibility, but acknowledged that this depends on the actual biological knowledge. Biological plausibility is as important as ever; insight in to the underlying biological mechanisms might also help to focus interventions on particularly susceptible workers. In the same way as Hill's [7] viewpoints help to move from association to causation, we explore moving from association to intervention effect. As new methods in study designs and statistical modelling emerge, Hill’s [7] viewpoints such as the strength of the association, consistency, biological plausibility, and dose-response gradient remain as important as ever. Although the epidemiologic evidence offered by Hill [7] are saddled with reservations and exceptions [15], in the case of application of less optimal study designs there could be a plea to take into account these nine viewpoints as a check or guide.

Plausibility in terms of successfully targeting the ODs specific risk factors and executing the interventions as planned can be strengthened by process evaluations on behavioral change. For example, the reduction in needle stick injuries (Example 1) was aligned with behavioral change in facilitation (the traditional injection needles were replaced by the safety needles) and safety culture, i.e., better communication between workers and management about safety risks, rules, and procedures [9].

In this short communication, we have taken the challenge to quantify, arbitrarily, the border between shorter and longer latency, as well as lower and higher ODs frequency in a population. This is solely done to be transparent to choose an optimal study design. To the knowledge of the authors, no evidence exists to help decide on that beforehand.

In summary, besides the experimental (e.g., randomized controlled trial) and plausibility characteristics, some others of Hill's [7] viewpoints for causal attribution could also be applied towards intervention studies. Controlled studies provide a stronger indication for intervention effects because of possibilities for establishing measures of strengths of association and dose (intervention)-response gradients. However, uncontrolled studies can also provide an indication for intervention effects based on a temporal coincidence and appropriate analysis (meta-analysis on interrupted time series, e.g., Example 3) or based on analogy, specificity, and consistency (case series, e.g., Example 4). In the end, the evaluation of each intervention on ODs can be a combination of above mentioned study designs—often depending on available occupational data—and attributing to Hill’s [7] viewpoint of coherence.

Conflicts of interest

All contributing authors declare no conflicts of interest.