Abstract
Background
To support occupational physicians (OPs) in the implementation of workers’ health surveillance (WHS), a training programme was developed.
Aims
(i) To evaluate the effects of a WHS training programme for OPs on knowledge, self-efficacy and skills to implement WHS. (ii) To evaluate to what extent a WHS training programme is acceptable and feasible for implementation in practice.
Methods
A single-blinded randomized controlled trial with waiting-list control group was used. The WHS training programme consisted of an e-learning and a 4.5-h online training session. OPs completed a knowledge test (0–8), self-efficacy questionnaires on knowledge and skills (6–60), and vignette assignments (0–16) to measure skills. OPs completed the questionnaires, either before and after the WHS training programme (intervention group), or before the training programme (control group) while receiving the training programme after the waiting period. All OPs completed questionnaires about the training’s acceptability, and feasibility for implementation in practice. ANCOVA and Poisson regression analyses were conducted.
Results
The self-efficacy score (M = 44.1 versus M = 37.2) (P < 0.001) and skills score (M = 9.6 versus M = 8.3) (P < 0.05) of OPs in the training group (N = 16) were higher than the control group (N = 23). No effect was found on knowledge. Evaluation of acceptability and feasibility showed that 21 (58%) OPs were very satisfied with the training part on initiating WHS, and 29 (85%) would recommend the WHS training programme to colleagues.
Conclusions
This WHS training programme has a positive effect on self-efficacy and skills of OPs to implement WHS, and may be acceptable and feasible to implement in practice.
Workers’ health surveillance (WHS) is aimed at the prevention of work-related diseases. This study shows that a newly developed WHS training programme had a positive effect on self-efficacy and skills of occupational physicians (OPs) to implement WHS. Because the training showed to be acceptable and feasible, enhancing educational programmes with specific WHS training may support OPs to initiate and implement WHS.
Key learning points.
What is already known about this subject:
Work-related risk factors can lead to health complaints among workers, which in turn can lead to (long-term) sickness absence or presenteeism.
A workers’ health surveillance programme can be implemented for detection and prevention of early health complaints, and risk factors among workers.
Occupational physicians have a need for development of their knowledge and skills to implement workers’ health surveillance in companies.
What this study adds:
A workers’ health surveillance online training programme was developed to support occupational physicians in the implementation of workers’ health surveillance programmes.
A workers’ health surveillance online training programme resulted in an increase in skills and self-efficacy of occupational physicians to implement workers’ health surveillance.
What impact this may have on practice or policy:
A workers’ health surveillance online training programme increased self-efficacy and skills of occupational physician. It may be useful to include the online training programme in educational programmes for occupational physicians to support them in the implementation of workers’ health surveillance.
Introduction
Risk factors in the workplace can lead to work-related diseases among workers. High job demands or inadequate social support, for example, are risk factors for the development of common mental health problems [1]. Work-related health problems may result in (long-term) sickness absence, and presenteeism or productivity loss [2,3].
Work-related diseases and injuries can be reduced by implementation of prevention measures [4]. For example, cognitive behavioural interventions might reduce stress [5]. To be able to implement preventive interventions appropriate for the specific work-related risk factors and health complaints in workers, systematic early detection of health complaints and risk factors is needed. One preferred strategy is workers’ health surveillance (WHS).
WHS can consist of a periodic assessment of workers’ health [6], aimed at detection of adverse health effects resulting from occupational exposures as early as possible, so that appropriate preventive measures can be implemented. The findings from WHS can also be used to implement prevention on group level [7].
WHS was addressed in Article 14 of a European Council Directive [8]. The directive was adopted by the majority of EU countries in national law; modification of the directive was allowed [9]. In most EU countries, employers are obliged to offer WHS to their workers, while the actual implementation is performed by occupational physicians (OPs) [9].
In the Netherlands, the employer is obliged to offer WHS, and workers can voluntarily participate. In practice, every company has its own contract with an occupational health service or self-employed OP about the number and content of WHS programmes without subsequent regulatory enforcements [10]. The main goals of WHS are (i) prevention of work-related diseases and injuries, and (ii) monitoring and promoting the health of workers [11]. The content of the WHS should correspond with the occurring health risks, e.g. based on a risk inventory and evaluation at company level.
In the Netherlands, OPs’ role is crucial to the initiation and implementation of WHS in companies. To support OPs in the implementation of WHS, the Netherlands Society of Occupational Medicine (NVAB) has developed a guidance document [11]. However, a survey among OPs in the Netherlands shows that WHS is not implemented by all OPs [12], and that OPs feel a need for further development of their knowledge and skills on WHS [12]. Because education or training to initiate and implement WHS was not available, a WHS online training programme was developed.
The aim of this study was to evaluate (i) the effect of a developed WHS online training programme on knowledge, self-efficacy and skills of OPs using a randomized controlled trial (RCT) and (ii) the acceptability and implementation of the WHS online training programme.
Methods
The Medical Ethics Committee of Academic Medical Centre found that a comprehensive evaluation was not required, as the study was not subject to the Medical Research Involving Human Subjects Act (W21_155 # 21.170). The Consolidated Standards of Reporting Trials statement [13] was used reporting the results.
An RCT with a waiting-list control group, with an allocation ratio of 1:1 as to training and control group, was used.
Participants were eligible if they were OP or OP in training. In April 2021, we recruited OPs from a cohort study. The cohort study aimed to investigate the implementation of WHS by OPs, and consisted of 128 OPs who signed up to complete four waves of annual questionnaires on WHS. Some OPs who were not participating in the cohort also signed up after they received the invitation from colleagues in the cohort. OPs were invited by e-mail. In the invitation, six longitudinally planned date options were presented. The first three options were scheduled for the training group, and the other options for the control group. The e-mail included an attachment that provided information on the content and duration of the e-learning and training programme, the informed consent form and the questionnaires participants had to complete as part of the study.
OPs could sign up by e-mail, and indicate two date options on which they could participate.
Data were collected using the survey system Castor (Castoredc.com), which created a participant number for each participant, ensuring that data could be pseudonymized, and anonymously analysed.
The training programme was developed on the basis of the behaviour change wheel [14,15]. Based on the guidance document, the SPIN selling model [16] and motivational interviewing [17], a new approach consisting of five steps was developed for (i) the initiation of WHS, (ii) to determine the content of a WHS and (iii) to conduct preventive consultations with individual workers (Table 1).
Table 1.
5-step approaches used in the WHS online training programme
| Initiating WHS | Determining the content | Preventive workers’ consultations |
|---|---|---|
| ▪Step 1: Clarify expectations ▪Step 2: Gain insight into the situation within the company ▪Step 3: Gain insight into possible bottlenecks and the employer’s dissatisfaction with the situation ▪Step 4: Allow the employer to gain insight into the effects and/or consequences of the perceived bottlenecks for workers and thus for the organization ▪Step 5: Let the employer gain insight into the benefits of WHS as a possible solution for bottlenecks |
▪Step 1: Gather information about: ➣ Job requirements, exposures to risk factors for developing health complaints ➣ Use risk inventory reports, or literature ▪Step 2: If available, collect information about special job requirements or job requirements from mandatory medical examinations ▪Step 3: Check whether there is evidence that the job demands pose an increased risk for the occurrence of health complaints ➣ If yes, include in WHS ▪Step 4: Check whether the prevalence/incidence of the health complaint is higher compared to the general working population ➣ If yes, include in WHS ▪Step 5: Check whether the workers’ performance is directly affected by the health complaint ➣ If yes, include in WHS |
▪Step 0: Preparation of the consultation ▪Step 1: Welcome the worker and clarify expectations ▪Step 2: Discuss results of the screening (e.g. does the worker recognize the results) ▪Step 3: Problem Inventory of the context of the health problems (e.g. possible cause, severity of complaints, possible influence on work functioning) ▪Step 4: Discuss advice suitable for the formulated problem ▪Step 5: Continuation and conclusion |
OPs in the training group participated in the training programme in May and June 2021. The training programme consisted of online self-study in the form of an e-learning module, and a 4.5-h online session on Zoom (Zoom.us). In the e-learning, OPs were informed about the developed 5-step approaches (5SA), main goals of WHS and examples of WHS modules.
In the online training session, role-play exercises in small groups using cases were conducted to practice skills in using the 5SA. The trainer discussed OPs’ experiences and the role-play exercises, and provided feedback (Appendix 1, available as Supplementary data at Occupational Medicine Online).
OPs in the control group also received the training and e-learning, after completing the measurements, in June 2021.
Participants in the training group received a knowledge test, a self-efficacy questionnaire and vignette assignments, approximately 3 weeks before the training started (T0), and after participation in the online training session (T2). After OPs had completed the questionnaires at baseline, they received the e-learning programme. The knowledge test was also conducted at the start of the online training programme (T1).
OPs in the control group received the first knowledge test, self-efficacy questionnaire and vignette assignment approximately 6 weeks before the training started. The second and third knowledge tests and second questionnaires on self-efficacy and vignette assignments were sent approximately 3 weeks before the training programme started.
Knowledge tests, consisting of eight true/false statements, were conducted. A total of eight points could be earned. Three versions of the knowledge tests were developed. A counterbalanced design was used in which two groups had version A on T0, version B on T1 and version C on T2. The other two groups started with version B or version C.
A self-efficacy questionnaire was conducted, consisting of six questions in which OPs had to assess their level of knowledge and skills separately to (i) initiate WHS, (ii) determine the content of WHS and (iii) conduct preventive consultations with workers, on a scale from 1 to 10. A maximum of 60 points could be earned.
To measure OPs’ skills, six vignette assignments were used. Vignettes described fictional situations, and assignments for OPs concerned the use of different components of the 5SA (Appendix 2, available as Supplementary data at Occupational Medicine Online). Two similar versions of the six vignette assignments and scoring criteria were developed by the research team. A counterbalanced design was used, in which three groups completed version A at T0, and version B at T2, while the other groups started with version B. The scoring of the vignette assignments was done in duplicate, by the first author (F.S.L.), and by one of the other authors. F.S.L. compared the answers, and differences were resolved by discussion. In total 16 points could be earned. The answers on the vignette assignments were extracted from the datasheet in SPSS, and only a participant number was available.
To measure acceptability, participants indicated on a 5-point Likert scale to what extend they were satisfied with various components of the training programme. Furthermore, participants indicated on a 5-point Likert scale to what extend they agreed that the training programme had contributed to their knowledge and skills.
To measure feasibility, participants provided feedback on positive and negative components of the training programme, whether they would apply the acquired skills and the developed 5SA (yes/no), and whether they would recommend the training programme to colleagues (yes/no). Open questions were used for explanation of their answers. Questionnaires, vignette assignments and knowledge tests were all developed by the research team.
OPs in both the training group and the control group completed questionnaires on acceptability and implementation after they had participated in the training programme.
We were not able to calculate a sample size based on earlier information about our outcome measures. We aimed to include 60 OPs.
OPs were enrolled, and randomized by the first author F.S.L. using a random number generator to assign each participant to a date option in the training or control group. A single-blind randomized block design was used. It was not explicitly stated in the invitation which date options were scheduled for the control group. Participants in the training group were informed that they would receive the electronic questionnaire including informed consent form pre and post training. Participants assigned to the control group were informed that they would receive two electronic questionnaires before participating in the training programme.
The researchers and trainers in the training programme were aware of the allocation to the training group or the control group. OPs in the control group were on a waiting-list for the training programme.
Participant characteristics were described using descriptive statistics; differences were analysed using an independent sample t-test, or Fishers’ exact test. For normally distributed data, ANCOVA analyses with correction for baseline level were conducted for knowledge, self-efficacy and skills after training, between the training group and the control group. For non-normal data, a Poisson regression analysis was conducted. Descriptive statistics were used to describe the results on acceptability and implementation of the training programme. Open answers were summarized by F.S.L.
Results
In total, 58 OPs registered, and 46 OPs started, of whom 23 OPs in the training group and 23 in the control group. Finally, 16 OPs in the intervention group, and 23 in the control group completed the study. A flow chart is shown in Figure 1.
Figure 1.
Flow chart RCT.
OPs were employed in a large variety of sectors, such as healthcare, education and construction companies. Furthermore, 21 OPs were male, and 11 were OP in training. In total 15 OPs were self-employed and 24 worked at an occupational health service. On average, OPs were 50.3 (SD = 10.3) years old, and had 16.8 years (SD = 10.7) of working experience (Table 2).
Table 2.
Demographic characteristics
| Total (n = 39) | Training group (n = 16) | Control group (n = 23) | P-value | ||||
|---|---|---|---|---|---|---|---|
| N (%) | M (SD) | N (%) | M (SD) | N (%) | M (SD) | ||
| Age | 50.3 (10.3) | 51.7 (8.3) | 49.4 (11.5) | NS | |||
| Years of working experience | 16.8 (10.7) | 18.6 (9.9) | 15.6 (11.2) | NS | |||
| Working hours per week | 33.7 (7.1) | 31.5 (8.3) | 35.1 (5.9) | NS | |||
| Sex | NS | ||||||
| Male | 21 (54) | 8 (50.0) | 13 (57.0) | ||||
| Female | 18 (46) | 8 (50.0) | 10 (43.0) | ||||
| Function | NS | ||||||
| OP | 28 (72) | 12 (75.0) | 16 (70.0) | ||||
| OP in training | 11 (28) | 4 (25.0) | 7 (30.0) | ||||
| Self-employed | 15 (38) | 8 (50.0) | 7 (30) | NS | |||
| At an external OHS | 19 (49) | 4 (25.0) | 15 (66) | ||||
| At an internal OHS | 5 (13) | 4 (25.0) | 1 (4.0) | ||||
| Companies with fewer than 50 workers | 8 (21) | 3 (19) | 5 (22) | NS | |||
| Companies with 50–250 workers | 19 (49) | 11 (69) | 8 (35) | NS | |||
| Companies with more than 250 workers | 32 (82) | 12 (75) | 20 (87) | NS | |||
The T1 knowledge level was not significantly different in the training group (M = 5.6, SD = 1.5) compared to the control group (M = 5.3, SD = 1.5), adjusted for T0 [F(1,36) = 0.57, P = NS, η2 = 0.016]. The T2 knowledge level did not significantly differ between the training group (M = 5.8, SD = 1.3) and the control group (M = 5.4, SD = 0.9), adjusted for T1 [F(1,36) = 0.89, P = NS, η2 = 0.024] (Table 3).
Table 3.
Mean, SD and range on knowledge scores (0–8), self-efficacy concerning knowledge and skills (1–60) and skills (0–16) on T0, T1 and T2
| Training group | Control group | P-value | |||||
|---|---|---|---|---|---|---|---|
| N | M (SD) | Min–max | N | M (SD) | Min–max | ||
| Knowledge | |||||||
| Baseline (T0) | 16 | 5.6 (1.1) | 4–7 | 23 | 5.3 (1.6) | 2–8 | |
| Post e-learning (T1) | 16 | 5.6 (1.5) | 3–8 | 23 | 5.3 (1.5) | 2–7 | NS |
| Post training (T2) | 16 | 5.8 (1.3) | 3–8 | 23 | 5.4 (0.9) | 4–8 | NS |
| Self-efficacy concerning knowledge and skills | |||||||
| Baseline (T0) | 16 | 35.1 (9.0) | 14–47 | 23 | 37.8 (9.1) | 12–54 | |
| Post training (T2) | 16 | 44.1 (4.0) | 37–51 | 23 | 37.2 (10.8) | 6–56 | P < 0.001 |
| Skills | |||||||
| Baseline (T0) | 16 | 7.8 (1.5) | 5.5–11 | 23 | 8.4 (2.0) | 4–12 | |
| Post training (T2) | 16 | 9.6 (2.7) | 3.5–13 | 23 | 8.3 (2.0) | 5.5–12 | P < 0.05 |
M, mean; SD, standard deviation.
For the training group (M = 44.1, SD = 4.0), self-efficacy scores T2 were significantly higher compared to the control group, adjusted for T0 (M = 37.2, SD = 10.8), [Exp(β) = 1.25, P < 0.001, 95% CI = 1.13–1.39] (Table 3). This was also the case for self-efficacy concerning knowledge or skills separately.
At T2, skills of OPs in the training group (M = 9.6, SD = 2.7) were significantly higher compared to the control group (M = 8.3, SD = 2.0), adjusted for T0 [F(1,36) = 4.5, P < 0.05, η2 = 0.11] (Table 3). For the vignette assignments on initiating WHS separately, the skills of OPs in the training group (M = 3.1, SD = 1.0) were significantly higher compared to the control group (M = 2.4, SD = 0.9) adjusted for T0 [F(1,36) = 5.6, P < 0.05, η2 = 0.13]. There was no significant difference in separate assignments on determining the content of WHS, or assignments on performing preventive consultations with workers.
In total 21 OPs (58%) out of 36 OPs who started the acceptability and feasibility questionnaires were very satisfied with the online training part on initiating WHS in organizations, while 15 OPs (42%) were very satisfied with the e-learning part on initiating WHS (Table 4). Moreover, 27 OPs (75%) agreed and 7 (19%) completely agreed that the training programme contributed to their skills to initiate WHS (Appendix 3, available as Supplementary data at Occupational Medicine Online).
Table 4.
Scores on a 5-point Likert satisfaction on statements about the WHS online training programme (very unsatisfied–very satisfied)
| Statements | N | Min–max | Mean Likert score (1–5) | Satisfied, N (%) | Very satisfied, N (%) |
|---|---|---|---|---|---|
| How satisfied are you with… | |||||
| The online training on initiating WHS | 36 | 2–5 | 4.4 | 10 (28) | 21 (58) |
| The e-learning part on initiating WHS | 36 | 3–5 | 4.2 | 13 (36) | 15 (42) |
| The online training on determining the content of WHS | 36 | 2–5 | 4.3 | 10 (28) | 18 (50) |
| The e-learning part on determining the content of WHS | 36 | 3–5 | 4.1 | 11 (31) | 14 (39) |
| The online training on preventive consultations with workers | 36 | 1–5 | 4.0 | 14 (39) | 13 (36) |
| The e-learning part on preventive consultations with workers | 36 | 2–5 | 3.9 | 13 (36) | 11 (31) |
| The level of the training programme | 36 | 3–5 | 4.5 | 9 (25) | 23 (64) |
| The interactive exercises | 36 | 2–5 | 4.3 | 12 (33) | 18 (50) |
| The plenary explanation | 36 | 3–5 | 4.6 | 8 (22) | 24 (67) |
| The case exercises | 36 | 2–5 | 4.4 | 11 (31) | 20 (56) |
| The group size | 36 | 2–5 | 4.9 | 2 (6) | 33 (92) |
| The length of the training session | 36 | 2–5 | 4.3 | 9 (25) | 20 (56) |
Respectively, 33 OPs (94%) would use the 5SA, and 32 (94%) the acquired skills to initiate WHS with employers. Twenty-nine OPs (85%) would use the 5SA, and 32 (94%) the acquired skills to determine the content of WHS in practice. Finally, 24 OPs (71%) would use the 5SA, and 26 (77%) the acquired skills to conduct preventive consultations with workers. In total 29 OPs (85%) would recommend the training programme to a colleague, and 29 OPs (94%) found that the training programme could be implemented in practice (Appendix 4, available as Supplementary data at Occupational Medicine Online).
Open-answer questions revealed that positive aspects of the training programme were attention to the specific needs of participants; and variety between short presentations and the practicing of skills. OPs wanted to use the 5SA, as it provided structure. OPs would recommend the training programme to colleagues because they learned useful skills for the implementation of WHS.
OPs also indicated that using Zoom was not convenient, and more time was needed to practice role-play exercises for each case. Some OPs indicated that they already used the methods provided in the 5SA. Some OPs would not recommend the training programme, as they found in-depth information on specific topics was lacking.
Discussion
We found a statistically significant effect of the training programme on skills and self-efficacy to implement WHS. No effect of either the e-learning or online training session was found on knowledge. The majority of OPs were satisfied with the training programme.
No effect on knowledge was found from both the e-learning and training session. The study of Hugenholtz et al. did find a positive effect on knowledge level after conducting an e-learning course for OPs [18]. The majority of the OPs in the current training programme were recruited from the WHS cohort study. As the aim of the cohort study was to investigate the implementation of WHS, OPs participating in the cohort possibly had a higher basic knowledge level on the implementation of WHS. In accordance, some OPs in our study indicated that they needed more in-depth information about specific WHS topics.
The increase in skills can mainly be explained by the increase in skills to initiate WHS in companies. In accordance, OPs were most satisfied with the training part on initiating WHS. This is in line with our findings in an earlier study that lack of opportunities within organizations of employers might be barriers to the implementation of WHS [12]. This indicates that, besides the capabilities of OPs, opportunities provided by stakeholders on different organizational levels [14,19], such as employers, can also be of influence on the implementation of WHS.
Closed-ended questions have been used in many studies to evaluate the effect of educational interventions. However, this may not be suitable to test skills. In this study, vignette assignments were conducted to assess OPs’ skills because they have been proved to be a suitable method of measuring physicians’ performance [20]. The vignette assignments in this study were developed by the research team. Using these vignettes gave the opportunity to assess specifically how OPs would apply the conversation techniques from the provided step-by-step approaches. It can therefore be considered a strength of the study. However, an actual conversation between employer and OP is not simulated with these written case vignette assignments. Assessing newly acquired skills in real-life conversations with employers might be even more accurate to investigate the effect of the WHS training programme [21].
A possible strength of the study is that the self-efficacy questionnaires were developed by the research team. This ensures that self-efficacy on implementation of WHS is more specifically reflected than in general self-efficacy questions of validated tools. However, the lack of validated tools could also be a limitation of the study since the predictive ability remains unknown [22]. An important deficiency of using self-reported instruments is the possibility of social desirability bias that can occur. The results of OPs who had participated in the training programme could have been influenced by the expectation that the training had contributed to an increase in knowledge and skills. Using a validated instrument is a strategy to overcome this bias [23]. However, a study on communication skills of medical students found that self-efficacy scores did not significantly differ from scores reported by observers and patient assessment in the majority of items on the self-efficacy scale [24]. Nevertheless, in addition to measuring self-efficacy, we also measured skills and knowledge by vignette assignments and knowledge test. The vignette assignment score to measure skills was also increased after the training programme.
A limitation of the study is the relatively large loss to follow-up. In total 46 OPs started the study, while only 16 OPs in the training group completed the study. An independent sample t-test revealed that OPs who did not complete the vignette assignments post training were older (M = 56.7, SD = 5.9) compared to OPs who completed the whole study (M = 50.3, SD = 10.3) (P < 0.05). Possibly, older OPs have more experience in implementing WHS, which made them less willing to complete their participation in the study. Consequently, the results of the study may be most applicable to starting OPs.
The knowledge tests in the control group on T1 and T2 were conducted consecutively. OPs in the control group may have learned new knowledge from the test on T1, which they could apply in the test on T2 [25]. However, no feedback between knowledge tests was provided, which is essential in learning from tests [26]. In addition, no effect on knowledge was found for either the training group or the control group. Therefore, we do not think that a learning effect was influencing the results of the control group.
The results of the study are a first indication that the training programme contributes to improved capabilities of OPs to implement WHS. However, to study the actual effect of training on implementation of WHS by OPs, further studies are needed.
In this study, we trained OPs in having conversations with employers about the implementation of WHS, without directly involving employers in the intervention. Future studies should develop and evaluate an intervention that also focuses on the role of employers in the implementation of WHS.
The training programme did increase OPs’ skills and self-efficacy to implement WHS, while an effect on knowledge was lacking. OPs were satisfied with the several components of the training programme, and the majority would implement the acquired skills.
Supplementary Material
Contributor Information
F S Los, Amsterdam UMC, University of Amsterdam, Department of Public and Occupational Health, Coronel Institute of Occupational Health, Amsterdam Public Health Research Institute, 1100 DD Amsterdam, The Netherlands.
C T J Hulshof, Amsterdam UMC, University of Amsterdam, Department of Public and Occupational Health, Coronel Institute of Occupational Health, Amsterdam Public Health Research Institute, 1100 DD Amsterdam, The Netherlands.
A G E M de Boer, Amsterdam UMC, University of Amsterdam, Department of Public and Occupational Health, Coronel Institute of Occupational Health, Amsterdam Public Health Research Institute, 1100 DD Amsterdam, The Netherlands.
H F van der Molen, Amsterdam UMC, University of Amsterdam, Department of Public and Occupational Health, Coronel Institute of Occupational Health, Amsterdam Public Health Research Institute, 1100 DD Amsterdam, The Netherlands.
Funding
This work was supported by the Amsterdam UMC, location AMC.
Competing interests
None declared.
References
- 1. Harvey SB, Modini M, Joyce Set al. Can work make you mentally ill? A systematic meta-review of work-related risk factors for common mental health problems. Occup Environ Med 2017;74:301–310. [DOI] [PubMed] [Google Scholar]
- 2. Salvagioni DAJ, Melanda FN, Mesas AE, González AD, Gabani FL, Andrade SM.. Physical, psychological and occupational consequences of job burnout: a systematic review of prospective studies. PLoS One 2017;12:e0185781. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Si S, Lewkowski K, Fritschi L, Heyworth J, Liew D, Li I.. Productivity burden of occupational noise-induced hearing loss in Australia: a life table modelling study. Int J Environ Res Public Health 2020;17:4667. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. van der Molen HF, Frings-Dresen MHW.. Occupational diseases: from cure to prevention. J Clin Med 2019;8:1681. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Ruotsalainen JH, Verbeek JH, Mariné A, Serra C.. Preventing occupational stress in healthcare workers. Cochrane Database Syst Rev 2015;2015:Cd002892. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. International Labour Organization. Technical and Ethical Guidelines for Workers’ Health Surveillance. Report No.: 72. 1998. [Google Scholar]
- 7. Koh D, Aw TC.. Surveillance in occupational health. Occup Environ Med 2003;60:705–710, 633. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. European Council. Council Directive 89/391/EEC—OSH ‘Framework Directive’. Off J Eur Commun 1989: No. L183/1–8. 1989. https://eur-lex.europa.eu/legal-content/EN/TXT/PDF/?uri=CELEX:31989L0391&from=EN (2 January 2023, date last accessed). [Google Scholar]
- 9. Colosio C, Mandic-Rajcevic S, Godderis L, van der Laan G, Hulshof C, van Dijk F.. Workers’ health surveillance: implementation of the Directive 89/391/EEC in Europe. Occup Med (Lond) 2017;67:574–578. [DOI] [PubMed] [Google Scholar]
- 10. PAGO (Periodic Occupational Health Surveillance): Ministry of Social Affairs and Employment. https://www.arboportaal.nl/onderwerpen/arbobeleid/pago (2 January 2023, date last accessed).
- 11. Sluiter JWA, Hulshof C.. Guideline Preventive Medical Examinations of Workers. PMO: Leidraad voor preventief medisch onderzoek van werkenden | NVAB, 2013. https://nvab-online.nl (2 January 2023, date last accessed). [Google Scholar]
- 12. Los FS, de Boer A, van der Molen HF, Hulshof CTJ.. The implementation of workers’ health surveillance by occupational physicians: a survey study. J Occup Environ Med 2019;61:e497–e502. [DOI] [PubMed] [Google Scholar]
- 13. Schulz KF, Altman DG, Moher D.. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Br Med J 2010;340:c332. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Michie S, van Stralen MM, West R.. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 2011;6:42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Los FS, van der Molen HF, Hulshof CTJ, de Boer A.. Supporting occupational physicians in the implementation of workers’ health surveillance: development of an intervention using the behavior change wheel framework. Int J Environ Res Public Health 2021;18:1939. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Rackham N. SPIN—Selling. 1st edn. London: Routledge, 1995. [Google Scholar]
- 17. Rollnick S, Miller WR.. What is motivational interviewing? Behav Cogn Psychother 1995;23:325–334. [DOI] [PubMed] [Google Scholar]
- 18. Hugenholtz NI, de Croon EM, Smits PB, van Dijk FJ, Nieuwenhuijsen K.. Effectiveness of e-learning in continuing medical education for occupational physicians. Occup Med (Lond) 2008;58:370–372. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Hendriks AM, Jansen MW, Gubbels JS, De Vries NK, Paulussen T, Kremers SP.. Proposing a conceptual framework for integrated local public health policy, applied to childhood obesity—the behavior change ball. Implement Sci 2013;8:46. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Peabody JW, Luck J, Glassman Pet al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med 2004;141:771–780. [DOI] [PubMed] [Google Scholar]
- 21. Ram P, van der Vleuten C, Rethans JJ, Grol R, Aretz K.. Assessment of practicing family physicians: comparison of observation in a multiple-station examination using standardized patients with observation of consultations in daily practice. Acad Med 1999;74:62–69. [DOI] [PubMed] [Google Scholar]
- 22. Axboe MK, Christensen KS, Kofoed P-E, Ammentorp J.. Development and validation of a self-efficacy questionnaire (SE-12) measuring the clinical communication skills of health care professionals. BMC Med Educ 2016;16:272. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc 2016;9:211–217. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Ammentorp J, Thomsen JL, Jarbøl DE, Holst R, Øvrehus ALH, Kofoed P-E.. Comparison of the medical students’ perceived self-efficacy and the evaluation of the observers and patients. BMC Med Educ 2013;13:49. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Binks S. Testing enhances learning: a review of the literature. J Prof Nurs 2018;34:205–210. [DOI] [PubMed] [Google Scholar]
- 26. Larsen DP, Butler AC, RoedigerHL, 3rd. Test-enhanced learning in medical education. Med Educ 2008;42:959–966. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.