Enabling Effective Implementation of Occupational Safety and Health Interventions

Abstract

Background

The design, implementation, and evaluation are three important stages of occupational safety and health (OSH) interventions. Historically, there has been a tendency to prioritize implementation, often neglecting detailed design and rigorous outcome evaluation. Currently, much has changed, and contemporary approaches recognize the interdependence of these stages, considering them integral to the success of any intervention. This work presents a comprehensive procedure for implementing interventions, not only to ensure short-term effectiveness but also their long-term sustainability through continuous monitoring. The focus is on a national OSH project introducing a near-miss management system (NMS) in Italy.

Methods

Initial meetings were convened among project partners, complemented by interviews with diverse stakeholders, to plan implementation steps and test the NMS. Tailored questionnaires were designed for diverse stakeholder groups – initial promoters, company managers and employers, and employees – facilitating targeted implementation, and three case studies were started in Italian regions to assess the structured implementation, involving intervention promoters and collaborating companies.

Results

The primary outcome is the development of practical tools, specifically three questionnaires, which are considered valuable for establishing an effective human-centered implementation strategy, meticulously designed to facilitate ongoing monitoring of processes and continual enhancement of instruments intended for NMS integration within companies.

Conclusions

This work lays the foundation for successful NMS implementation in Italy and, although the outlined procedure had specific objectives, it also provides valuable insights applicable in enhancing the effectiveness and sustainability of interventions across diverse contexts. It underscores the importance of comprehensive planning, stakeholder engagement, and continuous evaluation in achieving lasting OSH interventions.

1. Introduction

The effectiveness of Occupational Safety and Health (OSH) interventions is a critical fact in intervention development and discussing with academics and practitioners this is felt as a huge issue [[1], [2], [3]]. A discussion has emerged in the academic literature regarding how to make interventions more effective and practitioners have started to assess various field interventions from an operational standpoint. Nevertheless, do we actually have evidence of the interventions’ effectiveness? Deriving generalizable insights from the literature has proven challenging, primarily due to unique dynamics [[2], [4]]. Several systematic literature reviews have analyzed interventions in an attempt to uncover shared patterns and strategies for improving effectiveness [1,[5], [6], [7]]. Unfortunately, many of these reviews failed due to the high variability of the environment, rendering comparisons too heterogeneous to be meaningful [1,6,7].

Given the above considerations, it is worth analyzing interventions not as black boxes but by taking into account the various factors that influence them [8]. Some authors have emphasized the importance of delving into the mechanisms – what has or has not worked –of the interventions [1,5,9]. Contextual factors play a paramount role in all phases of the design, implementation, and evaluation of interventions [2], and assessing and managing these factors is essential to enhance the likelihood of success [10]. The selection of the most appropriate approach is typically contingent on the specific subject matter and environment [11]. Furthermore, considering the promotion of human-centric technological development, it becomes crucial to move from interventions that prioritize economic maximization towards ones that aim to harmonize the human, social, and environmental implications with economic and financial factors [4].

The development of each intervention comprised three equally important stages: design, implementation, and evaluation [3,8,12]. In the past, the implementation process has received more attention and OSH interventions tended to neglect both detailed design and rigorous evaluation of the outcomes achieved [8]. Currently, much has changed, and the implementation of interventions often goes with the discussion of the other stages – design and evaluation – recognized as necessary for the success of the intervention. By looking at the interventions’ development in the literature, some macro-areas of documents are distinguishable.

One common section is dedicated to the intervention design (e.g., [[13], [14], [15], [16], [17], [18]]), with a specific section dedicated to the participatory approach, which can also apply to all the stages of the intervention development (e.g., [[19], [20], [21], [22], [23]].). A shared opinion among researchers identified, participation is a valuable element as engaging in participatory organizational interventions (POIs) would ensure higher results in improved employee well-being and overall intervention effectiveness [[24], [25], [26], [27]]. POIs are tailored to target specific contexts, proactively address issues at their root causes – not reacting after implementation when it is already overdue – and take into account the relations among various organizational levels. This approach has the potential to identify areas requiring change at multiple levels [26,28], thereby facilitating a co-creation process in which participants contribute value from the outset, rather than being mere recipients of it [29]. Another group of authors consider the context around intervention development by considering factors – positively or negatively – affecting their development and sometimes proposing a framework for the factors’ assessment (e.g., [30,31]) and evaluation (e.g., [32]). Finally, a substantial number of works refer to the intervention’s effectiveness theoretically by reviewing, for example, past interventions (e.g., [[33], [34], [35]]) or practically by providing tools for the evaluation of specific interventions (e.g., [36,37]). As proof of the topic’s relevance in intervention development, a few authors have also brought in theories and methodologies applied in other domains to better explain, implement, and evaluate OSH interventions (e.g., [[38], [39], [40]]), which proves that OSH research is moving toward more theoretical concepts for higher generalizable and effective results.

According to the literature, all the stages of intervention development have been investigated with more focus on the practical implementation of the interventions. However, as we move to the real field, there is still much to do, and leading interventions to higher effectiveness is a critical issue as they are still rarely properly designed and monitored over time and intuitively developed by OSH managers, who often select the most common and not the most effective measures [41].

In the context of technology development, the Value-Sensitive Design (VSD) is an approach that prioritizes human values in the design process [42], which could prove highly beneficial in the OSH field for an effective design of interventions. VSD enables designers and developers to consider the values of direct stakeholders, i.e., the users, as well as indirect stakeholders, i.e., who may be impacted by the system’s use [43]. The values’ hierarchy builds on this but works to facilitate the translation of abstract human values into tangible design requirements. Hence, the VSD consists of three iterative phases – conceptual, empirical, and technical investigations – like the inspiration, ideation, and implementation phases of the design thinking approach [44]. The conceptual phase identifies stakeholders, and understands their values, while the second phase of empirical investigation ensures that design choices align with stakeholders’ needs. Technical investigation evaluates the feasibility of aligning designs with human values within technological constraints.

As a result, this study, leveraging evidence from the literature for intervention design, shows a procedure for the implementation of an intervention, detailed below, with the primary intent to propose a structure that not only ensures the intervention’s effectiveness in the short-term but also sustains it over time, by keeping it continuously monitored, including its outcomes and retention [45]. According to Chambers et al [46], the sustainability of interventions relates “to the extent to which these interventions can continue to be delivered, while institutionalized within settings, and having the necessary capacity built to support their delivery.” The chosen intervention is particularly relevant to the aim of this research as a national OSH intervention for introducing a Near-Miss Management system (NMS) in the Italian country has been taken as a reference. Near-misses are events that could have resulted in an injury but luckily did not. Analyzing them can be an effective strategy for identifying risk factors and preventing accidents. It was funded by the Italian national insurance institute through a project called CONDIVIDO which stands for “developing an intelligent tool to support virtuous ecosystems for knowledge and sharing management of near-misses in industrial sectors”. It has been developed in collaboration with different stakeholders throughout the Italian territory: two Italian research centers and three local health and safety departments (ASLs), which are the centers of public healthcare in Italy under the National Healthcare Service. In brief, this intervention hash provided Italian companies, especially small and medium-sized enterprises (SMEs) as well as large companies, a practical tool, an online platform, for the daily collection, analysis, and sharing of near-misses (website: https://www.condivido-nearmiss.it). The CONDIVIDO project fits into a broader framework of actions to improve injury and accident prevention in the workplace. Through the analysis of near-misses, risk factors are identified, and new effective tools are developed to support companies in workplace safety management.

With regard to the implementation phase, this intervention has laid the foundation for a structured development and spreading of the building of the NMS. The true significance of the CONDIVIDO project goes beyond the NMS tool itself as it resides in the manner in which it has been developed for effective implementation. This becomes particularly important if the typical outcomes of ordinary interventions are considered. Moreover, to let this project not die once the funds ended, a second project had been promoted and has already started, namely PMP 5.0 (Prevention Plans) “development of technical and organizational tools to support prevention interventions for the development of resilient network ecosystems”.

Considering such background, this study shows a way of implementing effective interventions by showing which processes, activities, and stakeholders have been selected for the implementation of the NMS in the Italian country. Although the considered intervention had a defined focus, the proposal of a structure for its implementation offers insights into how to develop effective OSH interventions and promote their sustainability in diverse settings.

Finally, the following sections are structured as follows. Section 2, starting from the mentioned background, details the methods applied to set the procedures for the implementation of the national supportive NMS for the Italian industrial sector. Section 3 proposes activities set for the implementation of the intervention. Section 4 discusses previous findings. Section 5 draws conclusions by discussing the implications and future development of the proposed procedure.

2. Materials and methods

Consistent with the aim of this research, the VSD approach was taken as a reference by iteratively applying conceptual, empirical, and technical investigations and actively involving several stakeholders. Several meetings among the project’s partners have taken place to define the steps and activities for the implementation phase of the CONDIVIDO project. The first implementation had the intent to test the tool developed for near-miss assessment, and at the same time, test the network of stakeholders for its deployment and development, what we call the ‘ecosystem’. The implementation was thought to bring an effective intervention that could last over time. The partners of the project were all involved in the definition of the implementation phase, which also directly involved some of them as promoters of the activity (e.g., ASLs).

Hence, the applied methods for setting the implementation of the CONDIVIDO project are based on collaboration, interaction, and engagement of experts from project partners. Collaboration is crucial to ensure that all involved parties have an active role in the decision-making process and the definition of intervention strategies [25]. Collaboration facilitates the exchange of knowledge and optimization of available resources. Partner interaction gives the provision of diverse and complementary expertise, enabling a more comprehensive approach to challenges related to near-miss management in workplaces. Each partner contributed with their specific experiences and knowledge to define a set of effective tools for project implementation. The involvement of people experts in various industrial sectors ensured a multidisciplinary perspective in the analysis and definition of solutions, considering different viewpoints and evaluating sector-specificities to ensure the effectiveness of measures and the achievement of health and safety goals in the workplace. Involvement is crucial at every stage of the CONDIVIDO project, from the initial design of the project to its actual implementation. This process (of designing the intervention) began with three workshops in the initial phase that involved many stakeholders in different regions of the Italian country. Active participation during these meetings contributed to more effective design.

During that phase of the project,–the definition of the implementation, active participation and continuous dialogue were ensured by the involvement of all project partners in the definition of the methods, and tools, for the startup of the implementation. The steps for the implementation have been defined as well with tools, specifically a questionnaire ready for submission to collect the voice of the developers and users of the online platform (website: https://www.condivido-nearmiss.it). Questionnaires provide an effective means to collect specific feedback, allowing targeted adaptation of the designed NMS to evolving needs, thus making the intervention effective – and sustainable – over time. In addition to questionnaires, interviews served as pilot studies to test the feasibility of the developed questionnaires to understand whether the required information was clear and collectable. The inclusion of interviews has been considered a valuable element in the engagement process because they provide more detailed insights, address complex issues, and gather nuanced opinions. The questionnaires were developed for different stakeholders involved in the project. Three main prominent roles in the implementation of the project were selected:

• •the initial promoters, the ASLs;
• •the managers inside the companies implementing the developed NMS;
• •the users (employees) of the online platform because they daily use the online platform for near-miss management.

Details on the questionnaires are provided in the next section (Results).

To this end, three case studies were chosen for the initial assessment of the structured implementation. The testing phase and the identified methods were applied to three specific Italian regions. These regions were selected because there the ASLs, involved in the project CONDIVIDO as initial promoters, operated. These ASLs, in turn, collaborated with various stakeholders, including companies and other entities such as employers’ associations within their respective areas of influence, for the validation and application of the networked ecosystem and online platform.

3. Results

The following results are presented with the intention of setting the stage for an effective implementation phase, which has been designed to be continuously monitored over time and ensure the ongoing improvement of the tools, developed as part of the CONDIVIDO project, for the introduction of the NMS inside companies. To achieve this objective, three questionnaires have been devised. They serve the purpose of collecting specific information regarding company involvement, training, platform implementation, mentoring during the trial phase, and feedback and suggestions for improvement (Fig. 1). Therefore, while certain sections of these questionnaires are tailored to the project’s content, they can also be used as valuable guidelines for monitoring the effectiveness of the implementation of other OSH interventions in different contexts.

Fig. 1.

Tools for a controlled implementation phase.

The first questionnaire is directed toward the promoters of the intervention, ASLs in the CONDIVIDO project, and concerns the planning of the activities for the implementation of the NMS within participating companies. It aims to assess all phases of NMS introduction, following a plan-do-check-act approach for continuous improvement, which implies proposing a change in a process, implementing it, measuring the results, and taking appropriate action. The questions pertain to the launch, organization of informational and training events, testing of the online platform, and gathering feedback and proposals for improvement.

The second questionnaire addresses the OSH managers inside companies, who were identified as responsible for managing near-misses and the online platform proposed by the CONDIVIDO project. The questionnaire addresses general company information, the presence of a pre-existing NMS, and assesses how the new activity of near-miss management would be performed referring to the involvement of stakeholders, the reporting and information-gathering phase, and finally, the data analysis and processing phase. This study leverages a previous work, Vitrano et al. [13], referring to the CONDIVIDO project, which defined NMS based on processes, scenarios, and stakeholders. Fig. 2 reports the NMS and this questionnaire particularly targets the first two phases occurring at the company (local) level, as illustrated in the operational management area.

Fig. 2.

The near-miss management system [retrieved by Vitrano et al. [13].

The third questionnaire is intended for final users of the tools developed for managing near-miss management. The literature suggests various indicators for evaluating user experience but there is no shared consensus on the best set dependent on several factors [47]. In this study, we adopt the scales recommended by the user experience questionnaire, which covers both classical usability aspects (efficiency, perspicuity, dependability) and user experience aspects (originality, stimulation) [48]. The questionnaire includes all these aspects except for originality, and specifically, the questions investigate the efficacy of the online platform in terms of timing for filling out the questionnaire, availability of the platform, clarity and completeness of the information, usability of the application, and overall user satisfaction.

4. Discussion

This study leverages a previous work , Vitrano et al. [13] [reference omitted, Safety Science 2023] referring to the same project where the NMS were defined according to the processes, scenarios, and stakeholders. Fig. 2 reports the NMS and will support the following discussion of the results presented in this work. The main pillars that guided the design of the NMS also proved to be valuable in shaping the implementation phase and the theoretical and practical implications will be discussed below.

In terms of theoretical implications, it is crucial to prioritize an accurate design, as it directly impacts the effectiveness of an intervention [3,13]. The VSD approach was chosen as a reference because it ensures alignment between design choices and stakeholders’ needs. That is why the planning of the implementation phase took some time and involved many stakeholders across the Italian country. Related to this, the importance of collaboration in questionnaire design has emerged; indeed, the participation of workers and other stakeholders in the decision-making process and in promoting safe practices contributes to a safer work environment and the prevention of accidents [[24], [25], [26]]. The CONDIVIDO project adopted an iterative and participatory approach, involving various experts to ensure the representation of diverse points of view and horizontal decision-making among the people involved. This enabled the integration of different knowledge and improved the effectiveness of the questionnaires used. The questionnaires were indeed designed to be flexible and adaptable to different situations. For this reason, they offer multiple response options and adapt to companies’ specific characteristics. This is crucial to ensure the proper functioning and high adaptability of the NMS. During the implementation of the tools, collecting feedback from participants and improvement proposals is considered fundamental, as this is valuable in identifying critical issues and areas for improvement and making necessary changes accordingly.

From a practical standpoint, it was noted that the strategic area of control (Fig. 2) plays a fundamental role also in ensuring the effectiveness of the questionnaires. To ensure the effective and sustainable operation of the entire NMS over the long-term, questionnaires have been developed for various stakeholders, including the initial promoters, managers, and users (employees) involved in the NMS. This approach enables proper control/effectiveness of the activities and not only the efficiency of the tools developed for end-users. Continuous evaluation of the implemented NMS, e.g., through questionnaires, is crucial to assess pre- and post-intervention results and make any necessary adjustments or improvements over time.

Furthermore, the importance of bidirectional exchange between the front-end and back-end processes of the NMS (Fig. 2) is emphasized. The double information flow between the design and maintenance and operational management areas plays a pivotal role in ensuring the sustainability and resilience of the entire system. The processes at the back-end provide essential support to the front-end (user) processes, and in turn, the front-end processes furnish the system with fresh data, which drives continuous improvement through back-end processes. Indeed, the primary objective of the front-end area is to provide evidence of the positive impact of the system on end-users, simultaneously encouraging more users to join and increasing overall system utilization, which is also a goal of the back-end area. In the implemented NMS, even companies that do not directly report near-misses still benefit from access to the aggregate data. This, in turn, can motivate new companies to participate, thereby expanding the user base that contributes data to the system, creating a self-reinforcing cycle.

As a result, questionnaires were both developed for front- and back-end processes to ensure proper control and effectiveness of the activities implemented for the NMS. Specifically, questionnaires directed at companies allow for evidence collection of the positive impact of the NMS and encourage greater participation among them. They also are structured to gather useful information for the continuous improvement of the built NMS. Besides, the questionnaire for end-users has been intentionally devised to actively involve employees in the process of near-miss reporting and analysis as active employee engagement is essential for improving health and safety conditions at work [27]. Employee participation in the decision-making process and reporting of hazardous situations helps identify and resolve issues promptly. This active participation can be encouraged through incentive programs, training, and effective communication.

In conclusion, considering the entire CONDIVIDO project, the value of a multidisciplinary approach to near-miss management has emerged. In any OSH intervention indeed, collaboration with partners from different industrial sectors provides unique contributions and perspectives that enrich the decision-making process and the definition of solutions. This multidisciplinary approach promotes the consideration of different viewpoints and the integration of diverse skills and knowledge to address health and safety aspects more comprehensively. For example, within the CONDIVIDO project, the collaboration between universities, national authorities, and on-site OSH practitioners (e.g., independent consultants, from employers’ associations) has facilitated a comprehensive understanding of the problem. This included a blend of both theoretical and practical perspectives, with the NMS being meticulously designed to address not only theoretical considerations but also to effectively tackle practical challenges that emerged during extensive discussions with field-based OSH practitioners. This exchange of know-how will indeed contribute to ensure a more complete and in-depth spread of the devised NMS.

5. Conclusion

This study sets the basis for implementing in a sustainable, effective, and efficient way of the NMS defined by the CONDIVIDO project. Although the considered intervention had specific objectives, the proposal of a structure for its execution provides valuable insights into the development of effective OSH interventions across various contexts and the enhancement of their sustainability.

The CONDIVIDO project, in full, demonstrates the fundamental importance of collaboration and the involvement of relevant stakeholders in the design process to ensure the sustainability and effectiveness of the intervention in near-miss management. The multi-phase design process developed within the CONDIVIDO project has proven to be effective in promoting the integration of diverse perspectives and knowledge, thereby enhancing the effectiveness of the intervention. Furthermore, the monitoring in all the phases, implementation included, has emerged as crucial, playing a fundamental role in keeping track of the effectiveness of the built NMS and supporting adjustments and improvements over time. Continuous evaluation is essential to assess the implementation of the intervention and ensure long-term success. The questionnaires presented in this study are valid tools for gathering specific information from various parties involved in the NMS, and the testing of these tools has already begun with the launch of three ‘ecosystems’ in the territorial realities of the project operational units. Overall, the CONDIVIDO project has demonstrated, emphasizing the importance of stakeholder involvement, that collaborative design, continuous evaluation, and flexibility are essential for the sustainability and effectiveness of an OSH intervention as shown in this case for the NMS. Therefore, these findings provide valuable insights for the development of OSH interventions in different contexts.

On the other hand, it is important to recognize the limitations of this study and consider how these may influence the interpretation of results. Some limitations pertain to the individuals engaged in the NMS design process and the development of the questionnaires as well. These participants were chosen by the three ASLs involved in the project, were implicitly active promoters of OSH inside companies, and were also favorable to the introduction of near-miss management inside companies; consequently, their responses may be biased when considering the broader population of OSH practitioners and stakeholders. This implies that the current NMS might still fail to address certain crucial issues that were not encountered by the project’s original collaborators, which could become apparent once the NMS is made available to the wider interested audience. For the same reasons, relying on a selected sample of involved individuals rather than a larger and more diverse group, the questionnaires might fall short of capturing all the essential elements needed to evaluate and track effectively the progress of the intervention’s development.

Based on these limitations, several areas could be explored in future research developments. Conducting a longitudinal study that follows participants over time could provide a better understanding of dynamics and changes in the deployment of the NMS, and specifically, in near-miss reporting preferences. This could identify trends or changes over time and help develop more effective and sustainable near-miss management strategies. Additionally, integrating a qualitative approach into the study could allow for a deeper understanding of the motivations and factors influencing near-miss reporting preferences. This could include interviews or focus groups to explore participants’ thoughts, emotions, and perceptions in more detail, and certainly, assessing the long-term effectiveness of the built NMS, in terms of tools provided and established networks of stakeholders (ecosystems), which enable a comprehensive evaluation of the impact on performance and workplace safety culture. This could include collecting data on accident and injury trends over time, as well as measuring employee engagement and perceptions of workplace safety.

Funding

This research presents some of the results of the projects CONDIVIDO (BRIC ID 01/2019) and PMP 5.0 (BRIC ID 01/2022), designed and developed by Politecnico di Milano, Università del Salento and the Department of Occupational and Environmental Medicine, Epidemiology and Hygiene of INAIL (the Italian National Institute for Insurance against Accidents at Work) and granted by INAIL itself.

Conflicts of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

CRediT authorship contribution statement

Gaia Vitrano: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Supervision, Visualization, Writing – original draft, Writing – review & editing. Davide Urso: Formal analysis, Investigation, Writing – original draft. Guido J.L. Micheli: Conceptualization, Investigation, Methodology, Project administration, Supervision, Writing – review & editing. Armando Guglielmi: Formal analysis, Investigation. Diego De Merich: Formal analysis, Investigation. Mauro Pellicci: Formal analysis, Investigation.