Advancing safety analytics: A diagnostic framework for assessing system readiness within occupational safety and health

Abstract

Big data and analytics have shown promise in predicting safety incidents and identifying preventative measures directed towards specific risk variables. However, the safety industry is lagging in big data utilization due to various obstacles, which may include lack of data readiness (e.g., disparate databases, missing data, low validity) and personnel competencies. This paper provides a primer on the application of big data to safety. We then describe a safety analytics readiness assessment framework that highlights system requirements and the challenges that safety professionals may encounter in meeting these requirements. The proposed framework suggests that safety analytics readiness depends on (a) the quality of the data available, (b) organizational norms around data collection, scaling, and nomenclature, (c) foundational infrastructure, including technological platforms and skills required for data collection, storage, and analysis of health and safety metrics, and (d) measurement culture, or the emergent social patterns between employees, data acquisition, and analytic processes. A safety-analytics readiness assessment can assist organizations with understanding current capabilities so measurement systems can be matured to accommodate more advanced analytics for the ultimate purpose of improving decisions that mitigate injury and incidents.

1. Introduction

Safety incidents have a major impact on the workforce and organizations (Bureau of Labor Statistics [BLS], 2018). There are approximately 2.8 million injuries and illnesses within the private work sector in the United States annually (BLS, 2018, 2019, 2020). Costs associated with these work injuries go beyond workers’ compensation. The National Safety Council (2019) estimates that work injuries cost companies $161.5 billion annually, from expenses such as wage and productivity losses ($50.7 billion); medical expenses ($34.3 billion); and administrative expenses ($52.0 billion).

Initiatives and regulations introduced by agencies in the United States, such as the Occupational Safety and Health Administration (OSHA), have done much to reduce injuries and illnesses (OSHA, 2012). Organizations are required to notify OSHA when injuries and fatalities occur in the workplace and store large amounts of data that can be presented as proof of compliance with regulations when necessary (OSHA, 2001). As such, many industrial organizations have large amounts of safety data available that could be analyzed statistically.

Further, current analytics research within occupational safety has demonstrated the predictive capabilities of (a) demographic information, such as age, gender, and worker experience (Chi et al., 2014; Stewart, 2013), (b) job-related information such as industry, equipment, job risk, and training (Lingard et al., 2017), and (c) behavioral information, such as the use of personal protective equipment, hazard identification, and housekeeping (Mistikoglu et al., 2015) in predicting adverse safety outcomes. Some of these models are quite sophisticated and have a high degree of accuracy. For example, Carnegie Mellon University (Predictive Solutions, 2012) incorporated 112 million safety observations and safety incident data from over 15,000 work sites to predict incidents with accuracy rates as high as 80–97 percent and R² as high as 0.75 between actual and predicted incidents.

Using predictive models such as these, organizations can direct scarce resources to the locations and work teams that are at highest risk of safety incidents (Schultz, 2012). For example, Deloitte analyzed five years of data on injuries, demographics, production, operations, and weather from Goldcorp, a gold-mining company (Stewart, 2013). Statistically significant relationships were found between injury rates and compensation, age, job roles and other operational factors. Goldcorp used these insights to increase managerial training, write new policies, and focus supervisor support on employees with higher risk profiles.

Despite the demonstrated success of safety analytics, these applications remain the exception and not the rule. Organizations have made limited progress in developing analytics for safety (Tan et al., 2016), and many safety professionals lack an understanding of what is possible with existing data pools (Ferguson, 2018). In this paper, we provide a thorough primer on data attributes, analytic sophistication, and the application of data strategies to Occupational Safety and Health (OSH). We then highlight system requirements for advanced analytics and describe safety-specific challenges. In doing so, we develop a framework for assessing current safety analytics capabilities, maximizing analytic value, and making prioritization decisions for investment in system improvements.

1.1. Features of big data

Safety professionals have more data than ever at their fingertips (Canitz, 2019). In addition to artifacts of employee reporting, there is a flood of data being created by new safety devices that provide detailed information on people, machines, and the environment (Ferguson, 2018). These increasingly large datasets approach the size of big data, defined as a lot of different types of data handled in new ways, or large datasets that tax system capacity (Dastjerdi et al., 2016). Big data is a term used ubiquitously within the analytics zeitgeist, yet it is distinguishable from other data forms and statistics through five defining features: volume, velocity, variety, value, and veracity (Ramadan, 2017). While big data often refers to data that is unmanageable by traditional architecture (Goel et al., 2017), these five features (see Fig. 1) can be useful in identifying the characteristics of any size data that contribute to sophisticated analytical output.

Fig. 1.

Five Characteristics of Big Data.

Overall, analytics within OSH will be more successful (e.g., value) in identifying trends that lead to risk mitigation when there is more (e.g., volume) high quality (e.g., veracity) data of varied type (e.g., variety of data type) over time, considering both archival and continuously updated current data (e.g., velocity), originating from safety as well as other organizational units (e.g., variety of source; quality control, production, human resources).

Volume is defined as the quantity and size of data that are generated and stored. Increasingly, OSH is collecting data in volumes that are difficult or impossible for traditional databases and statistical methods to handle. Text analysis in R, for example, can only analyze a certain number of observation comments. For less advanced analytics, such as identifying trends by frequency count, volume is typically less of a concern. However, as OSH increases exponentially in data volume, constraints to conducting analytics may include man hour costs for personnel and technical expertise necessary to collect, aggregate, store, and analyze the high-volume data. Thus, as volume increases, OSH may need to invest in a dedicated position or allot significant time for incumbent staff to manage the safety data.

Velocity is defined as the speed at which data are generated and processed. In the past, high velocity was less of a challenge, as incident reports happened relatively infrequently, and other data collection mechanisms required manual entry. There are new technologies within safety (e.g., wearable ergonomic belts), however, that provide constantly streaming real-time information via sensors. Though high velocity data collection is not necessary for all analytics, data that are collected more frequently will allow for more precise temporal output of predictions, as continuously updating information makes real-time predictions possible. High velocity can lead to high volume, however, and OSH professionals will face challenges in maintaining pace in decision-making with the continuously updated information. In addition, practitioners will need to make decisions about timing of analyses (i.e., frequency of insight dissemination to managerial personnel) and retention periods.

Variety is defined as the heterogeneity of data type and nature, including the different sources of information across processes and functions, forms of data (e.g., electronic monitoring, fillable forms, surveys), and types of database structures utilized within the organization. This wide variety of data can make aggregation and analysis challenging. For example, data may be structured (e.g., text observations in a relational or tidy format), semi-structured (e.g., csv files) or unstructured (e.g., images and video). OSH faces obstacles in gaining or utilizing the technologies and infrastructure that allow for this data to be stored, aggregated, and analyzed despite these different formats.

The final features of big data—veracity and value—are essential characteristics of all data, not just big data (Ramadan, 2017). Veracity is defined by three dimensions of information quality: data objectivity, truthfulness, and credibility (Lukoianova & Rubin, 2014). The validity and reliability of data are rarely formally assessed yet analytical output is only as accurate or as trustworthy as data inputs; a concept succinctly summarized by the popular adage “garbage in, garbage out.” Veracity impacts data quality and analytic value, and big data is often plagued by biases, ambiguity, or error (Lukoianova & Rubin, 2014). Safety data relies heavily on employee and front-line supervisor reporting where rater errors in reporting can occur due to concerns over negative outcomes on their jobs such as discipline for rule violations, extra personal attention by management, or added cumbersome safety rules and equipment. These workplace variables may cause a degree of “pencil whipping” (Ludwig, 2014) whereby information entered into safety reports are underreported or not factual (Probst & Graso, 2013). In other instances, employees may not have the technical expertise to correctly categorize safety events and/or adequately describe the event. These and other rater errors can impact the validity and reliability of the data. Luckily, big data is typically of such volume that trends are often still useful. There are also methods of managing data veracity, such as identifying suspicious outliers, automated detection and sorting of quality using machine learning, or crowdsourcing techniques (see Assiri, 2020, Table 1).

Table 1.

Variable List consisting of the 26 important variables that may impact safety outcomes, along with their definitions.

Variable	Definition
Safety Metrics
Safety Metrics Hazard ID	The metrics surrounding the reports made by employees of hazardous environments that could lead to safety incidents.
Audits/Inspections	These metrics track the frequency of audits and inspections of workplaces and workgroups to measure if the work is being done safely and if the environment is safe.
First Aid/Minor Injury Reporting	These metrics track minor injuries, such as small cuts, trips, or falls that may need first aid.
Near Miss/Close Call Reporting	This metric measures the number of instances where employees report that a safety incident did not occur, but almost did.
Environmental	Environmental metrics track adverse conditions such as weather, heat, wind, storms, etc.
Behavioral Observations	Metrics collected on behavioral observations may include video or checklists, either paper and pencil forms or electronic.
Fatigue	Measures tracking the point at which employees are likely to have a safety incident due to strain and fatigue caused by the nature of their work/workload.
Safety Participation	Measures of safety participation track indirect and voluntary behaviors that contribute to a safe work environment, including participating in shift discussions, safety meetings, close call/near miss reporting, or investigating incidents.
Safety Culture	Safety culture measures the extent to which employees and managers think about, talk about, and are committed to having a safe workplace.
Production
Volume Trends	The quantity of product output at a given time.
Scheduled Events	These metrics relate to changes to the production that managers may make, such as switching a machine from one product to another.
Staffing Loads	These metrics relate to accounting for people working at a given time, and how much those employees are working.
Calendar Events	These are events that may center around a date on a calendar, like a holiday or vacation.
Quality	A measure of excellence or a state of being free from defects, deficiencies, and significant variations. These metrics may also measure the opposite, such as errors or reworks.
Cost/Budget	These metrics would measure estimated costs, revenues, and resources over a specified period.
Maintenance
Failures (Equipment)	These metrics track what machines have failed, how often, or why.
Action Item Backlog	These metrics measure unfinished tasks that need to be completed.
Preventative Maintenance	Data centered around equipment and facilities by tracking systematic inspection, detection, and correction of incipient failures either before they occur or before they develop into major defects.
Procedures
Change Management	These metrics could track what occurs after a change in procedure, product, or could track leadership changes over time. These organizational metrics may cross-functions and help to provide a strategic lens.
Human Resources
Safety Knowledge	Knowledge of how to perform tasks safely; affected by T&D practices.
Job Attitudes	Job satisfaction, organizational commitment, etc.
Safety motivation	Willingness to exert effort to enact safety behaviors.
Turnover	A measurement of the number of employees who leave an organization during a specified time period.
Absenteeism	The measure of the number of employees that are absent from their scheduled shift.
Employee Characteristics	Personality characteristics (e.g., Big 5, locus of control, risk taking propensity, demographics of the workforce that is being measured (e.g., tenure, sex, age, education).
Culture	The measure of the cultural norms of the workforce, including what they prioritize, talk about, and behave.

The last feature of optimal data is also related to quality, but for outcomes. Value is defined as the measurable impact of data-driven insights. Value can thus refer to both inputs and outputs; data itself (input) has a value in how it can contribute towards data-driven insights, and those insights (output) have value when actions derived from data have a measurable impact (e.g., reliably predict risk profiles and mitigate injury). Each piece of information is more or less important to an analysis in terms of its relevancy to the question of interest, as well as how much value the information adds to the output statistically (e.g., increases in R² or explained variance). For example, within OSH, the data collected from smart wearable ergonomic belts are important in determining risk profiles for back injuries, and likely make up the bulk of the necessary information to determine risk factors, along with behaviors and equipment associated with the task. In contrast, hazard identification data would contribute to a risk profile for the same job but add less value to the prediction of back injuries. The “value” of analytics is thus created when the right objective and credible data (e.g., the sensory information from smart ergonomic belts) drives action (e.g., new standard operating procedures or targeted observations) that has measurable impact (e.g., fewer back injuries).

While these five features of big data are typically used to describe the challenges of data management on a huge scale, we argue that these characteristics can also be used to determine the level of readiness of smaller OSH datasets to successfully engage in advanced analytics. In addition, there exist common obstacles (e.g., difficulty in aggregating data of different forms) and techniques for mitigating these challenges (e.g., ‘data lake’ management systems for unstructured data). This comparison between big data characteristics and analytics within OSH may also aid organizations in determining the level of statistical methods or analytics that organizations can aspire to, given the constraints of their data characteristics.

1.2. Levels of analytics

There are five levels of analytics that offer increasing degrees of sophistication in terms of their ability to drive improved decision-making (Fred & Kinange, 2015): (a) descriptive analytics, (b) diagnostic analytics, (c) predictive analytics, and (d) prescriptive analytics, and (e) cognitive analytics (Canitz, 2019; Lepenioti et al., 2020). Descriptive analytics answer questions about what has happened in the past. Data are analyzed for characteristics and relationships through data visualizations and statistics such as sums, means, and averages (Huang et al., 2018). In OSH, descriptive analytics may look at the number of behavioral observations or equipment inspections in a month and distribution curves across different departments in the company. Dashboards and reports are also a form of descriptive analytic technique.

Diagnostic analytics provide clues about the reason for such past occurrences. These types of analytics (e.g., correlational analysis) describe relationships between variables to provide context. Diagnostic analytics use historical and past safety performance to identify reasons for the success or failure of initiatives—or explanations for specific outcomes—by investigating relationships, outliers, and sequences (Huang et al., 2018). An example of a diagnostic analytic technique in OSH would include a root cause analysis (Canitz, 2019). Safety data analytics at this level has focused on correlates to workplace injury such as external pressures, internal social context, and organization characteristics including job demands (e.g., environmental conditions, scheduling and workload, physical job demands, and the overall complexity of work; Barling et al., 2002), leadership (e.g., relationship with the manager, leadership style, trust, and accountability; Fogarty, 2004), and organizational commitment to safety (Fogarty, 2004).

Predictive analytics (e.g., regression analysis) attempt to prognosticate future outcomes to answer questions about which incidents are likely to happen and why. In addition to historical data, predictive analysis incorporates current information in an attempt to predict the likelihood of an event (Huang et al., 2018). Predictive analytics in OSH can include both short-term and longer-term predictions. For example, Lingard et al. (2017) used injury rate and other data from a large construction company’s safety program (e.g., toolbox talks, prestart meetings, safety observations, hazards reported, etc.) to identify a lagging predictive pattern between indicators. Their findings suggested that injuries were often followed by an increase in preventative measures (e. g., toolbox talks) which subsequently decreased until the occurrence of the next injury. In another example, a construction contracting company in Singapore used five different types of machine learning models on project- and safety-related variables to predict conditions under which no accidents would be likely to occur, conditions which may cause minor accidents, and conditions which may cause major accidents. The most effective of these, a random forest regression, a type of decision tree, predicted these conditions with 78% accuracy (Poh et al., 2018).

Prescriptive analytics take results from predictive analytics and utilize real-time data streams to provide more accurate guidance for decision-making (Mousanif et al., 2014). Safety data, mathematical formulae, safety rules, and machine learning are used in continuously updated models to suggest the most advantageous real-time decision options based on the identification of future opportunities or risks (Huang et al., 2018). For example, Ayhan et al. (2018) built a Viterbi algorithm variant (i.e., a dynamic programming algorithm that finds the most likely of events within a sequence) to reduce the chances of air traffic and collisions by predicting when an airplane trajectory may infringe on the protected zone of another aircraft. The prescriptive model detects and resolves these potential conflicts before aircraft even depart, resulting in safer operations, higher efficiency, higher capacity, and reduced air traffic controller workload.

Finally, cognitive analytics represent a further advancement in capabilities. Cognitive analytics refer to the integrative process of acquiring and transforming heterogeneous data sources into real-time actionable insights using models and systems inspired by the mechanisms and intelligence of the human brain (Gudivada et al., 2016; Osman & Anouze, 2014). By integrating substantial amounts of knowledge, these systems can reason, perceive and reflect on their capabilities and behavior, learn and improve from past experiences, plan, and respond quickly and efficiently to system shocks or surprises (Bannat et al., 2011). Cognitive analytics differ from the other levels because processes are fully automated and operate without human oversight (Gudivada et al., 2016). For example, cognitive architecture for unmanned surface vehicles (e.g., cars or nautical equipment) engages in human-like behavior and decision-making to prevent hazardous or catastrophic events to equipment and personnel, such as sensing the environment, maintaining safe distances from obstacles, and responding to stimuli appropriately and timely (Dreany & Roncace, 2019).

The levels of analytics require increasingly complex data characteristics. For example, prescriptive analytics require data that updates in real-time (e.g., high velocity), while descriptive analytics can use archival data to assess past trends (e.g., low velocity). The techniques implemented in the different analytic strategies described above, however, are varied and similar techniques can be used to answer questions in a descriptive or prescriptive phase. For example, one could use multiple linear regression to identify trends in existing data or to predict future patterns of injuries or near misses. There have been a variety of techniques implemented in the extant literature including classification and regression trees (Shirali et al., 2018), support vector machines (Sarkar et al., 2017; Sarkar et al., 2016), decision trees (Ajayi et al., 2020; Sarkar et al., 2020a), and k-nearest neighbors (Poh et al., 2018). Additionally, organizations may be able to create functional models yet have relatively low analytic capabilities. Successful analytics, then, is not about the sophistication of the analytic technique but is rather defined by the value of the outcome.

Not all firms who make large investments in analytics achieve these improvements in performance and value, however (Cosic et al., 2012). Indeed, the successful use of data analytics within occupational health and safety has been lagging other industries due to many unique challenges. One issue is that large amounts of safety data are reliant on worker observations and voluntary reporting. In many cases employees may be unwilling to report on or take the time to identify hazards. Additional obstacles such as disparate data platforms, data input errors, or the lack of support staff to organize, clean, and run statistics on the data (Gao et al., 2015), may also prevent organizations from running more advanced levels of analytics (Hadaya & Pellerin, 2010).

While many organizations may be interested in the valuable outcomes safety data analytics might provide, they may not know if their safety measurement systems are adequate for analytical techniques nor how they might mature these systems. Determining current capabilities will allow organizations to utilize the most efficacious level of analytics (i.e., the level of analytics that provides the most accurate and actionable output given system constraints) to inform decision-making and identify the components of the data analytic process that need to be further developed (Foreman et al., 2020; Lepenioti et al., 2020). Therefore, organizations need to be able to reliably predict their level of success at analytics (i.e., descriptive, diagnostic, predictive, prescriptive, cognitive). Accordingly, this paper describes a diagnostic framework that identifies data analytic system readiness.

2. Data analytics readiness assessment framework for safety

Readiness assessments evaluate an organization’s current stage of system and process maturity across individual components to provide recommendations for how to improve readiness (Klievink et al., 2017). Other industries have developed frameworks to assess data readiness, including healthcare (Snyder & Fields, 2006), education (Arnold et al., 2014), city planning (Barham & Daim, 2020), and supply chain operations (Nemati & Udiavar, 2013). Snyder & Fields (2006) developed a data readiness assessment in the healthcare industry to assist in the utilization of predictive variables to create safer environments by reducing medication errors and adverse drug outcomes. Another readiness assessment within healthcare, the Healthcare-Analytics Pre-Adoption Readiness Assessment Instrument (HAPRA; Venkatraman et al., 2016) guides organizations to self-rate their maturity across medical technologies, IT, user adoption of technology, data quality, and management. Within supply chain operations, analytic readiness assessments have also been useful to prepare data structures for analytics to improve efficiency, quality, and supply chain strategies (Nemati & Udiavar, 2013). Industry-specific readiness assessments provide value by aligning the focus of the readiness assessment to variables, relationships, and strategies/goals of importance in that industry.

Research has yet to develop an industry-specific framework for assisting OSH in identifying current capabilities for data analytics. The first task for such a framework is to identify common predictor and outcome variables within organizations. These variables will be the minutiae upon which larger organizational factors (e.g., data quality and norms of data collection) are assessed.

2.1. Safety variables for analytics

Organizations planning on engaging in analytics should create their own comprehensive list of available and aspirational metrics considering a variety of safety and other operational measures. These measures should be organized in meaningful predictive groupings suggesting hypothetical relationships in a path that may be predictive of OSH outcome variables (e.g., injuries).

Important variables within the safety function are typically dichotomized into two categories: leading and lagging. Safety leading indicators are defined as “proactive, preventative, and predictive measures that monitor and provide current information about the effective performance, activities, and processes of a safety management system that can drive the identification and elimination or control of risks in the workplace that can cause incidents and injuries” (The Campbell Institute, n.d., p. 2), whereas lagging indicators are reactive measures that track outcomes or events after they have already occurred (Usrey, 2016). Data collection coverage of leading and lagging indicators has been an established measure of OSH program effectiveness (Wurzelbacher & Jin, 2011).

Researchers focused on identifying current enterprise- and establishment-level analytics within safety found that most studies reported only on variables contained within injury reports (Foreman et al., 2020). These variables, collected after the occurrence of an injury, are classified as examples of lagging indicators (Lingard et al., 2017; Ludwig & Laske, 2020; Pawłowska, 2015). Lagging indicators are limited in their ability to predict future incidents because these types of reports generally have low velocity and variability due to the low base-rate occurrence of injuries. Additionally, prediction relies on causal mechanisms; to reliably say that a variable predicts an outcome, it must both covary statistically with that outcome (i.e., implying that the two pieces of information are related) and happen prior to the outcome. Although lagging indicators do describe pre-incident variables (e.g., causes of incidents) lagging indicators are post-incident artifacts (e.g., text descriptives of the accident) and cannot be used to predict risk or injury with great confidence, as they are not causally related to those incidents.

Some safety analytic studies reviewed by Foreman et al. (2020) assessed leading indicator data (e.g., near miss reports) that represent the current environment in predicting risk of incident (e.g., Dhalmahapatra et al., 2019; Sarkar et al., 2019; Sarkar et al., 2020b; Verma et al., 2014; Verma et al., 2017). Leading indicators include variables such as incident reports, surveillance, and surveys covering hazard identification, injury and first aid reports, near miss reporting, inspections/audits, and behavioral observations.

Safety data used for analytic studies should come from a variety of safety management system sources covering both leading and lagging indicator variables (Foreman et al., 2020). Numerous studies, however, have demonstrated the importance of variables generated outside of safety in business operations such as planning, maintenance, training, management (e.g., Bevilacqua et al., 2010), and culture (e.g., Goh et al., 2018). Moreover, studies on construction site planning (e.g., Elbeltagi et al., 2001; El-Rayes & Khalafallah, 2005; Sanad et al., 2008; Zhang et al., 2016), process safety (e.g., Baek & Choi, 2019; Li et al., 2019), equipment maintenance (e.g., Tan et al., 2011) and structural engineering (e.g., Zhao et al., 2020) delineate the relationship between operations and safety outcomes. Therefore, additional operational variables should be considered across common business functions such as production (e.g., quality, volume, cost, and budget), maintenance (e. g., equipment failure, action item backlogs, and preventative maintenance), procedures (e.g., change management and leadership) and human resources (e.g., safety attitudes, job attitudes, safety motivation & knowledge, turnover, absenteeism, hours worked, or employee characteristics; see Christian et al., 2009).

We created an example Risk Metrics Framework using a cause and effect diagram (Ishikawa, 1982) outlining potential groupings of relevant organizational metrics and safety outcomes such as incidents, near misses, and fatalities (see Fig. 2). Although safety analytics is focused on reducing these adverse outcomes, a full breadth of variables must be analyzed for safety analytics to discover novel relationships and provide impactful insights. Organizations seeking to understand their current capabilities should assess a variety of these metrics across analytics readiness factors.

Fig. 2.

Risk Metrics Framework of Variables Affecting Safety Outcomes.

These common safety and operational metrics are compiled into a variable list which summarizes common metrics within safety measurement and other operational areas that impact risk and injury. The variable list in Table 1 was devised by safety Subject Matter Experts (SMEs) from two multinational manufacturing organizations along with a comprehensive review of the literature (Forman et al., 2020) to provide an example of organizational metrics that might be considered for inclusion in safety analytics. As organizations engage in building analytics capability, they can use the proposed variable list as a comparison or antecedent in deciding which of their own metrics to build, collect, and analyze. However, note that this is not an exhaustive list and may not include metrics more common in alternative industries (e.g., construction) or culture- and regulation-specific variables commonly measured in organizations based outside of the United States.

While having a wide and comprehensive array of metrics and variables will lead to analytics that are better able to identify a more holistic network of risks, this coverage in itself does not contribute to analytic capability. Instead, there are several factors of the organizational system that ultimately contribute to data analytics readiness.

2.2. Analytics readiness assessment factors

Established analytic readiness frameworks (e.g., Arnold, et al., 2014; Comuzzi & Patel, 2016; Eybers & Hattingh, 2017) suggest that assessments focused on optimizing analytics must evaluate the adequacy of individual measurement data features (i.e., volume, velocity, variety, veracity, and value) as well as the organizational capacity to manage and analyze larger data sets. For example, in a readiness assessment developed for education, Arnold et al. (2014) emphasize the assessment of data itself but also include an evaluation of personnel expertise (e.g., ability), infrastructure (e.g., resources), and culture. In city planning, factors associated with the success of big data projects include people, technology, politics, and the organization (Barham & Daim, 2020).

Based on common themes across readiness assessments in other industries and the body of safety analytics literature, we propose organizing data and organizational factors related to safety analytics readiness into four key factors that affect an organization’s system capabilities: data quality, rules and operations, infrastructure, and safety measurement culture (see Fig. 3). Each of these readiness factors are comprised of several components which are evaluated individually; the combined assessment of these components provides evaluation of the higher order readiness factor. The readiness factors and their components are summarized in Table 2.

Fig. 3.

Applicability of Readiness Components to the Data Analytics System.

Table 2.

Summary of Readiness Factors with their Components and Definitions.

Readiness Factors	Components	Definitions
Data Quality	Validity	Refers to the extent to which measures accurately represents the “real world” phenomenon targeted
	Reliability	Refers to consistency of measurement across time and units
	Variability	Refers to the ability of a measure to detect differences across time and units
Rules and Operations	Adequate Coverage	Measures the extent to which the things we want to look for in relation to safety outcomes are represented in data collection
	Velocity	Refers to the frequency with which data are collected, entered, and updated in our databases
	Harmonization	Refers to having common demographics (e.g., who, what, where, when variables) across datasets that allow for data to be linked
Foundational Infrastructure	Personnel Infrastructure	Refers to the availability of key personnel with the necessary expertise to carry out technical processes of working with big data
	Centralized Database	Refers to the degree to which data variables are stored or can be readily combined into a central database
Measurement Culture	Employee Participation	The extent to which employees participate in the process and reporting of safety matters
	Management Concern	The extent to which managers support and encourage employees to participate: includes transparency about the purpose of reporting

department collects information electronically on the hours worked beyond a total of forty, whether that begins on the third day of the workweek or the fifth (i.e., format and scaling).

2.2.1. Data quality

High-quality data are necessary to realize valuable outcomes (i.e., outcomes which can improve decision-making) from analytics (Cai & Zhu, 2015). Unfortunately, archival databases of organizational safety measures are often created for a purpose other than conducting analytics and this can affect the quality of the data and analytics process. Many safety processes such as inspections collect high volumes of information but are only used to find equipment issues to fix. Human resources employment data, such as overtime, are collected for payroll but not used to assess the amount of time an employee is working and how it may relate to injuries. Such measurements were not designed to satisfy the quality requirements for the proposed analyses. Accordingly, the framework assesses the quality of the data across three components (i.e., validity, reliability, and variability) for its adequacy to conduct different levels of initial and advanced analytics.

Validity assesses the extent to which inferences from the data accurately represents the “real world” phenomenon targeted by the measurement (Jugulum, 2016; Sechrest, 2005). Validity can be a concern in the safety industry because many metrics are reliant on employee reporting, which may be affected by culture or biases (Salas & Hallowell, 2016).

Reliability refers to consistency of measurement across time and units. Disparate databases and collection methods may lead to decreased consistency in variables being named, defined, formatted, and scaled in the same manner across systems. For example, data collected on “overtime” may be consistent definitionally in that each measure contains information about time spent at work beyond a predetermined schedule of time (i.e., name and definition), but inconsistent in scaling when, for example, one department collects daily punch card information such as the minutes worked beyond eight hours in a day and another department collects information electronically on the hours worked beyond a total of forty, whether that begins on the third day of the workweek or the fifth (i.e., format and scaling).

Finally, data must contain enough variance to conduct statistical analyses. Variance is defined as the divergence of a set of measurements from the mean of the total sample within a variable. Standard deviations, mathematically calculated as the square root of variance, are commonly used in statistics to describe dispersion or how spread out the observations are (i.e., data variability). Another way to check dispersion is to assess kurtosis. If range restriction occurs, causing an abnormal distribution shape (e.g., leptokurtic distribution shapes contain greater amounts of measures very close to the mean) with a very low standard deviation, then the chance of finding a correlation between variables becomes limited (Type II error). Many statistical analyses based on the general linear model rely on adequate variance in distributions in order to find relationships.

A few variables in safety, such as outcome variables measuring injuries, naturally have very little variation due to the low frequency of occurrence. In addition, there may be restrictions on the variance of predictor and precursor data because measures require individuals to voluntarily recognize the event(s) and record/enter the data. For example, the reporting of minor injuries, close calls, and at-risk behaviors may be truncated because workers (a) don’t perceive their importance, (b) forget to stop their work to report the event, (c) perceive personal negative outcomes in retaliation for reporting the information, or (d) engage in extreme rater response styles like pencil whipping (Ludwig, 2014). Finally, in reality most variables assessed using safety measures are regarded as safe, which reduces the variance typically introduced by risk. For these reasons, the variability in safety measurement is often reduced to a small standard deviation.

2.2.2. Rules and operations

Rules and Operations refers to organization-level processes of data collection (Comuzzi & Patel, 2016; Eybers & Hattingh, 2017; Gao et al., 2015). This readiness factor is assessed across three levels. First, in order to run analytics, the organization must have access to a range of variables that can be used to predict outcomes (i.e., adequate coverage). Second, the data must be collected with consistent and common variables so that un-centralized data can be connected (i.e., harmonization). Third, the speed at which the data are updated is assessed (i.e., velocity).

Adequate coverage measures the extent to which targeted safety variables in the variable list (e.g., pre-incident “leading” indicators, “lagging” outcome variables, and process measures such as behavioral observations and inspections) are adequately covered within the data. Additional variables from cross-functional areas should be included and evaluated if trends identify a relationship with important outcomes.

Data harmonization refers to the ability to link and combine disparate variables and databases. For example, Microsoft Excel^© uses the VLOOKUP formula to combine data from different sheets; the formula requires a common, unique variable by which the data sets can be matched. The same process is used within big data, which necessitates the availability of a common variable to match across datasets for combination into a single database. These most likely include demographics such as names, employee numbers, departments, dates, tasks, etc. These variables can be as specific as employee names/ID numbers, can be aggregated by intermediate levels like work team or department, or can be as general as a calendar unit (e.g., week, month, or quarter).

Velocity refers to the frequency at which data enters a database and is updated (SAS, 2021). Examples of data velocity are the frequency of interval updates (e.g., daily, weekly, monthly, quarterly, or annually) of historical, batch, and real-time data feeds. Historical data, and data updated in larger intervals, are sufficient to describe what events have occurred in the past (e.g., descriptive and diagnostic analytics) and make predictions on what may occur in the future (e.g., predictive analytics), but in order to run prescriptive or cognitive analyses, real-time data updates are essential for the ability to make automatic adjustments based on nuanced changes in the data stream. Within OSH, certain technologies such as sensors can provide real-time information measuring pressure, ambient temperature, speed, and direction or angle of body movement. Most safety data, however, is collected through manual reports on paper or in an electronic system, and this delay between occurrence and report may prevent some organizations from being able to use those data sources for higher-level analytics.

2.2.3. Foundational infrastructure

Foundational infrastructure refers to the maturity of the organizational environment and technological processes devised to acquire, store, manage, and extract knowledge from all the different sources of data proposed in Table 1 (Comuzzi & Patel, 2016; Eybers & Hattingh, 2017; Gao et al., 2015). Included in the readiness factor of foundational infrastructure are (a) personnel with technical skills to manage and analyze data, and (b) centralization of the data, which refers to how compatible platforms are for integration, whether raw or aggregated data are available to be extracted from the system, or what variables are chosen to be stored locally versus shared cross-functionally due to the original purpose of the data collection.

The availability and expertise of key personnel (i.e., the personnel infrastructure component) necessary to carry out the technical processes of working with large data sets should be considered. Such expertise includes: (a) ensuring availability of data while minimizing cost (e.g., data management), (b) developing and maintaining predictive and forecasting models while establishing common analyses and reusable processes to reduce execution time and cost (e.g., analytics modelling), and (c) leadership oversight to define strategies and tactics that ensure relevance of analyses.

The degree to which data are stored or can be readily combined into a central database is a necessary component for efficient and effective analytics as the models must have access to all the variables of interest (i. e., centralized database component). It will be rare for organizations to have one database for storing big data, unless the organization is utilizing advanced technologies such as data lakes (e.g., Apache Hadoop) for storing unstructured data or disparate databases, which can be restructured, aggregated, and transformed as later required (Quix et al., 2016). A more common scenario is for organizations to have multiple technological platforms for their data. For example, OSH departments may house their data (e.g., Velocity) differently than human resources (e.g., Workday), finance, operations, or other functions. In addition, some data are stored at the worker level (e.g., payroll information), while other data may be stored at the project level (e.g., inspection rate), causing further challenges to data centralization (Pereira et al., 2020).

2.2.4. Safety measurement culture

Law and Ruppert (2013) describe the collection of data as a social system where culture, which may be called safety measurement culture, impacts the entirety of the process through which data are collected and used to make decisions. When people interact with data entry forms (or not) in the context of their work or use what emerges from the data to mitigate safety issues they are engaging with the safety measurement process. Law and Ruppert (2013) posit that these processes are heterogeneous arrangements between technology and humans. Therefore, active social patterns emerge concerning data collection (e.g., employees conduct more observations and log reports at the end of a quota cycle), and the communication of data can be political in their circulation (e.g., management purposefully may not discuss data findings with front-line workers), thereby affecting willingness to collect data in the first place.

A company can have the best data infrastructure possible, but the system will be ineffective for analysis and improvement if employees are not willing to participate in that system with integrity. Noncompliance with existing protocols may be affected by potential ramifications (e.g., negative job outcomes), how data are presented to employees, and how improvements made based on the data are marketed (Beer, 2015). The culture surrounding reporting (e.g., the employee’s comfort with speaking about incidents or risks, or protection from retaliatory practices) and management use of the data to find problems and make positive changes may impact the employee’s willingness to report accurate information (Kagan & Barnoy, 2016).

Safety measurement culture, then, refers to the extent to which employees and management are inclined to engage in the voluntary extra-role activities that may contribute to improved measurement systems, in addition to meeting minimum requirements for compliance, which refers to engaging in core baseline measurement activities (e.g., compliance versus participation, Griffin & Neal, 2000). Safety measurement culture assesses the willingness of employees to regularly and voluntarily record information correctly, honestly, and in a timely manner.

Safety measurement culture has some conceptual similarities to safety voice behavior, defined as “explicit communication that is (1) discretionary, (2) aimed at improving a perceived unsafe situation, and (3) addressed to others of equal or senior status” (Noort et al., 2019, p. 381). However, they do differ in a few important ways. First, safety voice describes discretionary behaviors, whereas safety measurement culture is comprised of both compliant (e.g., documentation of the administration of first aid) and discretionary (e.g., consistently submitting near miss reports) efforts. Safety voice is singularly focused on prohibitive messages (i.e., voice behaviors that mitigate harmful outcomes) where safety measurement culture would also include positive feedback or ideas for improvement among its outcomes. Further, safety voice is singularly focused on upwards or horizontal verbal communication (Bazzoli & Curcuruto, 2020; Noort et al., 2019), while safety measurement culture considers the bidirectional (i.e., interactions between employees and managers) cultural impact of perceptions, behavior, and communication message content on data artifacts (e.g., quality, timeliness, and frequency). We have, accordingly, separated perceptions, behaviors, and communication content about measurement practices (i.e., safety measurement culture) into two primary components: employee participation and management actions.

Employee participation in safety initiatives like reporting has been found to improve safety outcomes (Hagge et al., 2017). Within OSH, employees are a necessary component of data collection for hazard identifications, near miss identifications, observations and checklist completions, audits, and inspections. The extent to which employees participate in the reporting process can impact both safety outcomes and quality of data. This can be affected by positive cultural perceptions (i.e., “my reporting can help reduce injuries”) as well as negative cultural perceptions. For example, a factor that often impacts data quality related to employee participation is pencil whipping. Pencil whipping happens for many reasons, such as within mandatory quota systems (i.e., compliant yet untrustworthy reporting), where an employee may feel pressured to fill out a certain number of reports regardless of whether a recordable event took place or fear of reporting accurately due to negative repercussions from management (Ludwig, 2014). Therefore, several different perceptions of participation must be assessed to determine which cultural mechanisms are affecting employee reporting behavior.

Safety measurement culture is additionally impacted by management actions that demonstrate concern for worker safety (Frazier et al., 2013). Employee participation in reporting processes is impacted by visible manager behaviors that reinforce a consistent narrative of safety importance. OSHA (2016) recommendations for building safety culture are also applicable to the data collection and management process: (a) encourage employees to participate in reporting, (b) encourage workers to report safety and health concerns, (c) involve workers in all aspects of a safety program, and (d) remove barriers to participation and reporting. Managers and organizations can show their commitment through financial investments in OSH departments, training, and tools. Similarly, the number of employees in the OSH department, time permitted for OSH reporting versus production expectations, and positive reward programs are signals of manager commitment to safety (Paz, 2019). Finally, key behaviors of management concern complete a feedback loop; for employees to feel that their time spent reporting is valuable, managers must communicate to employees the positive changes, improvements, and impact that their reporting has had on the organization and safety outcomes.

An evaluation of each of these readiness factors and their components will diagnose organizational capabilities regarding safety analytics. As each organization improves their systems and are capable of more advanced analyses, the field of safety analytics itself will be improved, leading to initiatives and improved industry standards that are targeted and more efficient, improving worker health and safety.

2.3. Capability evaluation: Addressing the final V(alue)

As we have discussed previously in this paper, the promise of data analytics lies in improved decision-making resulting from valuable data-driven insights. Top performing organizations tend to use analytics at five times the rate of lower performers (LaValle et al., 2011). This is corroborated by empirical research, which has found that high capacities for data volume, variety, and veracity lead to valuable insights that drive firm performance (Cappa et al., 2020). While analytic models can improve efficiency, prevent injury, and reduce costs within OSH, victory can only be declared when such models are used to create new value (Morison, 2013; Veeramachaneni, 2016). Often, however, there is a push to advance technologically without pausing to consider how to evaluate those models for value creation.

The data analytic process captured in Fig. 3 describes an information value chain: data is captured, aggregated, integrated, and analyzed with the promise of gaining information that can guide future action (Kiron et al., 2013). As we have described in our discussion of analytic readiness factors, this process can be stymied by issues upstream at collection (e.g., poor quality or coverage of metrics), aggregation (e.g., harmonization) and storage access (e.g., centralized databases) phases, as well as downstream where the data is analyzed and disseminated (Kiron et al., 2013). Our framework for assessing these key phases of the analytic process thus offers an additional method of evaluation for the ultimate goal—value creation.

2.3.1. Output evaluation

The readiness framework is designed to give organizations an understanding of their current capabilities. This extends beyond an assessment of components of the analytic process to a measure of the integrity of analytic output. As the levels of analytics increase in sophistication, the system requirements also increase. Thus, more advanced analytics require higher maturity across the readiness factors. In this way, “capabilities” refers to the capabilities required for optimal results rather than the capabilities necessary to run a model.

While an organization may be able to run an advanced model, the results may not be trustworthy, relevant, or valuable. The analytic output will only produce value if each component in the information value chain contributes at the level of maturity required for the analytic sophistication. Prior statistical analyses have demonstrated some of the necessary components for each level of analytics. For example, Vater et al. (2019) found that prescriptive analytics in manufacturing required optimal data acquisition, connectivity, data storage, data processing and control. With enough high quality data collected over time, OHS can trust that their descriptive analytics (e.g., trend analyses over time) can provide trustworthy insights for reducing risk. Correlations (e.g., diagnostic analytics) can provide incremental value over trends when there is additional coverage of variables (i.e., potential covariates) and variability. Without variability across outcome measures, however, and more frequent data collection (i.e., velocity), predictive analytics may not be reliable. In this way, understanding current capabilities using our framework aids organizations in identifying the level of analytics that will be the most accurate and actionable, thus providing maximum value for decision-making given system constraints.

Of course, additional value created by our framework lies in the identification of maturity across the smaller components of our framework. By diagnosing areas of both strength and opportunities for improvement, our framework provides an action plan for improving system sophistication and increasing capabilities.

2.3.2. Improving capabilities

The path to improved analytic capability is incremental. Implementation can begin in a smaller scope within areas of strength, as these improvements are often more achievable and less overwhelming to budget and staff (LaValle et al., 2011). Often, there is not a need to overhaul entire systems; rather, improvements can be made to existing process management tools. In fact, some of the more successful organizations take smaller, staggered steps that allow leaders to focus efforts and resources on the areas that will provide the most value, leading to exponential increases in analytic capability over time (LaValle et al., 2011).

Conversely, organizations may encounter large deficits that prevent value generation using analytics. For example, data veracity or quality is foundational; organizations cannot trust the insights gained from analytics if data inputs aren’t trustworthy. Our framework will aid organizations in identifying areas that need significant improvement. Improvement in these areas, while requiring more time investment and higher costs, may provide significant and large gains.

Organizations should not be discouraged if they discover deficiencies in their data processes. Value creation can be achieved even at early and less mature stages (LaValle et al., 2011). Perfect data, infrastructure, or sophisticated analytic strategies aren’t necessary for organizations to gain insights that can improve decision-making. Rather, understanding current capabilities can aid organizations in choosing strategies that will maximize value creation at any level of sophistication. In addition, this readiness framework can be used to incrementally increase value by aiding organizations in identifying strengths which can be developed and pain points that need investment. As organizations transition from one level of analytic capability to the next, the frequency with which analytics are used and resultant decision support insights will increase (LaValle et al., 2011).

3. Discussion

This safety analytics readiness assessment framework proposes four major readiness factors and corresponding components which may impact the ability of OSH to engage in analytics with insightful and meaningful output. These factors are likely to adapt and change as literature and participating organizations discover additional relationships between variables and important safety outcomes. Additionally, these factors likely differ in importance and priority. For example, data quality may be the most essential for insightful analytics. One could argue that it is not worthwhile to perform analytics unless the organization has developed high quality data; again, that old adage warns us that garbage data will only lead to garbage insights, leading to wasted resources in performing the analytics. This logic may mean that data quality is more important to conducting analytics than a centralized database.

Additional research must be conducted to develop appropriate assessment and scoring methodology to (a) assess if these readiness factors are indeed predictive of success across the elementary to advanced levels of analytics, (b) evaluate if analytics readiness can be predicted across disparate organizations and industries, and (c) compare the relative importance of these factors in contributing to analytics readiness. The following section proposes future research that may aid in the development of this system.

3.1. Self-Assessment of data analytic readiness

Future researchers should attempt to discern the best methodology for an organization to self-assess the readiness factors. Such a rating system could include a numeric scale based on maturity model criteria (Pfleeger, 1995) where each readiness factor is rated from baseline to optimal stages. A low maturity or baseline rating in the Rules and Operations readiness factor, for example, may be categorized by ill-defined inputs (e.g., a lack of definition and consistency) where data variables can only be loosely connected to expected outcomes (Pfleeger, 1995). At this lower rating of maturity, organizations are just starting to explore what can be done with current capabilities and analytic results are difficult to explain and interpret (Comuzzi & Patel, 2016). Ratings in the middle of this continuum may reflect analytic systems that have defined activities within the analytics process, including collecting metrics, defining variables, designing systems and code, and testing results. Defined activities increase the consistency, replicability, and efficiency of analyses. There may, however, continue to be challenges with metric and data validity. The highest rating of optimal maturity would be applied to organizations who continuously collect and centralize quality information on the entire safety management process (e.g., input, management, and outcomes) allowing for the most sophisticated forms of data analytics. Specific text descriptions, like those in the Rules and Operations example above, may serve as anchors for maturity ratings specific to each of the readiness factors. The development of this anchored rating scale might facilitate increased fidelity of the scoring system over a Likert-type scale.

Finally, a process will need to be developed to adequately prepare for and use the readiness assessment. Such a process should guide assessors in creating their own list of key variables hypothesized to have relationships with injury outcomes and then describe the method of rating maturity of these variables against the analytics readiness factors. These ratings can then be summarized by factor to provide diagnoses of current capabilities and distinguish areas of high optimization that need little investment from areas which need improvement. Ideally, the assessment could be taken periodically, which would provide additional information for how investments in various capabilities enhance analytics readiness over time.

A final consideration for the development of an instrument based on this framework should be on the reliability of the raters. While safety professionals may be well versed on data collected within their function, they may find it more difficult to assess the maturity of data and processes within other functions. Additionally, an acquaintance with the data itself may not imply an understanding of the data management mechanisms that contribute to analytic success. Finally, there may be concerns for inflated self-ratings; excitement about the possibilities of analytics in reducing injuries and fatalities may cause a kind of optimism halo effect whereby safety professionals rate their capabilities more strongly. For these reasons, further investigation is needed on the level of expertise required for an assessor.

3.2. Potential roadblocks and limitations

While we believe the practice of measuring and tracking safety analytics readiness and the related safety metrics has great potential in improving occupational safety outcomes, we also acknowledge that there are factors to consider before attempting such an initiative. First, for many organizations, the time, effort, and monetary costs involved in setting up and maintaining these systems may effectively hinder such efforts. While having some measurement is better than having none, this point is still worthy of consideration. Importantly, our framework is not built to be implemented in an all-or-none fashion. Rather, we argue that organizations of all sizes can endorse the overarching idea of creating a safety measurement system and take the pieces from our framework that are within their capabilities (Ogbuokiri et al., 2015). Indeed, the popular press has been encouraging small business owners to take advantage of big data in fostering a competitive advantage (e.g., Polakoff, 2020), and the fundamentals of our framework may help guide system implementation and improvement even at these smaller scales.

Second, and more importantly, we believe that it is very important for organizations to secure the buy-in of all the parties involved in the implementation of these systems, such as the supervisors and IT professionals who will manage such systems and especially employees whose data will be collected as part of these initiatives. It is possible that employees’ privacy concerns and concerns regarding the use of advanced analytical tools such as artificial intelligence may influence employee participation levels and even cause legal issues between the employees and/or their unions and the organizations that implement such systems. In response to some of these concerns, there has been a practitioner movement dubbed “AI Ethics” (e.g., IBM, 2020), which aims to embed fairness in all models, ensure that outcomes have no adverse impact on minority groups, and promote transparency about data inputs and algorithms used in analyses. Organizations can follow these principles to augment perceptions of fairness in treatment, processes, and outcomes related to data analyses (e.g., justice perceptions; Gilliland, 1993).

Finally, we acknowledge that the development of this framework has been impacted by the use of SMEs from a singular industry—manufacturing. As we mentioned earlier in the paper, this may impact the applicability of this framework to other industries or to organizations in countries with differing safety processes and norms. However, a comparison of our framework to analytics readiness assessments in other industries (e.g., LARI for education analytics; Arnold et al., 2014) does provide evidence for the universality of some necessary elements (e.g., data quality). We are thus encouraged that our framework will provide value in safety functions across industries but advise future research to apply this framework in various contexts to assess generalizability

4. Conclusions

Unlike other organizational functions, OSH has just begun to explore advanced analytic techniques and methods such as machine learning (Vallmuur et al., 2016) and data lake management (Guo et al, 2016). With safety analytics being at such a nascent stage, it is imperative that organizations assess their current capabilities and make improvements to their safety systems. This proposed safety-industry readiness assessment framework gives managers and safety professionals an overview of safety analytics and the areas that drive this capability. This framework will allow organizations to identify areas of strength and weakness in their existing data management systems, thereby aiding organizations in allocating their resources to areas in which improvements will have the largest impact. Ultimately, this will enhance an organization’s ability to gain analytical insight, leading to improved data-driven initiatives that target and reduce risk.