Where do we start? Guidance for technology implementation in maintenance management for manufacturing

Abstract

Recent efforts in Smart Manufacturing (SM) have proven quite effective at elucidating system behavior using sensing systems, communications and computational platforms, along with statistical methods to collect and analyze real-time performance data. However, how do you effectively select where and when to implement these technology solutions within manufacturing operations? Furthermore, how do you account for the human-driven activities in manufacturing when inserting new technologies? Due to a reliance on human problem solving skills, today’s maintenance operations are largely manual processes without wide-spread automation. The current state-of-the-art maintenance management systems and out-of-the-box solutions do not directly provide necessary synergy between human and technology, and many paradigms ultimately keep the human and digital knowledge systems separate. Decision makers are using one or the other on a case-by-case basis, causing both human and machine to cannibalize each other’s function, leaving both disadvantaged despite ultimately having common goals.

A new paradigm can be achieved through a hybridized systems approach — where human intelligence is effectively augmented with sensing technology and decision support tools, including analytics, diagnostics, or prognostic tools. While these tools promise more efficient, cost-effective maintenance decisions, and improved system productivity, their use is hindered when it is unclear what core organizational or cultural problems they are being implemented to solve. To explicitly frame our discussion about implementation of new technologies in maintenance management around these problems, we adopt well established error mitigation frameworks from human factors experts — who have promoted human-systems integration for decades — to maintenance in manufacturing. Our resulting tiered mitigation strategy guides where and how to insert SM technologies into a human-dominated maintenance management process.

1. Introduction

The era of big data and Internet of Things (IoT) in manufacturing – with low cost sensors and cloud-based solutions – has left many manufacturers with a plethora of data in many different forms. With the recent buzz around more accessible, easy to use Artificial Intelligence (AI) solutions, some manufacturers are asking themselves “can we just throw our data in an AI?” Other manufacturers might ask “how can we get smart with new technologies?” However, AI and other Smart Manufacturing (SM) technologies are not one-size-fits-all solutions for all data types or problems, especially when there are many human-centered aspects in the workflow. Most AI and digital solutions do not work out-of-the-box and cannot directly replace personnel in many situations. Within manufacturing, maintenance is inherently one of the most human-centric processes, but is uniquely suited for an approach designed to intertwine human and digital capabilities. A new paradigm is needed that involves the human, AI and other advanced technologies working collaboratively and efficiently within the maintenance workflow. Achieving this paradigm requires an understanding of how and why this fails to happen in current maintenance practice. This paper dissects the maintenance workflow into the tasks that are performed by personnel, so that commonly occurring errors can be analyzed in a unifying error framework. Using this framework enables a tiered approach to technology implementation, guidance which is useful when manufacturers do not necessarily know where to start.

The rest of the paper is structured as follows: the remainder of Section 1 discusses maintenance in manufacturing, including the maintenance management workflow, maintenance strategies, and issues that occur in practice; Section 2 presents well established human factors research and how it will be applied to maintenance in manufacturing; in the subsequent sections, the maintenance workflow is broken down into three high level tasks: 1. Preparing for Maintenance (Section 3), 2. Performing Maintenance (Section 4), and 3. Discovering Maintenance Needs (Section 5). Within each of these sections, the applicable research and technologies are discussed, with high level tasks being further decomposed into sub-tasks to determine the types of errors that can occur. At the end of each subsection, the mitigations for these different example errors are discussed. Section 6 summarizes the steps from Sections 3–5 to generalize the error mitigation so manufacturers can follow similar steps. Lastly, Section 7 presents conclusions and future work opportunities.

1.1. Maintenance in Manufacturing

Maintenance is a collection of “actions intended to retain an item in, or restore it to, a state in which it can perform a required function” [1]. It is estimated that in 2016, US manufacturers spent $ 50 billion on maintenance and repair, which is between 15 % and 70 % of the cost of goods produced [2]. This estimate includes outsourcing of maintenance and repair, but does not include expenditures on labor and materials or the value of lost productivity due to unscheduled downtime. Estimates suggest that employing smart technologies can reduce maintenance cost from 15 % to 98 % with a high return on investment (ROI) [2]. Within the aerospace industry, examples of specific savings include an estimated return on investment of 3.5:1 for moving from reactive to predictive maintenance on electronic display systems [3] and a 56 % savings in costs from switching from reactive to predictive maintenance for train car wheels [4, 5].

The practice and delivery of maintenance has evolved over the last fifty years. During the late 1960’s Nolan and Heap’s [6] investigation of failures in the airline industry led to the development of reliability-centred maintenance (RCM), a process still widely used today. Building on RCM, a well defined theoretical and practical structure for maintenance management now exists. This is documented in standards [7], textbooks [8, 9, 10] and by professional societies [11, 12].

In the 1970s Japanese manufacturers introduced the concept of Total Productive Maintenance (TPM) [13]. The elements of TPM are 1) a focus on maximizing equipment effectiveness, 2) establishing a system of preventive maintenance for the equipment’s entire life, and 3) the participation of all employees in maintenance through a team effort with the operator being responsible for the ultimate care of his/her equipment [14]. TPM is widely adopted in mature manufacturing organizations with well documented benefits [15]. While RCM and TPM are not competing frameworks, they have different goals: RCM determines an appropriate maintenance strategy while TPM is concerned with managing how maintenance is executed.

In the late 1990s Lean maintenance became popular, which built on the concepts of TPM and RCM and promised a transformation in manufacturing management through standardized workflows, value stream mapping, justin-time (JIT) and Kanban ”Pull” systems, Jidoka (Automation with a human touch), Poka Yoke (Mistake proofing), and the use of the plan-do-check-act process [16]. Despite the promised benefits of lean maintenance mentioned earlier, a literature review by Mostafa et al. found that research on applying lean principles into maintenance had not provided convincing evidence of success [17].

1.2. Maintenance Strategies

Two artifacts of the maintenance management system are particularly noteworthy. First is the maintenance work order (MWO). This concept refers to the archival record of the maintenance event from its inception to its completion and is shared along the way throughout the workflow. All maintenance work should be associated with a work order. The second concept is the Computerized Maintenance Management System (CMMS). This system supports maintenance management with a record of the maintenance work orders and through access to documentation of the assets, resources, and other relevant information. In the maintenance workflow we present here, the CMMS is a hypothetical system and any actual implementation of a CMMS will vary. A MWO is generated, tracked, and eventually archived in the CMMS. The CMMS generates reports documenting the tasks that are due. The maintenance strategies are dependent on when MWOs are acted on and how they are planned through the CMMS.

Preventive maintenance is defined as the “actions performed on a time- or machine-run-based schedule (sometimes referred to as interval based) that detect, preclude, or mitigate degradation of a component or system with the aim of sustaining or extending its useful life through controlling degradation to an acceptable level” [12]. Preventive tasks and intervals are often found in manuals from original equipment manufacturers and are usually a requirement as part of warranty. Over time many asset management organizations develop their own preventive maintenance tasks and intervals as they gain knowledge about their assets and systems.

Condition-based maintenance is defined by Society of Maintenance and Reliability Professionals (SMRP) [12] as “an equipment maintenance strategy based on measuring the condition of equipment against known standards in order to assess whether it will fail during some future period and taking appropriate action to avoid the consequences of that failure. The condition of the equipment can be measured using condition monitoring, statistical process control, equipment performance or through the use of human senses.” Maintenance personnel have been using inspections, process variables, vibration analysis, thermography, oil analysis, ultrasonic analysis, and other techniques for over 30 years. Predictive maintenance is defined in this paper as involving physical, statistical, or machine learning models that combine historical reliability and/or performance data with current condition assessment to generate a probability of failure and/or failure event prediction interval. These machine learning models are used to support condition-based maintenance programs and to inform interval selection for preventive maintenance tasks.

Despite best efforts at proactive maintenance, the stochastic nature of asset degradation means that failures do occur and reactive maintenance is necessary. These failures result in corrective work, which as will be seen in detail below disrupts the maintenance management process. Depending on the consequence of the failure, corrective work may need to be executed immediately (unstructured work). Otherwise, work can be passed into the planning process (structured work). In the manufacturing domain, corrective work is often referred to as unstructured reactive work [2].

1.3. Maintenance Management Workflow

A major factor for the efficiency of maintenance management is whether the work is structured or unstructured. To describe the preferred maintenance structure, reliability engineers have broken down the maintenance workflow into six major steps: 1) Analyze, 2) Select & Prioritize, 3) Plan, 4) Schedule, 5) Execute, and 6) Complete.

Analyze

The Analyze activity relies on the data documented in the work order. Planners, maintenance and reliability engineers use this data to inform their respective tasks. These include reviewing inspection and as-found condition reports to determine whether asset deterioration meets expectation and when the asset has deteriorated past that expected threshold reviewing existing strategy or interval settings for inspection and maintenance, updating data for reliability and risk calculations, and updating optimization models. Analysis is involved in many of the maintenance management processes.

Select & Prioritize

Maintenance work can be identified by many agents, such as operators, maintainers, engineers, and data analysts, by events (e.g., safety incidents), as well as from strategies stored in the CMMS, and in asset management plans, which include recommended routine maintenance schedules. There is always more work to do than can be done in any one planning and scheduling period, and hence work needs to be prioritized. Ideally there should be a risk-based process to prioritize work for each planning cycle. New work notifications arrive each cycle and are reviewed alongside work orders already on the backlog and scheduled preventive maintenance work orders due in the next maintenance work cycle. From these work orders a list of tasks is prioritized and high priority tasks are moved to the planning stage.

Plan

Planning is done by a maintenance planner. For each task, the planner needs estimates for the following types of questions: How long will the job take? How much and what types of labor will be required? What parts and materials will be required, and are they on hand? What are the costs? What tools, equipment, or other resources, including external contractors, will be required? What permits will be required? What are the job hazards, and how will they be managed? Many tasks, such as inspections, periodic condition monitoring, and tasks with a safe work procedure and bill of materials, need limited planner input, but others, such as major asset shutdowns, need considerable input from the planning team. Ideally planning happens some weeks before the time period in which work is due to be executed as part of a well-regulated planning cycle. Once all the information is gathered a work order is planned and it can be scheduled.

Schedule

Scheduling is the temporal organization of tasks for execution. It is a complex optimization problem with constraints such as the number of maintenance technicians available, limited ancillary equipment such as cranes, operational requirements limiting access to equipment, and system connectivity meaning that some work cannot occur at the same time as others. In addition, priority work must be balanced with preventive work so that preventive work does not fall behind over time.

Execute

Considerable investment is incurred prior to execution due to the resources involved in planning and scheduling. Value from this prior work is added when the proposed maintenance work is executed by maintenance technicians through repair and replacement tasks to restore the required functionality of the assets. Good quality maintenance work restores the asset function to some required level or function, either as-good-as-new or some level between that and the current state. Poor quality maintenance work or work that is unnecessary can destroy value by introducing defects and cost money for little gain.

Complete

When the technician has completed an assigned task, an important but often overlooked step is to capture data about the maintenance work with the as-found and as-left condition of the asset. This is documented on the work order, reviewed by the maintenance planner who is responsible for closing the work order, and stored in the CMMS.

Consequently, structured work refers to work that follows this entire maintenance management workflow. Structured work is planned and scheduled in longer time scales (normally planning sessions happen once a week and provide a time in the future to execute the maintenance). Unstructured work is often referred to as “reactive work”, as these jobs result from failures that are identified by asset operators and executed immediately. These unstructured jobs are still completed and analyzed but do not pass through the formal planning and scheduling stages. Because unstructured work is executed immediately, it often results in other structured jobs associated with preventive and condition-based strategies to be rescheduled.

While these activities are the focus of maintenance reliability experts, this structure makes it difficult to discuss the human role in maintenance. The human actors within this workflow perform different tasks dependent on the situation (i.e., unstructured vs structured work). These roles and the responsibilities are described in Table 1.¹ However, while the person performing the task may change (e.g., a planner calculates time estimates for a job in structured work, whereas a technician might calculate time estimates on the fly for unstructured work), the tasks themselves largely remain the same. Regardless of context or situation, a human must 1. Prepare for the Maintenance Job, 2. Perform the Maintenance Job, and lastly 3. Discover Maintenance Needs. This distinction highlights how personnel actually perform each task and the types of errors that might occur in doing so, with a subsequent mapping from the task performed to the corresponding activity in the maintenance workflow for both structured and unstructured work. This task-based analysis is necessary due to the issues that still exist in manufacturing maintenance practice.

Table 1.

Personnel in Maintenance

Job Title	Description of Responsibilities
Operator	Operates machines or monitors automated machines – can be responsible for one machine or multiple machines depending on the size of the organization and level of automation.
Technician	Used here to refer to the person performing minor maintenance jobs, for example routine inspections. These jobs can be done by both operators and maintainers.
Planner	Estimates time, cost, resources and documents for maintenance jobs, purchases parts and contracts.
Scheduler	Coordinate all planned jobs for a specific period into a realizable schedule.
Analyst	Analyzes and models data about equipment, operational and maintenance management performance.
Maintainer	A trade-qualified technician competent to perform tasks in their area of expertise.
Engineer	A degree-qualified individual who provides technical support for front-line staff such as operators and maintainers.

1.4. Issues in Practice

The SMRP Best Practices Committee suggests a distribution of maintenance work types as follows: for all executed maintenance work hours, 10 % to 15 % should be on improvement and modification work, 30 % on structured work - split between 15 % on predictive/condition-based work and 15 % on preventive work. Corrective maintenance hours derived from structured work should be 50 %, 15 % from preventive maintenance inspections and 35 % from predictive maintenance inspections. Only 5 % should be associated with corrective maintenance from unstructured work with a buffer of 5 to 10 % for other work. [12] In practice many manufacturing operations do not achieve these levels.

Small-to-medium sized enterprises (SMEs) still mainly employ a mixture of unstructured and structured maintenance strategies [18]. Once again, it is important to note that manufacturers often refer to corrective work as only unstructured, when in fact not all corrective work is unstructured work. Larger companies are employing preventive maintenance strategies, but unplanned maintenance jobs are still frequent [18]. Alsyouf [19] found that in Swedish manufacturing firms, 50 % of maintenance time was spent on planned tasks, 37 % on unplanned tasks, and 13 % for planning the maintenance tasks. Even though preventive maintenance strategies are more prevalent in larger companies, these maintenance jobs are not always performed correctly. It is estimated that one-third of maintenance jobs are improperly done or unnecessary [20]. Another study mentions that preventive maintenance is estimated to be applied too frequently in 50 % of all cases in manufacturing [21].

So why are so many SMEs employing mainly reactive unstructured maintenance strategies? Why are larger manufacturers still dealing with unstructured maintenance and often incorrectly performing preventive maintenance procedures? In a survey to manufacturers, the main barriers to adopting advanced maintenance strategies were cost (92 % of respondents), technology support (69 %), and human resources (62 %) [18]. This illustrates the need to help manufacturers find the most cost-effective path toward balancing technology solutions with human-driven tasks to improve maintenance procedure and reduce unplanned work. To effectively achieve such a paradigm, it is necessary to examine tasks within the maintenance management process to identify how to implement new technologies effectively by accounting for human knowledge and expertise.

2. Human Factors and the Maintenance Workflow

Incorporating a focus on human interaction with complex systems by applying human factors principles is not a new idea, and is rapidly gaining traction in sectors where implementation of new systems carries significant overhead, whether finiancially or culturally. In a 2011 report, the U.S. Department of Defense published a Human Systems Integration (HSI) Plan, [22] beginning with the following overview:

The human and ever increasingly complex defense systems are inextricably linked. [...] High levels of human effectiveness are typically required for a system to achieve its desired effectiveness. The synergistic interaction between the human and the system is key to attaining improvements total system performance and minimizing total ownership costs. Therefore, to realize the full and intended potential that complex systems offer, the Department must apply continuous and rigorous approaches to HSI to ensure that the human capabilities are addressed throughout every aspect of system acquisition [...] In summary, this means that the human in acquisition programs is given equal treatment to hardware and software.

To accomplish this, human factors engineers will review functions and tasks within a system, which at their most basic assign responsibility of some activity to personnel, automated systems, or some combination thereof [23]. The primary goal of defining these tasks is to better understand not only the specific roles of personnel, but also how these will shift under implementation of proposed changes to the system.

Defining the role of human actors within a maintenance workflow has already been a core, if controversial, topic of interest under the existing theoretical frameworks. Maintenance practices in the manufacturing sector center on the importance of individual authority versus the needs for centralized planning and scheduling of maintenance tasks. For example, the ideas of TPM focus on high levels of individual ownership of the asset by the operator with responsibility to adjust and maintain the unit. This will be largely an undocumented process with some work being done at the discretion of the operator to optimize their asset’s performance. In this approach, the operator is empowered to take responsibility over the domain. This contrasts with the maintenance management view in which maintenance is centralized and the aim is to minimize costs across all equipment and resources and only to touch equipment if the work has been prioritized, documented, planned, and scheduled. With the introduction of more automation requiring individuals to assume responsibility for larger segments of the operation and with many highly-knowledgeable operators aging out of the workforce, the more centralized approach is gaining traction. However, the need for the human knowledge and expertise is greater than ever before and needs to be factored into the management process so as to optimize their contributions.

In systems engineering the incorporation of a human in any automated system is often designed as a fail-safe mechanism. Designing for all possibilities and failure modes is an impossibility, so designers and maintainers exploit one of humanity’s greatest strengths: our ability to problem solve in unfamiliar situations and environments. However, this ability to reason from first principles has a high cognitive load, so humans try (where possible) to use rules and heuristics — mental shortcuts — to relax decision making and reduce the process of forming justifications into more habitual, routine tasks. Certainly, heuristics can be informed by prior observations of performance patterns and the success of previous solutions or approaches, but heuristics do not always work well to anticipate or mitigate failure: the human performing a task is left without enough context to recover from where the heuristic left off, or to estimate risks under unknown system behaviors or personal biases. Ironically, these situations tend to arise more often as systems become more automated—failure contexts become more complex and observations of particular situations become increasingly rare. The implementation of technologies, while intended to support technicians, will also require them to learn new ways of working. It will take time to build new sets of heuristics for each scenario. Digitization of equipment, for example, can decrease physical accessibility to manufacturing systems, along with altering the skill-set required to perform technical troubleshooting. The tension between a drive for automation to compensate for human error, and the necessity for humans to compensate for increasingly complex automated-system failures, should be dealt with up front by explicitly accounting for human failure modes that are causing the errors, in the original implementation plan. Orienting the function of emerging technologies in manufacturing maintenance around the causes of errors opens a path toward efficient and holistic implementation of those technologies.

This paper is not intended to serve as a sweeping guideline for implementing human factors, or for performing HSI, within maintenance in general—this would be far outside the scope for a single paper. Rather, we focus on specific pain points encountered in existing maintenance workflows, specifically in the context of human error before and after implementing some of the recently developing technologies in the space. We hope to provide initial guidance on augmenting specific functions/tasks within the maintenance workflow through certain types of technologies, based on how their strengths and weaknesses mesh with the strengths and weaknesses of critical personnel.

2.1. Human Factors Background

To analyze the maintenance management workflow, the role of the human in the maintenance paradigm must be understood. This paper uses the research by Jens Rasmussen and James Reason to provide a framework for estimating prime insertion points of new technology into the maintenance workflow. [24, 25] This framework provides guidance towards a hybridized maintenance workflow with both the human and technological systems working harmoniously. The framework centers around Skill-, Rule-, and Knowledge-Based error occurrences in the maintenance workflow.

Rasmussen introduced the Skill-, Rule-, Knowledge-Based Human Performance model in 1983. At the time when computers were becoming more mainstream, Rasmussen understood that the introduction of new digital technologies required “consistent models of human performance in routine task environments and during unfamiliar task conditions.” This need for a human performance model ultimately led to the Skills-, Rules-, Knowledge-Based model of human behavior. Rasmussen proposed that human activity was a complex sequence of activities that depend on whether the activity was in a familiar or unfamiliar environment. He argued that, in a familiar environment, a human strives towards some high level goal through unconscious thinking based on similar situations. If the goal is not met, they use a set of “rules”, which have perhaps been previously successful. In an unfamiliar environment, when proven rules are not available, a human makes different attempts – often in their head – towards a successful sequence to reach a goal.

Skill-Based Behavior

A skill-based (SB) behavior takes place without conscious attention or control (e.g., tracking tasks in real time). A majority of the time, human activity can be considered a sequence of strictly SB actions or activities. SB behavior is an unconscious action implying difficulty or redundancy for a person to explain what information is required to complete the action.

Rule-Based Behavior

When a sequence of actions is controlled by a rule or procedure derived from previous occasions, this is a rule-based (RB) behavior. The boundary between SB and RB behavior depends on the level of training and the attention of the person completing the task. Higherlevel RB behavior is based on the human’s explicit understanding of the problem and the rules used in accomplishing the task, while The “rules” in RB behavior can be derived empirically from previous attempts at solving a problem, communicated from another person’s know-how, or may be prepared on occasion by conscious problem solving and planning. These rules are dependent on the knowledge of the environment.

Knowledge-Based Behavior

When faced with an unfamiliar situation a human may need to rely on building new reference knowledge: this is knowledge-based (KB) behavior. A KB behavior involves explicitly formulating a goal based on an analysis of the environment and the aim(s) of the task. An individual develops different plans and tests them against the goal – either by trial and error or conceptually through understanding the environment and predicting the effects of the various plans – to determine the best course of action. This understanding requires mental models of the task and environment to predict the impact a specific plan might have on achieving the goal.

Error Classification

While these different categories of human behavior are very useful in human reliability research, determining the appropriate category for individual tasks in a workflow is difficult in general. Thus, Reason [24] takes an error modeling approach toward the use of Rassmussen’s skill, rule, and knowledge in his Generic Error Modeling System (GEMS). Rather than assigning a classification to each task, it is often more efficient to classify the error modes (which can occur while performing each task) into skill-, rule-, or knowledge-based errors. In the interest of using technology to cost-effectively address errors currently existing within a maintenance workflow, we focus implementation strategies around these errors explicitly. Figure 1 displays how this workflow is implemented in GEMS through the different levels of human performance, as well as providing some examples of various errors that can occur in the maintenance management workflow. The GEMS mapping of skill-, rule-, knowledge-based behavior onto errors enables an examination of activities within maintenance tasks by focusing on events when the system is not performing as desired — quite similar to system investigation through Failure Modes and Effects Analysis (FMEA). This discussion can help to determine the context-appropriate technologies that can be inserted into the maintenance workflow in a way that augments a maintenance practitioner’s ability to successfully complete a task both efficiently and effectively.

Figure 1.

SRK Framework. Adapted from Reason [24].

1. Does the practitioner know that something is amisswhile it is happening (has an attentional check occurred)?

   No The errors involved will be SB level. Mitigations for these would help him/her perform the attentional check (notice the error), or make noticing at this part of the workflow unnecessary through anticipation.

   Yes Problem is being investigated at a RB or KB level

2. Does the practitioner believe they have a way to solvethe (noticed) problem?

   Yes Errors will be RB level. The selected “rule” may not actually be appropriate, and mitigations should provide more (or better) sources of data and pattern discovery, e.g., sensor outfitting, machine learning models.

   No They are actively searching for a new rule, making the relevant errors KB level. Context and causally sensitive models would be helpful to teach or suggest new solutions, like simulation, schedule optimization, or expert systems.

Using this model, the same failure in an activity may have distinct causes — understanding at which level the failure occurs can help to address it. An SB error stems from the inherent variability of human action with familiar tasks. Commonly referred to as slips and lapses, they generally occur without immediate recognition that something is wrong. It is only after an “attentional check” that one might notice something has gone awry, and begin applying some known rule or pattern that addresses this problem.

RB errors typically are the misclassification of situations, leading to the use of an inappropriate rule or incorrect recall of procedure. However, once the problem-solver realizes that none of their existing rules apply, he/she begins modeling the problem space (e.g., by analogy) and searching for context clues that relate the problem to past successful rules. KB errors arise from the incomplete or incorrect knowledge of the problem-solver and stem from situations that represent the highest cognitive demand. Reason indicates that this state will quickly revert to SB once a satisfactory solution is found, and that this is a primary cause of sub-optimal solutions. GEMS postulates humans behave in such a way as to minimize their cognitive load and that many errors are a result of this tendency. Once an error is recognized, the person will move to the next higher level of cognition to resolve the error; once resolved, they will quickly retreat to lower cognitive effort.

2.2. Error Mitigation Framework for Efficient Technology Adoption

This process of discovering a problem and the validity of the current solution strategies is the same process that will be applied at a management level when implementing new technologies into the maintenance workflow. Unless it can be demonstrated that 1) there might be a problem with the current situation (SB) and 2) the solutions currently in place are sub-optimal (RB), practitioners in the existing maintenance workflow will be unlikely to use solutions that attempt to improve performance through suggestion of new modes of operation or behavior (KB). Knowing how to frame technology implementation in terms of these steps is key. Cultural momentum and the power of the status quo is consistently overlooked, and failing to understand or adapt to it is nearly always the primary cause for technology implementation failures [26]. While this may sound extreme, in the context of system maintainers, it makes sense: if trusted personnel in the human-centric maintenance workflow do not believe that there is a problem, or do not think their solution is insufficient, the possible performance of new technology will not come to fruition—no matter the expenditure that went into implementation.

This paper does not exhaustively enumerate all potential errors, their probabilities, or the factors that affect them. Starting on that path would require a more sophisticated Human Reliability Analysis (HRA). ² We leave this worthwhile task for future work. Instead, we illustrate some common errors that can occur in the maintenance workflow and how GEMS can be used to highlight common opportunities and pitfalls when implementing technologies meant to assist the maintenance management process. Sequentially progressing through SB, RB, then KB errors and deciding the risk and mitigation possibilities of each through available resources provides valuable guidance, especially when choosing a starting point can be difficult in the face of capital or personnel costs. Importantly, we

1. classify example errors that commonly occur during maintenance tasks, and

2. discuss how the features of each task tend toward more or less errors of a given type.

Going through a similar exercise prior to selecting or implementing new hardware or software systems will assist in matching use-cases, as well as mitigating consequential errors effectively, since different technologies are designed to address widely different error types.

A tiered error-mitigation strategy, based on patterns observed in literature and industrial application, structures our discussion of inserting advanced technologies into a maintenance workflow. Improvement opportunities are aligned to the consequences of the problems they address, balanced with feasibility of implementation in terms of cost, logistics, and organizational maturity. In this strategy, errors are addressed based on their cognitive load.

The discussion that follows centers around the different errors in the maintenance workflow and the human actor who commits them. It is important to note that these errors are committed by a variety of roles, and not necessarily by just the technician (as is often thought by maintenance managers). The errors presented are often discussed on an individual basis (e.g., one technician does not notice an alarm); however, manufacturers must view these errors at a systematic level to understand the true “pain points” in their factory. A single technician not noticing an alarm is not typically high risk, but having a majority of technicians systematically miss alarms is a larger, more important issue to recognize. As such, while it is tempting to focus on individual actors as problem sources, better guidance is needed that assists manufacturers in tracking and estimating errors across the entire factory.

In the following sections, applicable research and technologies are discussed that apply to the tasks in the maintenance workflow. Examples of errors are described for each task: 1. Prepare for Maintenance (Section 3), 2. Perform Maintenance (Section 4), and 3. Determine Maintenance Needs (Section 5), and errors are mapped to the sub-tasks in Tables 2, 3, 4. Each table maps sub-tasks (Column 1), to example errors (Column 2) and their corresponding GEMS classification (Column 3). The errors and mitigations as presented are intended to be exemplary of common errors practitioners will encounter in the maintenance workflow.

Table 2.

Prepare for Maintenance Tasks: Errors and Mitigations

Sub-task	Example-Error Description	Error Type
Identification Tasks
React	Failure not noticed	SB
	Failure ignored		RB
Anticipate	Incorrect interval for replacement		RB
Assess	Incorrect condition for replacement		RB
	Operator misses condition	SB
	Misunderstand condition calculation			KB
Planning Tasks
Estimate	Error in calculation	SB
	Match task to wrong SWP		RB
	Estimate from wrong sources of data or experience		RB
Procure	Misevaluate available resources (when to purchase parts/use in-house)		RB
	Mismatch contractors or pick the wrong parts		RB
Prepare/Document	Miss/overlook a necessary document	SB
	Include or require wrong documents		RB
	Proposed procedure is non-optimal			KB
	Ignore previous feedback from execution team			KB
Scheduling Tasks
Prioritize	Poorly track resource availability	SB
	Misevaluate interrelationships between different maintenance tasks			KB
Assign	Poor match of skills of technicians to the job		RB
	Lack of communication between executing team and scheduler			KB

Table 3.

Perform Maintenance Tasks: Errors and Mitigations

Sub-task	Example-Error Description	Error Type
Assessment and Diagnostic Tasks
Assess	Overlook symptoms that indicate poor equipment health	SB
	Incorrect condition features used in assessment		RB
Compare	Incorrectly assume validity of assigned work		RB
Diagnose	Unfamiliar symptoms lead to incorrect fault diagnosis			KB
	Incorrect diagnostic conclusion due to lack of experience			KB
Execute Maintenance Action Tasks
Perform	Forget necessary tools needed to complete job	SB
	Lapse in execution quality due to focus constraints	SB
Triage	Mistake nature of work for similar type having distinct solutions		RB
	Attempt execution without requisite experience, tools or supervision			KB
Completion and Recording Tasks
Recall	Technician does not remember significant symptoms	SB
	Recalled features are not relevant to analysis		RB
Record	Work performed is entered incorrectly, or schema structure is incomplete		RB
	Technician gives up searching prior to finding appropriate problem-code		RB
	Technician misunderstands or is unaware use-case and functionality of the data structure (e.g. controlled-vocabulary)			KB

Table 4.

Discover Maintenance Needs Tasks: Errors and Mitigations

Sub-task	Example-Error Description	Error Type
Data Extraction, Transformation, and Loading (ETL) Tasks
Extract	Inappropriate data quantity or type	SB
	Not planning for volume	SB
	Collect wrong data for desired analysis		RB
Transform	Dimensions/units/feature compatibilty error	SB
	Discard potentially useful data		RB
Load	Choosing wrong hardware		RB
	Misunderstanding software tools			KB
Modeling and Analytics Tasks
Detect Trends	Not noticing trends	SB
	Detecting false trends (overfitting)	SB
	Insufficient communication of trends	SB
Define Patterns	Inappropriate model or modeling assumptions		RB
	Misinterpret correlation as causation		RB
Identify Causality	Unknown relationships driving unknown failure modes			KB
	Lack framework for synthesizing model output into actionable strategies			KB

3. Prepare for Maintenance

Prepare for Maintenance involves a number of actions to enable execution of maintenance work. The tasks performed by the human actors in maintenance are largely the same, but are performed in different stages of the maintenance management workflow and are performed by different people depending on structured versus unstructured work. For structured maintenance events, the required tasks are prepared by a maintenance planner/scheduler over a period of days, weeks or months. During unstructured maintenance events, the jobs and required actions are identified, often by an operator or technician while in the field, and usually under time constraints as the component may have already failed. A number of research efforts center around the prepare for maintenance task, as discussed in the following subsection.

3.1. Applicable Research & Technologies

Maintenance preparation is a very human-centric operation relying on tacit knowledge of how similar jobs have been planned in the past, what has worked well, and what has not. Various efforts have codified this knowledge using safe work procedures, bills of materials, and post-work reviews [10, 9]. As the balance of work to plan moves from corrective to preventive and predictive work, the opportunity for semi-automation of the maintenance planning process will increase.

Despite the considerable academic focus on maintenance scheduling under the umbrella of maintenance optimization, the levels of transaction automation and the use of simulation models in this process are small. Maintenance optimization uses mathematical models to find either the optimum balance between costs and benefits of maintenance or the most appropriate interval or moment to execute maintenance. An overview of the maintenance modeling approaches and examples of their applications are available in Dekker (1996 and 1998), Marquez, and Jardine [30, 31, 32, 33]. Both engineers and mathematicians have contributed to the area. Due to the complexity of these models they have not been easy to apply to real world manufacturing systems in practice [30].

Maintenance simulation models are classified in a number of ways. First, are they for planning or scheduling? The vast majority of optimization models in the literature address scheduling. Secondly, at what level is the maintenance decision being taken: organizational, plant, system, unit, or component? A consequence of the level consideration is that decisions at higher levels need to take all lower levels into account. Many different types of dependencies must be considered and these can only be accounted for in a simplified and often inaccurate way [31]. Finally the model needs to consider time scale. Is the model to support a decision with impact in the near or long term? Are we thinking about the next schedule period or something that could have long-term, but deferred impact, on the life of the asset?

Discrete event simulation is widely used to model maintenance systems and the uptake of new optimization methods, such as genetic algorithms, has been rapid. However, a review by Alragbhi [34] found only a few real life case studies were published and the academic cases that dominate the literature, such as a single machine producing a single product, are oversimplified and do not reflect the complexity and interactions of real systems.

Scheduling practice differs greatly depending on the type of system. Scheduling practices in manufacturing differ depending on whether the operation is a batch process or continuous process and on the availability of buffers. The presence of buffers in the system allows for a more flexible approach for minor maintenance and for opportunistic maintenance to occur [35, 36, 37, 38]. Examples of work on maintenance in a manufacturing context mainly have focused on management of preventative maintenance activities [39, 40] or maintenance resources [41] to optimize manufacturing system performance.

Practical and applicable models are needed that derive a set of optimized maintenance schedules offering a range of trade-offs across the objectives from which managers can select for their immediate needs. The modeling system needs to be able to adjust schedules on a real-time basis as circumstances and/or priorities change. Maintenance optimization is a complex problem, with multiple possible objectives such as system reliability, cost, availability and various combinations of these (many plants easily involve over 100,000 periodic activities). The complexity of the optimization has often precluded the use of decision guidance systems in real-time under current practices. Emerging technological advances are enabling better support in these systems, however new solvers are required to develop and solve the proposed models. Too often in the past engineers have focused on optimizing a particular asset or subsystem, where the complexity is more manageable, rather than considering the entire maintenance management system.

Digital twins are an emergent focus for many in manufacturing and are an integral part of Smart Manufacturing [42, 43, 44]. A digital twin is a digital model of the asset system. It is constructed using digital information of the physical asset and its environment and can be continuously updated from sensor data. This should enable better planning, prediction and simulation of future outcomes.

Despite the widespread use of discrete event simulation models, commercial and research interest in the potential of agent-based simulation approaches is increasing, particularly when organizational and human factors need to be incorporated [45, 46].

As discussed earlier, it is not simple to incorporate these technologies seamlessly into the maintenance management workflow. By decomposing the perform maintenance task into sub-tasks, we can better analyze the types of errors that occur and the potential mitigations. These sub-tasks include: 1) Identification – considering steps necessary in the maintenance execution process, 2) Planning – determining the required resources to perform the jobs, and 3) Scheduling – determining the schedule, when the job will be performed, and in what sequence with other jobs. The typical errors for this stage are described in Table 2.

3.2. Identification Task Errors

The identification of work occurs through the structured processes and also during reactive work as described in Section 2. In the latter case the maintainer must identify the work to be done when he/she gets to the failed asset. Similar human processes are involved in both examples of ‘identification’. Some examples of errors that occur during the identification task are below.

SB A Condition Based Maintenance (CBM) technician fails to notice the vibration sensor is not adhered properly so the data collected is wrong. (Assess Sub-task)

RB A CBM technician identifies a high peak in vibration when collecting data on Pump 1 but the source is actually the adjacent pump. (Assess Sub-task)

KB CBM monitoring technician generates a work order that machine X1’s “lead-screw’s vibration is high”. This failure mode has not previously been seen and there are no visible symptoms. The proposed work is subsequently overridden by planner who believes the analysis is inaccurate. (Assess Sub-task)

Traditionally the approach to dealing with SB errors on the plant floor is through increased surveillance with the use of sensors, process control and alarms. These techniques give more than one person the opportunity to identify the fault. Another common approach has been the development of checklists. However, these checklists can promote mindless completion of forms – even if the form is incorrect or incomplete – for the sake of just completing the form because they are told to complete it, instead of mindfully completing the forms to properly and accurately document the activity. A proliferation of unqualified checklists can also create issues and, where they are useful, should be replaced by centrally developed and version controlled maintenance procedures. Currently, few technical solutions address when the planner is dealing with paperwork when many distractions and other calls on his/her time exist. Improved supervision, workload management and team support are often key to improving concentration and mindful execution of routine work [47, 48].

RB and KB errors (See Table 2) in identification of work often result from different mental models of the failure or its consequence between parties involved. As in the illustration above, the technician assumes the vibration data is from Pump 1, according to his/her previous experiences, and so assumes the data is correct even though it is incorrect.

These errors can be mitigated through investment in digitization of the prioritization and approval processes for the planner or through improved training (for the maintainers). Ensuring that the initial SB errors are mitigated first where possible, enables the higher-level RB and KB errors to be addressed through these more advanced approaches.

3.3. Planning Task Errors

Planning involves estimation of necessary resources, time, cost for each job based on historical organizational data, rules and practices and the experience of the human planner. Parts must be procured and maintainers and tools selected, and documents are prepared to assist in execution. The tasks within Planning are similar for both structured and unstructured work. However, for structured work the tasks are performed on a longer time scale and by a dedicated planner. This contrasts with the shorter time scale (often right when a failure occurs) associated with the unstructured work done by an operator or technician. A summary of common errors and their classifications are located in Table 2 and some examples are discussed below.

SB When planning a rebuild on Machine X the maintenance planner miscalculates the time estimate for job 1. (Estimate Sub-task)

RB The planner orders all the same parts as used in the last Machine X rebuild rather than considering the work specifically identified for this rebuild. (Estimate Sub-task)

RB The new maintenance planner contracts Company A for Machine X rebuild, because of a past relationship, but Company B should also have been considered. (Procure Sub-task)

KB The planner miscalculates the downtime required for Machine X rebuild by failing to take account of resource constraints. (Prepare/Document Sub-task)

SB errors during this task, such as forgetting a necessary document or making a slip during an estimate calculation can be aided by centrally managed and controlled procedures that are easy to use. Many of the errors during this task are RB errors that involve matching an aspect of the work order to some necessary document or resource. These type of errors could be well suited for machine learning solutions because these algorithms can learn the important features of the maintenance task and match to the correct previous solutions to provide estimates of resources, time, cost, etc.; however, these solutions can be difficult to implement because of the way in which data about the tasks is stored (natural language) and because of the variety of different contexts in which the same task can be executed. If these SB errors are dominant, investment in the search-based solutions enabling planners to locate information on previous similar tasks can assist.

3.4. Scheduling Task Errors

Once the tasks are planned, they must be scheduled for a specific time. For structured work, this task is completed by a scheduler. However, it can be done in the field by a technician or operator when a failure occurs and an unstructured job is initiated. Coordinating the scheduling of many machines, people, parts, contractors and production equipment requires consideration of many permutations for optimal solutions. This complexity can potentially lead to a number of higher level errors such as those seen in Table 2 and discussed below:

SB Maintenance scheduler forgets that Team A has a safety Day and schedules work when they are not available. (Prioritize Sub-task)

RB Technician 1 is chosen for an A-type job, as always; Technician 2 has recently been A-certified, and is not being assigned A-type jobs. (Assign Sub-task)

KB The scheduler reuses an old schedule for rebuild of Machine A even though analysis the last time this rebuild was done resulted in a 50% time overrun. (Assign Sub-task)

Given the complex nature of scheduling, a high incident of KB errors are likely to occur. These types of KB errors can benefit from investment in project planning software and scheduling and optimization models. SB errors, such as when a scheduler has a lapse determining availability of a technician or asset, can be mitigated through scripts to assist in availability calculations from calendars. RB errors that occur in matching a work order to an appropriate technician can benefit from the analysis and modeling solutions discussed in Section 3.3 on Planning tasks. Scheduling is one of the more difficult tasks to provide easy-to-implement solutions; however, it can benefit from planning tools that enable a variety of schedules to be tested and various constraints to be incorporated.

4. Perform Maintenance

The Perform Maintenance stage consists of executing the maintenance actions, and recording the necessary information about the job. The majority of tasks are similar whether the work is structured or unstructured. The research efforts in this space are described in the following subsection.

4.1. Applicable Research & Technologies

The perform maintenance task includes the maintainer documenting the as-found and as-left conditions of the equipment as well as the work that was done. The steps in executing maintenance work have changed very little in the last 40 years. Many of the same tools and processes are used. There have been advances in support tools, for example laser alignment to replace dial indicators, auto-lubers, and greater use of digital interfaces to support troubleshooting for electrical/electronic equipment but the nature of the way work is executed today would be familiar to many retired technicians.

Each time a maintainer or technician interacts with equipment, he or she expands their own expertise of the asset and captures a textual description of observations of the asset and records what was done, when, and how. The lack of correct and complete data in work order records to support analysis is widely acknowledged. Recent work to better understand factors that affect data quality of maintenance work orders include [49, 48]. Work orders typically contain unstructured text with jargon, abbreviations, and incomplete data. Primary interests for analysis are information to establish the as-found condition, the causality of failure including the failure mechanism, and a description of the maintenance work executed and parts used. This data is often in the work order texts, but it is not extracted in a machine-readable way. As a result maintenance staff rely heavily on personal expertise, word-of-mouth, and ad-hoc data exchange, consulting the records when these other methods fail.

Research from several different academic perspectives has been conducted on the execution of maintenance work; however, these types of studies have seldom translated into meaningful change on the maintenance shop floor. Human factors specialists have looked at how maintainers interact with assets [50]. The impact of human error on maintenance outcomes has been of significant interest [51] and spurred attention from other organizational psychologists in exploring how culture affects motivation and the execution of quality work and consistency in following procedures [52, 47]. Considerable interest exists now in the potential for mobile technologies such as assisted reality and GPS tracking to better understand and support maintainers in the field both in the execution of their work and in how data about the work is collected [53]. The latter is of vital interest to engineers as a maintainer’s observations on the as-found condition of an asset can be vital input to validating condition-based work orders.

The explosion of current technology dealing with multimodal data sources is particularly relevant to maintenance management. The information about asset condition, failure cause, and maintenance work extends beyond what is captured using language in written work order records. While work orders are central to maintenance processes, maintainers communicate with each other and others using a wide variety of media, including photos, videos, emails, text messages and phones, in addition to other resources such as sensor data. Support systems are emerging to provide access to critical information about maintenance issues from disparate sources. Given the emergence of alternate ways of collecting data from maintainers with mobile devices containing cameras and audio to sensor data from machines, methods to efficiently process and synthesize these different modes of data capture to provide asset health status assessment are needed. Assisted and augmented reality (AR) head-set systems are emerging into the market that provide maintainers with access to audio and visual support in the field and the ability to look at drawings and other relevant information in a head set visor [54, 55, 56].

Technical developments are needed to enable maintainers to efficiently capture, retrieve, absorb, process and exchange knowledge about equipment and maintenance work. One of the most exciting recent developments is in natural language processing to enable work order texts to be read and analyzed more efficiently by computers. Examples of recent work in this area include [57, 58, 59, 60]. This work is complemented by developments in semantic knowledge representation technologies to capture data and contextual relationships between data. This will support the development of inference engines capable of performing basic reasoning over maintenance operations enabling better decision support and improved quality control.

Another notable development is the emergence of ontologies for maintenance that are aimed at addressing different needs such as data integration, semantic interoperability, and decision support in maintenance. For example, several ontological approaches have been proposed to overcome the problems of heterogeneity and inconsistency in maintenance records through semantic data annotation and integration [61, 62]. When formal ontologies are used for annotating vast bodies of data, this data can be more easily retrieved, integrated and summarized. Also, the annotated data can easily be exploited for purposes of semantic reasoning. In a recent initiative, referred to as the Industrial Ontologies Foundry (IOF), an international network of ontology developers are working towards developing a set of modular, public, and reusable ontologies in multiple industrial domains [63, 64]. Their work includes a reference ontology for maintenance.

To discuss how the technologies within this stage can be implemented in the maintenance workflow, the perform maintenance task is decomposed into the following sub-tasks: 1) Assessing and Diagnosing, 2) Executing the Maintenance Action, and 3) Completing and Recording the Action. The set of typical errors for this stage is in Table 3.

4.2. Assessment and Diagnostic Task Errors

The Assessment and Diagnostic Tasks depend on the type of work required, such as assessing the equipment condition to see if condition of the asset is as expected from the work order and if the task described in the work order is appropriate. These tasks rely heavily on the tacit knowledge of the maintainer, whether heuristics or rules-of-thumb, that can be applied in uncertain or developing circumstances. One way to think of this is if an assessment does not match the assigned work, similar diagnostic and assessment tasks are required as in the reactive identification tasks, discussed above. This means that many of the SB and RB errors mentioned previously (specifically for unstructured work) apply to this task as well.

In supporting these tasks maintenance technicians may face specific challenges. If they are in the field (this may be remote from the maintenance shop such as on the factory floor), they can be isolated from reference material or knowledge bases and their team and supervisors. In addition ergonomic constraints often exist — e.g., using a touchscreen is difficult with gloves, which can make access to digital support tools challenging. In addition, digital support tools need to be rugged to survive dust, water and unsecured work places that can be present in maintenance situations. This isolation from easy-to-access reference material differentiates this step as having a high concentration of possible KB errors, for example:

SB There is a noise in a pump-motor unit, the technician notices the noise as assesses it as a potential failure but gets side-tracked and fails to report it. (Assess Sub-task)

RB The engineer investigates the noise in the pump-motor unit in the field but decides it is ‘normal’ when it is not. (Assess Sub-task)

KB The vibration analyst diagnoses the pump has a bearing failure due to lack of lubrication but the cause was a seal failure. (Diagnose Sub-task)

The sources for error during Assessment and Diagnostic tasks that have significant impact are less often slips or lapses in memory (SB errors), but rather stem from the complexity in diagnosis of the cause of machine failures or sub-optimal performance. Machines of the same make and model can be at different life stages, have experienced different operating profiles and maintenance events. This means that all machines are subtly different and hence diagnostic rules that should work on one machine, do not always work on another. This situation results in RB and KB errors during execution. The need to address unfamiliar situations in the field, often in a remote location and without immediate access to knowledge bases, compounds the need for more sophisticated approaches toward technological enhancement of agents executing maintenance. Rapid advances have been made in assisted reality glasses and headsets for diagnosis. These tools need to be supported by trained people and new business processes.

4.3. Maintenance Execution Task Errors

The Execute Maintenance task occurs when the maintenance action is explicitly performed. Work can be a routine job (performed regularly) or non-routine, involving new steps that may not be familiar. The Execute task is also highly human-dependent. Routine work includes many SB errors, while non-routine work involves more RB and KB errors, as described below.

SB Technician 1 forgets to loosen the motor when installing new v-belts. (Perform Sub-task)

RB Technician knowingly replaces only one of the v-belts rather than the whole set as he/she had done this last time and there has been no adverse repercussions. (Triage Sub-task)

KB The technician knows that an adjustment in alignment needs to be made for thermal growth but cannot remember the rules and formula. (Triage Sub-task)

The errors out of Execute Maintenance are dependent on the type of work. Issues like forgetting a set of tools before reaching a job location, or accidentally forgetting to loosen the motor while in a hurry, are not necessarily able to be mitigated through direct automation. Rather, these tend to ease over time on an individual level with experience on the shop-floor and better planning. Obvious aides like digital assistants may be additions to speed up this process, however, automation systems are not capable of replacing a human at the skills-level in this manual, tacit, dexterity-intensive task. RB and KB errors in this task stem from lack of appropriate experience of the technician. One approach to mitigating these inexperience errors, especially where effective rules do exist in the expertise of senior staff, is a buddy system [65]. Such a system is increasingly being digitally augmented through the use of assisted reality as described in Section 4.2 or remote support systems which allows a technician to bring in an expert virtually.

For the KB errors, training is always useful, though it is impossible to train for every occurrence. Knowledge-bases tend to be of limited use here, since ergonomic constraints (like gloves, ambient/background noise, etc) make interfacing with traditional digital systems — or even documentation — rather difficult. However, recent developments in assisted- and augmented- reality displays (AR/VR) can by-pass this ergonomic problem, especially in preparation for jobs on difficult or seldom accessed equipment. It is important to remember that these displays do not provide such functionality out-of-the-box, and several supporting technologies, like interconnected data storage and digital twin reference models, will need to be successfully adopted prior to reaping benefits from the continually decreasing cost of this exciting technology.

4.4. Completion and Recording Task Errors

When addressing the state of data recording in maintenance, regardless of sensor-outfitting or other types of datastreams, one goal of recording maintenance information is to capture the activities of a person executing maintenance — their ability to diagnose and solve problems. Recording this information requires the technician to recall features associated with the work order that distinguish it from other work orders. Once these features have been recalled, they must be recorded by translating into a format acceptable for predefined data structure required by the CMMS.

Recalling features poorly is typically a sign of unstructured work. Slips and lapses are more likely to occur in this recall phase, e.g.:

SB Emergency MWO’s 18 and 19 were executed yesterday, but work pressures meant the information was not recorded until today. Cannot recall which of MWO 18 or 19 was the seal replacement on Machine A3 (Recall Sub-task)

Structured work tends to reduce the likelihood of this type of failure mode: pre-documented assignments that have been planned and scheduled a priori will have associated documentation that assists or automates a significant amount of feature recall, leaving only the most relevant human actions to be input by hand. However, simply using a work order generated by a CMMS does not by any means guarantee high quality data. Translating data into a CMMS will necessitate higher-level cognitive engagement, and associated errors quickly transcend the skill-based level:

RB Technicians have been asked to classify failures with specific names and codes to help with analysis but Technician A continues to use the names he/ she has used in the past instead. (Record Sub-task)

KB There are 5 fault codes and the technician struggles to find one that actually describes the fault, so selects the “miscellaneous” code but provides no further information. (Record Sub-task)

Addressing these types of errors is more difficult. Solutions to the SB errors (e.g., standard MWO structure, pre-filled MWO forms, designated time-slots for data entry after every work order, etc.) could provide significant return on investment to improve data quality, and should be in place prior to addressing the higher-level RB and KB problems. Recommendation systems and user-interface design can be helpful in improving potential value of the recorded data. Statistical summaries of common themes in existing “miscellaneous” work order records, through the use of Natural Language Processing, is potentially useful, though care to include expert judgments must be taken when processing technical, domain-specific, short-hand-filled language [58, 60].

Recommendation systems could be applied during the completion stage to augment a technician’s ability to rapidly sort his/her knowledge into the required format. [66] If sufficient effort has been made to create and maintain digital references for an entire line, real-time suggestions for recording related symptoms or components could provide a boost to both data-quality and the speed of experience-gain for the maintenance team. Given sufficient investment, these tools could provide additional input and context that assists technicians in creating the rules and knowledge they need — combating the errors induced from often-dense, difficult-to-navigate user interfaces for selecting from a complex web of controlled vocabularies that so often occur in this space.

5. Discover Maintenance Needs

Discover Maintenance Needs tasks involve the use of software tools to create value from existing data, and inform the future workings of other tasks. These tasks are independent of structured versus unstructured work, but the tasks performed inform future structured work. Technology and research in this stage are described in the next subsection.

5.1. Applicable Research & Technologies

Discovering maintenance needs should be the product of a maintenance strategy informed by on-going analysis of asset condition, performance, and failures. Maintenance strategy is informed by an understanding of the function of the component, its failure behavior, and the consequence of loss of function as determined by a FMEA [67]. A Risk Priority Number is produced based on the likelihood, consequence, and detectability of each functional failure. Maintenance strategies are developed for the most critical functional failures using a Reliability-Centered Maintenance (RCM) or similar process [68, 69] and described in a variety of standards [1, 70]. Common names for these strategies are design out (or improvement), predictive and condition-based, preventive, failure finding, and run-to-failure. Condition-based strategies produce tasks to collect and analyze the performance or condition data (but not to do the work arising from the analysis). Run to failure strategies, employed when there is low consequence of failure and the cost necessary to prevent it exceeds the cost of the failure, result in corrective maintenance work. For RCM, the interested reader is referred to Rausand [68] and examples from infrastructure applications, such as electric power distribution systems [71, 72], maritime operations [73], and wind turbines [74]. Although RCM is widely used in defense, automobile, aerospace, and electronics for product design, there appears to be limited, well-cited literature, such as Tu and Jonsson [75, 76], on the use of RCM for the equipment used in manufacturing processes. A potential roadblock to the implementation of novel sensing and analytics opportunities is a manufacturing plant’s lack of a well-framed and functioning maintenance strategy process.

The subject of analysis in maintenance work is vast and encompasses topics such as reliability analysis, health condition diagnostics and prognostics, predictive maintenance models, strategy selection models, maintenance performance, and spare parts modeling. Despite the growing number of papers published on these topics each year, the uptake of the various models by industry is low [77]. Much work, particularly in prognostics has been theoretical and restricted to a small number of models and failure modes. There are few published examples for manufacturing systems and, more generally, on systems exposed to a normal range of operating and business conditions [78]. Published models rarely examine their practical and theoretical limitations in sufficient detail to understand when and where the model should and should not be applied. Other issues include how to assess model performance and uncertainty quantification [79].

Although asset manufacturers and operators have used sensors and manual data collection for decades to collect health data on assets, developments towards IoT offer a new opportunity where data is transmitted from assets to the Cloud [80]. In this architecture, data for health estimation (e.g., condition monitoring, environmental condition data, previous maintenance work, and operating data) is more readily available for health monitoring and prognostic assessment to assist in identifying maintenance work. The cost and ease of sensor deployment also creates opportunity for more relevant data collection. This sharing of information across assets and platforms should enable the development of a system view and the flexibility to assess and manage existing and emerging risks [80].

The potential for these developments to impact manufacturing maintenance is widely evident. Asset-owning companies are implementing software platforms to better understand their maintenance needs especially for predictive maintenance applications. The market for software platforms to integrate data from multiple sources and support the use of this data in analytics and access to the results through web applications is dramatically growing. Examples include, but are not limited to, Dassault Systems 3DExperience [81], General Electrics Predix [82], PTC ThingWorx [83], Inductive Automation Ignition [84], and Siemens Mindsphere platforms [85]. Beyond the commercial market, free, and accessible open-source statistical and machine learning algorithm packages, online training, software platforms and visualization applications are making these capabilities accessible to manufacturers on a lower budget.

While these technologies expand the potential for better maintenance practices, they are not without complication. The growth in research in the application of the technologies is enormous but the path to wide-spread application has not yet been paved. Challenges exist with both identifying the opportunities for better prediction and with creating the infrastructure needed to get the right data at the right time.

There is no shortage of predictive maintenance models proposed in the scholarly literature with over 25,000 ‘predictive maintenance’ papers listed in Google Scholar in 2018 alone. As a result of the growing choice of available diagnostic and prognostic models, a number of papers have been written to provide guidance on model selection, for example in Leep, Lee, Sikorska [86, 87, 78].

Machine learning technology is proving useful in this context. Machine learning models train on large amounts of data to provide output predictions given new input from many large historical datasets (e.g., Neural Networks, Support Vector Machines, Bayesian Networks). Provided relevant and sufficiently-sized datasets, these data-driven models can be good at detecting and predicting poorly understood or poorly modeled system behavior without a strong dependency on the relevant physics or other dynamics. However, the nature of failure datasets creates particular challenges. Failures, particularly of critical equipment, are rare. Most equipment is replaced in whole or in part before the end of life. As a result, failure datasets are unbalanced and sparse. In addition, for reliable analytics the datasets need to be assigned meaning, or labeled, which can be an onerous task. Without this labeling, the ground truth for validation often does not exist. Furthermore, condition monitoring data, when available, is often collected on assets using different methods at different time intervals which complicates the analysis process. Poor quality data results in greater complexity of the analytic models that at best muddies inference and at worst misleads inference and produces persistent prediction bias. These contextual issues, if not dealt with rigorously in model selection and validation practice, lead to poor performance and a loss of trust by decision makers.

Another challenge in deploying analytics is that each data-driven model is developed for a specific application, resulting in the need for a plethora of models depending on the scale and complexity of the manufacturing system. At a minimum a prognostic or diagnostic model needs model selection justification, validation, application limitations, and uncertainty quantification [77]. Developing a model for each dominant failure mode involves significant time and cost, which is increased by maintenance and validation of the model as the asset ages or operating conditions change. Considerable opportunities exist to develop new processes, platforms, and standards –an ecosystem– to support these models enabling them to be more widely adopted.

To aid in technology insertion, the determine maintenance needs task is broken into sub-tasks: 1) Data Extraction, Transformation, Loading (ETL) for organizing data and 2) Modeling and Analysis. The tasks are mainly performed by engineers and data scientists, requiring system architecture knowledge, expert elicitation, and mathematical or physical modeling assumptions. The common errors for this stage are described in Table 4.

5.2. Data Extraction, Transformation, and Loading Task Errors

Data does not exist in a vacuum, and cannot provide value without intermediary steps. Collecting, storing, processing, and serving data to analysis tools are all core parts of data engineering and are relevant to MWO data. The tasks required of data analysts are typically organized as ETL. Extraction refers to getting data from relevant data sources, such as machines on the shop floor. Transformation is the act of preparing the data, such as cleaning, data type selection, or feature engineering. Loading is the process of sending the data to the another system for modeling and analysis. Because the goal of a properly implemented ETL system is the automation of data transfer and organization, sources for KB errors are typically few, occurring around misuse of software or functionality. Rather, key errors possible in these tasks are largely SB and RB:

SB Data tables do not record units for the sensor readings. (Transform Sub-task)

SB Data engineer merges two tables using the wrong join process. (Transform Sub-task)

RB The data analyst uses an Excel spreadsheet from a colleague without checking if cells are updating properly. (Load Sub-task)

KB Reliability engineer when loading data puts zeros into empty NA cells which skews all the subsequent analysis. (Load Sub-task)

Lapses in design specification, like forgetting to use proper and consistent units throughout a pipeline, are often found when communication between ETL architects and the end-users (such as engineers and data analysts) is weak. Using standard formats for things like time-stamps (e.g., ISO 8601 [88]) is a common way to build interoperable data stores, but care must be taken to account for both present and future data storage needs of the enterprise when deciding to adopt formats or processes. One way to account for these data storage and transformation needs can be the use of standard data exchange protocols [89] and manufacturing information standards [90]. Protocols should be designed through strong relationships between management, planners, and data architects, along with the providers of any digital tools being used. Because experimentation with digital pipelines can be low-risk in the early stages (i.e. does not immediately impact production), it may be worthwhile to allow acceptable errors during technology adoption phases while exploring possible ETL architectures. This ensures that the final implemented design will correctly fit organizational needs, while exploiting the most recent techniques in ETL’s rapidly shifting landscape.

Because planning for future storage and processing need is so critical, cloud computing through services like Amazon Web Services (AWS) [91] or Microsoft’s Azure platform [92] present opportunities for flexible expansion of capabilities, that can scale with the needs of a system as it grows. Likewise, understanding exactly what functionality does and does not exist in adopted or purchased software stacks is key to planning ahead. If an engineer creates custom software to facilitate specific needs of an organization, good documentation practices are necessary for future understanding of the software. However, the engineer may never create this documentation due to time constraints, thus leading to the innate knowledge of that software retiring when they leave the company. By encouraging knowledge transfer (e.g., to recent hires) and guideline creation, risk of relying on customized software and code for critical tasks can be reduced.

5.3. Modeling and Analytics Task Errors

Once data is loaded for analysis, it is analyzed through use of statistical summaries, model training, and data visualization. While there is no universal procedure for data analytics, there are practices to follow when modeling and analyzing data: 1) detection of trends in data, 2) defining of patterns between data types, and 3) identification of causal relationships and application potential. These practices map well with the GEMS, since the goals of each stage are similar to the goals of a problem solver — noticing a problem, noticing patterns that fix the problem, then understanding the mechanisms that cause the problem.

SB Bearing sensor data from the past 5 months can be accessed via disparate spreadsheets, which indicate nominal system health over time, despite a 2-week period of increasing vibration (Detect Trends Sub-task)

RB Analyst estimates tool wear overhead with a physics-based model that calculates a mean time to failure metric for a part; this model is not calibrated for one of the required depth-of-cut + diameter combinations (Define Patterns Sub-task)

KB A neural network trained with infrared maps of steel heat predicts a quality drop is imminent. The analyst is unable to determine a probable cause of the failure mode from the model, despite previously good model accuracy. (Identify Causality Sub-task)

The first example is an oversight and can be addressed by making the data easier to see and accessible to more people. Numerous no cost, open source tools exist to perform initial statistical summary of data for reporting the key indicators of a need for work, and to visualize the data in ways that make trend detection at multiple time- and length-scales much more obvious. Starting to standardize or automate data pipelines that explicitly result in basic plots for machine performance summaries can be beneficial in building trust in an automated system [93, 94]. Addressing these errors is intertwined with proper ETL solutions, as discussed above: having a controlled data repository that structures and cross-links information makes designing and deploying such visualizations much easier. When well designed ETL is combined with dashboards (typically centralized displays of real-time streaming information), these types of well designed visualizations can take steps into mitigating RB and KB errors, by gathering disparate data sources into a single, easy-to-reference location which is accessible to multiple people. This makes the error of missed trends much less likely, and informs the creation of new rules for standard work.

A second major mitigation strategy, especially for the RB analysis errors, is the use of data-driven or hybrid data + physics models, for the detection and exploitation of patterns in observed equipment or system behavior for predicting health or performance. These are typically considered as part of Prognostics and Health Monitoring (PHM), a rapidly advancing sub-field of reliability engineering as described in Section 3.1. A key trade-off for using such models is that while high-accuracy predictions can be made when high quality and high quantity data is available, the models are not always interpretable, can be over-fitted, and may not be indicating causal (but rather coincidental) links between inputs and outputs. It is obviously better to be alerted to a prediction of failure than not, but if actionable strategies based on causal relationships are required, significantly more effort may be needed. False positive alarms reduce trust in the analysts and their predictions [95, 96]. Some forms of semantic and causal reasoning is possible, perhaps through design of custom ontologies or high-fidelity physics simulations, but implementing these tailor-made solutions presents a barrier, in infrastructure, labor, and research costs. Fortunately, expert knowledge can sometimes be applied to identify causation.

Based on this, investment only in the analysis stage starts out with high potential returns, but reaches a horizon as the needed technology to infer context and causality reaches the edge of the state-of-the-art. Readily-available technologies can assist analysts in addressing SB and RB errors is an efficient way to encourage them to use their own domain expertise in determining causality and potential strategies. This lays the groundwork to enable higher impact improvements in KB-intensive tasks, like execution and scheduling.

6. Discussion

Sections 3–5 provide a high level task and error analysis of the maintenance management workflow. Some common errors are classified according to Reason’s GEMS framework (skill-, rule-, and knowledge-based errors). Careful consideration is made to distinguish between structured versus unstructured job tasks and errors. While the tasks and errors had much overlap, often they were performed by different roles within the organization and at different time scales. The errors are largely the same, but they occur more often with unstructured work. Unstructured jobs require decisions made in near real time by roles in the organization that are not meant to be making these decisions (e.g., a maintainer estimating severity of a failure on the fly) and in high stress situations (e.g., during a machine failure that can lead to production impacts). Shifting towards a more structured maintenance paradigm is important for an organization’s success with new technology insertion. The steps provided in this paper are a beginning in this direction, however, as discussed, technological solutions are not the only mitigation strategy; cultural shifts are necessary as well [97].

Mitigation strategies are discussed for commonly occurring errors, independent of structured or unstructured jobs. These mitigation strategies range from necessary cultural changes to advanced AI solutions, however, these errors and mitigations do not represent every possible situation at different manufacturing facilities. How does a manufacturer repeat this same process and how should they implement their own technological solutions?

If one approaches modernizing a factory with new digital technological solutions as a problem solving situation, in a similar process to GEMS and the above discussion, the first step begins when stakeholders in the organization begin to perform an attentional check [25]. This step must happen before any problem can be solved because, by definition, this check identifies SB errors that are occurring without conscious recognition that something is wrong by the human actor.

For example, an operator not noticing an alarm that indicates failure is a potential SB error identified in Table 2. One solution to this problem, could be to install a sensor visualization dashboard to display the performance of the system for many to view. This solution could potentially solve the problem, but requires new sensors, logic for failure identification, visualization packages, etc. Does this solution always mitigate the error of not noticing a failure? By capturing new data and creating a new visualization solution, will the visualization clearly indicate failure so that the error does not occur? If the operator can miss an alarm, he/she can very easily not notice an icon in a visualization dashboard. It might not come to light that this solution is poor until a high investment in both time and cost is sunk into the project. A low technology solution might be a better answer to initially solve this error. For example, implementing a prototype visualization using existing data sources to ensure the operator can adapt to the new technology, or instilling a cultural enabler that could include a buddy system, so the operator can learn from more experienced colleagues could alleviate this error.

One of the most common causes for technology implementation failures, such as new CMMS, is an inability to make the necessary cultural changes [26]. Too often proponents of new software systems believe that the software implementation is the change, rather than putting effort in appropriate organizational change management processes to support the installation. For example, if operators and technicians are making many SB errors, such as not remembering significant symptoms from a maintenance job, a CMMS system will not immediately solve this problem. These types of errors often occur because the technician has no incentive to enter correct, long-form data about the maintenance job. In fact, the poor organizational culture encourages bad data entry, as these technicians are judged on how well and how quickly they solve the problem, not on the quality of the data [49]. If a CMMS system was installed, without addressing the SB errors, the technicians would still follow bad data entry practices, albeit on a much more expensive system. However, by discovering and attempting to mitigate SB errors, we enable a more efficient investigation into more emerging, sophisticated technologies that hold greater promise to automate systems and more directly assist human decision making.

As SB errors are mitigated, a trust for new technologies builds. By alleviating the SB errors, the differences between the RB and KB errors will also emerge. It is typically the most knowledge intensive tasks for which humans are required, making these errors some of the most difficult to completely mitigate with new technologies. The next step after discovering and mitigating SB errors is investigating the emerging technologies for RB errors.

RB errors, by their nature, involve patterns and rules that are misapplied or an inappropriate rule. The avenues to identify the occurrence of rule-based errors include the use of digital pattern recognition and recall processes. For example, routine tasks can be aided through machine learning technologies that can learn the important features of the work. This technology augments the planner, scheduler, engineer and technician, who can use the knowledge to make appropriate decisions and focus on other tasks in their job.

Once the SB and RB errors are mitigated, manufacturers can attempt to address KB errors. As stated above, KB errors are difficult to mitigate with automation and are better suited for augmentation technologies that aid the human in the task. For example, imagine a technician attempts to solve a problem that he or she has never encountered before. To completely replace the human actor, in this scenario, with a robot is not realistic with the current technology solutions. It may be cost and time-effective to investigate AR solutions that can link with more experienced technicians and Computer Aided Design (CAD) drawings of the asset to visualize and talk through the current problem. However, while the solutions themselves might be low cost, creating an environment that connects CAD drawings, technicians, and visualization tools with assisted reality headsets, is often difficult for manufacturers to tackle if this is the first SM technology they employ.

While this procedure of error identification and technology mapping can help manufacturers, how can researchers push forward and create solutions that are used by manufacturers? Researchers are needed to reduce the cost of entry to these solutions, both in time, monetary cost, and required expertise. The exercise of identifying tasks and errors can leads to a better understanding of a manufacturer’s trouble areas and provide more concrete use cases for researchers; however, scenarios are often not enough for some data-driven techniques. Realistic datasets, that are analogous to the data that occurs during maintenance, are necessary to train and prepare the data-driven models [98]. These types of datasets would support the development of open source data analysis and visualization tools that can greatly benefit manufacturers.

As the technologies are further developed and current technologies are deployed, guidelines for when and how to use various technologies are necessary. This paper ultimately provides guidance on what types of errors are dominant throughout the maintenance procedure, but stops short of discussing at length the pros and cons of each technology solution. Researchers can create and contribute to standard guidelines on what tools work and why for specific types of manufacturing datasets and problems. Guidance is also required to determine how to turn the outputs of the data analysis tools into actionable intelligence in a consistent manner. Lastly, manufacturers need to share their success stories in implementing these technologies for maintenance management. As shown in [2], the ROI of Smart Manufacturing technology implementation in maintenance ranges from 15 % to 98 %. As many manufacturers are nervous of the cost of these technologies, more rigorous studies of ROI are necessary to pave the way for other manufacturers. This paper can provide a first step in a Smart Manufacturing journey in maintenance.

7. Conclusions and Future Work

This paper analyzes each step of the maintenance workflow: both reviewing current industrial implementations of research for each maintenance activity and providing a framework for determining the most cost effective points of entry for emerging technologies in Smart Manufacturing. The maintenance activities are broken down by tasks and potential errors are identified using Reason’s taxonomy. The errors are classified according to Rasmussen’s skill-, rule-, knowledge- performance model. This classification provides a framework to discuss the most effective areas to introduce emerging Smart Manufacturing technologies. Low- technology solutions, particularly cultural changes, can sometimes be employed to rectify skill-based errors; AI-driven solutions may solve rule-based errors; and knowledge-based errors will need high effort, high cost, and high fidelity system models to pull together many disparate data sources that form the human expertise.

Several potential areas for future exploration follow from this work:

More complete task analyses of the maintenance process —

Further human reliability research should provide a more sophisticated breakdown of tasks, sub-tasks, and the associated potential errors. This type of analysis is necessary to fully recognize the relationship between the human actors in maintenance and the technology solutions applicable to a manufacturing facility. Using a more complete task analysis on the maintenance workflow, will allow researchers to better understand the interrelationships of the human and technology within the maintenance workflow.

Systematic error identification and tracking —

Solutions are needed to provide manufacturers with guidance on how to perform this analysis across the entire manufacturing facility. A key aspect of recognizing error severity is the determination of key performance shaping factors: environmental or other contextual influences that modify error likelihood (also called common performance conditions, see [29]). Having a repository of manufacturing maintenance errors, perhaps taking a cue from the U.S. Nuclear Regulatory Commission Human Event Repository and Analysis (HERA) database [99], could prove useful for more efficient error modeling, going forward.

Human models and assistance through machine learning —

In areas like maintenance that require human engagement, and tend to generate smaller data compared to other domains, up-and-coming advances in machine learning that can handle a lack of large training datasets will have an understated impact on our ability to model and assist relevant aspects of human behavior. These types of models, whether focused on reliability prediction, ergonomic optimization, or performance measures, are becoming possible through hybridized learning techniques, which exploit existing basic knowledge about some model while still adapting to new circumstances in reasonable ways. This provides a mechanism for ML to assist less experienced practitioners in learning their domain: “intelligence augmentation” over “artificial intelligence” [100]. Techniques like restricting predictions to a learned space of useful results [101, 102], discovering computational models for difficult-to-quantify user preference in decision making [103, 104], and many more, can be directly applied to better model and assist maintenance practitioner’s diagnostic and execution behavior.

Guidelines on tools that are available in Smart Manufacturing and the potential benefits and drawbacks of each method or tool —

This paper provided examples of tools that are available in industry, but did not enumerate every potential Smart Manufacturing technology. More work is necessary to discuss how and when to use specific techniques for the appropriate problem in manufacturing, including not only potential benefits but also drawbacks.

Reference datasets from manufacturers for analysis comparison —

Within manufacturing, publicly available datasets mimicking real world scenarios are lacking. Without these datasets, it is difficult for analysis to provide solutions that works in real manufacturing environments.

Standard guidelines on how to perform this analysis consistently within manufacturing —

While this paper provides the first steps in this process, this work can be forwarded through standard organizations to provide simple-to-follow guidance allowing manufacturers to perform this analysis on their own in a structured way.

As the factories of the future become more and more automated, the skills required to support manufacturing operations will shift from operations to maintenance. In this environment, manufacturers need to understand the best place to start with implementing these emerging technologies. The optimal path forward with technology for maintenance is not to replace a human in the workflow. A solution that augments the human’s abilities will take advantage of the human’s cognitive capability while removing the reducing errors. In fact, in the future, Intelligence Augmentation (IA) might become a more practical approach, compared to AI, since it supplements human’s cognitive process at different levels of Bloom’s Taxonomy while keeping the human at the center of the decision-making process [105]. This paper allows manufacturers to stop asking how to get “smart”, but instead allows manufacturers to ask how can we “smartly” implement new technologies in maintenance with the highest probability for success by accounting for the errors the technology will alleviate.

Nomenclature

AI

Artificial Intelligence

AR

Augmented Reality

CMMS

Computerized Maintenance Management System

DES

Discrete Event Simulation

ERP

Enterprise Resource Planning

ETL

Extract, Transform, Load

FMEA

Failure Modes and Effects Analysis

HMI

Human Machine Interface

IDEF

Integrated Computer Aided Manufacturing (ICAM) Definition for Function Modeling

IoT

Internet of Things

KB

Knowledge-Based

MES

Manufacturing Execution System

ML

Machine Learning

MTBF

Mean Time Between Failures

MTTR

Mean Time To Repair

MWO

Maintenance Work Order

NLP

Natural Language Processing

OEM

Original Equipment Manufacturer

RB

Rule-Based

RCM

Reliability Centered Maintenance

ROI

Return on Investment

SB

Skill-Based

SM

Smart Manufacturing

SME

Small-to-Medium Enterprise

SMRP

Society of Maintenance and Reliability Professionals

SOP

Standard Operating Procedure

SRK

Skill-, Rule-, Knowledge-Based

SWP

Safe Work Procedure

TPM

Total Productive Maintenance

VR

Virtual Reality