Enhancing worker-centred digitalisation in industrial environments: A KPI evaluation methodology

Abstract

Recently, the European Commission announced Industry 5.0 as a strategic initiative toward a value-driven industrial transformation. This new paradigm coexists with previous Industry 4.0 revolution that has guided the efforts towards technology driven industrial digitalisation in the past ten years. As part of this Industry 4.0 strategies, numerous KPI-driven evaluation methods were proposed to cover the multiple pillars of smart industry assessment. However, they do not incorporate human workers and actors in a systematic way as drivers for digitalisation processes, as the new Industry 5.0 paradigm argues. This paper addresses this gap by proposing an evaluation methodology that incorporates multiple human actors in the digitalisation process. The final objective of this methodology is to evaluate the direct and indirect benefits of the technology-driven transformation process to achieve the goals of human workers and other human stakeholders. To this end, our methodology provides the basis for proposing assessment tools and instruments for technological and infrastructure integration, process optimisation, new functionalities and human factors benefits, and four core indicators that have been applied to a real case comparing the digitalisation processes of three different companies.

1. Introduction

During the past decade, Industry 4.0 has been widely adopted, transforming manufacturing models. Boosted by strategic government initiatives and supported by significant research efforts, a high-tech guided revolution has progressed toward the concept of Smart Factory [1]. At the heart of this innovative concept, new data-driven technologies, such as IoT and Big Data, enable seamless data sharing to make intelligent decisions through real-time communication and cooperation among manufacturing entities [2], with the aim of achieving production efficiency and flexibility.

Regarding the adoption of Industry 4.0 technology, engineering research provides several comprehensive reviews that confirm the need to measure how new digital methods, tools and systems affect the manufacturing environment [[3], [4], [5]]. Initially, performance assurance was closely related to product quality assurance, with tools and methods being translated from one to the other. In this way, system performance assurance is defined in relation to the specific objectives and goals achieved, similar to product quality assurance methodologies [6]. All these methodologies consider that the measurement procedure plays an predominant role in the evaluation process [7]. In addition, they are closely related to management methodologies such as LEAN or Six Sigma [8], which serve as a response to the need to react to unpredictable events that could affect the manufacturing environment. These methodologies propose a set of measures, such as performance, that can be used to control and monitor the manufacturing processes in accordance with the organisational goals, modify organisational behaviour, and influence key decision making. The definition of performance indicators is usually dynamic and involves a multidisciplinary approach that integrates different areas and visions [9]. The incorporation of different International Organisation for Standardisation (ISO) standards in the companies’ strategic plans, add new dimensions to the performance evaluation methodologies such as social responsibility and sustainability. In recent years, the COVID-19 pandemic and successive global economic and political crises have accelerated the adoption of quick-win solutions that respond and adapt to the new global scenario [10]. Hence, this unprecedented situation underscores the prevailing focus of current digitalisation strategic plans on economic and organisational aspects, overshadowing the crucial social and human dimensions necessary for a truly human-centre digital economy.

Such circumstances can present challenges for companies as they strive to adapt to the dynamic economic, environmental, and social conditions of today's rapidly changing society. As a response, a new concept aimed at accompanying the industrial digital revolution emerged. Industry 5.0 places the focus on the industry as a key driver of economic and societal transitions, but places industrial innovation at the service of a sustainable and human-centric transition. While Industry 4.0 represents a technology-driven transformation, existing performance evaluation methodologies predominantly focus on addressing societal needs and human-centricity through consideration of anticipated impact [11,12]. This opens an unprecedented opportunity in which two different revolutions co-exist, generating synergies with each other. The socioeconomic challenges of the technology while the need for rapid adaptation to the increase in the production of goods and competitive advantage and cross-border business opportunities requires a specific assessment process adapted to each company [13,14].

In the absence of a set of widely accepted performance measures and transferable methodologies that guide the human centre digitalisation process, it is very difficult to globally assess the effectiveness of the new digitalisation processes. In fact, companies have often developed their own indicators to evaluate the success of initiatives. However, these have been used in the methodologies of Industry 4.0 development, and have not taken into account the added value for the different human actors involved in the process, limiting the measurement of transition in terms of human-centric and resilient, as requested by Industry 5.0 [15]. In addition, they mostly refer only to the company in question and are able to provide objective conclusions whether a solution, technology, process or practice that certain effects in a shopfloor have could have similar effects in others. As the number of initiatives aimed at reusing technological components to increase the adoption of digital technologies, especially in small businesses (SMEs), increases, such as SHOP4CF, SMART4ALL, DIH-HERO research projects, among others [16], there is a need for the development of a methodological approach that allows the comparison of the digitalisation process among companies facing similar problems or utilising similar technologies. This approach would facilitate the adoption of these solutions and enablers. Hence, this study aims to propose a human-centred performance measurements system to enable the evaluation of industrial digital transformation processes incorporating innovative technologies in the assessment loop. It also aims to develop a normalisation method that allow to compare digitalisation and transformation processes from different types of industrial environments, identifying facilitators, best practices and supporting, in this way, the development of strategies from management decisions and business practices that accumulative integrate the human-centric, sustainability and resilience as core values. The proposed methodology is applied to the case of three different companies in three different countries and industrial sectors. The different nature of the digitalisation process initiative in each of the companies and the variety of use cases make it very difficult to analyse the processes from a holistic perspective, compare them, or extract best practices to use in future planning of transformation processes. Consequently, the application of this methodology was to provide a global performance comparison and evaluation of the transformation behind the Industry 4.0 and Industry 5.0 revolutions. The research is based on the formulation of a theoretical and conceptual methodology and a case study to apply and test the developed methodology.

The paper is structured as follows. Section 2 presents the conceptual methodology of the study and the relevant literature review on performance measures in industrial digital transformation processes. Section 3 describes the results of applying the methodology to the proposed multi-case study. Section 4 documents the application of KPIs to a case study that involved three different industrial environments in three different countries, discusses the results, and identifies future research lines and finally Section 5 concludes the article.

2. Methods and materials

2.1. Background

In the Industry 4.0 paradigm, automation is at the core of the smart transformation process. Cyber-physical systems (CPS) allow for decentralised decision making with the final goal of generation greater flexibility, efficiency, adaptability and improving communication between consumers and productors, creating a smart connected industry. To ensure the smart connected industry purpose those aspects, a performance measurement system plays a vital role in the transformation processes management. Therefore, the digital revolution introduced by Industry 4.0 has radically changed the collection of traditional performance measurement systems, from isolated and static historical information sources to a highly proactive collection of information, adaptable to the needs of the management system. This opens the potential to anticipate future performance instead of reacting to past problems, as traditional performance measurement systems do [17]. Research has shown the need for new adaptable performance measurement methodologies that can be applied to the changing context of the smart industry [4,18,19]. The heterogeneous nature of Industry 4.0 transformation processes adds difficulties to the performance measurement definition process. This challenging task was solved by some researchers applying performance measures from other highly complex digitalised industries, such as aerospace industry, to other less complex manufacturing industries, such as textiles or components [20,21]. In other cases, researchers combined various metrics consistent with the different stages of lean manufacturing [22,23]. Based on this approach, the literature has numerous studies to identify qualitative and quantitative performance measures [3,11,[24], [25], [26]]. However, most of them focus on the supply chain, while other areas of performance measurement research, such as smart manufacturing or human-centred transformation, have been largely scarce [3].

The way Industry 4.0 is characterised based on its technology directly determines the quality of the digital transformation at the corporate and value change levels. The literature commonly characterises associated transformation processes based on the adoption of technology trends, in which the Internet of Things (IoT), CPS, and digital twin technologies could be considered higher-tier technologies [27]. However, there is evidence that current performance measurements in industrial digital transformation processes are not aligned with disruptive technologies that accompany the Industry 4.0 transformation [4,28,29]. Reference Architectures (RA) as being adopted as blueprints for building and interoperating their systems. The main benefits of RA include increased interoperability between systems/subsystems, reduction of development costs/time by enabling reuse, reduction of development risk, improvement communication, and adoption of best practices [30,31]. However, RA has not solved the elicitation of standard requirements that facilitates the customisation of the digitalisation process for specific smart factories, as the supporting technologies and tools. This makes it impossible to understand the real capabilities of Industry 4.0 enabling technologies, anticipate barriers, or mapping supporting tools capabilities with the involved processes to be included in a systematic way into the management processes. Nevertheless, many standardisation initiatives, research projects demonstrate RAs contribute to better understanding of Industry 4.0 transformation processes from a technology perspective [32,33]. One of the main impacts of these RAs is the identification of a clear overview of the main technological challenges addressed by Industry 4.0 [30,31]. This could open up an opportunity for alignment with the technical nuances and particularities of each of the use cases with a focus on the quality of the processes. Some authors highlighted the need to change the mindset regarding reference architecture, since they should address not only issues related to existing technological development, but also the contextual situations in which these architectures will be deployed, including the combination of human-technologies relationships that make the process successful [34,35].

In recent years, the need to empower industrial transformation to take a step forward in productivity and efficiency as a key enabler to achieve current societal and environmental goals, placing the wellbeing of workers at the centre of the production process, incorporated techno-social discussion in the research arena. When the research gaps in the evaluation of transformation processes in Industry 4.0 were not yet solved, Industry 5.0 incorporates human centricity, sustainability, and resilience as the main driving forces for industrial transformation processes. With the advances of technology, especially Artificial Intelligent, the intelligent manufacturing systems incorporate human-cyber-physical systems (HCPS) in which humans, cyber systems, and physical systems collaborates with the aim of achieving specific goals at an optimised level [36], with the human more engaged in more valuable creative and intellectual work. However, when considering HCPS operations, most of the economic and environmental factors are considered first for regulation and measuring, the social and human dimensions are still underrepresented [37]. Incorporating HCPS and adaptive automation, with aims at a social-orientated workforce, is proposed in some human-centric requirements elicitation methods [38,39], but they are still paradigms under development. Even the human-centric manufacturing research was found in some of the key research publications [[40], [41], [42]] is a technological driven paradigm, while Industry 5.0 focus shifts from individual technologies to a systematic use of technological methodologies [15]. The novelty of the approach made very few studies address the specific problem of defining and assessing performance measures in industry 5.0 transformation processes [[43], [44], [45], [46]].

Most of them use current methods to complement and extend the features assessed in Industry 4.0 and focus on supporting the development of transparent, trustworthy, and quantifiable technologies that support a rewarding working environment driven by real workers’ needs. Furthermore, these approaches do not support the management processes of companies that have already started their journey with Industry 4.0 and may need to repurpose it considering the core values of Industry 5.0 [47]. This justifies the need to review and validate relevant performance measures for digitalisation transformation processes in industrial environments.

2.2. Methodology development study design

The background description highlights some of the main aspects that should be considered during the methodology design process and that could be used in the context of developing a new measurement performance system. Since the use of KPIs in industry is a common practice, with well-known generic indicators to assess and measure performance and quality, a case study research method was followed [48]. The first step was the descriptive work of the cases. A focused descriptive work approach was followed to perform the analysis of the output based KPIs for various technologies, incorporating the KPIs from the RA. The summary of these analyses is described in the background summary of Section 2. All of these KPIs methodologies have shown their usefulness in managing digital transformation efforts.

Before starting with the theoretical-heuristic work, a second step was to analyse their intrinsic limitations from a human-centric perspective. To achieve this objective, the approach from Ref. [43] was followed in order to understand the limitations of each of the approaches followed. The analysis illustrated the complex landscape of Industry 4.0 initiatives, technology-centric, and process-specific metrics that emerge when applying convention measure performance approaches. The lack of a systematic assessment of the impact of the transformation process with a worker-centre perspective limits the results. As a consequence, using case-specific goals and technological deployment evaluation do not achieve with the industrial-wide transformation goals, neither of Industry 4.0 nor Industry 5.0. Table 1 summarises the main identified limitations.

Table 1.

Summary of the limitations of current methodological approaches found during the literature analysis.

Identified problem	Limitation	Suggestions
Metrics based on particular deployed technology or service	Direct metrics do not capture indirect benefits or inconveniences from workers	Incorporate human workers as a variable during the assessment process
Metrics focused on vertical domains, with the goal of evaluation components, technologies or specific services	The metrics do not take into account the needs of workers reskilled or engagement	Incorporating human workers as a system to evaluate in the process
Companies that have similar KPIs with differing infrastructure and RA	Uncertainty in identifying best practices for evaluation and assessment and scale up practices	A process that allows the normalisation of the KPIs allowing to compare different processes
Lack of information sharing and fragmentation in the assessment processes	Unfeasibility of comparing process even in the same company. Best practices sharing not possible	Scalability-related information could enable repurposing and reusing practices and IoT deployment, facilitating the incorporation of SMEs to the transformation loop
There are not enough mature models to ensure the uptake of Industry 4.0 in some sectors and/or industry types	Lack of normalised assessment processes that allow the best practices sharing	A KPIs-based assessment that allows us to compare models would help to inform leaders and motive decision makers
It is difficult to replicate successful management processes in other companies, even in the same sector, with similar characteristics	Lack of normalised assessment processes that allow the best practices sharing	The KPIs Methodology can be used to incentivise replicability in using KPIs in the same domains

With the analysis performed in the previous stages, the third stage of the process was the identification of the elements of an effective KPI methodology. Following a grounded theory building, the structure of the proposed KPI Methodology anticipated by this study is based on conventional KPI methods [49] but without the limitation observed in the previous stage. As a result, the identification of the main common domains addressed by industrial digital transformations emerged. Previous studies have attempted different techniques to identify relevant domains. These domains detection supports rigorous identification of KPIs while providing a replicable method for self-assessment tailored to the needs of each particular industrial setting. In this study, the identification of domains takes advantage of RAs. This has an additional advantage, as it allows the identification of human stakeholders involved in the transformation process and identified as users, consumers, or producers in those Ras [31]. The next step is to formulate the models of functional relationships between human roles and KPIs.

The final step is to validate the proposed model in a multi-case study. This provides a more robust and significant validation due to the comparison of different industrial contexts and sectors [50]. The following sections describe the research carried out in detail.

2.3. The human-centred digital industry KPI methodology

The Human Centric Digital Industry (HCDI) KPI Methodology is a measurement methodology for objectively self-assessing human-centric digitalisation factory model. The methodology focusses on key elements to achieve the right balance between cost-effective automation, repetitive tasks, and human workers’ participation in value-added responsibilities. In this way, the HCDI-KPIs Methodology assesses two main components of the process: human-centricity and smartness.

The aim of the HCDI-KPI evaluation methodology is to provide a comprehensive method to generate aggregated and normalised worker-centred industrial digitalisation indicators that avoid fragmentation and enable the application to different industries, independently of their sector, size, or location.

To adapt the methodological approach to any type of industry, the HCDI-KPI Methodology embraces the uniqueness of each particular industrial environment. The key element of the HCDI-KPI methodology is that each industry is unique, considering details such as location, workforce, sectorial context, infrastructure, and more. To this end, the methodological tool takes into account the following baseline assumptions [51]:

• •Each industrial company is unique, which means that there is no ‘one-size-fits-all’ option that works across different companies and use cases.
• •KPIs should be able to adapt to the evolving needs of companies in a rapidly changing society. The selected KPIs may need to be easily revised and updated.
• •The selection of KPIs is an iterative cycle in which there are multiple pros and cons.

The novelty in the HCDI-KPIs methodology is the assessment of data at four interacting levels: integration, process, functionality, and human factors. While the first three levels are common technological challenges addressed by industrial reference architectures [30,31], the level of human factors incorporates the human-centric vision incorporated in the transition from Industry 4.0 to Industry 5.0 [15]:

• •The integration level focusses on technological, legal, and infrastructure constraints. This includes ensuring the safety of the worker, securing all information and protecting intellectual property, integrating with existing enterprise software, and adhering to company-specific deployment rules [52]. Data analytics at this level focus on technology, services, and process metrics such as network capacity, sensor coverage, conformance, security, privacy, etc. In many industrial processes, integration-level components and requirements support essential functionalities that enable high-level services such as compliance with safety and security regulations, data sharing services, interoperability with existing hardware and software, and others. Effective performance metrics at this level support infrastructure management and new application deployment.
• •The process level bundles processes that enable an industrial company to function, such as product quality, lead time, cycle time of production, and efficient utilisation of raw material [53]. Data analytics at this level are related to measuring infrastructure functions and outcomes and must consider accountability and the industrial model, as well as dependencies between services and departments.
• •The functionality level specifies the expectations for the individual components of the digitalisation loop and covers all the areas addressed by Industry 4.0-enabled technologies, including robotic manipulation, mobility, augmented reality, and artificial intelligence-enabled predictive maintenance and decision making [54]. Performance indicators at this level must consider the varying levels of technological maturity in the different services and infrastructures.
• •The human factors level focuses on processes and applications that benefits the workforce and provide worker-centred benefits access [47]. Examples include personal safety and security, well-being, productivity, and quality of life, among others. Data and analytics at this level focus on the experiences of workers, and measurements are human-centric. Subjective measures can also be included as a way to gauge changes in worker perceptions and individual satisfaction.

Establishing and assessing interactions at these four levels of analysis is a central component of this method. For example, a data sensor network deployed at the integration level can contribute to improving multiple processes at the process level and enable new automation applications at the functionality level, ultimately impacting the worker at the human factor level. Therefore, the KPIs associated with a sensor network deployment must encompass all levels of the infrastructure. To establish these interactions, information flows between levels are taken into account. This process helps identify the original use of specific data and potential additional users of specific data types beyond the original use. In addition to the interactions between levels, the methodology includes the concept of actors, defined as the relevant individuals or entities directly involved in the implementation/adoption of the digitalisation use case under review.

The methodology comprises the following metrics:

• (1)Alignment of company priorities within and across actors: A company and its workforce can set multiple goals, such as increasing productivity, improving data management, and facilitating worker participation, among others. These goals should be prioritized to guide resource management.
• (2)Actor perceived benefits: This index recognises the importance of workers in the transformation cycle towards Industry 4.0 and how their contribution and commitment contribute to the company's goals in relation to the digitalisation process.
• (3)Data flow access improvement: This index recognises the importance of accessing to the new data resources due to the digitalisation process.
• (4)Perception of the quality of the digitalisation process: This index recognises the added value of the digitalisation process for the human actors involved in the process.

2.3.1. Alignment of company priorities within and across actors

This first metric focusses on managing the alignment of the level KPIs with the overall goals and priorities of the company. The purpose of these metrics is to align the KPIs with the company's overarching goals and priorities. To calculate the Alignment Index (AI), both the company as a whole and the actors involved in the KPI assessment provide prioritisation values to the selected KPIs. For example, the actors involved in a process X in company Y include the manual worker, the supervisor, and the manager. Each of them assigns a prioritisation value to each KPI based on their respective goals and objectives using requirement bazaar based technologies [55]. The general management also provides a prioritisation value based on the overall objectives of the company. The sum of all prioritisation values per actor is one. This process should be added to the specific context of each of the companies and tools to obtain the prioritisation values could be vary from one company to another.

The prioritisation factor is calculated using formula (1). Higher AI values indicate a higher priority in the digitalisation process.

$${AI}_i=|\frac{\sum _i^n{PF}_c-\frac{\sum _i^n{PF}_i}{n}}{n}|\%$$ (1)

where PFc is the prioritisation rate of the company, and PFi represents the prioritisation values of each of the actors.

2.3.2. Human resources investment perceive benefit

The second metric distributes the investment in digitalisation effort according to the work position of human workers and the priorities of each actor in relation to the different KPIs. For example, when pursuing a wellbeing KPI, the perception of effort may be lower for manual workers compared to managers who are responsible for adapting the workforce and may experience resistance to change. The proposed Human Resources Investment Index (HRI) provides a measure of how investment in human resources (in terms of people involved in the transformation process, training, and adaptation of workers) is distributed across the defined KPIs and aligned with the distribution of these investments in different implementation phases. In this case, the raking values could be obtained combining quantitative data (investments, costs, saving hours, etc) and qualitative data (perceived effort, resistance to change, etc). This metric allows for a comparison of how effectively human resources are distributed based on the needs of the various actors involved in the process. Higher values indicate better alignment and a more human-centred approach to the digitalisation process.

$${HRI}_i=\sum _{j=1}^n\frac{{HRI}_{ij}}{n}$$ (2)

where HRI_i is the Human Resources investment indicator for each defined KPI.

2.3.3. Data flow improvement (DFI)

This metric addresses the vision of the new industrial environment in which the data of physical, operational, and human workers are integrated in a decentralised, flexible, and self-organising manner. However, the number of data flows or connected devices is not what defines a connected shop floor. This metric provides a way to normalise the availability of data flows by considering the number of new actors who have relevant access to that data flow for their daily activities. The data could be obtaining from the quantitative data of the company (i.e. direct number of number of databases shared). In this way, the success of the digitalisation process is not solely determined by the absolute number of new data sources or data flows enabled through digitalisation but rather by the usefulness of these new data accesses. Higher values indicate a better digitalisation process.

$${DFI}_i=\frac{NºDatabase+ConnectedDevices}{Nºofinvolvedstakeholders}$$ (3)

2.3.4. Perceived benefit

This last metric, Perceived Benefit Indicator (PBI), aims to gather information on the perceived benefits of the digitalisation process for each stakeholder and KPI. The resulting factors provide a measure not only of how well the digitalisation process meets performance indicators on average, but also of how much an individual target group perceives the process and incorporates it seamlessly into their daily workflows. In this case, quantitative values obtained by different collaborative methods inspired by the requirements bazaar techniques could be used to make the ranking. This information allows us to identify barriers, resistances to change, and other aspects that could potentially hinder the digitalisation processes or specific tools and services for the actors involved. Higher values indicate fewer barriers and greater commitment from the actors as a whole.

$${PBI}_i=\sum _{j=1}^n\frac{{PBI}_{ij}}{n}$$ (4)

where PBI_ij is the Perceived Benefit Indicator for each defined KPI and each target group.

3. Results

The HCDI KPIs Methodology can be applied to a variety of industrial domains. As an example, this method was applied to validate the usefulness of adopting the SHOP4CF platform as a reference architecture for developing technological projects that support human beings in the manufacturing industry.

3.1. Preparatory activities

Before deployment, a baseline assessment of the pilot sites was performed. The HCDI KPIs Methodology provides a means for cataloguing existing performance indicators and linking them to current company priorities. During this process, it offers a structured procedure to create an inventory of current requirements, associated technologies, and deployments. The flexibility of this process allows for adapting the data collection and inventory processes to enable meaningful comparison with other companies and use cases, adjusting the measurement procedures to the specific needs of each of the involved production lines. In this way, data related to specific requirements or the need to comply with company rules or standards can be collected according to the specification of each pilot site. Initially, the different pilots involved in SHOP4CF were used to collect requirements from each of their use cases for industrial digitalisation processes. This process was done following Agile methodologies [56]. Their different nature and context provided a wide range of different requirements and needs. Afterwards, the requirements were clustered to specify capabilities, features, and stories which define digitalisation processes in response to their context input and outputs. Collectively, these artefacts provide the solution intended behaviour of the solution. As a result of the clustering four categories were proposed following a traditional system hierarchy model [57] as shown in Fig. 1. Integration characterised the input features in the digitalisation system contexts, process describing the system to be digitalised, and functionalities describing the output of the system. In addition, the human factor domain was included as part of the human-centric vision of the whole process. In this way, in the integration level category, all the requirements arising from the existing legal and technical constraints were included. This includes ensuring safety of workers, securing all information and protecting IP, integrating with existing enterprise software, and adhering to the company-specific deployment rules. The process level bundles the requirements related to the manufacturing process itself, the quality of the products, the lead time and cycle time of production, as well as the efficiency of using materials and methods. The functionality level includes the expectations towards the individual elements supporting the digitalisation process and covers all the technologies around this type of processes, from robotic to AI-Preventive maintenance and decision making. Finally, human factors incorporate the vision of human workers in the process from their wellbeing and safety vision. The results of applying this preliminary phase within the SHOP4CF pilots are shown in Fig. 1, where the data collected during the baseline assessment are classified and characterised according to domain-relevant KPIs and their subcategories according to the suggested hierarchical model.

Fig. 1.

KPI hierarchy at the four levels defined by the methodology.

For each of the defined KPIs, an associated data source and data collector are assigned to proceed with the verification process associated with the received data field. In this preparation process, it is necessary not only to catalogue existing data sources and associate them with different KPI categories, but also to link them to current company goals and priorities. Once this process is completed in the different companies involved in the SHOP4CF project, the methodological proposal allows a meaningful comparison between them to be drawn from relevant experiences elsewhere, overcoming the expected differences between the different companies and the workplaces under comparison.

Once the data sources are defined, the next step is a comparative evaluation of the options. For this purpose, the proposed metrics are used to generate quantitative measurements that go beyond individual improvements in KPIs values, as they introduce alignment of the current digitalisation process with the needs, goals, and expectations of the actors involved in the process. In this case, the comparative aspects are related to the main objectives of the SHOP4CF project and are orientated towards generating evidence values in the four described Industry 5.0 transitions: Integration, Process, Functionality and Human values.

3.2. Use cases and KPI description

As a representative example of the application of the proposed methodology, we have selected three of the nine initial use cases implemented in the SHOP4CF project in three of the four main pilots. Table 2 provides a summary of the use cases using this methodology. To obtain meaningful results for this study, we have chosen the cases that have more similar KPIs and involve similar roles in the process.

Table 2.

Summary of the use cases involved in the study.

Company	Description of the process	Objective of the process	Country
Company A	The use of a robotic arm built on an AGV platform is implemented to feed the production lines of the USS6 production area in the plant. These lines have resources that are difficult to schedule for on-demand utilisation and usually require manual feeding by workers dedicated to other tasks.	Automate repetitive tasks, such as the feeding task of these lines, while human workers can dedicate themselves to other tasks.	Spain
Company B	A tool is used to reduce the need for programming different interfaces for the creation of digital models and enables the display of specific information related to assembly, thereby reducing cognitive load and stress associated with the task.	Assist human workers in training robots and support human workers in calibration and inspection tasks.	Germany
Company C	Supporting intralogistics engineers in preparing data for the (semi-) automatic generation of an intralogistics simulation (digital twin) model.	Allow for the analysis of both operators and machines and make recommendations for changes in setting positions, timing, routes, etc.	Poland

The actors involved in the three use cases have similar roles. However, new roles could be added depending on the characteristics of the industries involved in the process:

• (1)The company serves as the leader and manager of the digitalisation process. Some needs related to production improvements with a focus on enhancing workflows in the selected process were previously detected.
• (2)A shopfloor manager collaborates with manual workers and manages human and non-human resources to achieve the production objectives.
• (3)Manual workers are the end users in the proposed use case; they need to adapt their current workflows to the newly deployed digitalised solution within the use case.

The selection of use-case KPIs depends on each company's plans and goals. To this process, they have used the current KPIs methods. The methodology supports companies in the process of selecting relevant data sources in different domains to explain the achievements on the selected KPIs. Table 3 summarises the categorisation of data sources per company and the defined KPIs categories, as shown in Fig. 1. To better understand the process, Table 3 provides an overview of the KPIs defined by company and the proposed categories.

Table 3.

Data sources selected to measure the different KPIs in the different companies involved in the use case example.

Company	Integration Requirements
A	T1.3	Deployment	Reduce the stress of the human worker working collaboratively with the robot.	Downtime due to lack of material
B	T1.3	Deployment	Flexible degree of automation, based on human worker preferences and skills, and taking into account performance and quality requirements.	Workload measurement [workload]
C	T1.3	Deployment	Flexible degree of automation, based on human worker preferences and skills, and taking into account performance and quality requirements.	Workload measurement [workload]
	Process Requirements
A	T2.1	Quality	Workers can focus on their main tasks in production without taking on additional tasks.	operators number/shift
B	T2.1	Quality	Reduction of quality issues, relevant to improper assembly, robot teaching or calibration	KPI: Inventory [pcs/hours]
C	T2.1	Quality	Reduction of faulty produced vehicles	Number of faulty vehicles
	Function requirements
A	T3.5	Decision making	Efficient use of equipment without compromising automation.	Downtime due to lack of material
B	T3.5	Decision making	Provide more information to operator technicians and maintenance, enabling more informed decision-making	Number of available information sources
C	T3.5	Maintenance	Optimise the schedule and management of the workers who take care of the maintenance task	Number of kilometres without failure
A	T3.7	Data Traceability	Increase material and process traceability during transport.	Downtime due to lack of material
B	T3.7	Data traceability	Improve information on failures during the manufacturing process	Number of failures
C	T3.7	Data traceability	Reduction of the lack of parts so that I can stop production.	No parts
A	T3.1	Manipulation	Reduce the stress of the human worker need to feed the production line	Down time
B	T3.1	Manipulation	Reduce the cognitive load of the worker in following up with the robot during the assembly task, as well as for the robot calibration task	KPI1: Set-up time [min] K
C	T3.1	Manipulation	Reduce the stress of the human worker, by allowing the post-inspection and confirmation of the correct calibration of the robot.	Workload measurement [Workload]
	Human related values
A	T4.4	Acceptance and Usability	Motivation	Downtime due to lack of material
B	T4.4	Acceptance and usability	Acceptance of the system	System usability [usability score]
C	T4.4	Acceptance and usability	Acceptance of the system	System usability [usability score]
A	T4.3	Autonomy and Control	Worker autonomy	Improving the work of the staff
B	T4.3	Autonomy and control	Decision making-problems solving	Improving the work of the staff
C	T4.3	Autonomy and Control	Worker autonomy	Improving the work of the staff
A	T4.6	Safety	Reduce the stress of the human worker working collaboratively with the robot.	Workload measurement [workload]
B	T4.6	Safety	Reduce the stress of the human worker working collaboratively with the robot.	Workload measurement [workload]
C	T4.6	Safety	Reduce the stress of the human worker working collaboratively with the robot.	Workload measurement [workload]

3.3. HCDI-KPI methodology index calculation

The next step is to calculate the indexes for measuring the human-centric nature of a digitalisation process in an industry use case. The following sections outline the concepts for measuring HCDI-KPI for each of these metrics. Measurements are made for each metric and each KPI for each target group, resulting in a table of results.

Table 4 illustrates the calculation process of the alignment index (AI) for Company A, including the alignment factors per KPI and target group. Each group prioritises, using requirement bazaar and co-creative methods (i.e., focus groups, workshops, etc.) the KPIs based on the definition of the KPI definition and their own needs and goals. The sum of these prioritisation rates per worker group should be 1 to normalise the raking for both the company and the actors involved. In the company involved, the KPI rankings were obtained during a session that explained the use case objectives and characteristics, along with a presentation of the KPI definition. After this session, each group of involved workers was asked to prioritise the KPIs selected by the company for that use case, considering their needs, goals, and daily work objectives. The values were grouped by worker position (in the table: manual worker, manager, and company representatives), and the final values were calculated as the mean value of the responses from workers in the same work category. Table 5 presents the AI results per KPI in Company A.

Table 4.

Example of the calculation of alignment index with prioritisation rates per actor in Company A.

KPI identification	Manual worker	Manager	Company	Alignment Index (%)
T1.3	0.1	0.2	0.05	0.15
T2.1	0.05	0.15	0.15	0.3
T3.1	0.05	0.1	0.3	3.51
T3.5	0.1	0.1	0.2	2.19
T3.7	0.2	0.1	0.15	1.41
T4.3	0.1	0.05	0.1	0.7
T4.4	0.1	0.25	0.025	0.23
T4.6	0.3	0.05	0.025	0.23

Table 5.

Results of the final alignment index per KPI in each company.

	T1.3	T2.1	T3.1	T3.5	T3.7	T4.3	T4.4	T4.7	AI mean
Company A	0.15	0.3	3.51	2.19	1.41	0.7	0.23	0.23	1.09
Company B	2.58	3.36	1.33	1.4	2.42	0.39	0.55	0.47	1.56
Company C	2.73	0.94	2.11	0.86	1.64	0.78	0.47	0.156	1.33

According to the results, Company B has the highest alignment between the digitalisation process and the needs and preferences of the actors, as indicated by its higher AI index (1.56).

The procedure for obtaining the prioritisation values was the same in all three companies involved, and the final results for Companies A, B, and C are included in Table 4. It also includes mean AI as an indicator of the overall alignment index of the KPIs in each company.

Furthermore, this calculation also allows for identifying the elements within the use case that are collectively aligned with the overall goals of the company, including the workforce and other relevant internal actors. The higher the alignment of each actor's goals, the greater the numerical value of the AI Index. By tracking this index, we can observe how to optimise resources to meet different needs. For example, in Company A, the prioritized KPIs are mainly related to the functionality domain, which are distinct from the others. In contrast, in Company B, the highest prioritisation value is given to quality-related issues, as well as aspects of deployment and data traceability, all of which are closely related to the final production. This makes sense since the assessed use case focuses on meeting the worker's needs in simplifying the deployment of the new product production process. Finally, in Company C, data deployment and traceability receive the highest ratings. This could be due to the need to support digital twin functionalities as defined in the use case.

Table 6 illustrates the process of calculating the Human Resources Investment Index (HRI) for Company A, including the alignment factors according to the KPIs and target group. The process of obtaining individual KPI rates was the same as in the previous index calculation, involving the direct participation of a representative sample of workers from each target group. This procedure was consistent between the three companies involved. The resulting HRI index reflects the degree to which the perceived distribution of efforts across KPIs and actors aligns with the distribution priorities. For example, in the case of Company A, the highest value is obtained for decision making and safety in it is shown T3.5 and T4.7.

Table 6.

Example of HRI calculation with the prioritisation rates per actor in Company A.

	Manual worker	Manager	HRI
T1.3	0.1	0.05	0.075
T2.1	0.05	0.1	0.075
T3.1	0.05	0.2	0.125
T3.5	0.15	0.3	0.225
T3.7	0.1	0.1	0.1
T4.3	0.05	0.1	0.075
T4.4	0.15	0.1	0.125
T4.7	0.35	0.1	0.225

The final results for Companies A, B and C are included in Table 7. It also includes the mean HRI as a value indicator of the general perception regarding the resources required for the digitalisation process in each company. Higher index values are indicated better alignment and measure the effectiveness of the human resources investment strategy. In this case, Company A appears to have the most human-centred investment strategy.

Table 7.

Final HRI results per KPI in the companies involved.

	T1.3	T2.1	T3.1	T3.5	T3.7	T4.3	T4.4	T4.7	HRI mean
Company A	0.075	0.075	0.125	0.225	0.1	0.075	0.125	0.225	0.225
Company B	0.125	0.075	0.1	0.15	0.125	0.125	0.125	0.2	0.128
Company C	0.15	0.125	0.1	0.1	0.1	0.15	0.1	0.175	0.0875

Table 8 illustrates the calculation process of the Data flow improvement (DFI) for Company A, including the alignment factors per KPI and target group. In this case, the companies, along with their technical staff, defined the number of databases or new data flows generated that would be involved in the process, as well as how and who would benefit or have access to each of the parties involved. The results, illustrated in Table 9, demonstrate that the more parties have access to the data or benefit from the data access, the greater the overall benefit for the process.

Table 8.

Example of DFI calculation with the prioritisation rates per actor in Company A.

	Nº dataflows/databases	Manual worker	Manager	DFI
T1.3	4		X	0.25
T2.1	3	X	X	0.67
T3.1	5		X	0.2
T3.5	4	X	X	0.5
T3.7	2	X	X	1
T4.3	2	X		0.5
T4.4	2	X	X	1
T4.7	3	X	X	0.67

Table 9.

Final results of DFI per KPI in the companies involved.

	T1.3	T2.1	T3.1	T3.5	T3.7	T4.3	T4.4	T4.7	DFI mean
Company A	0.25	0.67	0.2	0.5	1	0.5	1	0.67	0.6
Company B	0.5	0.1	0.25	0.15	1	1	0.25	0.3	0.44
Company C	0.5	1	0.75	0.1	0.5	0.25	0.67	0.15	0.49

The final results for Companies A, B, and C are included in Table 9. It also includes the mean DFI as a value indicator of the overall effectiveness of information flows within the company.

Table 10 illustrates the calculation process of the Perceived Benefit Index (PBI) for Company A, including the factors according to the KPIs and the target group. The process to obtain individual KPI rates was the same as that used in the Alignment and the Human Resources Investment indexes, involving a representative sample of workers from each target group. The procedure was consistent between the three companies. The final results for Companies A, B and C are included in Table 11, which also incorporates the mean PBI as a value indicator of the overall alignment index of KPIs within the company.

Table 10.

Example of the PBI calculation with the prioritisation rates per actor in Company A.

	Manual worker	Manager	PBI
T1.3	0	0.1	0.05
T2.1	0.1	0.1	0.1
T3.1	0	0.25	0.125
T3.5	0.05	0.1	0.075
T3.7	0.1	0.15	0.125
T4.3	0.25	0.2	0.225
T4.4	0.25	0.05	0.15
T4.7	0.25	0.05	0.15

The resulting PBI index is a measure not only of how perceived benefits meets performance of the new process, but also of how much each of the involved actors' benefits can vary from the target.

Table 11.

Final PFI results per KPI in the companies involved.

	T1.3	T2.1	T3.1	T3.5	T3.7	T4.3	T4.4	T4.7	PBI mean
Company A	0.05	0.1	0.125	0.075	0.125	0.225	0.15	0.15	0.125
Company B	0.2	0.1	0.15	0.15	0.125	0.15	0.25	0.2	0.156
Company C	0.15	0.05	0.125	0.075	0.05	0.25	0.15	0.15	0.125

Table 12 presents a summary of the different indexes and indicators calculated within the application of the proposed methodology in the different companies, providing a comparative analysis of the results. Based on these findings, Company A demonstrated greater success in implementing management practices, as indicated by its higher HRI value compared to the other companies. This success could be attributed to Company A's strong performance in improving data flow access, as evidenced by its highest DFI index value. Company B achieved better results in terms of aligning the involved parties and perceived benefits. This suggests a high level of commitment from all stakeholders in industrial processes, mitigating potential barriers such as resistance to change or lack of interest from workers. On the other hand, while the results for Company C are not unfavourable, the low values in the HRI index imply that the implemented changes may require significant effort and the direct benefits for workers and other stakeholders may not be clearly defined. Enhancing the management of resource investment could address these issues, promoting greater commitment, reducing resistance to change, and potentially shortening training periods in the future.

Table 12.

Summary of the indexes associated with each of the three companies.

	AI	HRI	DFI	PBI
Company A	1.09	0.225	0.6	0.125
Company B	1.56	0.128	0.44	0.156
Company C	1.33	0.0875	0.49	0.125

3.4. Limitations addressed by the proposed methodology

Previously to the work presented, the authors performed a study analysis of current methodologies summarise in section 2.2. This work aim was to overcome the limitations found in this analysis with the objective of providing a tool for guiding industrial digitalisation process from a human centric perspective. Table 13 highlights how the proposed methodology overcomes the current barriers in the literature and proposes a method for addressing the gaps in the available performance measurements incorporating the principles of the Industry 5.0.

Table 13.

Limitations found in the current performance indicators methodologies, addressed by the HCDI KPIs Methodology, incorporating the foundational principles of Industry 5.0

Industry 5.0 Principles	Gaps in the current methods	HCDI KPIs Methodology proposal
Human centric	Performance indicators guided by technology	Human workers are part of the performance indicator measurements
	Performance indicators focused on vertical domains	Systematic incorporation of hidden aspects as resistance to change, human and society perceived benefits
	Companies that have similar KPIs with differing infrastructure and RA	Technology and service agnostic metrics definition
	Unique focus on a specific industrial process	Incorporation of societal aspects as part of the measurement process through the direct involvement of the human workers
Sustainability	Unfeasibility of comparing process even in the same company. Best practices sharing not possible	Generation of normalisation indicators that allows to compare processes even of the different nature and sectors
	Unfeasibility of replicating successful management processes in other companies	Domain identification and normalisation indicators defined as incentivizes for replicability
Resilience	There are not enough mature models to ensure the uptake of Industry 4.0 in some sectors and/or industry types	Flexibility and systematic identification of the factors (technological, managerial, human and societal) that could influence in the adoption process
	Poor adaptability of the current methods. Process dependant.
	Fragmentation in the assessment process	Normalisation and harmonisation of KPIs for comparing results between different process, companies and sectors

4. Findings and discursion

Industry 4.0 has changed the relationships between workers and the workplace; however, there is no systematic process to assess the deep transformation in human, process, management, quality assessment, data collection, or environmental processes around any digitalisation process, considering these changing relationships [58]. In the current post-pandemic scenario, human-centred industrial digitalisation becomes a novel vision that seeks to prioritise human workers at the core of the technological industrial revolution, even more than Industry 4.0, more focused on data-driven and automation-driven technologies. This concept has garnered recent attention in the literature, incorporating into such complex processes around Industry 4.0 transformation many other interrelated factors that would need to be classified and normalised in order to compare practices, understand success factors, and facilitate the adoption of enabling technologies and practices [59].

However, the development of assessment methodologies and performance indicators that effectively contribute to the evaluation of this human perspective and facilitate comparisons between different use cases is still in the research phase [60]. Meanwhile, the inception of different context conditions for human centre research promoted by the new Industry 5.0 paradigm increases the demand to incorporate the commitment in the digitalisation process, with the final objective of allowing employees to rely for making human centre digitalisation processes a success. The HCDI KPIs Methodology provides four indices for self-assessing the human-centred and smartness aspects in industrial digitalisation processes. The indices can be used individually if the industrial digitalisation actors may choose to place greater emphasis on one of the indices or they can be considered collectively to rate the system success. The method was built to be flexible, adaptable, and extensible. This allows us to incorporate additional metrics using the same underlying methodology. The first step in developing or revising a digitalisation process is to conduct a baseline assessment. The methodology facilitates this process, providing a comprehensive procedure to categorise existing technological deployment performance indicators into four umbrella dimensions (integration, process, functionality, and human factors). Additionally, it provides a means to link these performance indicators with the goals and priorities of current human actors. Finally, the normalisation of the indices allows comparisons to other digitalisation processes, including the ability to respond to differences between industries. The second step is a comparative evaluation of different options when a digitalisation process in a specific industry is faced. This enables the quantitative comparison of options in which each option can be modelled in the context of the final system solution. Comparison metrics include not only cost/revenue analysis, but also efficiency of use of previous investments, alignment with the priorities of human actors’ priorities, and effects between the various actors involved in the process.

4.1. Theoretical implications

Industrial digital transformation processes require a clear planification and monitoring to prevent issues related to economic, social, sustainability, and other aspects that could harm the results of the transformation and sustainability. Previous studies in the literature have focused on identifying processes drivers and barriers, providing different performance measurements to be integrated in different management methodologies [61]. However, they fail to provide a compressive method to guide and evaluate the transformation implementation taking into consideration the nuances and distinctions of technologies and context, and neither human involved in the process. The main contribution of this study is to propose a methodology to systematically recognise and involve key human actors immersed in the digital transformation process and guide them in the identification of alignment facilitators that facilitate the success of the initiating processes. The proposed HCDI-KPI Methodology is a novel contribution of the study. It provides a comprehensive analysis of complex multi-technological based industrial digitalisation process where human and robots interact with each other, and it enables quantitative self-assessment of available and commonly used performance metrics, including across the different levels of interaction. These four levels of interactions align with the current challenges of the Industry 5.0 revolution and introduces a new set of indicators associated with each level, namely integration, process, functionality, and human factors.

The AI introduces a new reference point for characterising the human-centred industrial revolution, as it incorporates the individual needs of the different human actors involved in the assessed process (the closer the alignment, the greater the numerical value of the index). This enables the identification of the diverse needs and goals of the actors and aligns them with the company's strategic vision. The results reveal that companies with greater discrepancies in the rating values of the groups of actors different KPIs between the involved obtained lower mean values in the AI. This highlights the importance of aligning needs, goals, and objectives to achieve successful digitalisation processes. By adapting KPIs to meet the needs of these actors, the AI provides a quantifiable element that effectively manages the process and aligns resources with the overall workforce and company priorities. Provides one measure of how effective the resource and workforce strategy for the digitalisation process may be.

This alignment requirement is also evident in the HCI calculation. The HCI defines a measurement that quantifies the objective and subjective investment of human and economic resources to achieve a specific process. In contrast to traditional investment-related KPIs [17], the HCI incorporates the social and human aspects of economic/resource investments, such as perceived effort, behaviour change, or the learning process. Thus, the index incorporates a social and individual dimension alongside absolute economic values that solely measure benefits, payback periods, or economic efficiency. Our results demonstrate that higher HCI values are obtained in companies where all parties assign similar importance to the resources needed to successfully complete the digitalisation process. In addition, this index provides a means for generating an alignment for all the defined KPIs across all the elements. For example, it can be used to assess how effective was the integration activities to needs the needs of specific workers roles while staying aligned with the priorities of the company. Higher index values are indicative of better alignment of the strategy proposed by the company.

The DFI normalises the availability of data flows, considering the number of human actors. Although alignment is less critical in the calculation of this indicator, it significantly influences the overall process. Given that the essence of Industry 4.0 lies in leveraging information to enhance automation processes and generate production benefits, this index facilitates the meaningful identification of the data flows assessed for each KPI and each involved actor in the process. This provides a means of assessing the information available to evaluate each of the KPIs or as an overall index for all the KPIs. In this case, higher values are an indicator of better access to information, facilitating the integration of data silos that could hamper the digitalisation process or generating resistance to change among human actors.

Finally, the PBI evaluates the extent to which perceived benefits align with performance targets, while also measuring individual effort through actor-specific assessments. Similar values between groups of involved actors can indicate greater commitment from all parties in the process. The results in our use case show that the better the alignment and perceived need for resources, the greater the perceived benefits by the actors in general, thus indicating a more successful initiation of the digitalisation process.

The study extends the literature in two significant ways. On the one hand, it contributes to the research of the Industry 4.0 transformation by offering a performance measurement system that can be used to identify areas of transformation in four identified domains according to the technological challenges proposed by industrial reference architectures. This incorporates the type of innovative technology as part of the factors to be assessed in the evaluation. In another, it incorporates the core values of Industry 5.0 into the loop by identifying key human actors in the assessment loop, allowing alignment of their different vision on the process. In addition to these two main contributions, the proposed methodology normalises the validated performance indicators to evaluate transformation performance, facilitating, in this way, the comparison between processes, the identification of best practises, and sharing previous experiences. The findings of this study will lead current digitalisation and transformation processes in the industry to move forward with a more human-centre, sustainable, and resilient industrial revolution.

4.2. Managerial implications

Collectively, this work is intended to enhance the ability of companies to use advanced technologies efficiently and effectively in improving their pathway toward a digitalisation and automation process with a focus on human workers. The findings of this study have several managerial implications for decision makers of companies involved in digital transformation processes. The challenges introduced by Industry 5.0 discussed in this study and incorporated into the proposed HCDI-KPI methodology enable managers to understand the potential challenges they may face in an attempt to adopt the industry 5.0 principles in their digital transformation processes. The proposed HCDI-KPI Methodology enables a comparative and individual evaluation of industrial digitalisation processes, with a specific focus on the participation of humans rather than only technology or processes. This contribution paves the way for future research efforts in this area while providing managers with tools to monitor their processes and identify best practices from others that fix with their specific needs and challenges. Hence, the results of this study are very useful for understanding context factors and barriers, included human related, for the implementation and the relationships between these factors, while normalising them to facilitate a real comparative environment in which the differences between companies, working places, workforces, management strategies, etc. emerges. The results also suggest that decision makers should also focus on the importance of aligning the vision of different human actors for the success of the initiated processes. This provides a complementary tool for designing horizontal digitalisation strategies between actors and contexts.

Based on this methodology, the practical requirements of the industrial process are used to structure the KPIs and their related measurement elements. This facilitates the establishment of relationships between KPIs at different levels, leading to a better understanding of the performance process. The agnostic KPI measurement procedure allows the use of various appropriate measurement tools and techniques to obtain evaluation results. The proposed indexes then normalise these results, enabling process verification and comparisons with other use cases within and outside the company. Consequently, although each company can define the measurement of its KPIs differently, as demonstrated in Table 6, the proposed methodology offers a comprehensive measurement approach that takes into account variations in technological capabilities, needs, goals, and workforce across the four domains. The finding implies that policymakers and decision makers in companies would benefit from robust information flow provided by Industry 4.0 transformation processes to structure strategic planning and operation management. Collectively, this study aims to enhance the ability of decision makers to use advanced technologies efficiently and effectively in creating more human-centre industries that contribute to the quality of life of our societies.

Furthermore, direct involvement of the different actors in the KPI priority activities, which is used to calculate the different indicators, allows the inclusion of multiple perspectives beyond the company's own vision, incorporating a more human-focused digitalisation process. This is a design principle in Industry 5.0 but traditionally management and evaluation methodologies have not been systematically taken into account. While the KPI evaluation Methodology evaluates the procedures according to companies' own business practises [62], our findings demonstrate the influence of different human factors in the efficient and effective real deployment of the transformation process. These different perspectives can be observed in our example, where the KPIs in the function domain received higher prioritisation rates among company representatives, while the human-related factor domain was rated highest among manual workers. These results highlight the importance of key role identification and the alignment of needs and requirements to ensure uniformity in the efficient management of digital transformation processes. The results emphasise the importance of clearly identify the areas and domains to address the transformation and what the industry 4.0 driven technological areas are but demonstrates hidden factors could happen related with workers and other relevant roles that can hamper the applications of industry 4.0 driven technologies and procedures for efficient management of these technologies. Finally, the methodology supports companies in data collection and data source identification, allowing for the incorporation of new information elements resulting from the digitalisation process itself. This has allowed the companies involved in the project to dynamically adapt their assessment processes by identifying important characteristics discovered during the assessment.

4.3. Limitations and future research

The proposed methodology and indexes to assess different digitalisation use cases in various types of industries demonstrate the feasibility of the proposed methodology for evaluating these technology-driven changes from a human-centred perspective. The study had few limitations, which provides opportunities for future research. One major limitation of the study is that the selection of the different roles involved in the KPIs prioritisation matrix could vary from one company to another because of their different nature and internal management procedures. A systematic identification of roles and a methodological procedure to obtain the data might have resulted in a more robust solution. In addition, there might be biases in the prioritisation of the KPIs because the different organisation structures of the companies.

The future scope of this research includes extending the model to companies by identifying and involving more groups of actors. This will allow us to establish correlations between the indexes, which will support the practical implementation of contingency plans or actions in the event of poor index values, as well as the identification of factors that hinder the process. Additionally, it enables the identification of bottlenecks in the four assessed domains, thus facilitating process improvements. Incorporating computational tools along with the supporting systems is a challenge for future work.

5. Conclusions

The purpose of this paper is to develop a quantitative approach to evaluate digitalisation processes in diverse industrial environments, under different technological conditions, with a human-centric perspective. This aims at overcoming a gap in the current performance indicators methodologies to systematically incorporate the principles of the Industry 5.0: human centricity, sustainability, and resilience. The methodology intends to normalise the performance measures, allowing us to compare different processes and identify the drivers that promote digital transformation and under which material, management, and social conditions. This has a two-fold benefit. From a scientific perspective, the proposed conceptual method addressed the current fragmentation of information and data siles that hamper the sharing of best practices sharing and the scaling up of processes. In addition, the systematic incorporation of the human workers vision in the assessment processes provides a tool to research that posits a society served by technology to make an ideal technological society-centric use of new paradigms as AI and IoT. On the other hand, from an industry management perspective, the conceptual method proposed for assessing the digitalisation process enabled the identification of potential bottlenecks across the four domains: integration, process, functionality, and human factors. These domains serve as the foundation for quantitative metrics that measure the degree to which the process is human-centred. This method is an opportunity for companies to rethink their already Industry 4.0 processes by expanding the collaboration between humans and the automatic process in a more responsible, engaged and satisfied manner. The four-core metrics of the process are alignment index, human resources investment, improvement of data flow, and perceived benefit. All metrics incorporate the human workforce as the core of the measurement process.

This study aims to provide insights that may help companies, but also local and national governments planners, to define strategies for re-purposing digital transformation processes considering human centricity, sustainability, and resilience, core values of Industry 5.0. It is essential to adopt effective strategies that focus on the social value of the production process and shift the focus from welfare to well-being. Although most industries that initiate their digitalisation processes are currently adopting their strategy for Industry 4.0, it should be applied to every sector and every organisation given the current major challenges and crisis that we are facing. The methodology developed in this document can be easily applied to any type of company structure, from small SMEs to large industries. The results obtained by applying this methodology to three digitalisation use cases in three different manufacturing companies contribute to the direction of future research and enable the quantitative self-assessment of performance metrics that incorporate the needs, goals, and objectives of the various human actors involved in the process and demonstrate its potential to overcome current gaps in the massive uptake of Industry 5.0 principles. This has the potential to enhance the ability to advance in the technological industrial revolution while prioritising human efficiency and social effectiveness, thereby contributing to the development of resilient industries in our rapidly changing society.

Funding

This work was supported by the European Commission H2020 program under Grant 873087.

Data availability statement

The data supporting the findings of this study are available from the corresponding author, upon reasonable request.

CRediT authorship contribution statement

Patricia Abril-Jiménez: Writing – review & editing, Writing – original draft, Validation, Methodology, Formal analysis, Data curation, Conceptualization. Diego Carvajal-Flores: Writing – review & editing, Writing – original draft, Validation, Methodology, Investigation. Eduardo Buhid: Writing – original draft, Validation, Investigation. María Fernanda Cabrera-Umpierrez: Writing – review & editing, Writing – original draft, Validation, Supervision, Investigation, Funding acquisition, Conceptualization.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.