Wireless Network Design for Emerging IIoT Applications: Reference Framework and Use Cases

Abstract

Industrial Internet of Things (IIoT) applications, featured with data-centric innovations, are leveraging the observability, control, and analytics, as well as the safety of industrial operations. In IIoT deployments, wireless links are increasingly used in improving the operational connectivity for industrial data services, such as collecting massive process data, communicating with industrial robots, and tracking machines/parts/products on the factory floor and beyond. The wireless system design for IIoT applications is inherently a joint effort between operational technology (OT) engineers, information technology (IT) system architects, and wireless network planners. In this paper, we propose a new reference framework for the wireless system design in IIoT use cases. The framework presents a generic design process and identifies the key questions and tools of individual procedures. Specifically, we extract impact factors from distinct domains including industrial operations and environments, data service dynamics, and the IT infrastructure. We then map these factors into function clusters and discuss their respective impact on performance metrics and resource utilization strategies. Finally, discussions take place in four exemplary IIoT applications where we use the framework to identify the wireless network issues and deployment features in the continuous process monitoring, discrete system control, mobile applications, and spectrum harmonization, respectively. The goals of this work are twofold: 1) to assist OT engineers to better recognize wireless communication demands and challenges in their plants, 2) to help industrial IT specialists to come up with operative and efficient end-to-end wireless solutions to meet demanding needs in factory environments.

I. Introduction

INDUSTRIAL practices heavily employ operational technology (OT) domain data in their production activities, such as the asset performance monitoring, work flow optimization, and plant safety management [91]. The volume and categories of industrial data exhibit a remarkable growth. For example, a modern computer numerical control (CNC) machine already produces data in the order of 30 Terabytes (TB) per year [1], and it is estimated that by 2020 a smart connected factory will generate 1 Petabyte (PB) of data per day [2]. To feed the huge data demand, the Internet of Things (IoT) techniques are introduced into vertical industrial domains. The manufacturing sector is anticipated to occupy 33% of the total IoT applications in 2025 [89]. As the industrial variant of IoT, the Industrial Internet of Things (IIoT) provides customized architectures and standardized interfaces in data acquisition, transmission, and analytics for industrial applications [3]. IIoT is credited to boosting the visibility of production processes and the transparency of control decisions. It is also actively playing in the ongoing innovations of cyber-physical systems (CPS), known under labels, such as Smart Manufacturing [4] and Industry 4.0 [5], [117].¹

IIoT, as an open and scalable information technology (IT) platform, enables the exchange of machine-typed data between industrial devices and the on-premises/cloud computing facility in the local and wider-areas industrial operations [95], [118]. Wireless communication networks are playing an increasingly important role in such a machine-to-machine (M2M) communication paradigm. Compared to their wired peers, wireless networks have feature advantages, such as connection flexibility and cost efficiency, which facilitate IIoT operations, e.g., connecting massive industrial “things” in the field, conveying the system state within open and closed-loop control processes, and serving objects in motion such as mobile robots and parts/goods in logistics flows.

In the remainder of this section, we will first walk through the state-of-the-art wireless techniques for IIoT applications to briefly review current research and implementation progresses. Next, in Section I-B, we will discuss challenges and opportunities that motivate this work. Finally, we will identify this paper’s contributions in Section I-C and introduce how the following sections are organized.

A. State of the Art of Wireless IIoT Techniques

Identifying Use Cases

Wireless use cases in current IIoT implementations can be generally classified by (1) the associated applications in industrial sectors and/or (2) the quality-of-service (QoS) levels in wireless links. For example, wireless use cases have been identified in a variety of applications including asset performance monitoring [86], [113], [118], real-time process control [16], [104], [105], inventory and logistics management [88], [112], and safety [28], [69]. The numerology for wireless networks specifies qualitative and quantitative measures on the desired performance, e.g., the data rate, transmission range, latency, and reliability. For example, we can differentiated the delay (in-)tolerant and loss (in-)tolerant services in industrial applications and design different wireless networks to adapt to their respective tolerance to the transport delay and data loss [90]. Fig. 1 illustrates a few of exemplary wireless use cases with the identified performance requirements.

Fig. 1:

Exemplary performance requirements of industrial wireless use cases. (Wireless use cases are exemplary illustrations of possible system settings, the values of which are not mandatory thresholds in the deployments; the higher performance bounds of metrics are illustrated when applicable; the data rate per link is estimated by adding a protocol overhead which is as much as 50% of the payload; and the data rate per serving area assumes that the frequency reuse factor is 1 and all links are saturate, i.e., at the peak value. Specifications of use case a, b, and c are referenced from the 3GPP TS 22.261 [8, Sec. 7.2.7]; use case d refers to the specifications in [47]; use case e showcases the Tennessee Eastman (TE) chemical process control problem [73] with wireless settings discussed in [75]; use case f and g are identified in the 3GPP TS 22.804 [27, Table 5.3.2.1], which are also used by 5G-ACIA in its report as typical 5G wireless use cases [29]; use case h presents an example of massive wireless sensor networks collecting uncompressed temperature, humidity, and pressure sampling data [27, Sec. 5.3.8].)

Wireless standard bodies have been involved in specifying the representative use cases in industrial environments [8], [25]–[27]. The European Telecommunications Standards Institute (ETSI) performed a series of site surveys and concluded that industrial applications mainly carry machine-typed data which refer to the data generated and consumed by machines [26]. Meanwhile, ETSI also identified a number of industrial use cases with specific radio communication requirements. Based on this report [26], the International Telecommunication Union Radiocommunication Sector (ITU-R) released the “International Mobile Telecommunication (IMT) 2020 and beyond” vision on the fifth generation (5G) cellular mobile systems. In 5G systems, ITU-R considers two major usage scenarios for M2M communications, i.e., massive machine-type communications (mMTC) and ultra-reliable and low latency communications (URLLC), respectively [120]. The former refers to the cost-efficient and robust connections to massive devices² without overloading the network in industrial use cases such as process sensing and control, remote machine diagnostic, and over-the-air update; the latter applies in the use cases under more stringent latency and reliability rules such as motion control applications in factory automation, safety-related updates in robotic operations, and industrial virtual/augmented reality (VR/AR) applications. Meanwhile, ITU-R also defined typical wireless industrial automation (WIA) applications and posted performance requirements for specific services [25]. Mirroring ITU-R’s 5G recommendations, the 3rd Generation Partnership Project (3GPP) identifies industrial wireless use cases in its cellular service scenarios [8]. Ever Since its first release for 5G specifications, i.e., the Release 15, 3GPP has been conducting the study of 5G Communications for Automation in Vertical domains (CAV) applications and planning the roadmap to industrial cellular deployments [27].

Industrial users are normally playing a much stronger and more active role in deciding wireless services in their plants compared to personal customers in the wireless market. The newly founded 5G Alliance for Connected Industries and Automation (5G-ACIA) has provided some inputs from industrial manufacturers in the form of white paper. In its inaugural white paper [29], 5G-ACIA released new requirements on OT-driven metrics (in process-related aspects) in addition to the network-driven performance indicators (in link-related aspects) that had been raised by wireless industries [8], [25], [27]. In the recent industrial wireless guidelines published by the National Institute of Standards and Technology (NIST), surveying procedures for identifying a factory’s radio activities and wireless needs are suggested along with the terminology used in characterizing the industrial wireless applications [32]. As robots are widely used in the plant, wireless links have been increasingly employed to connect the industrial robots to its controller or the remote supervisor [114]. The American National Standards Institute (ANSI) and the Robotic Industries Association (RIA) are collaborating on drafting a series of safety-related standards for industrial robots which include the general wireless control use cases [115] and ongoing discussions on mobile robots [116].

Wireless IIoT Features

There are no one-size-fits-all wireless solutions for industrial use cases as the service requirements and operation environments may differ vastly from one another. Earlier industrial wireless networks were mainly developed for providing the connectivity in each single vertical manufacturing sector [99]. As a result, the solutions that function well under the specific service requirements and operating conditions they were designed for, may only yield limited value in different use cases. To replicate the wireless success in more emerging IIoT applications, wireless networks are expected to facilitate the wider and deeper digital contact with industrial systems and provide flexible interfaces and quick deployments while keeping data integrity. Current discussions exhibit a few distinct features in the design principles and operations.

The first keyword is machine-typed data. The coupling between IIoT and industrial systems is realized by the definition, exchange, and utilization of data occurring in various industrial areas and processes with respective functions in the applications [93]. As a result, IIoT data have unique requirements on their formats and transmissions. Specifically, in the upper layers, discussions on the OT communication traffic are looking for new ways of extracting the industrial data patterns from daily operations, especially for the QoS requirements. For example, M2M traffics require the deterministic packet delivery which needs the performance guarantees in latency and reliability over wireless links. Otherwise, it would result in error-prone control decisions which raise risks such as property damages and personnel injuries. Meanwhile, the IIoT data can also be labeled with one or more case-specific features such as massive access requests, high updating frequencies, and/or small payloads. Former models were found less effective in new IIoT cases where the analytical prediction calculated by these classic approaches often deviates from the real performance [121]. For example, the channel capacity is estimated in the communication theory based on the assumption that a packet can always increase its payload without bound [56]. In the IIoT links, as data usually have small payloads, typically of only a few bytes, such a classic assumption does not hold [121].

Second, IIoT exhibits unique characteristics in the radio environment. The diversity and variety of plant layouts and onsite activities have attracted multiple measurement efforts on characterizing industrial radio environments [33]–[36], [102]. Measurements have been taken in the sites of different types and sizes, such as automotive assembly lines, steam supply plants, small-sized machine shops, and collaborative robotic work-cells. Topographic differences have been captured and associated with the parameter settings in wireless channel models. The measurements observed a noticeable deviation in the settings, such as the delay spread profile and path lost exponent, from the models in non-industrial cases. Even in the same plant, wireless channel characteristics may also vary with the location-specific topographic pattern, e.g., walls and machines [33]. Moreover, a plant can accommodate multiple wireless systems whose transmissions may interfere with one another in the same or neighboring spectrum bands. It is impractical to solely rely on radio regulations to protect the spectrum usage considering the large number of legacy and unlicensed wireless applications and the delay. Therefore, it is important to recognize the coexistence issues from the use case and develop interference management schemes in different situations [23].

The last but not least, wireless IIoT techniques adopt an evolutionary roadmap in compliance with the IT innovations [95]. IIoT is looking for coherence in a factory’s IT system where wireless networks are an integral part. Wireless engineers need to leverage their design from pure discussions on air interfaces to the full-stack wireless solutions considering the end-to-end service provision. Currently the industrial IT infrastructure is moving forward to aggregating the massive data at the centralized computing facility for improving computational efficiency [94]. Meanwhile the control intelligence is being deployed closer to the field and utilizing proximate compute resources to realize low-latency responses and situational awareness [87]. As a result, wireless networks need to be capable of negotiating with the IT infrastructure on the end-to-end performance once industrial services are deployed in the cloud. Otherwise, a service level agreement (SLA) cannot be reached [85]. New system integration techniques, such as software-defined networking (SDN) and network function virtualization (NFV), are developed to adapt the IT resources and management strategies for end-to-end service provision.

Evolving Technologies and Standardization Efforts

Wireless networks are still evolving to address more IIoT features in their standardization efforts [89]. Candidate wireless communications protocols for serving IIoT use cases are found to be diverse and dynamic ranging from near-field communications (NFC), e.g., the product scanning with radio-frequency identification (RFID) tags, to long range wireless transmissions, e.g., smart meters in a wide area. These standards are enablers of IIoT services by innovating key wireless techniques. Huang et al. reviewed current standardization efforts on supporting wireless IIoT transmissions [84]. Generally wireless standards can be classified by their working channels and QoS levels in the wireless services, e.g., the data rate, coverage, and energy consumption. We will briefly discuss their respective features for IIoT practices.

Deployments in Unlicensed Spectrum:

There are a huge selection of wireless IIoT systems using the unlicensed spectrum bands, e.g., the industrial, scientific and medical (ISM) bands at sub-1 GHz, 2.4 GHz, and 5 GHz. The representative techniques include the low rate wide-area (LRWA) networks, BlueTooth and BlueTooth Low Energy (BLE) techniques, IEEE 802.15.4 and its industrial variants, and wireless local-area networks (WLAN).

The unlicensed LRWA IoT techniques, e.g., Long Range (LoRa) and Sigfox, provide the wireless connectivity mainly in the retrieval of data from field devices [141]. The radio part utilizes the sub-1 GHz bands and transmit over long distances (up to tens of kilometers). The link traffic is very light, i.e., there are only a few bytes in the payload and a few seconds of air-time per device per day. They usually connect the low-cost environmental sensors and meters to the data collector. The field nodes work outdoors in a wide area deployment and can survive for years on batteries.

BlueTooth added the support of mesh networks in its latest release, i.e, BlueTooth v5.0, to serve device-to-device (D2D) connections [122]. It can be used in wearable devices for workers and safety-related industrial applications [124]. Meanwhile, its low power version, known as the BlueTooth Low Energy (BLE), can work at only one tenth of the power consumed by the classic BlueTooth devices and last for months or years before the battery replacement. BLE also supports industrial wireless communications [125].

The IEEE 802.15.4 standard along with its industrial variants, e.g., WirelessHART [12], ISA100.11a [13], and Wireless Network for Industrial Automation Process Automation (WIA-PA) [14], support the low data rate (up to 250 kbps) and long durability wireless transmissions that carry process automation data. To meet up with the transmissions of mission-critical data, the medium access control (MAC) layer replaces the original carrier-sense multiple access (CSMA) design by the time-division multiple access (TDMA) scheme that allows the deterministic allocation of transmission slots for periodic industrial updates and increases the time resolution as short as 10 ms.

Recently the IEEE industrial electronics society (IES) has identified the new trend of expanding wireless adoptions from slow paced continuous processes to more time-sensitive discrete automation applications such as industrial robots and motion control cases [77]. The IEEE 802.15.4 or BlueTooth radios can only support the low rate data transmissions for real-time industrial control applications [92]. Such discussions have triggered the inventions of new wireless Ethernet techniques, such as wirelessHP [105]. Among them, IEEE 802.11 is a strong candidate to match the industrial Ethernet’s high throughput performance [126].

WLAN techniques, mainly from the IEEE 802.11 standard family, have received worldwide success in office and home wireless scenarios. However, the early versions, i.e., the IEEE 802.11b/g/n/ac specifications, use the CSMA scheme which does not fully support the IIoT data features in mission-critical applications, such as determinism and small payloads. The IEEE 802.11 working group (WG) has started drafting a series of new amendments to adapt WLAN to the IIoT use cases. IEEE 802.11ah was the first of such efforts [127]. Working in the unlicensed 900 MHz ISM band, it achieves a denser deployment, i.e., up to 8000 devices in a coverage range up to 1.5 km, with lower power consumption compared to legacy IEEE 802.11 networks [127, Table 1]. As another effort for supporting mission-critical data transmissions, IEEE 802.11ax adopts the orthogonal frequency-division multiple access (OFDMA) MAC layer which guarantees the explicit transmission scheduling for the low latency threshold. Mean-while, IEEE 802.11ax can also serve in more challenging use cases, such as robotic motion control and VR/AR, thanks to its diverse modulation and coding schemes (MCS) to cope with high dynamics in wireless links [128]. The industrial variants of WLAN are further encouraging the adoption of WLAN techniques in vertical industrial domains. For example, the Wireless Network for Industrial Automation–Factory Automation (WIA-FA) standard introduced the TDMA scheme into the IEEE 802.11 radio targeting the class of factory automation applications for the high throughput and low latency performance [15].

Deployments in Licensed Spectrum:

IIoT techniques in the licensed spectrum are mainly developed in cellular systems which can be further classified into the cellular IoT techniques and ongoing 5G efforts.

Cellular IoT refers to a subset of technical specifications in the 3GPP Release 13 and defines the IoT deployments in the 2G/3G/4G bands.³ There are three options: narrowband (NB)IoT [22], enhanced MTC (eMTC), and Extended Coverage-GSM-IoT (EC-GSM-IoT). These techniques are designed mainly for the outdoor connections that operate at the low to moderate data rate (ranging from around 100 kbps to 1 Mbps), have long standby time (for years), allow massive radio access (with typical 100 to 10000 nodes per cell), and transmit over long distances (from hundreds of meters to tens of kilometers) [129]. In the following Release 14, 3GPP further strengthened these techniques with new features including positioning, serving mission-critical IoT data, e.g., 1 ms latency in vehicle-to-everything (V2X) communications, the mobility support for service continuity, and reduced system overhead [129].

NB-IoT belongs to the LRWA techniques and was designed to compete with its unlicensed peers, i.e., LoRa and Sigfox. Compared to them, NB-IoT has its own advantages, such as the higher data rate (up to 150 kbps), additional support for downlink (DL) transmissions, and interference management in licensed cellular bands. However, cellular IoT relies on the local carrier’s network and spectrum resources to provide the service. The cellular IIoT techniques will focus on high-valued applications for mission-critical applications and wider area deployments where they would face less challenges from the peer wireless techniques, such as low-power low-cost indoor wireless techniques including WLAN, Bluetooth, etc [83].

In 5G systems, industrial data services are considered in two main usage scenarios, i.e., mMTC and URLLC, which provide connections for the long range IoT coverage and mission-critical applications, respectively [83]. The 5G air interface, known as 5G new radio (NR), has implemented new techniques in its physical (PHY) layer and MAC layer to support machine-typed communications [98]. 5G NR defines more dedicated radio resources, such as the sub-millisecond time resolution in the transmission time interval (TTI) and OFDM symbol durations. Meanwhile, it also introduces extra transmission redundancy to improve the link reliability and employs quicker allocation schemes, e.g., smaller HARQ reply messages, to better support industrial applications in the URLLC scenario [8], [101], [110].

5G networks consider the performance optimization from the entire system architecture. For example, the end-to-end service delay consists of the air-time delay over cellular links, the transport delay in the 5G infrastructure, and the service processing time. Using network slicing in NFV, 5G can preallocate the end-to-end resources for both communications and computing to guarantee the service availability and reliability [53]. Mobile edge computing (MEC) techniques were also developed to further reduce the transfers between gateway and application servers by deploying cloud compute resources close to operating sites. Spectrum bands and base stations can also be assigned with private cellular connections so that industrial data are transmitted in the reserved channels and secured with access privileges [11].

General Purpose Technical Enablers

Besides innovations in each standard’s protocol stack, there are also active discussions on many general technical topics on improving the industrial wireless performance. These topics mainly address problems and challenges in three aspects: system models, radio resource management (RRM) schemes, and protocol interfaces.

System Modeling and Verification:

System models can be created and verified through approaches including theoretical inference, empirical measurements, and simulation/emulation tests. Since IIoT is a complex system, the models of data traffic pattens and wireless environments serve as the important reference in the design work. New system models were developed [73]. Based on these models, system verification methods using co-simulation platforms [75], [136], [138], [140], hardware-in-the-loop (HIL) experiments [139], and testbeds [137] become popular in learning the industrial environment and service characteristics.

Radio Resource Management (RRM):

Radio resource management (RRM) schemes provide reliable services in dynamic and diverse wireless environments [18]. In the RRM topics, wireless coexistence becomes critical in the deployment site of massive industrial instruments [109]. The interference management and load balancing are the main issues in the design of coexistence mechanisms [23], [24]. Technologies using cognitive radios (CR) are also used in the industrial networks to sense the ambient radio environment and estimate the interference level [107]. Besides reliable links, wireless networks are also expected to achieve energy efficiency, especially in the cases where IIoT devices are working on batteries. Accurate energy models are the key. Meanwhile, the co-design approach helps fight against the harsh industrial radio environments by considering both the industrial process state and the transmission energy in search of optimal network operations [97].

Protocol Interface Design:

Besides the data plane performance in industrial data transmissions, general discussions are also underway in the control and management planes. Studies of designing and improving protocol (plane) interfaces are identified in both horizontal and vertical directions.

Horizontal interfaces refer to the ones between interconnected nodes to fulfill specific network functions, such as the clock synchronization between devices. Timing is critical regarding the real-time performance in industrial applications [106]. Time sensitive networks (TSN) protocols in local and metropolitan area networks (LAN and MAN) target the real-time performance for mission-critical information updates [19]. The IEEE 1588 precision timing protocol (PTP) and the IEEE 802.1AS protocol were initially developed for the device synchronization in the hardwired Ethernet. Recent studies have confirmed that these protocols can also serve in WLAN for the synchronization between distributed and heterogeneous IoT devices with mission-critical data [123], [126]. The other schemes developed in TSN, such as deterministic scheduling algorithms [20], [21] and MEC schemes [96], can further reduce the end-to-end service delay and ensure the real-time performance in industrial wireless transmissions.

Vertically functions in the control plane secure the data integrity and service consistency when the IoT information flow travels through the protocol stack. IIoT assigns a unique IP address to each network device as its identity in communications. The full-length address, such as an 128-bit IPv6 address, is a big overhead for many IoT devices that operate at the low power and transmit small-sized messages, just of a few bytes. Current industrial wireless practices usually adopt the shortened subnetwork addresses in their networks to save the header space. However, such proprietary addressing methods impair the interoperability of IIoT data in the wide area deployment and increase the processing delay to allow the upper layers to translate the address between different subnets. The Internet Engineering Task Force (IETF) released the encapsulation and header compression mechanisms, known as IPv6 over Low-Power Wireless Personal Area Networks (6LoWPAN) [131], which unify such operations in the IEEE 802.15.4 devices. IETF also released the similar approach for IPv6 over BLE devices [132]. Currently, there are further discussions on replacing the IP-based addressing by topic-based information distribution mechanisms, known as information centric networks (ICN) or named data networks (NDN) [133]. IIoT deployments and wireless networks are recently considered in such a new paradigm [134].

B. Motivations of Our Work

The wireless network design in emerging IIoT applications is a multi-disciplinary challenge. IIoT revisits strategies of data utilization in industrial CPS which (1) integrate the end-to-end application procedures into the “OT-IT-OT” data flows and (2) orchestrate vertical and networked industrial systems with case-specific configurations for the incoming orders [119]. Since key performance indicators (KPIs) and network bottlenecks are dispersed in production activities, factory environments, and on-premises/cloud IT platforms, precisely identifying and tackling them in the design involves close collaborations between OT system engineers, enterprise IT architects, and wireless network planners [31]. As discussed above, most of current design efforts are still focused on specific techniques used for separate use cases or design principles for general wireless communications, which have only partially addressed the complexity of such a problem.

Industrial wireless users, especially those in the small and medium-sized enterprises (SME), may lack of the required knowledge to comprehensively reviewed their wireless needs. Recently industrial wireless users are calling for new wireless design principles and guidelines to support their digital vision on CPS and Industrial 4.0 [32], [77], [105], [135]. Wireless industries have also echoed the importance of building such a common design language allowing different parties to effectively and efficiently exchange their knowledge and opinions [83].

Among the earliest attempts, NIST has been conducting a series of measurements and evaluation activities to standardize generic procedures of implementing industrial wireless networks [30], [32], [35]. As part of such efforts, we present a new design reference framework in this paper which reviews a variety of factors in a generic use case and identifies their impact on the design of efficient industrial wireless networks.

C. Contributions and Paper Organization

Fig. 2 illustrates the structure of the proposed reference framework. In a wireless use case, the design workflow consists of four stages that are performed sequentially including 1) survey & characterization of use cases, 2) measurement & quantification of system metrics, 3) modeling & verification of design problems, and 4) delivery & evaluation of solutions. Accordingly, the whole design problem can be divided into four concatenated subproblems which have their own tasks described by the names. Three distinct moves are identified between the consecutive stages which are impact mapping, requirement instantiation, and solution exploration, respectively. In each move, the former stage’s output is translated to the next one as the input. The details about the design framework and its components will be addressed in the following sections.

Fig. 2:

Reference framework for wireless system design in IIoT use cases.

The work presented in this paper has three main contributions as follows,

• First, the proposed framework decomposes the analysis of complex IIoT use cases into separate technical domains, i.e., OT applications, IIoT data services, and IT infrastructures, each of which is comprised of structured factors that impact the wireless network performance in the target use cases. By adopting such a framework, OT engineers, IT administrators, and network planners are able to address the design issues in their domain expertise and collaboratively contribute to the wireless design.

• Second, the wireless network design is formalized as a unified process. Design elements that represent generic service requirements and available resources are identified and selected as system parameters to serve in problem objectives and conditions. Such a modeling approach ensures that wireless techniques and solutions are (partially) reusable in different use cases if they are addressing the same design elements.

• Finally, enumerating the relationships between impact factors and design elements bridges the gap between the formerly isolated OT and IT engineering domains, which enriches the IIoT knowledge base and inspires new design, measure, and evaluation efforts on wireless practices within industrial plants.

The remainder of this paper is organized as follows. First, Section II provides basic concepts of wireless IIoT used in this work. The four stages of the proposed design workflow as shown in Fig. 2 are addressed step by step in the next four sections. Specifically, Section III characterizes wireless use cases by impact factors in Stage 1); Section IV enumerates system metrics in Stage 2) and discusses the mapping from impact factors to these metrics; Section V analyzes the problem formulation in Stage 3) and identifies different roles of system metrics in formating problem statements; and Section VI discusses the current progress in Stage 4), i.e., relating case-specific solutions to the identified problems. Section VII uses a few of typical wireless use cases as examples to showcase the proposed framework in addressing specific design issues. Finally, Section VIII concludes the paper with an identification of further research and practice opportunities.

II. System Model of Wireless IIoT

In this section, we provide a brief introduction on essential wireless IIoT concepts that are used in the remainder of this paper.

A. IIoT Components and Operations

The IIoT normally comprises three primitives by their functions, i.e., field instruments, compute and storage (C/S) resources, and networks.

Specifically, field instruments, often referred to as IIoT devices, include sensors and actuators that observe and interact with production processes and physical environments in the plants. C/S resources involve hardware and software that provide various process control and system management services. As field instruments and C/S resources are distributed on and beyond the factory floor, industrial communication networks link these devices and enable data services for various applications via wired or wireless means.

Communication networks can be further divided into core networks and access networks. Core networks are often referred to as “the infrastructure”. They accommodate C/S resources for factory applications and reach out to external C/S resources, e.g., providing the Internet connection to the cloud. Access networks connect the local field nodes together, e.g., in a work-cell or along an assembly line, and help relay data from remote nodes to the core network and vice versa, e.g., in a centralized control process.⁴

In the IIoT, industrial applications subscribe to sensors’ sensing and actuators’ manipulation updates and maintain the history in local and remote databases to feed the needs of industrial analytics and plant process control. Depending on control complexity and operational cost, OT decisions can be made either in local facilities, such as programmable logic controllers (PLC), or in cloud servers via Internet connections. Modern process controllers and analytics tools tend to serve OT decisions as online services in IIoT computing. The obtained control decisions are then fed back to factory processes in terms of command messages to field actuators or analytics results for policy makers, which complete the whole industrial information life cycle.

B. Wireless IIoT Architecture

Most of wireless use cases in the IIoT are purposed for providing connections to massive IIoT devices wirelessly or enabling mobile applications. Fig. 3 illustrates a generic architecture of wireless IIoT applications used in this work.

Fig. 3:

Illustration of wireless architecture in IIoT use cases.

Given the availability and difficulty of deploying wireless networks in industrial environments, we mainly discuss the design problems of radio access networks (RAN) in this paper. Generally, RAN are composed of wirelessly connected radio terminals, relay (optional), and access points. Specifically, radio terminals operate at the edge of RAN where the industrial information flows into the network with sensing data (in the upstream flows) while control messages travel reversely (in the downstream flows) to manipulate process actuators. The IIoT field instruments can join the RAN as radio terminals if wireless network adapters are available on board. Otherwise, IIoT devices need to send their data hardwired to the neighboring wireless nodes that are available to share their RAN connections. Access points are a type of wireless nodes that manage the RAN and serve as the gateway between RAN and core networks with stable backhaul connections for the end-to-end data flows. Radio terminals communicate with RAN access points directly or via the relay help. In the case with relays, the end-to-end path is formed, either spontaneously through the ad-hoc negotiations between wireless nodes and their neighbors, or assigned by the network manager based on instant and historic connection information.

Compared to the popularity of wireless links in RAN, core networks are usually connected by wires, such as optical fibers and Ethernet cables, that provide higher data rate and more reliable connections while at a higher cost.⁵

III. Factors Dimensioning Wireless Use Cases

The design workflow as shown in Fig. 2 indicates that the system design starts from recognizing various factors in wireless use cases. In this section, factors in wireless use cases, named dimensioning factors (DF), are identified and organized by their functions related to wireless networks and information-centric features. Fig. 4 illustrates this conceptual structure to locate these DF in a use case. Each DF may impact one or more system metrics in wireless network design which will be discussed in detail in Section IV.

Fig. 4:

Dimensioning factors (DF) in wireless use cases and their impact on wireless metrics. (The red lines indicate the connections between engineering DF and DF in the data service domain; the green lines indicate the functions of DF in the data service domain serving as the ports to the other domains. The details are explained in Section III-B.)

Generally, DF are grouped in three domains, i.e., OT applications, IIoT data services, and IT infrastructures, that form the top layer in the structure as shown in Fig. 4. From top to down in each domain, the affiliated factors can be further specified by domain knowledge in functional clusters. In the remaining part of this section, factors in each domain are enumerated along with discussing their respective impact on practical wireless problems.

A. Factor Analysis in OT Domain

Factors in the domain of OT applications address functions and activities of manufacturing systems and physical environments in a wireless use case, which vary with application tasks and plant sites. In the OT domain, the factors are classified into two clusters, i.e., environmental and engineering DF.

1). Environmental DF:

The cluster of environmental factors depicts the working environment of wireless networks and is mainly divided into topographic and mobility factors.

• Topographic factors characterize OT environmental features in a plant which include site types (e.g., indoor/outdoor), floor plans, layouts of machines and work-cells, and materials of the building (e.g., walls and beams). These factors are closely related with the radio environment and wireless transmission performance. For example, blocks and aisles formulate a radio frequency (RF) environment rich of reflections of electromagnetic waves which introduce multipath fading channels. The NIST guidelines recognize the impact of topographic factors on industrial wireless practices such as the antenna selection and installation in a plant [32]. Topographic factors may vary with their weight in individual factory environments and industries. For example, wireless channel models can be different based on measurement data from two separate sites, e.g., one is rich in highly reflective metallic materials while the other one is with highly absorptive paper products [36]. The impact can be found in the settings of channel models such as line-of-sight (LOS) and non-LOS (NLOS) parameters, the pathloss exponent coefficient, and fading statistics. The design of wireless networks depends on accurate channel models to identify primary topographic feature and characterize key impact factors in their working sites. As a result, at the beginning of a design it is important to obtain the site’s topographic information either by consulting the design under similar environments or perform a new site survey to better understand current situations.

• Mobility factors focus on depicting the impact of objects’ movements in wireless use cases. The plant environment is rich of body motions, such as raw materials and products that move in and out of production lines, operations with robotic arms, and activities of human staff. Wireless networks enable plant managers to collect real-time status data from industrial objects in motion and allow controllers to seamlessly send instructions to moving machines and workers, which used to be limited in wired networks. The mobility pattern in a plant affects the wireless system in multiple dimensions, especially in resource utilization. First, the movement of objects in industrial environments introduces the channel fading and distortion in spectrum, e.g., the Doppler effect, which reduces the quality of received signals and requires redundant resources to recover the lost data. Second, mobile devices may experience intermittent connections due to frequent location changes which could result in more handover operations and shorter connection sessions. Third, wireless devices are usually battery-powered so that the transmission range and computing capability are significantly affected along with additional complexity in power saving mechanisms. Last, mobility tends to increase the complexity of network control, e.g., the network has to reserve some network resources for incoming mobile users and/or cache their data at access points ahead to keep data flows out of interruption [37]. To predict the path of mobile nodes, Abboud and Zhuang survey different types of mobility models and classify them into micro, meso, and macro levels, each of which formulates different problems with unique resolutions on mobility effects [38]. Literally, the macro mobility model considers the node density in the serving area and the variations due to the patterns of motions. The micro model depicts the single mobile node behavior, such as its interaction with a neighboring node. The meso model considers the impact and transition between the performance in the macro and micro models.

The environmental factors are generally resource-oriented, i.e., they claim their influence on the availability and effectiveness of utilizing radio resources rather than explicitly raising requirements on data services. The topographic factors mainly function in the RF radio environment, while mobility factors shape the resource needs to support mobile nodes as discussed above.

2). Engineering DF:

Engineering factors address the OT engineering activities in the plant. Generally, these factors can be grouped into three sets which mirror the three facets of industrial systems and operations, i.e., depth of intelligence, function pyramid, and layouts and schedules. The facets indicate data-related features of industrial applications.

• Depth of intelligence depicts the usage of the process information and control data in industrial applications. The usage can be specified into multiple levels including information handling, passive problem notification, active problem notification, and decision making, which measures the integration level of application data (i.e., system intelligence) in the physical procedures [40]. Generally, the more interactions are observed within OT operations, i.e., between IIoT devices, people, and the infrastructure, the higher level of intelligence is required in the process implementations [41]. As a result, it needs a more capable wireless network to match the increased service level.

• Function pyramid identifies the level of an OT function that utilizes data in its operations. The pyramid model is commonly used to locate OT tasks in a hierarchical architecture and specify information flows between different functions. A simplified ISA-95 model (or known as the Purdue Reference Model [42]) is shown in Fig. 5 that indicates an exemplary definition on function layers. Specifically, it has four layers, from top to bottom, which are the enterprise resource planning (ERP) layer, the manufacturing execution system (MES) layer, the PLC layer, and the field. Related with data services, higher layers focus more on extracting meaningful operational insights out of a huge amount of data for the system design and policy making, while data operations in the lower layers tend to use data in a more direct way of controlling physical processes. Wireless networks can carry data for functions of different layers and follow their own transmission requirements within and between layers.

• Layouts and schedules is the third facet about OT applications. The plant layout, i.e., the arrangement of machines and production procedures on the shop floor, and the schedule, i.e., planning OT activities in time, define the orderly organization of production resources and map the flows of physical materials and logical information in production tasks. Therefore, they influence the distributions of data traffic and their loads in wireless use cases. Table. I illustrates three common layout methods and their features. These methods have their own pros and cons, and the selection in use cases largely depends on OT engineering concerns. For example, plant layouts can evolve with the maturity of production activities and the size of production orders. For new production lines with high value products but high failure rate, the job pattern may be the choice with more manual operations by experienced workers. Later on, once the production becomes mature along with larger market demand, the automated line can be used to increase volume and reduce per unit cost. The schedule on production tasks and work shifts of workers are closely bound with the plant functions and layout selection. Therefore they jointly indicate the concern of placement of production activities and resources.

Fig. 5:

Illustration of the Pyramid OT function classifications.

TABLE I:

Feature comparison between layout patterns

	Flow Shop (Product)	Batch	Job Shop (Process)
Production Spec.	Standardized parts	A group of work pieces in the same design (a batch)	Customized
Volume per order	High	Low per batch, multiple batch orders	Low
Diversity of jobs	Usually of the same type in new orders (quantitative increase)	Selected options for a new batch order	Unique per piece
Production cost per unit	Low	High	Highest
Machine arrangements [43]	Line-up in operation sequence	Concentrating machines with the necessary tooling for a batch in a cell	Concentrating machines with the same tooling method in a cell
Operation sequence	Progressively connected	Progressively connected in a batch, multiple batches in parallel in different cells	Disconnected, multiple jobs in parallel in different cells
Production automation	High (fully automated)	Medium (automated in a batch sequence)	Low
Response to different piece production	Full pause, limited options	Treat as a new batch order	Treat as a new order
Work schedules	Predetermined, 24/7	Flexible between batches, fixed per batch	Flexible
Industrial examples	Automated assembly line (e.g., electronic manufacturing, food, packing, and refining industry)	Customization of massive production (e.g., paint and trim in car assembly lines)	Machine shops, high add-on valued production (e.g., engine/aircraft assembly)
Wireless use cases	Asset performance monitoring, safety	Localization, job verification, Internet	Human-machine interface, Internet

Unlike environmental factors, engineering factors do not explicitly address their preference or requirements for the way of transmitting data, e.g., using wireless networks or optical communication links. Given the relative independence between OT and IT engineering efforts, the proposed framework defines a standard interface, i.e., data services, that ensures data transparency while encouraging design collaborations in cyber-physical systems. Engineering factors influence wireless systems through defining and customizing various data services, which is indicated as red dash lines in Fig. 4. Therefore, factors in the second domain, i.e., IIoT data services, are introduced next.

B. Factor Analysis in Data Service Domain

Data services serve as the interface between OT and IT systems to translate their different design languages to each other, e.g., from data requests by industrial applications to operation parameters in IT infrastructures. Therefore, the proposed framework treats factors about data services in an independent domain to highlight design issues in the process of matching up plant data needs with variable IT facilities.

As shown in Fig. 4, factors in the domain of data services are classified into three clusters, each of which represents one aspect of defining a data service, i.e., data semantization, utility indexing, and flow attributes. These aspects influence wireless network design in both design objectives and resource management. Specifically, data semantization takes care of data processing at the edge of wireless networks to format physical information with necessary context annotations and allow the sharing of inter-domain knowledge in the IIoT data. Utility factors map data updating/consumption patterns into different levels of services, which standardizes the measures of controlling QoS through service subscriptions. Since data flows may travel between heterogeneous communication, compute, and storage resources, flow attributes handle inter-system data dispatches so that the heterogeneity of IT systems is transparent to data services and OT applications.

1). Data Semantization:

It refers to adding general markups and notifications into the formatted data so that data are annotated with domain knowledge and context information about the served applications/processes [44]. Such add-on information enables different devices/systems to better interpret heterogeneous data that are shared in the IIoT. For example, Industry 4.0 formalizes the IIoT data of vertical industrial domains using OPC Unified Architecture (OPC-UA) as its common description languages [17]. Meanwhile, semantics also help to keep information consistent across heterogeneous systems in different applications. For example, as shown in Fig. 5, the product identity information can be used by the MES system in inventory check, and also in the field manufacturing station to retrieve the operation instruction on the incoming part. Data semantization serves as the control port of data services to the OT application domain that unifies various application data into some common description architecture shared by different IIoT devices. There are two factors in this aspect, i.e., domain knowledge and measurement system.

• Domain knowledge is used in the semantic annotation to associate the plain data, e.g., numerical values, with their target metrics in physical processes/systems, and provides necessary system background and context information to interpret the data. Domain knowledge varies with industries and regulations. By the description targets, the data can be associated with either objects or processes. The object data indicate status metrics of OT objects, such as an objects temperature, position, phase, and velocity, and the process data denote process or environmental variables which indicate the OT system performance, such as humidity, pressure, and percentage-of-completion of the process. Besides, using domain knowledge, data services can also identify the relation between different data flows to better understand and utilize them. For example, a control decision is usually made based on updates from multiple data sources, in which the received data value tagged with the identity can be weighted by specific system engineering information. Semantization mainly affects the service level, which in turn determines the set of QoS metrics in wireless systems. In the literature, common QoS metrics for industrial wireless networks can be found in [30], [45].

• Measurement system regulates the units of measurement in target OT applications as well as identifying the representation format in the annotation. Before any data service, the parties in sharing data should have agreement on metric units of raw data and annotations. Besides, semantic annotation also needs to standardize the definitions of expression precision and unit for the carried data in an implicit (between homogeneous devices) or explicit manner (between heteroeneous systems). For example, industrial systems should specify the adopted measurement system, e.g., the International System of Units (SI) or Imperial system, and the default base units in metrics, e.g., centimeter or meter, and dBm or mW, before starting any data service to transmit data [46]. The exchange, negotiation, and verification of measurement system information between the data source and subscriber consume additional network resources in data services.

Wireless systems can identify both service performance (from specific domain knowledge) and resource requirements (from measurement system overhead) out of data services as needed in wireless use cases. In [44], a comprehensive survey reviews recent progress on the standardization efforts in semantic expression formats and semantization procedures.

2). Utility Indexing:

Utility measures in data services share a set of perceptible performance metrics between the OT and IT systems in wireless use cases, which match various data requests of OT applications with service capabilities of IT resources at some common QoS level. Specifically, heterogeneous industrial data can be carried by some unified data services as long as they request the same level of QoS. Meanwhile, alternative IT solutions, e.g., wireless versus wired and cloud versus on-premises, can also support the same service as long as they have the same capability in utility measures. Therefore, the standardized data services filter out the heterogeneity of individual system instances in service definitions so that the IT and OT systems can focus on generic case solutions in their domains. Utility measures that depict different levels of transmission performance can be classified into three degrees, i.e., capacity, elasticity, and agility.

• Capacity regulates the boundaries of accumulative performance measures, such as the total number of users, the rate of multiplexed data streams, and channel bandwidth. The capacity-related metrics that label the service levels usually serve as the upper boundary in admitting new applications while acting as the lower boundary in recruiting new IT resources. For example, if a data service guarantees the data rate of 2 Mbps, it is possible to serve the plant application that requests the data rate to be ≤2 Mbps while it can only select the IT connections that support the data rate ≥2 Mbps.

• Elasticity evaluates the changes of performance measures in unit time, such as burstiness of packet arrivals, peak-to-average ratio, and error correction ratio. The elasticity-featured metrics identify the adaption and robustness of the system given the changes. For example, if a data service claims to support the error correction ratio (ECR) of 1/2, it can carry data for the application with ECR ≤ 1/2 while it would choose the device that supports ECR ≥ 1/2.

• Agility tests the recognition speed of changes in data traffics and transmission links, such as the response time to discrete system events.⁶ Agility metrics are largely related with the frequency of task scheduling and resource allocation. For example, if the data transmission is scheduled every 2 sec, such service can be used in the sensing task with the event frequency ≤ 0.5 Hz and it also requires the resource to be allocated ≥ 0.5 Hz.

As discussed above, utilities measures mainly affect individual service metrics to identify the QoS in wireless transmissions. Some typical QoS metrics can be found in different service dimensions, such as the maximum flow rate (by capacity), received signal strength of edge user (by elasticity), and latency threshold (by agility).

3). Flow Attributes:

Resembling the interactions between data semantization and OT applications, data services also have various flow attributes that serve as the control port to the IT infrastructure domain. As data flows may encounter conflicts between service demand and resource supply as well as uneven distributions of IT resources, flow attributes regulate data flows in the end-to-end connections from data sources to sinks, specify the forwarding strategy at intermediate relay nodes, and check the transmission quality in each relay hop to ensure the service satisfaction over heterogeneous IT resources. There are two factors identified as flow attributes, i.e., dispatch and tunneling.

• Flow dispatch refers to selecting the data flow paths for data services beyond a single IT system. Data flows often travel through heterogeneous IIoT devices and factory network appliances to reach remote data processing units. Network entities may use different communication protocols to disseminate data in the distributed network. Therefore, the exchange of data between different IT systems becomes increasingly common where messaging protocols are used to resolve service dispatch to heterogeneous systems. Data dissemination protocols, namely messaging protocols, employ various data transport patterns, such as point-to-point, publish-subscribe, request-response, and push-pull [48]. Traditional industrial applications usually adopt the client-server service model which uses the request/response messages to enable the paired information exchange with tight device coupling. Currently more and more IIoT data services are shifting to content-centric services. The publish-subscribe pattern fits better in such a case as it supports the many-to-many information exchange in a loosely coupled IT infrastructure. The selection of information dispatch pattern for industrial data dissemination influences wireless system design in defining network resources to support the dispatch of data flows. The impact of flow dispatch to the wireless network design is resource-oriented, which affects the meta overhead for syntax translation between heterogeneous systems (by message protocols) and the computing complexity of bridging data over different messaging interfaces. For example, when the process control flow goes wirelessly, the point-to-point synchronous message exchange between sensors and actuators requires more rigid connections for the ultra-reliable and low latency transmissions [49]. On the contrary, in the case where massive sensor data are obtained from the environment and processed remotely for analytics applications, the requested data scalability in the infrastructure leads to the adoption of the loosely coupled publish-subscribe protocol to share data. Last but not least, an IIoT use case may observe multiple data flows with corresponding dispatch patterns, it also challenges the radio resource management capability of the wireless network. For instance, WIA-FA employs three different transport communication models to exchange application data with different traffic patterns and service priorities [15]. The mapping between the higher-layer logical connections and lower-layer radio resources affects resource utilization along with scheduling between heterogeneous data flows. As an example, urgent alarm events, e.g., the emergency stop messages, should be able to override the bandwidth that has been allocated for periodic monitoring data.

• Tunneling of data flows refers to configuring various IT resources to fit data transmission requirements and ensure service consistency in the end-to-end paths. Good tunneling mechanisms are able to hide the heterogeneity of IT resources/solutions from data services. Since data services, e.g., sensing data collection, can run in different IT platforms that are available on the shop floor, e.g., RAN can use wireless connections in both cellular and Wi-Fi systems to reach field sensors, heterogeneous IT resources are programmed to serve data at the required performance level; meanwhile, the adjacent segments along the path also need to coordinate their operations with each other in translating performance metrics precisely and mapping system properties smoothly. This aspect in the data service domain mainly affects wireless systems in determining service metrics for each participating resource. For example, the delay estimated in the cellular links with scheduled time slots versus the one in contention-based Wi-Fi links are represented in different formats and at different levels of assurance to the flow performance. In such a case, the hard threshold in the former metric and the soft (statistical) threshold in the latter one should be unified into some common threshold to indicate their support on the target data flow and verified by the performance of individual layers/nodes/systems.

Fig. 4 indicates that factors in the data service domain are indispensable in wireless use cases where they have revealed the logical connections between OT and IT systems, i.e., the OT-to-IT conversions from application data to abstract service requests and the reverse ones from heterogeneous resources to unified channels, which closes the cycle of cyber-physical system design. OT engineering factors that are identified in Section III-A influence wireless system design through data services. Meanwhile, port factors in semantization and flow attributes have addressed case-specific adaptiveness and kept system-wide interoperability in the design of IIoT data services.

C. Factor Analysis in IT Infrastructure Domain

IIoT practices have significantly expanded wireless use cases in the plant from simply pairing two devices to coordinating heterogeneous industrial devices in complex factory automation applications, in which functions and operations of wireless networks have been deeply integrated into the factory’s IT architecture. On one hand, the network scale in the IIoT is being exaggeratedly extended which requires the IT infrastructure to be capable of increasing network connectivity to support data services beyond factory floors. Take the example of collecting data from multiple manufacturing sites. In such a case, data collected in local sites are loaded from RAN to the core network. On the other hand, nowadays sophisticated data applications are more likely to use online C/S resources to process and analyze the huge amount of data that are wirelessly collected from the plant. As a result, the IT infrastructure factors have become inevitably influential in wireless use cases.

The IT infrastructure generally comprises necessary resources of core networks, such as connections and C/S resources, that support data services in collecting, distributing, and processing industrial application data. Meanwhile, it also applies various management functions and policies over these resources in a particular architecture. The impact of IT infrastructures can be identified in the operations, which are divided into two types, i.e., service provisioning and architecture management.

1). Service Provisioning:

Service-oriented operations in the IT infrastructure refer to the admission, customization, and management of data services that vary diversely with their features as discussed in Section III-B. According to the processing order in a data service, these operations can be further classified into the subscription-related operations and customizations. The former one occurs before starting a new service type or when a new service flow is about to be added; and the latter one covers the life cycle of active data services to allocate and adjust resources given varying service demands.

• Subscription of data services stands for the process of examining the capability of IT infrastructures in support of incoming service requests. Besides various QoS requirements which have been elaborated in Section III-B as the flow attributes, there are also a series of requests regarding IT resources, which can be classified as M2M, machine-to-storage (M2Storage), machine-to-analytics (M2Analytics), and machine-to-process (M2Process). From left to right, the IT infrastructure is increasingly integrated with industrial applications and taking more weight in determining the performance of data services. Specifically, M2M services refer to data services close to the field where the IT infrastructure is mainly in charge of coordinating field nodes in data sharing. Only network signaling messages between them are carried in the infrastructure which does not intervene in the transmissions or processing of factory data. M2Storage allows the IT infrastructure to supervise the data plane operations by selecting proper data repositories for processing and service needs. For example, data warehouses consolidate data from various sources to obtain one version of the truth regarding the manufacturing activities; while data lakes, as another storage pattern, host variable data, both structured or unstructured, and transform them into the desired form when loading them to specific applications [50]. For M2Analytics and M2Process requests, more compute resources are expected in the application scenario as well as network management functions that coordinate operations between distributed nodes. Specifically, the service flow in the former one is unidirectional, i.e., the real-time data streams from the field to the infrastructure. In the latter case, two-way communication is enabled in the service between the field and infrastructure in a closed control cycle. To admit a service subscription which requests wireless connections and network resources in the end-to-end application path, the deployed wireless networks should work with the infrastructure according to their capability, such as the preset QoS boundaries, to evaluate the capability of serving wireless applications to meet their performance requirements in their capacity.

• Customization of the IT infrastructure refers to the operations of assigning and configuring various network resources, and it adheres to the subscription of data services and caters to resource requests in the admitted services. The main approach is network slicing, which literally separates the IT resources and management functions into virtually isolated sectors to accommodate different data services. Heterogeneous IT resources, e.g., across the infrastructures of multiple mobile network operators (MNO) [51], can also be integrated into the same virtual network for data flows along the dispatch paths. As resource allocation follows the tunneling attribute of data flows, the service performance is deemed as consistent. The customization operations identify the dynamics of service requests, e.g., the amount and type varying with time/location, and utilize predictive resource reservation mechanisms in the core network, which would impact RAN’s resource metrics that echo the infrastructure configurations in the end-to-end solutions.

2). Architecture Management:

Management operations in the IT architecture organize resources and functions structurally by their physical connections (e.g., relay, bridge, and switch) and logical relations (e.g., affiliation, supplement, and competition in conceptual domains). The operations are mainly divided into two classes of problems, i.e., the architectural support on IT management functions and the policies that regulate the ownership and authority in the IT deployment. Wireless networks are cooperating with the factory’s IT architecture in many IIoT practices. Therefore, the wireless network design follows the rules in the IT architecture to achieve interoperability between access and core network resources and service consistency.

• Resilience represents the modern design philosophy of maintaining and orchestrating the IT system architecture in treating function diversity and resource heterogeneity. Among recent innovations, SDN and NFV technologies are widely adopted in current and trending IT infrastructures [52], [53]. On the one hand, SDN technologies focus on implementing network control functions in software, e.g., scheduling data flows by their QoS priority, which used to be hard-coded in individual network devices, such as routers/switches, by the vendors. SDN controllers abstract these control functions, lift them above the device layer, and manipulate these functions in a centralized way through open control application programming interfaces (APIs). On the other hand, NFV technologies relocate network functions from dedicated appliances (e.g., routers, firewalls, and gateways) to generic servers and schedule them in a dynamic and on-demand manner. The IT architecture that deploys SDN/NFV technologies reduces the capital expenditure (CapEx) and operational expenditure (OpEx), speeds up the time-to-market of new applications, and provides flexibility and interoperability to serve variable traffic loads and operation tasks. The aforementioned customization for data services also benefits from a SDN/NFV-featured infrastructure. IT infrastructures vary with their SDN/NFV capabilities at different levels of functioning areas, e.g., in radio terminals, RAN, or core network appliances, which largely depend on engineering (e.g., the compatibility with legacy devices and functional availability in software) and economic considerations (e.g., the phased roadmap and deployment scale). As a result, SDN/NFV exerts influence to wireless networks in IIoT use cases in terms of the RAN’s compatibility with the IT architecture as well as the resource consumption and complexity.

• Procurement factors refer to the policy and economic factors in wireless use cases that are used in the selection of the appropriate usage model in the IT architecture. The IT infrastructure for IIoT use cases has become an integrated part of the ecosystem in manufacturing industries where short- and long-term investment-and-return calculations are taken. The ownership of infrastructures and factory data concerns the trade-off between economic factors and information security.⁷ Generally, the infrastructure can be deployed and managed as either the private on-premises systems, or the public cloud services. On-premises IT resources can be managed as elastic as cloud instances for factory computing needs thanks to SDN/NFV technologies. However, the capacity is never- theless restricted compared with emerging data-hungry applications, such as VR/AR and collaborative robots co-working with human staff. In such cases, depending on the depth of implementing cloud resources in the plant, the cloud services can be further classified as three layers, i.e., (IT) infrastructure, (unified network service) platform, and (data application) service. Accordingly, the Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Service-as-a-Service (SaaS) are provided in the named infrastructure-platform-service (I-P-S) cloud service architecture [54], [55]. Table II compares typical working patterns of the IT infrastructure in support of industrial communication services. The impact of IT infrastructure on the wireless network design includes the interoperability with a legacy industrial communication platform, scalability, and openness, which are mainly in the resource dimensions.

TABLE II:

Feature comparison between IT infrastructure options

	On-premises	Cloud (X as a Service)
		Infrastructure (IaaS)	Platform (PaaS)	Service (SaaS)
Virtualization	Network appliances	+ Compute and storage	+ Software API	+ Software (good-to-go)
Planner	Plant IT team	Third-party cloud solution providers and network operators
Services	End-to-end	Metal (MaaS), IaaS	PaaS	SaaS
Openness	Low	Medium	High	High
CapEx	High	Low (activation fee)	Low (activation fee)	Low/none
OpEx	Medium (utility + system updates)	Variable (subscription fee + pay-per-use)
Capacity/Elasticity	Bounded	Flexible
Scalability	Limited	Demand-specific
Data exposure	System capacity	Site layout and shift	Traffic pattern	Information interest
Exemplary usage	Proprietary IT infrastructure	“Plug-n-play” radio access networks (RAN)	Mobile Apps	Online data subscription (e.g., air quality, utility market price)

D. Remarks

The following observations are made to summarize the features of such a classification method by identifying individual factors along with their respective impact in wireless use cases:

1. Domains are divided following their separate roles in the IIoT service platform where OT applications are service subscribers that transmit sensing readings and distribute control commands, and IT infrastructures fulfill the hardware and software requirements to enable data flows. Data services are the logical container that carries the process and control information between the OT and IT systems, and standardize data flow patterns and formats for interoperability. The domain division secures the independence and transparency of intra-domain design while offering the opportunities of inter-domain collaborations.

2. Dimensioning factors capture diverse structural features and operations of wireless use cases, each of which has its own origin, class, and criteria. They are mutually exclusive in clusters and domains while collectively they can distinguish one use case from the others by enumerating commonalities and differences in individual factors. Since factors have different weights in formulating wireless problems, the domain expertise can be employed to address design requirements and concerns in respective dimensions.

3. A wireless use case can accommodate multiple domain instances in a complex industrial scenario, i.e., the multi-tenancy case. For example, multiple industrial applications can subscribe to the same sensing data service, e.g., the plant temperature reading, for different purposes, e.g., machine safety and product quality analytics. A service can also utilize multiple available IT infrastructures in the plant, e.g., local and cloud compute resources.

4. The proposed classification framework is open and scalable, which clearly illustrates the basic rules behind wireless use cases that assist industrial applications to find appropriate network resources for their data needs. Any unmentioned features in wireless use cases can be easily attributed to one or more factors in this framework. The identification of these factors shapes design principles in the problem formulation and solution development of wireless networks in the IIoT.

IV. System Metrics and Impact Mapping

Following the design workflow as shown in Fig. 2, once DF are identified in a use case, the next move is to capture how these factors impact wireless operations. In this section, we discuss the system elements that contribute to the problem formulation and illustrate how DF influence these elements. In Section V, the elements will be used as building blocks to form the design problem statements that address particular concerns in a wireless use case.

A. Wireless System Design Metrics

The goals and constraints in a design problem comprise a number of system metrics that depict the target levels of wireless service performance and system efficiency. These metrics can be generally divided into two major classes, i.e., services and resources. The former one indicates the level of service satisfaction in various measures; and the latter indicates the level of utilizing system resources.

1). Service Indicators:

Four element metrics, i.e., rate, delay, volume, and coverage, form the set of basic service performance indicators. Since wireless solutions need to guarantee QoS in industrial data transmissions, service metrics are widely used in formulating design objectives and setting the range of service parameters as the operational conditions. Each single element has its own performance focus that indicates the unique perspective of concerns in the wireless design.

Rate sets the amount of data in unit time as needed by data services. Different measures are used to identify service requirements on rate-related performance metrics, such as the total rate and good throughput, instant and average data rates in short- and long-term goals, and peak and minimum data throughputs as regulated in system conditions.

Delay, as another service indicator, regulates the data delivery speed and evaluates the timeliness of transmitting data packets, which is essentially important to industrial applications where mission-critical data, such as for real-time control or safety-related applications, have stringent delay boundaries and diminishing values as time elapses. Delay measures include the transmission delay, queuing delay, and jitter (i.e., the delay that varies over time).

Volume stands for the number of nodes that are admitted in a network or network event. The network scales up with the applications that range diversely from a few paired BlueTooth nodes to massive IoT deployments with hundreds or thousands of devices in a single network. Such a performance metric is usually closely related with estimating the load of network traffic, e.g., the number of service subscribers, and indicating the diversity gain among users, e.g., multiple relay candidates in route selection.

Coverage measures the range of wireless network connections for the provision of services. Generally, by the common transmission distance between nodes in a network, wireless networks can be classified as wireless personal area networks (WPAN), WLAN, wireless wide area networks (WWAN), and WMAN, which range from sub-1 meter to tens of kilometers.

The above four metrics serve as building blocks that form the base of indicating wireless service requirements, in which the first two focus on labeling data usage performance while the remaining two regulate the network-related considerations of supporting the services. Complex service requirements in wireless use cases can be represented by combinatorially selecting multiple indicators to form a composite service metric. For instance, wireless networks often consider the geographical distribution of traffic loads, which is jointly determined by volume and coverage. Another example can be found in defining reliability in a wireless link which can be decided under the service assumptions of the supported data rate and latency.⁸

2). Resource Indicators:

Service performance metrics can not play alone in a wireless design problem without specifying wireless network resources that are available along with their cost to build connections and transmit data. Therefore, another set of system metrics, i.e., resource indicators, are used to identify and regulate the measures and usage of network resources. By the type of manageable resources in wireless systems, resources are measured in four dimensions including time, spectrum, power, and network computing.

Time regulates the temporal usage of network resources, i.e., the order of nodes in wireless channel access as well as the length of each active wireless session/transmission. Wireless links can be granted for transmission time slots in a periodic way or on demand while data with prioritized QoS may have different access rights to the same time slot [56].

Spectrum defines another dimension of wireless channels in the frequency domain as well as the bandwidth of each channel. Since Shannon’s theorem indicates the relation between wireless channel bandwidth and achievable data rate [56], accommodating data of various services relies on spectrum resources that are properly utilized. Besides, in current spectrum policies, wireless systems that use different communication protocols are also usually bound with preset working frequency bands and channel divisions.

Power identifies both the instant transmission power at RF front ends, i.e., antennas, and the energy consumption rate at wireless nodes. The former one is closely associated with the quality of signal reception and wireless transmission range; and the latter one indicates the endurability of wireless systems in support of data services, especially when the networks are powered by batteries or renewable power sources.

Computing indicates the network resources, in both hardware and software, that coordinate wireless nodes and perform complex control algorithms in different network functions. Such a measure does not stand alone, and it should be paired with the other dimension(s) of resources. For example, the scheme of code-division multiple access (CDMA) assigns separate orthogonal spreading codes to different communication links so that multiple users can transmit in the same time-frequency resource block. The encoder/decoder complexity is independent from the time/spectrum dimension while it enhances the resource utilization within the network.

Among these four basic resource indicators, time and spectrum depict the physical availability and cost consideration of wireless network resources while power and computing are measuring the capability and complexity of supplying services with on-demand resources. In practice, wireless solutions usually utilize the resources in multiple dimensions to achieve the diversity gain in catering for various data services, e.g., the multi-antenna system uses power and complicated multi-input multi-output (MIMO) techniques to manipulate signal directions to improve wireless link quality. Better resolution in the resource space increases the flexibility in the design to agree with various service performance requirements, however, it may also enlarge the searching space of potential solutions and increase the complexity and cost.

B. Impact Mapping From Use Cases to System Metrics

Identifying key system performance and resource metrics in a wireless use case determines the complexity of the formulated problem and leads the search of optimal and heuristic solutions. As discussed in Section III, DF in different domains have the impact on separate system metrics as well as their own weights. A summary of their impact and how it is mapped to system metrics identified in Section IV-A is shown in Fig. 4.

The mapping between a factor and the associated system metric(s) can be further analyzed by the role of such factors in shaping system metrics in the design problem. Generally, three roles have been observed which are definition, characterization, and evaluation; and they are abbreviated as “Def.”, ”Char.”, and “Eval.”, respectively, in Fig. 4. Accordingly, the factors that hold such roles are referred to as defining/characterizing/evaluating factors. The defining factors identify and select the associated system metrics in the formulated design problem; the characterizing factors mainly work on the selected system metrics to further specify particular configurations in the deployed use case; and the evaluating factors instruct how to measure and evaluate the given system metrics in this system.

Fig. 6 illustrates how these roles are related. The functions of roles are conceptually organized as concatenated sets. In other words, the dimensioning factor with a higher role may also play the functions assigned for lower roles. For example, a defining factor may also specify eligible values of the selected system metric while instructing necessary evaluation approaches.

Fig. 6:

Roles of impact of dimensioning factors on system metrics.

V. Design Problem Formulation

Once system metrics are determined by the analysis of impact factors in a use case, a wireless design problem can be formulated.⁹ In this section, we propose a generic design problem model and use it to analyze how the identified system metrics contribute to the problem formulation.

A. Wireless Design Problem Model

A design problem that leads the exploration of feasible solutions in the solution space can be modeled as a set of statements of system metrics. As introduced in Section IV-A, these system metrics represent the performance requirements and operation features for a specific wireless use case or a type of use cases that share similar working patterns. Individual statements in the problem depict regulations on the metrics and the relationships between them. The statements can be classified into two sets based on their roles in the problem model: objectives and conditions. Therefore, the formulated problem can be denoted by P := G|C where G and C are the set of objective statements and the set of condition statements, respectively. Each set may contain one or more statements related with system metrics identified in Section IV-B. In the model, by default G ≠ Ø, and C ≠ Ø.

1). Objective Statements, G:

In the formulated problem set, the objective function identifies the goals of wireless system operations which can be directly related with service performance, improving the efficiency of resource utilization, or for both. The objective is usually denoted by statements on a vector of system metrics in respective dimensions, e.g., maximizing the multi-flow throughput and/or minimizing the whole network energy consumption. The subset of statements about service performance metrics in G is denoted by G_S :={G_S,1, G_S,2, …}. In the same way, the subset of resource-related statements in G is denoted by G_R := {G_R,1, G_R,2, …}. Thus, G := G_S∪G_R.

The format of each statement, G_S,i or G_R,i, can be either directive, i.e., maximizing service performance or minimizing the cost, or an answer to a validity check question, e.g., if a solution exists to meet the required performance or the given one meets all the conditions.

2). Condition Statements, C:

The set of condition statements hosts a list of conditions that indicate the relationships between system metrics, such as the channel rate versus the bandwidth, and the boundaries of individual metrics to keep them working reasonably, such as the minimum data rate of each active user and the total transmission power over the antennas. C contains two subsets of statements, C_S := {C_S,1, C_S,2, …} and C_R := {C_R,1, C_R,2…}, for target services and resources, respectively. C := C_S ∪C_R.

Condition statements are usually inherent from physical and logical features as specified by DF for different roles as shown in Fig. 6. The format of condition statement, C_S,i or C_R,i, also varies, which can be either quantitative or logical. For example, quantitative statements can define or characterize data types, valid ranges, and membership of system metrics; and logical conditions can specify the relationship between system metrics such as the conditional cost function of resources given the service level.

Remarks can be made by observing the set of problem statements:

1. System metrics of the same class, e.g., basic service (resource) indicators, are orthogonal to each other. This suggests that metrics can be combined to address design concerns of complex wireless systems in IIoT practices. Meanwhile, a multi-objective and/or multi-conditional wireless system design can be decomposed into parallel subproblems by separating statements related with different metrics so that domain knowledge is employed to search for solutions in each direction. Trade-offs can be made in between to evaluate alternative ways of tackling the original problem.

2. Both service metrics and resource metrics are necessary and complementary in the problem set. To define a wireless design problem, it should employ at least one element from each set in the statements as the problem objective and/or conditions. Otherwise, the formulated problem may introduce impractical solutions, e.g., an unsupervised improvement of the service performance without the resource limit, or a resource saving strategy by transmitting nothing without the minimum QoS definition.

3. Metric selection depends on the scope of problems that arise in the studied use cases. Take the network coverage as an example. In the IIoT paradigm, wireless data sources and subscribers may reside within different distances which are specified by individual applications. The coverage range is closely associated with the served industrial application. In a wireless use case, e.g., robot control, wireless personal area networks (WPAN) can achieve communications between robotic arms and proximity sensors within a few meters in the same work cell, wireless local area networks (WLAN) can coordinate neighboring robots in a discrete assembly line, wireless wide area networks (WWAN) can help connecting mobile robots to logistics jobs, and wireless metropolitan access networks (WMAN) can facilitate vendors in use cases of remote diagnostics.

B. Requirement Instantiation

To bridge the gap between system metrics and problem statements as shown in Fig. 2, requirement instantiation is the process of employing wireless system knowledge to establish the connections between service and resource metrics as identified in a wireless use case and associate them with the related statement sets. The mapping can be classified into four types of relationship between system metrics which are shown in Fig. 7:

Fig. 7:

Modeling wireless system design problems using identified system metrics and mapping to problem statements, a.k.a., requirement instantiation.

Utility functions, denoted by {C_R,i → G_S,j}, indicate the achievable service level varying with available resources. For example, the Shannon equation usually serves as the utility function of the data throughput in a wireless channel with the bandwidth as the input resource when the other settings are static [56]. Here, C_R,i sets the constraint on the available bandwidth, e.g., the total amount; while G_S,j can be the objective that maximizes the total throughput.

Budget functions, denoted by {C_S,i → G_R,j}, indicate the estimated resource amount as needed for the required service level which is the reverse function of the utility. Use the same example of the Shannon equation. In such a case, C_S,i can set the minimum service level as needed, e.g., the total data throughput should be greater than a given threshold; while G_R,j can be written as an indicator function, 1(·), which states the result of the validity check in (·), e.g., if the resource(s) to meet C_S,i would be feasible at the scheduler.

Screening functions, {C_S,I → G_S,j}, preselect the qualified candidate solutions that satisfy the service metric(s) in the condition before the exhaustive search. The condition and objective statements in such a mapping can be referred to the same service metric or different ones. In the former case, an example is that the objective is maximizing the sum of throughput in a network while as a condition the throughput has a minimum threshold for an active/scheduled link. In the latter case, the objective can be maximizing the throughput while the condition ensures the delay in each link, as a different service indicator, is smaller than a threshold.

Weighting functions,{C_R,i → G_R,j} enumerate all the resource alternatives and their respective weights in scheduling, such as availability, amount, marginal reward, and overuse penalty. Admittedly not all resources are treated equivalently in a system given their different operation cost and manipulation difficulty. Therefore, weighting functions provide some “price chart” to trade off resources in different dimensions for a given object. For example, C_R,i can indicate a condition that the total channel utilization, i.e., the ratio of the used time-frequency blocks to the allocated ones, should be smaller or equal to one. When G_R,j specifies the goal of minimizing the total energy consumption, the weighting function favors solutions that save more energy with the same time-frequency utilization.

The system requirements are mainly given by utility functions and budget functions in the studied wireless systems. In utility functions, the trends of system performance are measured by service indicators with the input of resources; while in budget functions, the needs of resources are calculated given the requested service level as the input. Screening functions or weighting functions only address single-side statements, either on services or resources. As a result, they can not be solo players in an established problem, but they may work as optional requirements to supplement utility/budget functions. As shown in Fig. 7, the problem set should contain at least one pair in the solid line. In other words, in the problem model of Section V-A, a necessary condition should be listed as G_S∪C_S ≠ Ø OR G_R∪C_R ≠ Ø.

The combinatorial pairs of service and resource metrics can characterize different types of wireless systems and guide the solution exploration. Fig. 8 illustrates some common problems identified with the joint service and resource concerns. In a multi-objective design problem, different variables may find their own gradients in the optimization approaches which are not aligned to one another. The metric sets and relation groups that are discussed above would suggest possible design approaches with practical design trade-offs, such as splitting the original problem into subproblems each of which takes care of separate service metrics powered with orthogonal resources. The search of subproblem solutions can be performed in separate solution spaces.

Fig. 8:

Exemplary wireless design challenges with paired system metrics.

VI. Solution Exploration

Once design problems have been recognized through the earlier procedures as shown in Fig. 2, we can start the search of efficient solutions for the studied IIoT use cases. Generally, the exploration of solutions can be performed in two directions: localizing turnkey solutions or developing new wireless techniques.

The former refers to the general case where the industrial wireless performance in the use case can be met by existing wireless standards/solutions developed in the information and communication technology (ICT) community for industrial applications which were discussed in Section I-A. In this case, the wireless network can be deployed using the off-the-shelf network equipment and software which accelerate the deployment [30]. Since there might be additional efforts on adjusting system configurations and/or ordering add-on stack layer functions from the vendor, this approach is referred to as the “localization” of generic standard techniques.

The latter applies in specific use cases where current “plug-and-play” solutions cannot fully support the required performance or major modifications are needed on enabling one or more network functions. New wireless techniques should be developed, deployed, and evaluated in the use cases. This approach would be increasingly used in emerging use cases given the fast grow of CPS and IIoT innovations. The design framework can serve as the key reference for OT engineers, IT architects, and wireless network designers to start their cooperations in the development and deployment of new techniques [32].

VII. Use Case Study

A. Overview of Use Cases

In this section, we select a group of industrial wireless use cases as examples and apply the proposed reference framework to identify key design issues in each of them. Four wireless use cases are discussed here addressing continuous processes, discrete factory control, mobile objects, and radio frequency harmonization in industrial sites. We reveal specific wireless design challenges in respective scenarios.

1). Tennessee Eastman Chemical Process:

In the first use case, we choose a typical process automation application known as the Tennessee Eastman (TE) process [73]. This use case represents one example of the continuous industrial processes, which can be widely found in oil and refinery industries, chemical and biomedical productions, and the food industry.

The TE process manages a chemical production plant by periodically collecting process data from distributed sensors while making decisions for actuators to run the process at the preprogrammed setpoints [74]. There are in total of 41 measurable process variables (PVs) (including temperature, pressure, flow rates, and product composition analysis data) and 12 manipulated PVs (including valve control, temperature control and feed/purge control) in a closed-loop control cycle. Wireless connections will link field sensors and actuators with the central controller which is typically a PLC, and carry the periodic data traffic containing PV updates. As PVs usually evolve slowly in continuous processes, e.g., state changes are observed in the scope of hours in the TE process, the update pace can be relatively slow in the order of seconds. In such a case, wireless network can replace the legacy wired networks to provide flexible cabling options and reduce operation costs. However, wireless networks also face a few challenges such as building and maintaining wireless connections in harsh industrial environments, scheduling diverse radio resources to provide deterministic and reliable services, and coordinating with core networks to ensure data sharing and safety. The details about the TE chemical process and one wireless network design based on the IEEE 802.15.4 radios can be referred to [73] and [75], respectively.

2). Industrial Printing Control:

The second use case covers a factory automation application of controlling industrial printing machines [47].

The industrial printer is equipped with 30 moving printing heads at the speed up to 3 m/s over distances up to 10 m. Each head has one on-board sensor to report the velocity and other real-time states to a central controller, and one 3-D actuator manipulated by the controller. The required cycle time is 2 ms in which each sensor transmits 20 Bytes to the controller and each actuator receives 20 Bytes from the controller, and the packet error rate (PER) should be less than 10⁻⁸.

The main design challenge in communication networks to support such mission critical applications is the limited link budget. To meet the low latency and high reliability requirements, current network solutions usually use the high speed Ethernet under practically-deployed wired control protocols, such as EtherCAT [76] and SErial Real-time COmmunication System (SERCOS) III [47]. When wireless is employed here, the error-prone wireless channels introduce additional uncertainty into the control loop. Moreover, as the network hosts distributed wireless nodes in the field, the access contention in the working spectrum band and potential interference between them further complicate the solution.

3). Scheduling and Safety Applications of Automated Guided Vehicles (AGVs):

AGVs, also known as mobile manipulators, are increasingly deployed in commercial warehouses and industrial plants to serve in plant logistics and inventory management [39]. These moving robots, unlike vehicles in the highway, need a closer control to function well in industrial operations. Since AGVs are mobile, wireless connections are inherently the first choice to link the moving nodes with the other players in the plant to exchange necessary information [79].

In this use case, we consider hybrid flow traffics in the wireless network for routine operations including AGV task assignment, navigation, and status collection [78], and for plant safety that carry emergency messages [80]. Routine operations are negotiated between AGVs and the task scheduler which resides in the infrastructure. Either side can initiate such actions. Take the example of new task assignments. The scheduler can poll the idling AGVs in the target work zones to assign new tasks, or it can request a bypassing AGV to load piggyback goods by updating its task list and planning the new route. Each AGV also needs to inform the scheduler about its status change, such as returning to the bay for recharging the battery. On the other hand, the safety-related messages, such as sending collision alerts or notifying an emergency, would cover a wider service area comparing to routine operation data. Not only the AGV nodes in the proximity will receive such emergency messages, but also the plant function modules related with the safety code will be triggered, e.g., downgrading production rates or evacuating the staff. Safety data flows normally have more stringent transmission requirements than scheduling data in terms of latency and coverage for the end-to-end flow design in the wireless architecture.

4). Industrial Spectrum Monitoring:

Industrial plants are normally adopting new wireless techniques in a way of introducing wireless connections in individual instruments and systems, e.g., in the above use cases. Multiple applications may claim the same popular wireless channels, such as the unlicensed WLAN bands, and run the radios to favor their own transmissions, e.g., operating at the high power and occupying more bandwidth. Besides incumbent industrial wireless networks, personal devices, such as cellphones and wearable devices for health monitoring, also leave their footprints in the wireless environments. Without a careful plan and effective control over diverse wireless networks in the plant, wireless activities in the same or overlapping spectrum bands would interfere with each other and degrade the supported OT operations.

Unlike the first three use cases where wireless nodes directly participate into production operations, the last one introduces a new wireless service, noted by spectrum monitoring, that supervises wireless usage in industrial plants. Spectrum monitoring services use wireless sensing instruments, e.g., spectrum analyzers [24] and software-defined radios [81], to measure and characterize wireless activities in target spectrum bands and support planter owners to plan and regulate wireless operations, which are an integrated part of spectrum harmonization in industrial environments [32], [108].

To capture wireless footprints spanning over time, frequency and space, the spectrum monitoring network consists of distributed sensing nodes and a central processor. The nodes report to the central processor routinely with updates of passive measurement data in the assigned RF channels, e.g., the received signal strength (RSS) samples in narrow-band frequency bins per sweep [24]. The sensors may vary with their sensing capabilities, such as scanning bandwidth, time/frequency resolution, sensitivity level, and data update rate [82]. Meanwhile, each sensed wireless channel may also give rise to spectrum specific requirements. For example, the 3.5 GHz Citizens Broadband Radio Service (CBRS) band in the U.S., one of the candidate spectrum bands for the industrial private LTE networks [11], has the “60 seconds” rule to silent secondary spectrum users if the incumbent signal is detected [70]. The central processor intelligently schedules the sensing assignment based on the interested spectrum bands and specifications of individual sensors. As the local data fusion point, it can run analysis programs to track wireless uses, identify violations and abuses of RF regulations (e.g., using restricted bands or operating beyond power limits), and provide off-line and real-time usage report for spectrum policy making.

Table III enumerates the exemplary specifications on dimensioning factors in the respective wireless use cases. In the reference framework, these features help to identify key system metrics that play key roles in each single use case, and provide insightful rationales on the objectives and conditions in formulating wireless design problems. In the remainder of this section, we will briefly compare the use cases and discuss the findings which are summarized in Table IV.

TABLE III:

Exemplary factors in the analysis of wireless use cases

			TE Process Control	Industrial Printing Control	AGV Schedul. and Safety	Spectrum Monitoring
OT	Environ-mental	Topographic	Industrial, indoor [35]	Industrial, indoor [35]	Industrial, indoor/outdoor [60]	Industrial, indoor [35]
		Mobility	N/A	< 3 m/s (velocity), < 10 m (range)	Walking speed [69], site-wide, e.g., 400 m × 400 m [35]	N/A
	Engineer-ing	Depth of Intelligence	Info. handling	Info. handling	Info. handling (routine), pass. notification (safety)	Active problem notification
		Function Pyramid	PLC	PLC	MES (routine), PLC (safety)	ERP/MES
		Layouts & Schedules	Flow, 24/7	Batch	Job, charging cycle	Site specific
DATA	Semanti-zation	Domain Knowledge	Chemical	Printing	Logistics and inventory control [61], safety [69], navigation [62]	Radio spectrum regulation [63], [64]
		Measurement System	SI, double float [65]	Vendor specs [66]	Vendor specs, e.g., [67]	Regulator specs [63], [64]
	Utility Indexing	Capacity	53 PVs, HART commands [68]	30 sensors, 30 actuators, 20 bytes per message	Site specific	Site specific
		Elasticity	Periodic	Periodic	Periodic (AGV states), on-demand (paging, task scheduling, safety)	Periodic (w/ tuned update report freq.), on-demand (control)
		Agility	sub-second	2 ms	< 50 ms (routine) [60], ≥ 125 Hz (safety) [69]	Spectrum specific, e.g., < 60 s in cleaning the CBRS band [70]
	Flow Attributes	Flow Dispatch	Pub-Sub	Pub-Sub (asset diagnosis)	REQ-RSP (routine), P2P (Safety) [15]	Pub-Sub
		Tunneling	Ethernet, e.g., OPC DA [71]	Ethernet, e.g., SERCOS III [47]	EtherCAT w/ OPC UA [72]	Ethernet
IT	Service Provisioing	Subscription	M2Storage	M2Storage	M2Analytics (routine), M2Process (safety)	M2Analytics
		Customization	N/A	N/A	Virtual firewall	Virtual network slice over multiple sites
	Architecture Mgmt.	Resilience	N/A	N/A	SDN/NFV by carriers	SDN/NFV by carriers
		Procurement	On-premises	On-premises	IaaS, e.g., enterprise private LTE [11]	SaaS by the trusted third-party [70]

Note: The specified values and thresholds in the table only serve as exemplary illustrations in respective applications.

TABLE IV:

Key system metrics in wireless use cases

	TE Process Control		Industrial Printing Control		AOV Scheduling and Safety		Spectrum Monitoring
	Service	Resource	Service	Resource	Service	Resource	Service	Resource
OT		Power (C, G↓)		Time (C) Power (C)		Time (C) Power (C)		Time (C) Freq. (G↑)
DATA	Coverage (G↑) Delay (C, G↓)		Delay (C, G↓)		Coverage (G↑) Rate (C)		Coverage (C, G↑) Volume (C, G↓)
IT					Delay (C, G↓)	Comput. (G↓)	Delay (C, G↓)	Comput. (G↓)

Note: The metric labeled by “G” stands for an objective in the ascending (↑) or descending (↓) trend.

The metric labeled by “C” stands for a system constraint.

B. Design Analysis in OT Applications

The harsh industrial environments, as the trademark of industrial wireless use cases, associate different resource constraints with the use cases. In use cases where the latency requirement is rigid such as in Use Cases 2 to 4, time serves as a key resource constraint since short time slots and flexible frame structures are needed to support scheduling schemes with low latency guarantees. In the TE process case, the relaxed control latency allows the wireless network to focus on the power-related resource metrics. One of the objectives in such process automation applications is to maximize the lifetime of the wireless networks with battery-powered nodes; and the energy consumption rate and residual energy indicator usually serve as featured conditions here.

In use cases with mobility, other resource considerations are also recognized. For example, the moving printing heads in Use Case 2 experience fast varying wireless channels between them and the central controller. In order to compensate part of the channel degradation due to the movement, the transmission power can be lifted within the allowed limit. In the AGV use case, the battery limit further shapes the behaviors of AGVs in the plant operations, e.g., the availability of mobile nodes between battery charges, which in turn need to be considered in the wireless network design.

In the spectrum monitoring service, wireless channels are treated as the served target instead of being used as a system resource. As a result, both the channels in use and those with potentials in plant operations would request spectrum monitoring services.

C. Design Analysis in Data Services

To ensure data services are available in the serving area, the coverage metric is widely identified in wireless use cases, e.g., in Use Case 1, 3, and 4. Since the central controllers of the TE process and industrial printer are normally located close to the operation sites, most of the control process data are going through wireless networks. As a result, data service factors place a clear definition on the latency performance requirements in Use Case 1 and 2 which serve as one design problem statement in Table IV. In contrast, in the AGV and spectrum monitoring cases, the processing and transmission delays in the infrastructure also perform a significant role in the latency considerations for data services. Therefore, latency-related metrics will be determined after incorporating the factors of the IT infrastructure in Use Case 3 and 4.

In the AGV case, the need of multiple data services with different operation purposes is catered for by the rate metric which serves as part of service provisioning requirements in use cases with heterogeneous data flows. In the spectrum monitoring network, a wireless channel may need a number of sensors at different locations to depict the spatial correlation of channel usage while a sensor can obtain the usage status of a few channels within one scanning at its listening spot. How to use the minimum number of spectrum monitoring sensors to perform sensing tasks, i.e., the volume optimization, is often discussed in the design problems of spectrum monitoring.

D. Design Analysis in IT Infrastructures

Largely out of practical concerns including effective plant asset control and OT data safety, plant owners still prefer on-premises solutions to build and operate their own IT infrastructure including wireless networks. For the process control applications in Use Case 1 and 2, the existing IT infrastructure may only give rise to a limited impact on the wireless network, e.g., providing the online storage of process history.

However, in more sophisticated plant operations, such as the AGV scheduling and spectrum monitoring, more C/S resources in the infrastructure are involved in wireless services. The conventional on-premises solutions to build such an infrastructure are often beyond the capability of individual plants or are not economical, especially for small and medium enterprises (SME). As a result, the consistency between wireless network settings and the IT infrastructure is often addressed as the computing resource condition in wireless design problems.

VIII. Conclusions

The discussions regarding industrial wireless use cases reflect the multi-disciplinary nature of IIoT practices with efforts from manufacturing industries, ICT community, standardization bodies, and government regulators, aiming to integrate OT and IT innovations on the factory floor for further improving production process efficiency and safety. A new design reference framework has been proposed in this paper which comprehensively reviews the critical factors in industrial wireless use cases. Such a framework has practical values as it standardizes the design cycle of wireless systems for industrial applications and reduces the time and costs involved in customizing design solutions for different wireless use cases. Useful directions for future research works include automating the problem formulation of industrial wireless use cases and standardizing the recommendation criteria on wireless techniques and solutions for the identified problems.