Online Distributed IoT Security Monitoring with Multidimensional Streaming Big Data

Abstract

Internet of Things (IoT) enables extensive connections between cyber and physical “things”. Nevertheless, the streaming data among IoT sensors bring “big data” issues, for example, large data volumes, data redundancy, lack of scalability and so on. Under “big data” circumstances, IoT system monitoring becomes a challenge. Furthermore, cyberattacks which threaten IoT security are hard to be detected. In this paper, we propose an online distributed IoT security monitoring algorithm (ODIS). An advanced influential point selection operation extracts important information from multidimensional time series data across distributed sensor nodes based on the spatial and temporal data dependence structure. Then, an accurate data structure model is constructed to capture the IoT system behaviors. Next, hypothesis testing is carried out to quantify the uncertainty of the monitoring tasks. Besides, the distributed system architecture solves the scalability issue. Using a real sensor network testbed, we commit cyberattacks to an IoT system with different patterns and strengths. The proposed ODIS algorithm demonstrates promising detection and monitoring performances.

I. INTRODUCTION

EXPLOSION of the Internet of Things (IoT) has been witnessed in diversified fields [1]. The number of IoT devices, as well as the generated data at different layers, are exponentially increasing. As reported, there will be more than 50 billion terminal devices worldwide, and the annual data generated will reach 847 Zebytes by 2021 [2]. “Big data” hereby becomes common in IoT applications [3], such as industrial manufacturing [4], smart cities [5], energy internet [6], wireless sensor network (WSN) [7], etc. However, because the IoT paradigm enables various connections between cyber networks and physical devices, vulnerabilities become an important issue [8]. Data-driven based IoT security solutions have been proposed, such as neural networks and deep learning based methods [9]. Nevertheless, the “big data” nature of IoT applications generates furthermore challenges, including the vast volume which will continuously grow [10], modeling complexity caused by large-scale processes [11], high-speed ubiquitous data collection [12], data redundancy introduced by multidimensional data collected asynchronously across distributed nodes [13], algorithm scalability [14], and so on. Thus, it is necessary to develop IoT security monitoring techniques in the “big data” era.

Our motivation is to effectively detect IoT system anomalies caused by cyberattacks under the big data circumstances, especially in WSN where multidimensional streaming data are gathered from networked sensors in a high speed [15], as shown in Fig. 1. The important anomaly detection and diagnosis information for IoT monitoring are typically buried in the system metrics, such as energy consumption [8] and system resource usages [9]. Thus, extracting useful information from data, especially unlabeled samples, is extremely important [16]. To fight against the data redundancy, finding the informative samples is highly desired for accelerating the computation and transmission processes of the high-speed streaming data. To effectively and efficiently extract informative samples, influential point selection (IPS) can be viewed as a data extraction approach to reduce the unnecessary energy consumption in IoT devices caused by redundant computations and system memory usages [17], [18]. Randomized data selection methods yield a high accuracy on model parameter estimation [17].

Fig. 1.

Multidimensional streaming “Big Data” from IoT systems. In a wireless sensor network (WSN), there are sensor nodes (gray dots) and sink nodes (blue hexagons). Besides sensing data (black cylinders), sink nodes also process data and exchange information among sink nodes (orange cylinders).

Based on the extracted influential points, how to understand the dynamic temporal and spatial/cross-sectional dependence structure of multidimensional streaming time series is still a challenge. The vector autoregressive (VAR) model, the most popular and fundamental time series models, provides a mechanism for capturing complex latent multidimensional dependency structures [19], [20], [13]. Since it is impossible to model the various unknown cyberattacks [8], we propose to model the normal system behaviors as an alternative solution. Thus, the system performance modeling accuracy is of great importance, and the accurate and robust data dependency modeling can facilitate this purpose.

In addition, IoT system essentially has a distributed architecture [21], where each sensor node only observes partial local information (a smaller set of relevant variables and features are analyzed locally [22]), but in together forms the analysis of the whole network. Note that, a distributed manner has additive benefits, such as more secure and enhanced robustness, since the attack behaviors are isolated. Thus, IoT security monitoring procedure should be designed in a distributed manner and the computation tasks can be assigned to individual nodes.

In this paper, we propose an Online Distributed IoT Security monitoring algorithm (ODIS) for multidimensional streaming big time series data. The whole workflow is shown in Fig. 2. Based on the efficient streaming time series data dynamic structure extraction through IPS, we can model the IoT data economically and accurately. In addition, thanks to the online distributed design, ODIS is suitable for the real-time large-scale multidimensional streaming IoT security monitoring. Finally, a complete algorithm is proposed so that the whole IoT system can have an end-to-end security solution. The contributions of this paper can be summarized as follows:

• We propose a novel online IoT security monitoring algorithm under the “big data” circumstances.

• Data science techniques such as IPS and streaming data modeling are proposed to extract intrinsic data structures efficiently and effectively. The IoT application data are collected across time and space, so the proposed approach considers and models spatio-temporal dependencies.

• Distributed algorithm design and the online streaming processing feature empower the ODIS algorithm the abilities of scalability and real-time applications for large-scale sensor networks.

Fig. 2.

The proposed ODIS algorithm with streaming big data. The detailed method descriptions and algorithms can be found in Section III.

The remainder of this paper is organized as follows.Related works are introduced in Section II. In Section III, we describe the proposed ODIS algorithm in detail with the related theoretical principles of every important operation. Using a real IoT testbed in Section IV, we analyze the performances of our proposed algorithm explicitly with comprehensive and quantitative analysis. In the end, a conclusion section is enclosed in Section V.

II. Related Work

IoT vulnerabilities arise due to the connected cyber-physical infrastructure [23], [24]. To eliminate IoT security threats, there is a high demand for solutions with a real-time, scalable, and distributed monitoring infrastructure [13]. Thus, the previous resilient approaches, such as, simple signal analytics based [25], Kalman filter [26], generalized likelihood ratio [27], the cumulative sum (CUSUM) [28], leverage score [29], Bayesian calibration [30], and machine learning [31], which need a center to monitor and control the entire system, are not suitable to be a distributed IoT security framework. In contrast, the required system should be distributed, where sensors only coordinate with its own neighbors within limited distances, and the multidimensional data could be collected asynchronously across distributed nodes [13].

In addition, typical data-driven approaches also suffer the lack of precise failure models in the device level as well as the system (network) level. For example, a resilient strategy was proposed to dispatch virtual power plant under cyber attacks in [32], where lower and upper bounds of the controller states are estimated in a distributed way. However, in the consensusbased energy management algorithm, false states, even minor ones within the given bounds, could cause large deviations [33], resulting in an ineffective resilient strategy.

III. Methodology and Algorithm

In this section, we briefly describe the principles of the ODIS algorithm. Every key step is introduced and the whole algorithm is summarized as well.

A. Symbol and Notation

The upper-case letters A and A are used for matrices and operators, and the curly capital letter $\mathcal{A}$ is for set or collection of sets. The vector is denoted by the lower-case bold letter a, and the scalar is denoted by the lower-case letter a. We write the transpose of a matrix A as A^′, the determinant of a matrix A as det(A), and matrix vectorization as vec(·). Specifically, $\mathbb{E}(\cdot )$ denotes the expectation, ≜, denotes equal by definition, ∥·∥₂ denotes the ℓ₂-norm for a vector, and ∥·∥_F denotes the Frobenius norm for a matrix. Moreover, $\mathbb{Z}$ denotes the integers, $\mathbb{R}$ denotes the real numbers, I_n denote the identity matrix of dimension n, and 1_{·} denotes the indicator function.

B. Multidimensional Time Series Modeling

The multidimensional time series modeling can effectively extract the temporal dependent information from the streaming multidimensional data, which is the key to understand and monitor the status of the IoT system.

The VAR family as the most important family of the time series models, is used to reveal the complex dependence structure in the streaming time series data [19]. The VAR model class quantifies complex temporal and cross-sectional interrelationship among the multidimensional time series. At the same time, the VAR model is flexible enough to be easily integrated into the distributed IoT system [13], providing treatment for the big data scenario.

The K-dimensional VAR model of order p (VAR(p)) of K-dimensional streaming data y_t can be written as

$$y_t=\sum _{i=1}^p{\Phi }_i{y}_{t-i}+{\epsilon }_t,$$ (1)

where $t\in \mathbb{Z}$, Φ_i’s are K × K model coefficient matrices, and ε_t is a sequence of independent and identically distributed (i.i.d.) random vectors with mean zero and finite non-singular covariance matrix. The VAR(p) model in Eq. (1) encodes the temporal and cross-sectional dependence structure between sensors in the IoT system through the coefficient matrices Φ_i’s, which are the key to understand the structural information from the streaming data. Learning these coefficient matrices can be done through Ordinary Least-Squares (OLS) estimate [19], [20]. However, two-fold challenges need to be conquered for the streaming big data setting. First, if given a fixed time period T, the computational cost of estimating the model coefficient matrices is O(TK²p²). It will pose a computational challenge for the entire IoT system when number of observed data T is huge or dimension of the data K is high. Second, for streaming data analysis, an online algorithm is needed so that we can achieve the real-time monitoring of the IoT system. In the following subsections, we will address these two issues.

C. Big Data Influential Point Selection

With the growing scale of the IoT streaming data, the huge volume of data challenges the computational and storage limits of the IoT system. For streaming IoT data monitoring, selecting the influential points reduces the processing time, energy consumption and thus be an effective road to the “big data” challenges. Data sketching and subsampling are popular tools to reduce the size of the data, with applications in online streaming analysis [13].

Extend the linear VAR(p) model in Eq. (1) to the form of general streaming non-linear model:

$$y_t^{\prime }=f{\left(y_{t-1}^{\prime },y_{t-2}^{\prime },\cdots ,y_{t-p}^{\prime }\right)}^{\prime }B+{\epsilon }_t^{\prime },$$ (2)

for observation up to time t, where $B={\left[{\Phi }_1^{\prime },{\Phi }_2^{\prime },\cdots ,{\Phi }_p^{\prime }\right]}^{\prime }$ is the Kp × K model coefficient matrix.

For a given function f(·), we de note $x_t=f{\left(y_{t-1}^{\prime },y_{t-2}^{\prime },\cdots ,y_{t-p}^{\prime }\right)}^{\prime }$, a column vector of length Kp. The model coefficient matrix then can be estimated as

$${\widehat{B}}_{OLS}={\left(\sum _t{x}_t{x}_t^{\prime }\right)}^{-1}\sum _t{x}_t{y}_t^{\prime }.$$ (3)

We value the importance of a data point y_t through its predicted value ${\widehat{y}}_t=h_{tt}{y}_t$, where

$$h_{tt}=x_t^{\prime }{\left(\sum _t{x}_t{x}_t^{\prime }\right)}^{-1}{x}_t,$$ (4)

is the t-th diagonal element of Hat Matrix, denoting the Mahalanobis distance of the t-th data [34], [35].

To effectively reduce the data size while maintaining the underlying data features, we select a subset $\mathcal{S}$ from time domain {1, …, T}, and use the subset data $\left\{y_t|t\in \mathcal{S}\right\}$ to efficiently estimate the model coefficient matrix. The least square estimator [36] then becomes

$${\widehat{B}}_S={\left(\sum _{t\in \mathcal{S}}{x}_t{x}_t^{\prime }\right)}^{-1}\left(\sum _{t\in \mathcal{S}}{x}_t{y}_t^{\prime }\right).$$ (5)

If the subset size $\left|\mathcal{S}\right|$ is much less than the data size T, i.e., $\left|\mathcal{S}\right|<<T$, the IPS will greatly save the computational time and cost to $O\left(\left|\mathcal{S}\right|K^2{p}^2\right)$.

The IPS is defined according to the selection rule

$${\mathcal{S}}_{IPS}=\left\{t\in \left\{1,\dots ,T\right\}:h_{tt}>r^2\right\},$$ (6)

where r is the selection threshold. The selection probability distribution follows the chi-squared distribution with degress of freedom $Kp,{\chi }_{Kp}^2$. Thus the selection threshold r is chosen as a square root of the quantile of ${\chi }_{Kp}^2$ distribution, that is

$$P\left(t\in {\mathcal{S}}_{IPS}\right)=P\left({\chi }_{Kp}^2>r^2\right),$$ (7)

where the selection threshold r is approximately proportional to the selection ratio, i.e. $r\propto \left|{\mathcal{S}}_{IPS}\right|/T$. The theoretical justification of the choice of r can be found in [13]. Alternatively, IPS can be described as, for data y_t observed at time t, if the Mahalanobis distance $\sqrt{h_{tt}}>r$, then we decide the data point y_t as the influential point and include t in subset ${\mathcal{S}}_{IPS}$. Fig. 3 visualizes the geometric property and corresponding Mahalanobis distance of IPS procedure, where the data points outside the ellipse are selected as influential points in the subset ${\mathcal{S}}_{IPS}$. IPS can be widely used to construct the importance sampling in big data analytic to reduce the data size, and it can also be applied in regression diagnostics to identify the outliers and the influential observations, see [37], [38].

Fig. 3.

IPS Illustration: One-dimensional data y_t are plotted with axes lag-2 values y_t−2 vs. lag-1 values y_t−1. IPS selection rule r is proportional to selection ratio, i.e. $r\propto \left|{\mathcal{S}}_{IPS}\right|/T$. The Mahalanobis distances larger than the ellipses (red: 10%; green: 5%) will be selected as the influential points. The influential points only account for a small amount of the whole dataset, e.g. 5 % or 10 %, but they represent the data structure.

D. Online Streaming IPS and Time Series Modeling

In practice calculation of the streaming IPS, we need to tackle the computational bottleneck of Mahalanobis distance in Eq. (4), the inverse of sample covariance matrix ${\sum }_t{\text{x}}_t{\text{x}}_t^{\prime }$ and update the it as the new data arriving. The benefit of calculatingP the Mahalanobis distance is that is provides the unitless and scale-invariant measurement to the influence of the data, and takes into account the correlations of the data. Such bottleneck makes the IPS computation as expensive as solving the original least squares problem in Eq. (3) in a streaming setting.

We propose an online streaming IPS adapting the single-pass streaming algorithm. As the new data arriving, we collect the first batch of data points as pilot sample to calculate a robust estimation on the sample covariance matrix $\Omega ={\left({\sum }_{t_0}{x}_{t_0}{x}_{t_0}^{\prime }\right)}^{-1}$ with time range t₀ taking from pilot sample batch. Then for t > t₀, the streaming IPS and the corresponding selection rule is replaced as

$${\tilde{h}}_{tt}\triangleq x_t^{\prime }\Omega x_t>r^2\text{ for }t>t_0.$$ (8)

Since x_t is constructed based on the VAR model, the streaming IPS procedure is a single-pass procedure that only requires linear computation time O(K_p) with respect to the VAR model dimension. It makes the streaming IPS scalable in the big data setting. Online streaming time series modeling also calls for an online real-time method to periodically aggregates historical information from previous data, updating the current estimation on the model coefficient matrix based on the arriving new data. More specifically, when the streaming data keep arriving sequentially, we update the estimate of the model coefficient matrix B adaptively.

The streaming IPS makes online real-time decision on selecting influential points ${\mathcal{S}}_{IPS}$ and IoT system monitoring. In other words, the online VAR modeling and optimization with respect to model matrix B become, for each time stamp t and selected influential points,

$$B_t=\underset{B}{\text{arg}\text{min}}\sum _{i\in {\mathcal{S}}_{IPS}\cap \left\{i\le t\right\}}\Vert y_i^{\prime }-x_i^{\prime }B{\Vert }_2^2,$$ (9)

where $x_i=f{\left(y_{i-1}^{\prime },y_{i-2}^{\prime },\cdots ,y_{i-p}^{\prime }\right)}^{\prime }$. Note that the estimation of B_t in Eq. (9) is an online optimization in a standard linear form. It can be solved by various optimization algorithms including the Kalman filter [39], recursive least squares [40], and gradient descent [41]. Our streaming IPS and time series modeling are independent from the choice of the solver to the optimization in Eq. (9). As long as the solver satisfies the one-pass property in online optimization and has the computational complexity linear in time t, the IPS monitoring and time series modeling will be scalable for the streaming big data setting.

E. Distributed Online Monitoring

Distributed computing infrastructure is intrinsic and necessary to the IoT tasks. Each sensor is observing a one-dimensional streaming data, and all sensors together form a network with a certain topological structure. The sensor network is observing the multidimensional streaming time series based on the topological structure. Such structure leads to a distributed computing environment. The streaming IPS and VAR modeling can be integrated into the asynchronous distributed computing environment. Fig. 4 illustrates the one-hop neighborhood diffusion strategy.

Fig. 4.

Diffusion strategy of the distributed network. At every time t, node k collects a measurement $y_t^{(k)}$ and neighborhood data.

The streaming IPS defined in Eq. (8) and selection rule defined in Eq. (6) can be implemented on each marginal dimension asynchronously and independently under the distributed setting. The selection rule ${\mathcal{S}}_{lev}^{\left(k\right)}$ for node k ∈ {1, …, K} becomes,

$${\tilde{h}}_{{\tau }_k}^{(k)}=x_{{\tau }_k}^{\prime }\Omega x_{{\tau }_k}>r^2,$$ (10)

where $x_{{\tau }_k}=f{\left(y_{{\tau }_k-1}^{\prime },y_{{\tau }_k-2}^{\prime },\cdots ,y_{{\tau }_k-p}^{\prime }\right)}^{\prime }$ is the k-th marginal local copy of the streaming data at local time τ_k.

Given the assumption that the multidimensional streaming data arrive sequentially in communication restricted distributed streaming environment, we exploit the VAR model structure so that the VAR modeling in Eq. (9) can be decomposed to K subproblems. For simplicity, we assume the functional f(·) as the linear form. We express the model coefficient matrix B as a block matrix with column vectors:

$$B=\left[b^{(1)},b^{(2)},\dots ,b^{(K)}\right]$$ (11)

with b^(k) being the k-th column of B for k ∈ {1, …, K}. For node k at local time τ_k, the k-th subproblem becomes

$$b_{{\tau }_k}^{(k)}=\underset{b^{(k)}}{\text{arg}\text{min}}\sum _{{\tau }_k\in {\mathcal{S}}_{IPS}^{(k)}\cap \left\{1,\dots ,t\right\}}{\Vert y_{{\tau }_k}^{(k)}-x_{{\tau }_k}^{\prime }{b}^{(k)}\Vert }_2^2,$$ (12)

where $y_{{\tau }_k}^{(k)}$. is the k-th element of $y_{{\tau }_j}$ at local time τ_k, for k ∈ {1, …, K}. The estimation of $b_{{\tau }_k}^{(k)}$ can be completed once all components of $x_{{\tau }_k}$ is observed at local time τ_k. The estimation of different nodes k ≠ k′ is calculated uncoordinately, which leads us the asynchronous algorithm in the IoT system.

Various distributed consensus optimization algorithms can be used to solved the subproblems Eq. (12), including distributed gradient descent [41], distributed ADMM [42], and distributed Kalman filter [39]. The framework of distributed recursive least squares [40] is adapted to solving the distributed problem in Eq. (12) as an illustration. When data from its one-hop neighbors sequentially arrived with some delay (see Fig. 4), the local recursive least squares is to estimate the local model coefficient $b_{{\tau }_k}^{(k)}$ for k-th node and local time ${\tau }_k\in {\mathcal{S}}_{IPS}$

$$b_{{\tau }_k}^{(k)}=b_{{\tau }_k-1}^{(k)}+\left[y_{{\tau }_k}^{(k)}-x_{{\tau }_k}^{\prime }{b}_{{\tau }_k-1}^{(k)}\right]k_{{\tau }_k},$$ (13)

where

$$P_{{\tau }_k}=P_{{\tau }_k-1}-k_{{\tau }_k}{x}_{{\tau }_k}^{\prime }{P}_{{\tau }_k-1},$$ (14)

$k_{{\tau }_k}\triangleq {\gamma }_{{\tau }_k}^{-1}{P}_{{\tau }_k-1}{X}_{{\tau }_k}$, and ${\gamma }_{{\tau }_k}\triangleq 1+x_{{\tau }_k}^{\prime }{P}_{{\tau }_k-1}{x}_{{\tau }_k}$ with $P_{{\tau }_k}$ as the k-th local estimate of the precision matrix. By transmitting the local estimation $b_{{\tau }_k}^{(k)}$ to its neighborhood, each node will form a complete model coefficient matrix estimate $B_{{\tau }_k}$, at time τ_k, by combining theses column vectors according to Eq. (11). We summarize the algorithm in Algorithm 1.

F. Consensus Hypothesis Testing

For the monitoring purpose, we develop the statistical hypothesis testing strategy to provide real time status and uncertainty quantification based on the distributed monitoring results B_t. In Algorithm 1, each node has been fused the consensus monitoring results B_t based on the diffusion strategy, see e.g. [39], which means every node will have the same monitoring results B_t when t is large enough. The purpose of hypothesis testing is to distinguish the attack status from the normal status in a quantitative way. Based on the VAR modeling, we construct the Wald type statistics for hypothesis testing with null hypothesis H₀: B_t = B_Normal against alternative hypothesis H₁: B_t ≠ B_Normal, where we ignore the superscript (k) since the consensus results of B_t. The Wald statistic has asymptotic χ² distribution with ρ degrees of freedom, where ρ is the rank of matrix B_t [20]. Then we provide a hypothesis testing with a p-value that quantifies the uncertainty of online attack monitoring statues. If we reject the null hypothesis based on the p-value, the current data suggest that there is a significant pattern change to make the system deviates from its normal status. After the distributed consensus hypothesis testing, the system can obtain a unified decision.

G. Proposed ODIS Algorithm

We realize the online distributed IoT security monitoring through the mentioned key steps. While Figure 2 shows the big picture of the whole workflow, Algorithm 2 shows the detailed comprehensive workflow of the proposed ODIS algorithm using the connections with key theories.

IV. Experiments and Evaluation

To evaluate ODIS algorithm, we carry out cyberattack experiments using a real IoT system, where smart sensors are connected within a wireless network. Different cyber attack strengths are tested to demonstrate the performances of ODIS in cyber attack detection and monitoring.

A. Testbed Setup

We use the Beaglebone Black boards (BBB) in our experiments¹ to implement a real IoT system consisting of wireless network connected smart sensors (embedded system). Fig. 5 shows the testbed in our study. Note that there are 36 available BBBs in the same cluster sharing the same mesh network, where distributed algorithms can be operated among nodes.

Fig. 5.

Real IoT device testbed built by BBBs connected via a wireless network.

B. Cyberattack Detection and Monitoring

Denial-of-Service (DoS) Cyberattack:

The IoT sensor networks are generally vulnerable to intrusion related to snooping, spoofing, masquerading, Denial-of-Service (DoS) attacks. DoS attacks impact the network communication partially or completely. As the sensor nodes of IoT are low powered and lossy, the impact of DoS attack is quite significant [43]. The DoS attack disrupts the communication between devices, making their unavailability. The DoS attack can be carried out externally or internally in IoT and is very hard to detect it unless the services have stopped working. For example, in the flooding attack, the attacker overflows the network through sending packets to disrupt the service of legitimate users. Its examples include DNS (Domain Name System) flood, ICMP (Internet Control Message Protocol) flood and UDP (User Datagram Protocol) flood. There are four common types of DoS attacks, volumetric, network transport, application and multi-vector. As volumetric attacks are the most common DoS attacks, we simulate volumetric DoS attacks in our experiments, which consume available network bandwidth between the target and the internet by overwhelming the target with a flood of data.

Data:

To monitor the WSN, we adopt the energy consumption measurement [8] of every node, which represents the whole system activities of the node. The total energy consumption consists of energy consumption from individual subcomponents, such as CPU, RAM, storage and data transmission, etc. In addition, cyberattacks result in the abnormal system behaviors [9] that can be observed from the energy consumption of the subcomponents. For example, DoS attacks significantly increase the amount of received data, resulting in the energy consumption of not only the network adapter but also the whole system increases. If the proposed ODIS algorithm can detect anomalies and monitor the attack variations based on the energy consumption auditing, the IoT monitoring system is fully functional.

Evaluation:

We compare the VAR modeling accuracy using IPS with Vanilla, and Bernoulli sampling methods. The Vanilla method uses all data points without data points selection for monitoring, which may result in delayed responses for IoT system. In Bernoulli sampling, a Bernoulli trail is conducted to randomly select data points at each time with a fixed success probability (p = 1/2) [13].

1). Experiment 1:DoS Detection:

Fig. 6 shows the relative weak cyberattacks. The attack strengths vary from 25 KBps to 10 MBps. Evey time, DoS attack happens 20 seconds then there is a 5 seconds interval. The time series data contain 36 dimensions (K = 36), and each dimension has around 24, 000 samples (23,985) with data interval 0.1 second.

Fig. 6.

DoS attack pattern in Experiment 1.

As defined in Eq (9), the VAR model should be accurately characterized using the streaming data. We compare the modeling performances in Fig. 7 using different sampling methods: IPS, Vanilla, and Bernoulli. Due to the sampling strategy used Bernoulli sampling, the modeling error is large. Vanilla and IPS sampling method generate relatively small modeling errors. Furthermore,due to the existence of inevitable noises in the real testbed, the modeling results from Vanilla could be influenced by the noise, whereas, IPS has a better robustness as only informative points are extracted and used.

Fig. 7.

Modeling errors using different methods in Experiment 1.

The estimated coefficient matrix Φ₁ under DoS attacks 25 KBps and 250 KBps are shown in Fig. 8. It is observable that there are minor off-diagonal unusual patterns indicating that the IoT system is under attack even the attacks are not strong.

Fig. 8.

The estimated parameter matrics Φ₁ under DoS attacks (a) 25 KBps and (b) 250 KBps.

2). Experiment 2: DoS Monitoring:

Fig. 9 shows the relative strong cyber attacks. The attack strengths vary from 10 MBps to 160 MBps. The time series data contain 36 dimensions (K = 36, and each dimension has around 24,000 samples (23,872) with data interval 0.1 second.

Fig. 9.

DoS attack pattern in Experiment 2.

Fig. 10 shows the modeling accuracy errors using different sampling methods: IPS, Vanilla, and Bernoulli. Similar to Fig. 7, the Bernulli sampling method generates larger errors than the other two methods. IPS is still the best approach. Note that because of the hardware design and system limitations, when the cyberattacks are strong, not only the network card performance is affected, the whole system does not behave normally. Then, there are more interference and noise mixed in the modeling process, so the modeling performances of Vanilla and IPS are not as good as those in Experiment 1.

Fig. 10.

Modeling errors using different methods in Experiment 2.

According to the consensus hypothesis testing in Section III-F, we use the Wald test [20] to monitor the streaming data structure variations. p-value is employed to reject the null hypothesis. We observe that p is close to 1 for the same attack strength, and when there is a system status change p value is small, for example, when the system changes from normal to under attack, p value can be as small as 0.0001, and when the DoS attack is strong, we observe p value’s unit could be 10⁻⁷.

3). Computation Efficiency:

Besides the modeling error characterization, we also compare the computation efficiency of the mentioned methods. Since the data amounts are close in Experiment 1 and Experiment 2, the total computation time consumption of two experiments does not vary significantly. It is clear that IPS is more efficient than Vanilla in both Fig. 11 and Fig. 12, as with less data the computation could be faster. However, because, besides the modeling computation, storage and data transmission also take time, even longer time, the time saving is limited. Nevertheless, the proposed approach is promising, as we can notice that more time saved for larger data, which is the very target for the “big data” processing. Compared with Bernoulli sampling, IPS has additional computations, so it is a little slower, but IPS can have a slightly better modeling accuracy even compared with Vanilla method. Thus, the proposed approach is the most promising.

Fig. 11.

The average elapsed time of IPS (blue), Bernoulli (red) and Vanilla (orange) in Experiment 1.

Fig. 12.

The average elapsed time of IPS (blue), Bernoulli (red) and Vanilla (orange) in Experiment 2.

V. Conclusion

To deal with the “big data” issues in IoT security, we propose an online distributed IoT security monitoring algorithm-ODIS. The proposed algorithm handles the complex streaming multidimensional time series very well. The latent streaming data dependency on both time and space can be effectively and efficiently extracted from the multidimensional big data. In addition, the online distributed algorithm design enables the real time IoT monitoring with affordable computation and communication burdens. Using the testbed with real IoT devices, we carry out experiments about cyberattacks (e.g. DoS) towards IoT sensor networks. The proposed algorithm is a general IoT cybersecurity solution, and shows promising performances in terms of cyberattack detection and monitoring.