Cognitive information measurements: A new perspective

Abstract

From a traditional point of view, the value of information does not change during transmission. The Shannon information theory considers information transmission as a statistical phenomenon for measuring the communication channel capacity. However, in modern communication systems, information is spontaneously embedded with a cognitive link during the transmission process, which requires a new measurement that can incorporate continuously changing information values. In this paper, we introduce the concept of cognitive information value and a method of measuring such information. We first describe the characteristics of cognitive information followed by an introduction of the concept of cognitive information in measuring information popularity. The new measurement is based on the mailbox principle in the information value chain. This is achieved by encapsulating the information as a mailbox for transmission where the cognition is continuously implemented during the transmission process. Finally, we set up a cognitive communication system based on a combination of the traditional communication system and cognitive computing. Experimental results attest to the impact of incorporating cognitive value in the performance of 5G networks.

1. Introduction

The increasing number of smartphones and multimedia services (e.g., virtual reality and augmented reality) and the development of the Internet of Things (IoT) are expected to have a higher requirements for the storage and computing ability of future communication systems. The number of users accessing the network is also expected to increase sharply to allow massive IoT participation. At the same time, the end-to-end delay will also be substantially reduced to only a few milliseconds [1–3]. However, the popularity of IoT could result in an explosive growth of mass data acquisition, which can put a tremendous pressure on the wireless communication network to satisfy growing bandwidth demands. Despite many advanced features, 5G networks may not be able to meet such a high demand.

Furthermore, it is known that transmitted data normally contains too much redundancies and useless data, such as continuous invariant video transmitted by video surveillance equipment [4]. Therefore, communication networks will need to achieve intelligent information transmission that is capable of selecting only useful information for transmission throughout the wireless network.

Fortunately, with the emergence of cognitive computing [5], we are required to have a different way of measuring information to define information from the perspective of the cognitive link. This contrasts with the traditional Shannon information theory [6], which assumes that the information size remains unchanged during the transmission process. Under dynamic conditions, a cognitive information theory can be established as a combination of cognitive computing and information transmission [7]. Since cognitive information regards the information value as one of the main indexes for evaluating the communication system, the core problem is how to measure the information value.

Bear in mind that when the information is generated, it will evolve continuously and the value density (i.e., information value of unit bit) will become higher and higher with cognition of the information. Eventually it will tend to converge. Meanwhile, upon receiving the information, users will further improve their understanding of the information according to their own knowledge.

Therefore, in this paper, we firstly provide the characteristics of cognitive information and information popularity that represent an evolutionary process of cognitive information. Then, to realize information cognition, we propose the mailbox theory and give its specific description. Finally, we design a cognitive communication system as a combination of the traditional communication system and information cognition.

In summary, the main contributions of this work are as follows:

• Introduce the concept of cognitive information value for intelligent transmission. This includes the characteristics of cognitive information involving dynamic, polarity, evolution, convergence, and multi-view, as well as a supporting analysis for information popularity.

• Develop a method of encapsulating the information as a mailbox based on the principal of mailbox theory, which recognizes the information value continuously during transmissions.

• Design a new communication system based on cognitive computing and a traditional communication system. Under these conditions, only the most valuable data is transmitted in order to reduce the amount of data transferred, and energy consumed.

The remainder of this paper is organized as follows. In Section 2, the evolution of cognitive information is introduced. Information measurement based on cognitive computing is given in Section 3. In Section 4, the mailbox theory for cognitive information is presented. In Section 5, the cognitive information communication system is evaluated. Finally, the conclusions are given in Section 6.

2. Evolution of cognitive information

In this section, we provide the motivation for using cognitive information and the importance of information cognition from the perspectives of network and data.

2.1. Motivation

The Internet Data Center (IDC) forecasts that global data will grow from 33 ZB in 2018 to 175 ZB by 2025 [8]. Thus, the processing for such a massive amount of data is quite challenging and would require sufficient storage and computing power. More importantly, it is required to develop an intelligent processing method capable of recognizing the meaningful information in real-time, which can be achieved best via information cognition. For example, in AlphaGo,¹ the game of Go uses the technology of deep learning, reinforcement learning, and Monte Carlo tree search, which would require training 30,00 0,000 sets of human data and 48 TPU distributed systems to defeat the Lee Sedol [9]. Thus, training AlphaGo requires a huge amount of data.

However, the game of Go follows an accurate game rule (i.e., cognitive information). On this basis, AlphaGo Zero does not use prior knowledge nor raw data. Its learning process is based on random games, such as using self-game reinforcement learning, rather than the human data. It takes black and white pieces on a chessboard as an input and applies neural networks to simultaneously represent policy and value, rather than using them separately, as in the case of human participants. To achieve a better performance than AlphaGo, AlphaGo Zero uses 64 GPU workstations and 19 CPU parameter servers for training, and 4 TPU to perform matching execution. Thus, the rules of Go are cognitive information, which plays an important role in establishing strong artificial intelligence.

Specifically, from the perspective of cognitive information, AlphaGo training requires a huge amount of data, but the quantity of cognitive information is very low. Under these conditions, it is essential to extract the value of the information so that the entire data can be simplified to a small amount. Therefore, the main objective of measuring the information capacity from the perspective of cognitive computing, is to reduce the amount of information as much as possible without compromising the basic rules of information theory. In this way, a new cognition of measuring the information value (e.g., a set of rules) is needed to reduce the amount of data.

2.2. Aspect of cognitive information

The data is first generated on a user terminal and then transmitted to the remote server by a communication network, where it is coded, analyzed, and processed. The remote server feeds the processed results back to the user in the same manner via different links. In this paper, we use a data transmission network and a processing terminal to realize the information cognition. It specifically consists of two parts, i.e., network cognitive optimization and data cognition.

Network cognitive optimization

Network cognitive optimization is mainly concerned with resource distribution, scheduling, and management to ensure the quality of service (QoS). The main objective is to create a suitable environment for cognitive information transmission to achieve a high reliability and low delay. It specifically aims at maximizing the mutual benefits between users and providers. A suitable optimization model can be used to enhance the systems’ throughput and spectral efficiency, as well as simultaneously minimizing the delay and energy consumption. Possible decision configurations include transmission power distribution, resource distributing and scheduling, routing decision, spectrum resource distribution, and other processes.

Generally, network optimization consists of the following three stages: i) performing data acquisition and analysis of the existing network on the premise of a full understanding of the networks’ running state; ii) discovering factors that can influence network quality, which can then be optimized using various methods such as machine learning, game theory, and other methods; iii) making a network to achieve its best running state by optimizing the resources.

Data cognition

Facing various types of information, humans first filter the received data to capture valuable information, and then evaluate its value. Similarly, for communication systems in the face of dealing with mass structured and unstructured data, a system needs to acquire valuable information to achieve the same goal. In traditional data processing methods [10], it is required to extract the data features manually. However, in an era of big data with booming machine learning, deep learning technologies can be exploited to extract data features automatically and achieve better prediction [11].

In deep learning, modeling and optimization would require recognition and understanding of mass data, so the volume of data is a key to improving the accuracy of the model prediction. Therefore, it is important to realize preprocessing and carry out certain screenings and discriminations before introducing the data into the construction machine learning model. This is mainly to prevent noisy data from impacting the model performance.

Furthermore, we take the user’s emotion detection as an example, and give the application of cognitive information in emotional data transmission. Generally speaking, emotional data detection models need to be trained in the cloud. Thus, users need to collect emotional data (e.g., multimedia data) offloaded to the remote cloud. In the traditional communication system, to improve the accuracy of emotional detection, the collected data need to be transmitted to the cloud. However, the data transmitted by this strategy contains a large amount of useless data that can impact the network load. When the cognitive information is applied to communication system, the transmitted data can be cognized, and then only valuable data will be selected for transmission to the cloud. This not only reduces the amount of data transmission, but also improves the accuracy of the model.

3. Information value measurement based on cognitive computing

In this section, we first describe the characteristics of cognitive information. Then, we present the measurement of information value from the perspective of information popularity.

3.1. Characteristics of cognitive information

According to the basic concept of information, each phase in the life cycle of information possesses a certain cognitive potential, which can be utilized to assess its information value. A value, which is contained in each piece of information, can be extracted to represent a quantified measure of cognitive information. It should be noted that cognitive information expands beyond communication systems and can be used as in combination with cognitive computing to solve many research problems, such as information acquisition, transmission, analysis, and application.

To be specific, information cognition is different from conventional information measurement. We describe information cognition as follows: when information is generated, it will be cognized by different users during transmission. Therefore, after being cognized, information eventually achieves the highest value density as it evolves. The characteristics of cognitive information are shown in Fig. 1 and described as follows:

Fig. 1.

The characteristics of cognitive information.

• Dynamic: As shown in Fig. 1, information value is comprised of an intrinsic value and an expanding value. The intrinsic value denotes the intrinsic natural attribute at the beginning of the information generation. The expanding value denotes a gradually formed social attribute under the influence of external factors during the information transmission process. To be specific, after generation the information is endowed with the intrinsic value, which is constantly transmitted to different users based on their own interpretations. During this transmission process, the information is commented and subsequently an expanded information value is generated. As for a information, its potential is rated differently by each user where its value is fully excavated, analyzed, and then utilized. It is noted that each user may have a different association and different understanding of the same information at different times. Thus, the cognitive information value is dynamically changing during the transmission process.

• Polarity: In traditional information theory, the measurement of information (i.e., Shannon entropy) is non-negative, but it possesses positive and negative polarities from the expanding value of cognitive information. During the transmission process and interaction with users, the associated information can be generated by using cognitive computing to reflect the expanding information value. It should be noted that certain information can have a positive impact on some users, but if wasteful, can have a negative impact on information transmission. For instance, for a primary school student learning that information relevant to scientific knowledge will play an important role in his/her growth and development will manifest the positive polarity of information. Conversely, if a primary school student focuses on illogical and meaningless information, it may have a negative impact on his/her cognition, manifesting the negative polarity of information. Thus, the cognition of information polarity is vitally important.

• Evolution: Once the information is generated, it will be continuously recognized during the transmission process. When the cognitive capability reaches a certain level (by analogy) information can be transformed between different dimensions. After transformation, it can be applied to the data level of other dimensions to generate new information and viewpoints. The cognitive system simulates human thinking during the training process, constantly enhancing intelligence through continuous learning, which gradually approaches the cognitive capability of humans. During the process of information transmission, the valuable information is expanded and then compressed based on the definite principles of the cognitive system, to enable the information to better meet the individual needs of users in multidimensional space. Thus, information possesses the evolutionary characteristic.

• Convergence: When the information is cognized, to some extent its value density tends to be stable from the value of the information, which illustrates that information has a convergence nature. For example, the information about the description of the motion of an object can be summarized as Newton’s three laws of motion. To be specific, according to the Shannon information theory, the amount of information transmitted through a traditional communication system in a time unit is restricted by the channel capacity. However, in practice, demands for high volume continuous information are always at odds with the physical channel capacity. Thus, it is required to continuously improve the communication system capability and enhance data transmission volume. Furthermore, by means of recognizing the information continually, it is possible to achieve the highest value density, which reduces the amount of information for transmission. Accordingly, to effectively reduce a burden on the communication system, the redundant data should be removed based on its information value. Namely, the removal of redundant information and conciseness of valuable information are not infinite and abide by the convergence principle on the premise of meeting the minimum reduction rule.

• Multi-view: From the user’s perspective, cognitive information can reach the largest value density when it converges. Upon receiving the information, the value of the same information can impact each user differently due to users’ different cognitive capabilities and demands. For instance, a high-level research paper can inspire researchers in related fields greatly and accordingly open a new research field. On the other hand, for a person who has poor scientific knowledge, such information has almost no value. This example clearly represents the information impact on different users. Since information has different convergence and value for different users, information has a multi-view nature.

3.2. Information popularity

In this section, for multimedia information (e.g., audio-video data), we can use information popularity as its measurement. To describe information popularity, we firstly introduce a definition of information. We assume that information i can be described by its size s_i. (given in bits) and the required computing resource (given in CPU cycles). To analyze the information value, we begin with defining the information life cycle. If t_i(t) is the time instance in which information i is generated, then the information life cycle at the time t is given by:

$${\eta }_i(t)=t-t_i(t).$$ (1)

We assume that the value of information i at time t is determined by function $h_i^t$. We further assume that information popularity is relevant to the information size, information life cycle, and information value. Thus, the popularity of information i at time t is defined as:

$$p_i^t=f\left(s_i,{\eta }_i^t,h_i^t\right).$$ (2)

where f(·) denotes the function relevant to s_i, ${\eta }_i^t$ and $h_i^t$.

4. Mailbox theory for cognitive information

In this section, we introduce the concept of the mailbox theory for cognitive information.

4.1. Overview of mailbox theory

In the traditional communication system information transmission process, only the redundant bytes will be added (based on the underlying data) in order to ensure transmission reliability. However, such a scenario is not user-centered and neglects the information value. Mailbox theory incorporates cognition of the information value in the transmission process. It counts on the realization of the user-centered information transmission by means of information cognition and filtering out valueless information where only higher value density information can be transmitted.

To better explain the mailbox theory, we apply the mailbox to the multimedia communication for the emotion detection application. In the emotion detection application [12,13], mobile users realize emotional detection by transmitting audio and video data to the cloud. Our goal is to improve the accuracy of the emotion detection model in the cloud by learning the emotional data of the mobile user. According to our proposed mailbox terminology, we only transmit data that can be employed to improve the accuracy of the model, hence reducing the amount of data transmission.

Specifically, we consider that information transmission is a dynamic process with entropy production [14]. Furthermore, to understand information cognition, we consider data generated by a user as an encapsulated mailbox, rather than static information. Also, we assume that users have different understandings of the information through reading and analysis. For instance, a user can perform a labeling operation on each piece of information by adding its own cognition and judgment. As an example, we consider emotional data transmission where each user can evaluate the contribution of the data and its possible impact on the accuracy of its emotion model. Based on cognition of the information, it will then decide whether or not to transmit the data. Data such as text, speech, or image, represents a mailbox where each mailbox has a different dimension and the information generated by interpretation is different for each user.

Next, we describe the detailed process of mailbox theory based on the example shown in Fig. 2. We assume that a batch of data at the source is encapsulated into a mailbox for transmission. Normally, in a communication system data packets may move along different edge nodes through their paths before reaching their destinations. A node with computing resources can open its mailbox and read its content to learn about the adopted policy and the algorithm in its path, and the information value. To guarantee high transmission efficiency generally, it is required to use an algorithm with lower complexity. Since high transmission efficiency and lower complexity are two contradictory optimization objectives, it is required to design algorithms to balance between the two.

Fig. 2.

The mailbox theory for cognitive information.

After completing the information value measurement (e.g., the contribution of data to the algorithm in the emotion detection model) at each node, the comments are dynamically added and then forwarded to the next node. After each node, including the raw data and the comments added by previous nodes are evaluated and then forwarded to the next node (if it decides not to drop it). Finally, at the destination node, the raw information and all comments are added to the data packet throughout its communicant link and can be analyzed.

4.2. Introduction of mailbox theory

We encapsulate the information into a mailbox, which can be then transmitted through the communication system. In the process of transmitting mailbox, due to the added comments in the packets, the mailbox size is changing at each node. Therefore, the end user (or server) is expected to receive data packets of different sizes. Moreover, due to dynamically adding cognitive comments to the raw information at each node the information value of each data packet is continually increasing. It is worth noting that during the mailbox information transmission the data packet can be unpacked directly at different communication nodes. However, in order to protect information transmission security, the comments on data can be added to the raw data only if a node has permission to modify the packet. This is mainly to ensure the integrity and safety of raw data. Thus, every node can gain the raw information transmitted from the source. Furthermore, when a data packet reaches the terminal, various characteristics of information (e.g., encapsulation format of data packets, coding and de-coding methods, etc.) should be consistent with those of the source-transmitting end. A series of processes for packet unpacking and adding comments are experienced during the transmission process, but the basic characteristics of the data packet must be consistent.

To describe a mailbox, we assume that a mailbox is defined by two important indexes, i.e., information value and information size. These two indexes can be changed during the information transmission process. The information value of an encapsulated mailbox i at time t, is defined by function $h_i^t$. Assume that n users label the information during the transmission process (i.e., from time t to t + 1). Also, assume that the change in the information value due to the labeling of a user j is Y_j. Thus, the information value at t + 1 is given by:

$$h_i^{t+1}=h_i^t+\sum _{j=1}^n{Y}_j$$ (3)

The average information value of information i from its generation to time t is defined by:

$${\overline{h}}_i=\frac{1}{T}\mathbb{E}\left[{\int }_0^T{h}_i^{t}dt\right]$$ (4)

Furthermore, the information value can be expressed by:

$${\overline{h}}_i=\underset{T\to \infty }{\lim }\frac{1}{T}\mathbb{E}\left[{\int }_0^T{h}_i^{t}dt\right]$$ (5)

Similarly, we assume that the information size of information i at time t is s_t, and that n users label the information during transmission process, such that the size of each label is x_j. Thus, the information size at t + 1 is given by:

$$s_i^{t+1}=s_i^t+\sum _{j=1}^n{x}_{{}_j}$$ (6)

Then, the ultimate information size is expressed by:

$${\overline{s}}_i=\underset{T\to \infty }{\lim }\frac{1}{T}\mathbb{E}\left[{\int }_0^T{s}_i^{t}dt\right]$$ (7)

5. Cognitive information based communication system

Considering the user requests for various information, traditional communication technology fails to meet such high communication demands. Thus, in this section, in order to optimize existing communication systems, we propose a new communication system based on cognitive information by exploiting the mailbox theory. The proposed cognitive information based communication system is aimed at multimedia communication, which takes into account that the existing multimedia data (e.g., audio and video data) is the main traffic in the existing network.

5.1. Proposed communication system

In this paper, we construct a novel communication system for transmission of emotion data that is based on mailbox theory. In the proposed communication system, the main objective of mailbox terminology is to reduce the data transmission by classifying the information by which only valuable information can be selected for transmission. To achieve it, we use cognitive information based on machine learning to obtain the information value. The architecture of the proposed communication system, which includes device layer, edge cloud layer, and cloud layer, is shown in Fig. 3. Taking emotional recognition as an example, we encapsulate facial data and speech data as mailbox. The data includes labeled and unlabeled data. Compared with the traditional method of offloading all data to the cloud, emotional recognition is directly carried out in the cloud. At first we recognize the information and after selecting and identifying the unlabeled data we offload optimal valuable data to the cloud. Thus, by recognizing the information, the intelligence and reliability of the proposed system can be realized.

Fig. 3.

The architecture of the proposed communication system.

In the following we will show how emotional data is recognized. In traditional communication systems, all data (i.e., information) needs to be offloaded to the cloud [15,16]. In our proposed communication system, only valuable data (after being identified by using machine learning) will be offloaded to the cloud [17]. It should be noted that in the emotion detection field, the accuracy of the training model depends on the data instances quality. In the classification problem, we can easily determine the classification confidence of a data instance by using the machine learning model. Under these conditions, we can obtain the information entropy of data by using the confidence value.

To be specific, the model framework and the training process are shown in Fig. 4. To recognize emotions, it is required that end users transmit large amounts of data to the cloud. For the data, only a small part of data is labeled, while most of the data does not have any label (i.e., unlabeled data) and contain much noise. Therefore, we first utilize the AlexNet [18] architecture which includes 5 convolution layers, 3 pooling layers and 2 full connection layers for emotion detection based on labeled data. Second, unlabeled data is used to further improve the accuracy of the emotion detection model. Assuming that the unlabeled data instance $x_i^u$ is predicted through AlexNet with a probability of $p_{x_i^u}=\{p_{x_i^u}^1,p_{x_i^u}^2,\cdots ,p_{x_i^u}^c\}$, where $p_{x_i^u}^j$ is the probability that unlabeled data $x_i^u$ is classified into emotion j. Thus, we can obtain the following formula:

$$E\left(p_{x_i^u}\right)=-\sum _{j=1}^c{p}_{x_i^u}^j{\log }_2{p}_{x_i^u}^j.$$ (8)

where c is the number of classes. From formula (8), we can see that when $E\left(p_{x_i^u}\right)$ is small, the data has less prediction uncertainty. Thus, the unlabeled data can be classified more easily and can be more valuable to model training and parameter iteration. When $E\left(p_{x_i^u}\right)$ is large, the data has greater prediction uncertainty so the contribution of this data to the model is small. Therefore, the amount of data transmission can be reduced by cognizing the data.

Fig. 4.

The information value measurement during AlexNet training.

To enhance computing ability to recognize data, we first set up the AlexNet model on the edge cloud and transmit the labeled data for model training in order to obtain an immature model. Then, we select partial data from the unlabeled data and transmit it to the edge cloud for model training where we can compute the information entropy by using (8). Next, we set an experiment which defines information value threshold to decide whether transmit the unlabeled data to the remote cloud based on the computed information value (if the information value is lower than the threshold, data will be transmitted; otherwise, the data packet will be abandoned). Finally, we select the unlabeled data continuously, until all the user data goes through value assessment. Thus, data selecting can alleviate the flow congestion in the network for every node in the network.

5.2. Experimental setup and performance evaluation

In the experiment, we encapsulate the facial expression and speech data generated by mobile users as mailbox, and cognize the data on the edge cloud and remote cloud. In the edge cloud, the value of data is measured by the contribution of emotional data to the accuracy of the model and using (8) for evaluation. On this basis, by discarding the data with low value density, we only transfer the most valuable information to the cloud for emotion detection model training. In our experiments, we compare the cognitive information system with the traditional communication system, which transmits all information to the cloud without cognizing the transmitted information. To be specific, at first, we compare the impact of the cognitive information system and traditional communication system on the accuracy of emotion detection model for transmitting the same amount of facial and speech data instances.

From Fig. 5(a) and (b), we can see that with the increase of facial and speech data instances, the accuracy of emotion detection is increasing. Furthermore, we can clearly observe that the accuracy of emotion detection in the cognitive information system is higher for a given facial and speech data instance. The accuracy of the model refers to the proportion of the predicted data instances labeled with real labels to the total data instances. From the figure, we know that in the cognitive system, by cognizing the unlabeled data, it can transmit data with a high value density (i.e., contribution to model accuracy) to the cloud, thus improving the accuracy of emotion detection. Through the cognition of information, the system proposed in this paper can transmit less data when transmitting data of the same value.

Fig. 5.

(a) Accuracy of emotion detection versus facial data instance under different communication systems. (b) Accuracy of emotion detection versus speech data instance under different communication systems.

Next, we evaluate the performance of the cognitive information system from the perspective of energy consumption. Compared with the traditional information system, the energy consumption of the cognitive information system not only includes transmission energy consumption, but also computing energy consumption involved in the cognitive processing of information. Fig. 6 shows the energy consumption of the two systems under the same accuracy of emotion detection. The experimental result shows that although cognitive information system requires extra computational energy consumption, the total energy consumption of the cognitive information system is still less than the traditional communication system under the same accuracy for the emotional detection model. This is mainly due to the scientifically reduced transmission rate of the cognitive information system in the cloud, which can result in an overall reduction of the energy consumption.

Fig. 6.

Energy consumption versus accuracy of emotion detection under different communication systems.

Finally, we investigate on the convergence rate of facial and speech data under different communication systems. We define the convergence of data as a degree in which new unlabeled data can no longer improve the accuracy of the emotion detection model in the cloud. Fig. 7(a) and (b) show a comparison of convergence rate of facial and speech data under the two systems, respectively. We can see that although the accuracy of the two systems eventually tends to become stable, the cognitive information system tends to converge faster. As soon as the value of information becomes stable, the transmission of information will be terminated.

Fig. 7.

(a) Convergence rate of facial data under different communication systems; (b) Convergence rate of speech data under different communication systems.

5.3. Discussion

Although this scheme can realize information cognition, its deployment depends on the following technologies:

• Artificial intelligence chip. The computing capability of a mobile terminal is constantly increasing to enable machine learning algorithms to be deployed at the mobile terminal where it can directly handle data computing and screening.

• Edge cloud computing. Using the storage and computing power of an edge cloud, data processing can be realized on an edge server in order to guarantee low latency and alleviate the traffic congestion of a core network.

• Network function virtualization. By using software-defined network (SDN) and network function virtualization (NFV), we can realize the coupling of control and user planes, as well as the coupling of hardware and software.

6. Conclusion

Considering information variability, in this paper we first introduce a cognitive information theory and its characteristics involving dynamic, polarity, evolution, convergence, and multi-view. We then provide the mailbox theory in cognitive information. Finally, combining the traditional communication system and cognitive information theory, we propose a new communication system. The results of the experiments show that the cognitive communication system has a greater performance than the traditional communication system.