Achieving Congestion Mitigation Using Distributed Power Control for Spectrum Sensor Nodes in Sensor Network-Aided Cognitive Radio Ad Hoc Networks

Abstract

The data sequence of spectrum sensing results injected from dedicated spectrum sensor nodes (SSNs) and the data traffic from upstream secondary users (SUs) lead to unpredictable data loads in a sensor network-aided cognitive radio ad hoc network (SN-CRN). As a result, network congestion may occur at a SU acting as fusion center when the offered data load exceeds its available capacity, which degrades network performance. In this paper, we present an effective approach to mitigate congestion of bottlenecked SUs via a proposed distributed power control framework for SSNs over a rectangular grid based SN-CRN, aiming to balance resource load and avoid excessive congestion. To achieve this goal, a distributed power control framework for SSNs from interior tier (IT) and middle tier (MT) is proposed to achieve the tradeoff between channel capacity and energy consumption. In particular, we firstly devise two pricing factors by considering stability of local spectrum sensing and spectrum sensing quality for SSNs. By the aid of pricing factors, the utility function of this power control problem is formulated by jointly taking into account the revenue of power reduction and the cost of energy consumption for IT or MT SSN. By bearing in mind the utility function maximization and linear differential equation constraint of energy consumption, we further formulate the power control problem as a differential game model under a cooperation or noncooperation scenario, and rigorously obtain the optimal solutions to this game model by employing dynamic programming. Then the congestion mitigation for bottlenecked SUs is derived by alleviating the buffer load over their internal buffers. Simulation results are presented to show the effectiveness of the proposed approach under the rectangular grid based SN-CRN scenario.

1. Introduction

Cognitive radio (CR) [1] has newly emerged as a promising solution to improve the spectrum utilization by allowing unlicensed secondary users (SUs) to access the idle licensed spectrum. In a CR network (CRN), SUs can periodically sense the licensed spectrum and opportunistically access the spectrum holes or spectrum opportunities (SOPs) unoccupied by primary users (PUs). Most of the existing research efforts in CRNs mainly focus on the issues of the physical and MAC layers for an infrastructure-based single hop scenario, such as spectrum sensing, spectrum access and sharing techniques [2,3,4]. In addition, SUs can also form a multi-hop ad hoc network without the support of infrastructure. In a cognitive radio ad hoc network (CRANET) [5], SUs can only access the SOPs by seeking to underlay, overlay, or interweave their signals with those of the existing PUs without significantly impacting their communications.

Spectrum sensing is one of the key enabling technologies for the establishment of CRNs, because it constantly allows for the opportunistic identification and use of the SOPs from a licensed primary network without causing harmful interference to the PUs. In order to improve the sensing performance, collaborative spectrum sensing has been proposed as an effective way to reliably detect the activity of PUs by addressing the issues imposed by the hidden PU terminal problem and the wireless channel impairments, such as the heavy shadowing and fading [6,7,8]. In this way, cooperation is achieved by allowing different SUs to collaborate and share their spectrum sensing results (SSR) through a fusion center (FC), which makes a global decision on the occupancy status of the licensed band. However, this centralized FC is not available in decentralized CRANETs. Clearly, each SU under this scenario must perform the distributed collaborative spectrum sensing, which is preferred to the centralized FC scheme because of its scalability, fault tolerance and flexibility [9].

In order to facilitate the spectrum sensing functionality, high sampling rates, high resolution analog to digital converters with large dynamic range, and high speed signal processors are required to be incorporated into an individual SU transceiver [10], which increases hardware cost and power consumption, especially for the double-radio sensing architecture of SU transceiver. An alternative approach is to adopt the cost-effective and dedicated spectrum sensor nodes (SSNs) that perform distributed collaborative spectrum sensing and report SSR to SUs acting as FCs in CRANETs [11]. Technically, s wireless sensor network can be naturally exploited to assist a CRANET by providing SSR about the current spectrum occupancy of PUs in a cooperation fashion. The concept of sensor network embedded into CRANETs has further called sensor network-aided CRANETs (SN-CRNs), which has been considered as one of the most appealing approaches to perform cost-effective spectrum sensing in CR systems [7,11,12].

Similar to most other traditional wireless networks or wireline Internet, network congestion in SN-CRNs will also occur when offered data load that exceed the available capacity of a SU due to buffer overflow caused by the data sequence of the SSR injected from SSNs together with the data traffic from upstream SUs. This therefore leads to energy consumption of SSNs, aggressive retransmission, queuing delay, and blocking of new flows from upstream SUs. Indubitably, a congestion control technique in the transport layer is essential to balance resource loads and avoid excessive congestion. However, the congestion control mechanism for the traditional Transmission Control Protocol (TCP) via the acknowledgement-triggered or window-based methods was initially designed and optimized to perform in reliable wired links with constrained bit error rates and round trip times (RTTs) [13]. A recent study [14] has reported that the performance of HTTP download deteriorates as much as about 40% under the TCP window control in an IEEE P1900.4 based cognitive wireless system by using User Datagram Protocol (UDP) and TCP transport protocols. On the other hand, some other research efforts about congestion control have also been conducted from the perspective of finding methods to modify the TCP protocol, such as TCP monitoring delayed acknowledgment, segment-based selective acknowledgement, TCP adaptive delayed-acknowledgment window, etc. [15], aiming to accommodate the challenging multi-hop wireless environments. Unfortunately, it has been shown that these methods of TCP modification and extension cannot be directly applied into SN-CRNs due to sudden large-scale bandwidth fluctuation, periodic interruption caused by spectrum sensing and channel switching [16].

Recently, there have also been previous works on congestion control for multi-hop CRANETs from a cross-layer design perspective. In [17], an end-to-end congestion control framework was proposed under the constraint of the non-uniform channel availability by taking into account the interactions from the physical layer to the transport layer. In [18], a cross-layer framework for joint optimization of MAC, scheduling, routing and congestion control was presented to maximize the throughput of a set of multi-hop end-to-end packet flows. However, the end-to-end control policy in [17,18] is ill suited for operation over wireless links characterized by higher RTTs. On the contrary, the hop-by-hop control reacts to congestion faster where the rates are adjusted at intermediate SUs by feedback information about the congestion state of congested SUs. A cross-layer framework to jointly achieve both congestion and power control through a non-convex optimization method was proposed in [19]. In [20], an optimization framework achieving tradeoff between energy efficiency and network utility maximization was devised, which can jointly balance interference, collision, and congestion among SUs by adjusting transmit power, persistence probability, together with data rate simultaneously via interaction between MAC and other layers. However, the proposed frameworks in [19,20] are just suited to mitigate the congestion caused by the data traffic from upstream SUs in multi-hop CRANETs, ignoring the impact of the data sequence of the SSR injected from SSNs on the congestion of SUs.

To the best of our knowledge, aside from some studies on congestion control for CRANETs as mentioned before, there is no related work reported in the literature related to congestion control over SN-CRNs. As a result, there is a strong motivation to explore congestion mitigation approach in SN-CRNs. Under this scenario, it is certainly not a surprise that the channel capacity between any SSN and FC is a concave function of the transmit power of this SSN and channel conditions [21]. In principle, effective transmit power control strategies have been widely used to maximize the total system capacity in conventional celluar wireless networks while adapting to the changing channel and interference conditions. Recent research efforts have achieved the capacity and energy efficiency maximization by devising the optimal power allocation on subchannels in two-tier femtocell networks based on orthogonal frequency division multiple access (OFDMA) [22], together with the optimal power control allocation and sensing time optimization in OFDMA cognitive small cell networks [23]. In addition, the transmission rate of this SSN always depends on channel capacity and is also a function of the transmit power according to the Shannon channel theorem. Thus, the congestion at FC can be controlled and mitigated through the transmission rate adjustment with the help of an optimal power allocation policy for this SSN in the physical layer. During a time interval, the amount of bits of the data sequence of SSR transmitted from this SSN to FC also approximately depends on the channel capacity [24]. For this observation it turns out that we can fully achieve the congestion mitigation for FC by reducing the amount of bits of the data sequence of the SSR transmitted from this SSN, aiming to release the capacity of the internal buffer for FC. In this paper, we propose a congestion mitigation approach by constructing a distributed power control framework for SSNs over the rectangular grid based SN-CRN. The main contributions of this paper are summarized as follows:

1. To evaluate the performance of local spectrum sensing, we present the relative divergence between the detection probability and the false alarm probability for each SSN under any uplink channel via the Kullback-Leibler divergence framework. By the aid of mathematical statistics, we obtain the detection probability and false alarm probability distributions for each SSN, and also model the stability metric of local spectrum sensing as the relative divergence by applying the entropy modeling framework.

2. We propose a distributed power control framework for SSNs from the interior tier (IT) and middle tier (MT) perspective in order to achieve the tradeoff between channel capacity and energy consumption. In particular, the power control problem is formulated as a differential game model by taking into account the utility function maximization together with the linear differential equation constraint with respect to energy consumption. We further present the theoretical results of the optimal solutions to this differential game model in a cooperative or noncooperative manner by using dynamic programming.

3. With the help of the proposed distributed power control framework, we attain the congestion mitigation for bottleneck SU by alleviating its buffer load over its internal buffer. We also rigorously analyze the impact of noncooperative and cooperative optimal transmit power for IT and MT SSNs on the internal buffer of bottleneck SU, respectively.

The rest of paper is organized as follows: Section 2 describes the system model. In Section 3, we present the spectrum sensing quality analysis method based on local spectrum sensing by SSNs. In Section 4, we formulate the distributed power control for IT and MT SSNs as a differential game model, and derive the noncooperative and cooperative optimal solutions. The congestion mitigation approach for bottleneck SU is analyzed rigorously in Section 5. Section 6 presents the simulation results. Finally, Section 7 concludes the paper.

2. System Model

2.1. Primary Network and Cognitive Radio ad hoc Network Model

We consider an underlay SN-CRN coexisting with a cellular primary network involving $N_P$ PUs and $N_S$ SUs in a torus area ${\Omega }_S={[0,\sqrt{N_S/\rho }]}^2$ ($\rho$ is the spatial density of SUs) sharing the spectrum within the same frequency band simultaneously, as depicted in Figure 1. Particularly, PUs have the full privilege of accessing their allocated frequency band whereas SUs can opportunistically utilize idle channels unoccupied by the PUs. In the cellular primary network, PUs send their data traffic to the primary base station (PBS) via the licensed uplink channels constituting a channel set $\mathcal{C}=\left\{1,2,\cdots ,N_c\right\}$. We employ the independent and identically distributed alternating ON-OFF process to model the occupation time length of PUs in uplink channels. Specifically, the OFF state indicates the idle state where the unoccupied uplink channels or the SOPs can be freely occupied by SUs.

Figure 1.

Illustration of the rectangular grid based SN-CRN. SU: secondary user; PU: primary user; PBS: primary base station.

In CRANET, SUs denoted by a set ${\mathcal{N}}_S=\left\{1,2,\cdots ,N_S\right\}$ can only leverage the OFF state to access the SOPs over the idle authorized uplink channels. Due to the randomness of data traffic and the dynamic behavior of PUs, we suppose that the SOPs are available for usage by SU i with a probability of ${\delta }_i$, for $i\in {\mathcal{N}}_S$. With the aid of the ON-OFF process to characterize the status of the uplink channels, the occupancy probability of the k-th uplink channel by PUs can be given by ${\alpha }_k/\left({\alpha }_k+{\beta }_k\right)$, where ${\alpha }_k$ is probability that the k-th uplink channel transits from OFF to ON state, and ${\beta }_k$ is probability that the k-th uplink channel transits from ON to OFF state, for $k\in \mathcal{C}$. We also assume that the occupancy probability of uplink channels by PUs can be acquired by SUs through a priori knowledge of the local spectrum sensing. By bearing in mind the mutually independent occupancy probability of the k-th uplink channel, the SOP usage probability ${\delta }_i$ of SU i during time interval $\left[t_0,T\right]$ is formulated as follows:

$${\delta }_i={\displaystyle \prod _{k=1}^{N_c}\left(1-\frac{{\alpha }_k}{{\alpha }_k+{\beta }_k}\right)},i=1,\cdots ,N_s$$ (1)

Let ${\vartheta }_i\left(t\right)$ and $B_i$ denote the amount of data traffic in the buffer of SU i at time $t\in \left[t_0,T\right]$ and the buffer size of SU i, respectively. The buffer of SU i generally consists of two buffer segments that can hold the offered data load including data traffic injected from upstream SUs and the data sequence of SSR transmitted from SSNs, respectively. Given a time interval $\Delta t$, the first buffer segment of SU i called as the forward buffer holds the amount of data traffic injected from upstream SUs denoted by ${\vartheta }_i^F\left(\Delta t\right)$. Meanwhile, the second buffer segment of SU i known as the internal buffer is used to store the amount of the data sequence of SSR transmitted from SSNs denoted by ${\vartheta }_i^I\left(\Delta t\right)$. Thus, the amount of aggregate data traffic from upstream SUs and SSNs in the buffer of SU i with a time interval $\Delta t$ can be formulated as ${\phi }_i\left(\Delta t\right)={\vartheta }_i^F\left(\Delta t\right)+{\vartheta }_i^I\left(\Delta t\right)$. Then, for a given time interval $\Delta t$, the amount of data traffic ${\vartheta }_i\left(t\right)$ in buffer of SU i at time $t$ evolves as follows:

$${\vartheta }_i\left(t+\Delta t\right)=\min \left\{{\vartheta }_i\left(t\right)+{\phi }_i\left(\Delta t\right),B_i\right\}-\left({\Lambda }_i\left(\Delta t\right)+{\chi }_i\left(\Delta t\right)\right)$$ (2)

where ${\Lambda }_i\left(\Delta t\right)$ and ${\chi }_i\left(\Delta t\right)$ stand for the amount of data traffic successfully delivered by SU i and the amount of the data sequence of SSR removed by SU i within a time interval $\Delta t$, respectively. It is worth noting that the received data sequence of SSR will be removed by SU i within a time interval $\Delta t$ in order to free storage capacity of the internal buffer.

Remark 1.

From Figure 1, there are $\left|{\mathcal{N}}_{IT}\right|+\left|{\mathcal{N}}_{ET}\right|$ hop-by-hop fluid flows of the data sequence of SSR injected from SSNs to a single SU (i.e., FC), in addition to the amount of data traffic from upstream SUs. Apparently, this single SU, also known as a possible bottlenecked SU, is a little more inclined to be a congested SU node as a consequence of its buffer overflow. For convenience, the terms bottlenecked SU is used in the following to describe a possible congested SU. It is worth noting that our work in this paper mainly concentrates on how to attain effective congestion mitigation for a single possible congested SU due to buffer overflow caused by the data sequence of SSR injected from SSNs by means of the proposed distributed power control framework for SSNs. However, it is conceivable that the amount of data traffic from upstream SUs may also lead to the congestion of bottlenecked SUs. This problem can be resolved by the specific congestion control technique (e.g., [25]) which is out of the scope of this work.

2.2. Sensor Network Model

As shown in Figure 1, we consider a rectangular grid based sensor network deployed in ${\Omega }_S$ to provide the SSR about real-time spectrum availability information to SUs. Each SSN is equipped with a single omnidirectional antenna, a predefined common control channel (CCC), and an energy detector that continuously senses the entire primary licensed uplink channels through individual local real-time measurement. We suppose that the distributed collaborative spectrum sensing is carried out by multiple collaborating SSNs to enhance the sensing performance. Also, each SU serves as the FC collecting the SSR and then makes a global decision on the availability of the monitored uplink channels via a decision fusion rule, e.g., OR-rule fusion mechanism [7]. All SSNs simultaneously communicate to SUs over a narrowband additive white Gaussian noise (AWGN) multiple-access channel with the channel bandwidth denoted by $W$. The horizontal or vertical distance between any SSNs in rectangular grid is initialized to be $d$. According to the location of each SU along with the distance between SSN and the corresponding SU, we define the set of SSNs center around the SU as a tier in rectangular grid. More specifically, the three-tier structure is exploited to organize SSNs into three groups due to the simplicity of implementation as shown in Figure 1, including an interior tier (IT) denoted by a set ${\mathcal{N}}_{IT}=\left\{1,\cdots ,n_{IT}\right\}$ with $n_{IT}=4$, a middle tier (MT) denoted by a set ${\mathcal{N}}_{MT}=\left\{1,\cdots ,n_{MT}\right\}$ with $n_{MT}=8$, and an exterior tier (ET) denoted by a set ${\mathcal{N}}_{ET}=\left\{1,\cdots ,n_{ET}\right\}$ with $n_{ET}=12$.

Remark 2.

Because of multiple SUs sharing the authorized uplink channels with PUs, one SSN may also belong to the different tiers based on the presented division criterion to devise the three-tier structure as stated previously. It is important to emphasize that our work in this paper is mainly aimed to study congestion mitigation approaches for one single possible congested SU under the scenario of the single three-tier structure of SSNs. The underlying scenario of the superimposed three-tier structures to organize SSNs is beyond the scope of this work. However, the results about our proposed distributed power control framework for SSNs are easily extendable to the superimposed three-tier structures.

We devise a time-slotted frame structure for sensor network, as illustrated in Figure 2, where each frame with duration $T_F$ is divided into different slots according to the types of tiers. Let $\phi$ be the number of time-slotted frames within time interval $\left[t_0,T\right]$. Clearly, the time-slotted frame duration is given by $T_F=\left(T-t_0\right)/\phi$. We assume that all SSNs from three tiers perform their operations simultaneously from the beginning of each time-slotted frame. We denote by ${\tau }_s$ the fraction of frame duration $T_F$ for the slot of spectrum sensing, and denote by ${\tau }_{rp}$ the fraction of frame duration $T_F$ for the slot of SSR reporting to the bottlenecked SU. We also use ${\tau }_f^{ET}$ to represent the slot of SSR forwarding from ET SSNs to neighbor MT SSNs, and use ${\tau }_{rc}^{MT}$ to denote the slot of SSR receiving by MT SSNs. During the remaining time, i.e., $T_F-{\tau }_s-{\tau }_f^{ET}$ for ET, $T_F-{\tau }_s-{\tau }_{rc}^{MT}-{\tau }_{rp}$ for MT, and $T_F-{\tau }_s-{\tau }_{rp}$ for IT, SSNs will go into sleep mode to save energy.

Figure 2.

The time-slotted frame structure adopted by dedicated spectrum sensor nodes (SSNs) in time-slot duration $T_F$. SSR: spectrum sensing results.

During the slot ${\tau }_{rp}$, the data sequence of the SSR will be transmitted by IT and MT SSNs to bottleneck SU. For analytical simplicity, we assume that the starting time and the terminal time of the slot ${\tau }_{rp}$ are equal to $t_0^{\prime }$ and $t_0^{\prime }+{\tau }_{rp}$ for both IT and MT SSNs, respectively. The instantaneous transmit power of the m-th SSN from IT or MT at time $t\in \left[t_0^{\prime },t_0^{\prime }+{\tau }_{rp}\right]$, denoted by $p_m\left(t\right)$, can be adjusted in a continuous way but is also limited by a maximum value $P_m^{\max }$, i.e., $0\le p_m\left(t\right)\le P_m^{\max }$. In the case of the simultaneous communications by IT and MT SSNs over an AWGN multiple-access channel, $p_m\left(t\right)$ should satisfy an average power constraint given as follows to mitigate the interference among IT and MT SSNs [26]:

$$\{\begin{array}{ll}\frac{1}{n_{IT}}{\displaystyle \sum _{m=1}^{n_{IT}}{p}_m\left(t\right)}\le P_{av}^{IT},& m\in {\mathcal{N}}_{IT}\\ \frac{1}{n_{MT}}{\displaystyle \sum _{m=1}^{n_{MT}}{p}_m\left(t\right)}\le P_{av}^{MT},& m\in {\mathcal{N}}_{MT}\end{array}$$ (3)

where $P_{av}^{IT}$ and $P_{av}^{MT}$ are the total average power assigned to IT and MT SSNs, respectively. We assume that each SSN knows its distance $d_{m\to b}$ from bottleneck SU b via the CCC and the channel path gain $h_{m\to b}$ from the m-th SSN to the bottlenecked SU b can be expressed as $h_{m\to b}={\left(d_{m\to b}\right)}^{-\kappa }$, where $\kappa \ge 2$ is the path-loss exponent and $b\in {\mathcal{N}}_S$. Thus, the channel capacity between the m-th SSN and bottlenecked SU b can be characterized by a concave function of the transmit power and channel conditions as follows [27]:

$$C_{m\to b}=W{\log }_2\left(1+\frac{p_m\left(t\right)K{\left|h_{m\to b}\right|}^2}{N_{0}W}\right)$$ (4)

where $K$ is a constant that depends on the transmission frequency and $N_0$ is the noise power spectral density. Under this channel capacity formulation, the signal-to-noise ratio (SNR) between the m-th SSN and bottlenecked SU b is given by:

$${\gamma }_m=\frac{p_m\left(t\right)K{\left|h_{m\to b}\right|}^2}{N_{0}W}$$ (5)

Remark 3.

It is assumed that the data sequence of the SSR will be forwarded by ET SSNs to neighbor MT SSNs via a single-hop fashion during the slot ${\tau }_f^{ET}$. Let $t_0^{″}$ and $t_0^{″}+{\tau }_f^{ET}$ denote the starting time and the terminal time of the slot ${\tau }_f^{ET}$, respectively. The transmit power $p_n\left(t\right)$ of the n-th SSN from ET at time $t\in \left[t_0^{″},t_0^{″}+{\tau }_f^{ET}\right]$ is limited to a maximum transmit power $P_n^{\max }$, for $0\le p_n\left(t\right)\le P_n^{\max }$ and $n\in {\mathcal{N}}_{ET}$. It is clear that the neighbor MT SSN with the shortest distance will be selected by an ET SSN as the next-hop SSN to save energy during the SSR forwarding period. In other words, the selection metric for the next-hop SSN by an ET SSN is only dependent on the distance between neighbor MT SSN and itself. Owing to the scenario of the rectangular grid based sensor network, it should be admitted that the shortest distance between ET SSN and neighbor MT SSN is equal to $d$ for any ET SSN. Therefore, the n-th SSN from ET is inclined to hold the same transmit power $p_n\left(t\right)$ at time $t$. Under an AWGN multiple-access channel, the average power constraint should also be satisfied for all ET SSNs, i.e., ${\displaystyle {\sum }_{n\in {\mathcal{N}}_{ET}}{p}_n\left(t\right)}\le n_{ET}{P}_{av}^{ET}$, where $P_{av}^{ET}$ is the total average power assigned to ET SSNs. Thus, our work in this paper primarily focuses on the distributed power control for IT and MT SSNs.

Remark 4.

The channel capacity $C_{m\to b}$ in Equation (4) will be rigorously guaranteed if the channel state information (CSI) including the channel path gain $h_{m\to b}$ and constant $K$ is perfectly known at the m-th SSN which transmits the data sequence of the SSR. In practice, the perfect knowledge of the CSI measured at the m-th SSN side cannot be available because of time-varying wireless channel impairments along with hardware limitations [28,29]. How to model the uncertain relation between the channel path gain $h_{m\to b}$ or constant $K$ and their estimates by taking into account the effect of imperfect CSI and outage constraint on distributed power control for IT and MT SSNs will be our further work in future.

2.3. Local Spectrum Sensing Model

We denote by ${\mathscr{H}}_1$ and ${\mathscr{H}}_0$ the binary hypotheses of the presence and absence of the PU on the uplink channel, respectively. Without loss of generality, we choose the m-th SSN from the three-tier structure of sensor network to describe its local spectrum sensing model during the slot ${\tau }_s$. This formulation can be easily extendable to the general case for any SSN from one of the tiers including IT, MT, and ET. The sampled signals that are received at the m-th SSN on the k-th uplink channel during the slot of spectrum sensing ${\tau }_s$ are given as:

$$y_{m,k}\left(u\right)=\{\begin{array}{ll}{h}_{m,k}s\left(u\right)+{\upsilon }_{m,k}\left(u\right),& {\mathscr{H}}_1\\ {\upsilon }_{m,k}\left(u\right),& {\mathscr{H}}_0\end{array}$$ (6)

where $s\left(u\right)$ denotes the signal from the PU on the k-th uplink channel with a sampling frequency $f_s$, ${\upsilon }_{m,k}\left(u\right)$ is the noise at the m-th SSN on the k-th uplink channel, $h_{m,k}$ is the channel gain between the PU and the m-th SSN on the k-th uplink channel implying Rayleigh fading. Then the number of samples that is collected by the m-th SSN on the k-th uplink channel can be defined as $U_s=f_s{\tau }_s$. We assume that the PU signal $s\left(u\right)$ satisfies an independent identically distributed (i.i.d.) random process with zero mean and variance ${\sigma }_s^2$, and the noise ${\upsilon }_{m,k}\left(u\right)$ is i.i.d. circularly symmetric complex Gaussian with zero mean and variance ${\sigma }_{\upsilon }^2$ [8]. Thus, the received SNR from the PU at the m-th SSN on the k-th uplink channel is given by ${\gamma }_{m,k}={\left|h_{m,k}\right|}^2{\sigma }_s^2/{\sigma }_{\upsilon }^2$. After collecting $U_s$ received signal samples on the k-th uplink channel, the m-th SSN obtains its test statistics given as $Y\left(m,k\right)=\left(1/U_s\right){\displaystyle {\sum }_{u=1}^{U_s}{\left|{\upsilon }_{m,k}\left(u\right)\right|}^2}$. Let ${\epsilon }_m$ denote a decision threshold by the m-th SSN to decide whether the channel is occupied by the PU. For the m-th SSN, the probabilities of detection and false alarm on the k-th uplink channel are approximately formulated as follows [8]:

$$p_d\left(m,k\right)=\mathrm{Pr}\left(Y\left(m,k\right)>{\epsilon }_m|{\mathscr{H}}_1\right)=\mathcal{Q}\left(\left(\frac{{\epsilon }_m}{{\sigma }_{\upsilon }^2}-{\gamma }_{m,k}-1\right)\sqrt{\frac{U_s}{2{\gamma }_{m,k}+1}}\right)$$ (7)

$$p_f\left(m,k\right)=\mathrm{Pr}\left(Y\left(m,k\right)>{\epsilon }_m|{\mathscr{H}}_0\right)=\mathcal{Q}\left(\left({\epsilon }_m-1\right)\sqrt{U_s}\right)$$ (8)

where $\mathcal{Q}(\cdot )$ denotes the right-tail probability of a normalized Gaussian distribution. Hence, the detection probability set ${\mathcal{P}}_d$ and the false alarm probability set ${\mathcal{P}}_f$ for the m-th SSN over the entire uplink channels can be further expressed as:

$${\mathcal{P}}_d=\left\{p_d\left(m,1\right),p_d\left(m,2\right),\cdots ,p_d\left(m,k\right),\cdots ,p_d\left(m,N_c\right)\right\}$$ (9)

$${\mathcal{P}}_f=\left\{p_f\left(m,1\right),p_f\left(m,2\right),\cdots ,p_f\left(m,k\right),\cdots ,p_f\left(m,N_c\right)\right\}$$ (10)

3. Spectrum Sensing Quality Analysis

In this section, our objective is to analyze the spectrum sensing quality of each SSN via a local spectrum sensing model, aiming to provide the quantification result with the emphasis to evaluate the spectrum sensing performance of each SSN. More importantly, the analysis results will be employed to formulate the distributed power control framework for IT and MT SSNs. By revisiting Equations (9) and (10) in local spectrum sensing model, we can observe that the detection probability $p_d\left(m,k\right)$ in ${\mathcal{P}}_d$ and the false alarm probability $p_f\left(m,k\right)$ in ${\mathcal{P}}_f$ can be referred to the random variables for the m-th SSN under the k-th uplink channel due to the uncertainty of the presence and absence of the PU. It is worth noting that the errors in spectrum sensing for a SSN will be generally considered negligible due to imperfect spectrum sensing [27], e.g., misdetection and false alarm caused by hardware capability of SSN and practical time-varying channel conditions. So the errors in spectrum sensing for a SSN will further incur the fact that different uplink channels will hold different probabilities of detection and false alarm. In particular, the higher the detection probability in ${\mathcal{P}}_d$, the better the PUs are protected; the lower the false alarm probability in ${\mathcal{P}}_f$, the more efficiently the uplink channel can be reutilized by SUs [7]. Based on this observation, the higher the relative divergence between $p_d\left(m,k\right)$ and $p_f\left(m,k\right)$, the better the performance of local spectrum sensing. It has been revealed that the Kullback–Leibler divergence is an effective measure of how one probability diverges from a second probability [30]. Hence, the relative divergence between $p_d\left(m,k\right)$ and $p_f\left(m,k\right)$ for the m-th SSN under the k-th uplink channel can be defined as follows based on a Kullback–Leibler divergence framework:

$$D\left(p_d\left(m,k\right)\Vert p_f\left(m,k\right)\right)\triangleq p_d\left(m,k\right){\log }_2\frac{p_d\left(m,k\right)}{p_f\left(m,k\right)}$$ (11)

With respect to the entire set of uplink channels, the relative divergence between ${\mathcal{P}}_d$ and ${\mathcal{P}}_f$ for the m-th SSN can be denoted as:

$$D_m({\mathcal{P}}_d\Vert {\mathcal{P}}_f)={\displaystyle \sum _{k=1}^{N_c}{p}_d\left(m,k\right){\log }_2\frac{p_d\left(m,k\right)}{p_f\left(m,k\right)}}$$ (12)

It is noticeable that the relative divergence between the detection probability and the false alarm probability just reflects the performance of local spectrum sensing by each SSN. Viewed from the SU perspective, we are also interested in the impact of the SOP usage probability on the spectrum sensing quality. To this end, we characterize the spectrum sensing quality factor which can be expressed by a function of two parameters including the relative divergence between ${\mathcal{P}}_d$ and ${\mathcal{P}}_f$ along with the SOP usage probability ${\delta }_b$ for bottleneck SU b. Specifically, the spectrum sensing quality factor ${ℱ}_{m\to b}$ of the m-th SSN with respect to bottleneck SU b can be defined as:

$${\mathcal{F}}_{m\to b}\triangleq {\delta }_b\cdot D_m\left({\mathcal{P}}_d\Vert {\mathcal{P}}_f\right),b\in {\mathcal{N}}_S$$ (13)

By using mathematical statistics theory, next we start by formulating the detection probability distribution ${\Upsilon }_d\left(m\right)$ and the false alarm probability distribution ${\Upsilon }_f\left(m\right)$, which have been derived from Algorithm 1.

Algorithm 1: Generation Procedure of Distribution ${\Upsilon }_d\left(m\right)$ and Distribution ${\Upsilon }_f\left(m\right)$
1:	Input: The detection probability set ${\mathcal{P}}_d$ and the false alarm probability set ${\mathcal{P}}_f$.
2:	Output: The detection probability distribution ${\Upsilon }_d\left(m\right)=\left\{{\Upsilon }_1^d\left(m\right),{\Upsilon }_2^d\left(m\right),\cdots ,{\Upsilon }_{\xi }^d\left(m\right)\right\}$
	The false alarm probability distribution ${\Upsilon }_f\left(m\right)=\left\{{\Upsilon }_1^f\left(m\right),{\Upsilon }_2^f\left(m\right),\cdots ,{\Upsilon }_{\varsigma }^f\left(m\right)\right\}$.
	The detection probability distribution generation:
1:	Sort the detection probability $p_d\left(m,1\right),p_d\left(m,2\right),\cdots ,p_d\left(m,N_c\right)$ in ascending order and constitute the sorted sequence as ${\widehat{p}}_d\left(1\right),{\widehat{p}}_d\left(2\right),\cdots ,{\widehat{p}}_d\left(N_c\right)$.
2:	$\forall X_d,Y_d>0$, for $X_d<{\widehat{p}}_d\left(1\right)$ and $Y_d>{\widehat{p}}_d\left(N_c\right)$.
3:	Divide interval $\left[X_d,Y_d\right]$ into $\xi$ equal subintervals, i.e., $X_d={\mu }_0^d<{\mu }_1^d<{\mu }_2^d\cdots <{\mu }_{\xi -1}^d<{\mu }_{\xi }^d=Y_d$.
4:	for $\ell =1\to \xi$ do
5:	${\mu }_{\ell }^d-{\mu }_{\ell -1}^d=\left(Y_d-X_d\right)/\xi$.
6:	Calculate the number of the detection probabilities within subinterval $\left({\mu }_{\ell -1}^d,{\mu }_{\ell }^d\right]$ denoted by $n_{\ell }^d$.
7:	Calculate the probability ${\Upsilon }_{\ell }^d\left(m\right)=n_{\ell }^d/N_c$.
8:	end for
9:	Return: ${\Upsilon }_d\left(m\right)=\left\{{\Upsilon }_1^d\left(m\right),{\Upsilon }_2^d\left(m\right),\cdots ,{\Upsilon }_{\xi }^d\left(m\right)\right\}$.
	The false alarm probability distribution generation:
1:	Sort the false alarm probability $p_f\left(m,1\right),p_f\left(m,2\right),\cdots ,p_f\left(m,N_c\right)$ in ascending order and constitute the sorted sequence as ${\widehat{p}}_f\left(1\right),{\widehat{p}}_f\left(2\right),\cdots ,{\widehat{p}}_f\left(N_c\right)$.
2:	$\forall X_f,Y_f>0$, for $X_f<{\widehat{p}}_f\left(1\right)$ and $Y_f>{\widehat{p}}_f\left(N_c\right)$.
3:	Divide interval $\left[X_f,Y_f\right]$ into $\varsigma$ equal subintervals, i.e., $X_f={\mu }_0^f<{\mu }_1^f<{\mu }_2^f\cdots <{\mu }_{\varsigma -1}^f<{\mu }_{\varsigma }^f=Y_f$.
4:	for $\ell =1\to \varsigma$ do
5:	${\mu }_{\ell }^f-{\mu }_{\ell -1}^f=\left(Y_f-X_f\right)/\varsigma$.
6:	Calculate the number of the false alarm probabilities within subinterval $\left({\mu }_{\ell -1}^f,{\mu }_{\ell }^f\right]$ denoted by $n_{\ell }^f$.
7:	Calculate the probability ${\Upsilon }_{\ell }^f\left(m\right)=n_{\ell }^f/N_c$.
8:	end for
9:	Return: ${\Upsilon }_f\left(m\right)=\left\{{\Upsilon }_1^f\left(m\right),{\Upsilon }_2^f\left(m\right),\cdots ,{\Upsilon }_{\varsigma }^f\left(m\right)\right\}$.

Owing to the fact that the number of the detection probabilities or the false alarm probabilities is calculated under the constraint of $N_c$, it is clear that the distributions ${\Upsilon }_d\left(m\right)$ and ${\Upsilon }_f\left(m\right)$ fall into the complete probability distributions, i.e., ${\displaystyle {\sum }_{\ell =1}^{\xi }{\Upsilon }_{\ell }^d\left(m\right)}=1$ and ${\displaystyle {\sum }_{\ell =1}^{\varsigma }{\Upsilon }_{\ell }^f\left(m\right)}=1$. Apparently, the entropy paradigm should be used for a measure of the uncertainty associated with a random variable of a distribution in information theory [31], and can be also applied to measure the uncertainty of the distributions ${\Upsilon }_d\left(m\right)$ and ${\Upsilon }_f\left(m\right)$. As a result, for the m-th SSN over the entire uplink channels, the uncertainty characterization of the distributions ${\Upsilon }_d\left(m\right)$ and ${\Upsilon }_f\left(m\right)$ can be respectively described as based on the entropy modeling framework:

$$H\left({\Upsilon }_d\left(m\right)\right)=-{\displaystyle \sum _{\ell =1}^{\xi }{\Upsilon }_{\ell }^d\left(m\right){\log }_2{\Upsilon }_{\ell }^d\left(m\right)}$$ (14)

$$H\left({\Upsilon }_f\left(m\right)\right)=-{\displaystyle \sum _{\ell =1}^{\varsigma }{\Upsilon }_{\ell }^f\left(m\right){\log }_2{\Upsilon }_{\ell }^f\left(m\right)}$$ (15)

In what follows, we are also interested in gaining a better understanding of how to apply this entropy measurement to evaluate the stability of local spectrum sensing. It should be admitted that the entropy tends to be larger when the change of the random variable values in given distribution is disorder or randomness [32]. That is, a more disordered probability distribution will result in larger entropy. Thus, the better performance of spectrum sensing for the m-th SSN will bring about the more ordered probability distributions ${\Upsilon }_d\left(m\right)$ and ${\Upsilon }_f\left(m\right)$. In this way, the stability of the distributions ${\Upsilon }_d\left(m\right)$ and ${\Upsilon }_f\left(m\right)$ will decrease because of more disorder for the values of the detection probability and false alarm probability in the distributions ${\Upsilon }_d\left(m\right)$ and ${\Upsilon }_f\left(m\right)$. Based on the insight, we model the stability metric of local spectrum sensing by the relative divergence between the entropy of the detection probability distribution and the entropy of the false alarm probability distribution. Thus, the stability metric of local spectrum sensing for the m-th SSN over the entire uplink channels denoted by $H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right)$ can be calculated as follows:

$$H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right)=\left|H\left({\Upsilon }_d\left(m\right)\right)-H\left({\Upsilon }_f\left(m\right)\right)\right|$$ (16)

4. Distributed Power Control for Spectrum Sensor Nodes

4.1. Problem Formulation

It is considered that the channel capacity between the SSNs from IT or MT and a bottlenecked SU is a concave function of the transmit power and channel conditions. Therefore, each SSN is expected to increase the transmit power in the physical layer to provide as much channel capacity that each flow of the data sequence of the SSR requires. However, the higher transmit power will result in the more energy consumption for SSN. Meanwhile, the average power constraint will not be guaranteed for all SSNs if each SSN aims to increase the transmit power. As a consequence, it is necessary to require a tradeoff between channel capacity and energy consumption by achieving an optimal power allocation for all IT and MT SSNs in the physical layer during the slot ${\tau }_{rp}$. Under the constraint of path-loss of wireless channel, the maximum transmit power of the m-th SSN at a distance $d_{m\to b}$ is approximately equal to [33]:

$$P_m^{\max }(\mathrm{dBm})=P_0(\mathrm{dBm})-10\kappa lo{g}_{10}\left(d_{m\to b}/d_0\right)$$ (17)

where $P_0$ is the receiving reference power of bottlenecked SU b at a reference distance $d_0$. Because of the limitation by a maximum value $P_m^{\max }$, the value of power reduction for the m-th SSN at time $t\in \left[t_0^{\prime },t_0^{\prime }+{\tau }_{rp}\right]$ can be expressed by $P_m^{\max }-p_m\left(t\right)$. Then the power reduction efficiency for the m-th SSN can be written as $\left(P_m^{\max }-p_m\left(t\right)\right)/P_m^{\max }$. To formulate the revenue for power reduction by the m-th SSN, we firstly define a pricing factor for power reduction by taking into account both the power reduction efficiency and the stability metric of local spectrum sensing according to the distributions ${\Upsilon }_d\left(m\right)$ and ${\Upsilon }_f\left(m\right)$. More precisely, we define a pricing factor by characterizing an efficiency-to-stability ratio for the m-th SSN at time $t\in \left[t_0^{\prime },t_0^{\prime }+{\tau }_{rp}\right]$, denoted by ${\wp }_m^R$, to describe the power reduction efficiency under the stability of local spectrum sensing, which can be defined as:

$${\wp }_m^R\triangleq \frac{\left(P_m^{\max }-p_m\left(t\right)\right)/P_m^{\max }}{H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right)}$$ (18)

Thus, the revenue of the power reduction for the m-th SSN at time $t\in \left[t_0^{\prime },t_0^{\prime }+{\tau }_{rp}\right]$ by attaining the product of the pricing factor together with the power reduction value, i.e.,

$$U_m^R={\wp }_m^R\left(P_m^{\max }-p_m\left(t\right)\right)$$ (19)

From Equations (18) and (19), it is worth remarking that the smaller of stability metric of local spectrum sensing will generate more pricing ${\wp }_m^R$ and also yield more revenue $U_m^R$ of the power reduction for the m-th SSN. On the other hand, it is also a natural idea for the m-th SSN to reduce its transmit power to obtain more revenue $U_m^R$ according to Equation (19). To depict the cost of energy consumption for the m-th SSN, we further define another pricing factor for energy consumption, denoted by ${\wp }_m^C$, by considering the spectrum sensing quality factor ${ℱ}_{m\to i}$ as:

$${\wp }_m^C\triangleq e^{-{\delta }_i\cdot D_m\left({\mathcal{P}}_d\Vert {\mathcal{P}}_f\right)}$$ (20)

Recall that the higher spectrum sensing quality factor will result in the better performance of local spectrum sensing by each SSN. As can be seen from Equation (20), it will be far more realistic to reduce the pricing ${\wp }_m^C$ for energy consumption for the m-th SSN with respect to the better performance of its local spectrum sensing, aiming to balance local spectrum sensing and the energy efficiency of SSN [34]. Let $E_m\left(t\right)$ and $E_A\left(t\right)$ represent the energy consumption value and the available energy value of the m-th SSN at time $t$, respectively. It is assumed that the available energy of the m-th SSN is derived from its battery of limited capacity and the harvesting energy from the renewable energy sources by exploiting the energy harvesting technology. Thus, the energy consumption of the m-th SSN evolves according to a linear differential equation given as:

$$\frac{d{E}_m\left(t\right)}{dt}=E_A\left(t\right)-{\tau }_{rp}{p}_m\left(t\right)-E_m\left(t\right)$$ (21)

Then the cost of energy consumption for the m-th SSN at time $t\in \left[t_0^{\prime },t_0^{\prime }+{\tau }_{rp}\right]$ can be given as:

$$U_m^C={\wp }_m^C{E}_m\left(t\right)$$ (22)

Therefore, based on the revenue and the cost formulated in Equations (19) and (22), the utility function of the m-th SSN at time $t\in \left[t_0^{\prime },t_0^{\prime }+{\tau }_{rp}\right]$ can be constructed as follows:

$$U_m=U_m^R-U_m^C={\wp }_m^R\left(P_m^{\max }-p_m\left(t\right)\right)-{\wp }_m^C{E}_m\left(t\right)$$ (23)

We denote the discount factor by $r$, for $0<r<1$. Our optimization objective is to maximize the utility function $U_m$ by choosing optimal transmit power $p_m^{OP}\left(t\right)$ of the m-th SSN during the slot ${\tau }_{rp}$ according to ${\wp }_m^R$ and ${\wp }_m^C$, at time $t\in \left[t_0^{\prime },t_0^{\prime }+{\tau }_{rp}\right]$, i.e.,

$$\mathrm{Maximize}:{\displaystyle {\int }_{t_0^{\prime }}^{t_0^{\prime }+{\tau }_{rp}}\left({\wp }_m^R\left(P_m^{\max }-p_m\left(t\right)\right)-{\wp }_m^C{E}_m\left(t\right)\right)}{e}^{-r\left(t-t_0^{\prime }\right)}$$ (24)

It is noteworthy that the discount factor $r$ is an exponential factor between 0 and 1 by which the future utility must be multiplied in order to obtain the present value with the underlying structure of differential game theory in mind. Therefore, each SSN is required to maximize its discounted utility $U_m$ function by discount factor $r$, implying that discount factor will have a stronger impact on the utility obtained by each SSN in the future. To this end, the power control problem for all IT and MT SSNs in the physical layer can be formulated as a differential game model defined by $\mathcal{G}=\left\{\mathcal{N},{\left\{p_m^{OP}\left(t\right)\right\}}_{m\in \mathcal{N}},E_m^{OP}\left(t\right),{\left\{U_m\right\}}_{m\in \mathcal{N}}\right\}$, where $\mathcal{N}={\mathcal{N}}_{IT}\cup {\mathcal{N}}_{MT}$ is the set of players involving all IT and MT SSNs, $p_m^{OP}\left(t\right)$ is the strategy of player m, ${\left\{p_m^{OP}\left(t\right)\right\}}_{m\in \mathcal{N}}$ is the set of strategies or strategy space related to all players, $E_m^{OP}\left(t\right)$ is the state variable associated with optimal transmit power $p_m^{OP}\left(t\right)$, and ${\left\{U_m\right\}}_{m\in \mathcal{N}}$ is the set of utility function of all players with their strategies.

4.2. Noncooperative Optimal Solution

We formulate a dynamic optimization problem $\mathcal{P}1$ to derive the optimal solution to the differential game model $\mathcal{G}$ with the objective of the utility function maximization problem under the linear differential equation constraint of the energy consumption for the m-th SSN. From Equations (21) and (24), the problem $\mathcal{P}1$ can be formulated as:

$$\begin{array}{ccc}\mathcal{P}1:\hfill & \hfill \underset{p_m\left(t\right),m\in \mathcal{N}}{\max }\hfill & {\displaystyle {\int }_{t_0^{\prime }}^{t_0^{\prime }+{\tau }_{rp}}\left({\wp }_m^R\left(P_m^{\max }-p_m\left(t\right)\right)-{\wp }_m^C{E}_m\left(t\right)\right)}{e}^{-r\left(t-t_0^{\prime }\right)}dt\hfill \\ & \hfill \mathrm{s}.\mathrm{t}.\hfill & \frac{d{E}_m\left(t\right)}{dt}=E_A\left(t\right)-{\tau }_{rp}{p}_m\left(t\right)-E_m\left(t\right)\hfill \end{array}$$ (25)

For the noncooperation scenario if all the players play noncooperatively, we aim at deriving an optimal solution to the problem $\mathcal{P}1$ in the distributed noncooperative power control (NoCoPC) problem for all IT and MT SSNs by employing the theory of dynamic programming developed by Bellman. Note that the players can abandon the cooperation due to their selfishness and own interests in the NoCoPC problem, e.g., the selfish behavior in forwarding the data sequence of the SSR to bottleneck SU. Specifically, we employ $p_m^{NC}\left(t\right)$ to represent the noncooperative optimal solution to the problem $\mathcal{P}1$, and assume that there exists a continuously differentiable function ${\mathcal{V}}_m^{NC}\left(p_m,E_m\right)$ satisfying the following partial differential equation:

$$r{\mathcal{V}}_m^{NC}\left(p_m,E_m\right)=\max \left({\wp }_m^R\left(P_m^{\max }-p_m\left(t\right)\right)-{\wp }_m^C{E}_m\left(t\right)+\frac{\partial {\mathcal{V}}_m^{NC}\left(p_m,E_m\right)}{\partial E_m}\left(E_A\left(t\right)-{\tau }_{rp}{p}_m\left(t\right)-E_m\left(t\right)\right)\right)$$ (26)

Proposition 1.

The function ${\mathcal{V}}_m^{NC}\left(p_m,E_m\right)$ for the m-th SSN in Equation (26) should be subject to the partial differential equation constraint as follows:

$$\frac{\partial {\mathcal{V}}_m^{NC}\left(p_m,E_m\right)}{\partial E_m}=-\frac{{\wp }_m^C}{r+1}$$ (27)

Proof. 

See Appendix A. ☐

Proposition 2.

The noncooperative optimal solution $p_m^{NC}\left(t\right)$ constitutes a Nash equilibrium solution to the problem $\mathcal{P}1$ if and only if the optimal transmit power for the m-th SSN can be expressed as:

$$p_m^{NC}\left(t\right)=P_m^{\max }\left(1-\frac{H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2\left(r+1\right)e^{{\delta }_i\cdot D_m\left({\mathcal{P}}_d\Vert {\mathcal{P}}_f\right)}}\right)$$ (28)

Proof. 

Recall the following expression of the noncooperative optimal solution in Equation (A1) on the basis of Proposition 1. By substituting the expression of function ${\mathcal{V}}_m^{NC}\left(p_m,E_m\right)$ in Equation (27) into Equation (A1), we can easily obtain the noncooperative optimal transmit power $p_m^{NC}\left(t\right)$ which constitutes a Nash equilibrium solution to the problem $\mathcal{P}1$ can be formulated by Equation (28). ☐

By observing Proposition 2, it is clear that an increased discount factor $r$ will enhance the noncooperative optimal transmit power $p_m^{NC}\left(t\right)$ for the m-th SSN. From Equation (28), the noncooperative optimal transmit power vector $P^{NC}$ for all IT and MT SSNs in the NoCoPC problem can be further combined as:

$$P^{NC}=\left\{\underset{\mathrm{for}{n}_{IT}\mathrm{IT}\mathrm{SSNs}}{\underbrace{p_1^{NC}\left(t\right),\cdots ,p_{\left|{\mathcal{N}}_{IT}\right|}^{NC}\left(t\right)}},\stackrel{\mathrm{for}{n}_{MT}\mathrm{MT}\mathrm{SSNs}}{\overbrace{p_{\left|{\mathcal{N}}_{IT}\right|+1}^{NC}\left(t\right),\cdots ,p_{\left|{\mathcal{N}}_{IT}\right|+\left|{\mathcal{N}}_{MT}\right|}^{NC}\left(t\right)}}\right\}$$ (29)

Given the noncooperative optimal solution $p_m^{NC}\left(t\right)$, by substituting ${\mathcal{V}}_m^{NC}\left(p_m,E_m\right)$ and $p_m^{NC}\left(t\right)$ into Equation (26), the function ${\mathcal{V}}_m^{NC}\left(p_m^{NC}\left(t\right),E_m^{NC}\left(t\right)\right)$ for the m-th SSN is further given by:

$${\mathcal{V}}_m^{NC}\left(p_m^{NC}\left(t\right),E_m^{NC}\left(t\right)\right)=\frac{{\wp }_m^C}{r\left(r+1\right)}\left({\tau }_{rp}{P}_m^{\max }-\frac{P_m^{\max }{H}_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}^2{\wp }_m^C}{4\left(r+1\right)}-E_A\left(t\right)-r{E}_m^{NC}\left(t\right)\right)$$ (30)

where $E_m^{NC}\left(t\right)$ is the noncooperative optimal energy consumption value of the m-th SSN at time $t$.

4.3. Cooperative Optimal Solution

In this subsection, we move on to explore the distributed cooperative power control (CoPC) problem for all IT and MT SSNs by building up the cooperation scenario if all the players play cooperatively via the differential game model $\mathcal{G}$. Note that this is a natural idea for the players aiming to achieve an optimal power allocation through full cooperation for their common interests. Under this scenario, our optimization objective is to maximize the sum of the utility functions of all players (i.e., ${\displaystyle {\sum }_{m\in \mathcal{N}}{U}_m}$) while the linear differential equation constraint of the energy consumption should also be satisfied for the m-th SSN. To be specific, we formulate a dynamic optimization problem $\mathcal{P}2$ as follows to attain the objective of maximizing the sum of the utility functions of all players:

$$\begin{array}{ccc}\mathcal{P}2:\hfill & \hfill \underset{p_1\left(t\right),\cdots ,p_m\left(t\right),\cdots ,p_{\left|\mathcal{N}\right|}\left(t\right),m\in \mathcal{N}}{\max }\hfill & {\displaystyle \sum _{m\in \mathcal{N}}{\displaystyle {\int }_{t_0^{\prime }}^{t_0^{\prime }+{\tau }_{rp}}\left({\wp }_m^R\left(P_m^{\max }-p_m\left(t\right)\right)-{\wp }_m^C{E}_m\left(t\right)\right)}{e}^{-r\left(t-t_0^{\prime }\right)}dt}\hfill \\ & \hfill \mathrm{s}.\mathrm{t}.\hfill & \frac{d{E}_m\left(t\right)}{dt}=E_A\left(t\right)-{\tau }_{rp}{p}_m\left(t\right)-E_m\left(t\right)\hfill \end{array}$$ (31)

Under the cooperation scenario, we use $p_m^C\left(t\right)$ to represent the cooperative optimal solution to the problem $\mathcal{P}2$, and assume that there exists a continuously differentiable function ${\mathcal{W}}_m^C\left(p_m,E_m\right)$ satisfying the following partial differential equation:

$$r{\mathcal{W}}_m^C\left(p_m,E_m\right)=\max \left({\displaystyle \sum _{m\in \mathcal{N}}\left({\wp }_m^R\left(P_m^{\max }-p_m\left(t\right)\right)-{\wp }_m^C{E}_m\left(t\right)\right)}+\frac{\partial {\mathcal{W}}_m^C\left(p_m,E_m\right)}{\partial E_m}\left(E_A\left(t\right)-{\tau }_{rp}{p}_m\left(t\right)-E_m\left(t\right)\right)\right)$$ (32)

Proposition 3.

The function ${\mathcal{W}}_m^C\left(p_m,E_m\right)$ for the m-th SSN in Equation (32) should be subject to the partial differential equation constraint as follows:

$$\frac{\partial {\mathcal{W}}_m^C\left(p_m,E_m\right)}{\partial E_m}=-\frac{{\displaystyle \sum _{m\in \mathcal{N}}{\wp }_m^C}}{r+1}$$ (33)

Proof. 

See Appendix B. ☐

Proposition 4.

The cooperative optimal solution $p_m^C\left(t\right)$ to the problem $\mathcal{P}2$ if and only if the optimal transmit power for the m-th SSN can be expressed as:

$$p_m^C\left(t\right)=P_m^{\max }\left(1-\frac{H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2\left(r+1\right){\displaystyle \sum _{m\in \mathcal{N}}{e}^{{\delta }_i\cdot D_m\left({\mathcal{P}}_d\Vert {\mathcal{P}}_f\right)}}}\right)$$ (34)

Proof. 

Substituting the expression of function ${\mathcal{W}}_m^C\left(p_m,E_m\right)$ in Equation (33) into Equation (A4), and applying the result of the indicated maximization operation in Equation (A4), we derive the cooperative optimal solution $p_m^C\left(t\right)$ to the problem $\mathcal{P}2$ as in Equation (34). ☐

Similar to Proposition 2, it is also revealed that an increased discount factor $r$ will enhance the cooperative optimal transmit power $p_m^C\left(t\right)$ for the m-th SSN. According to Equation (34), the cooperative optimal transmit power vector $P^C$ for all IT and MT SSNs in the CoPC problem can be also combined as follows:

$$P^C=\left\{\underset{\mathrm{for}{n}_{IT}\mathrm{IT}\mathrm{SSNs}}{\underbrace{p_1^C\left(t\right),\cdots ,p_{\left|{\mathcal{N}}_{IT}\right|}^C\left(t\right)}},\stackrel{\mathrm{for}{n}_{MT}\mathrm{MT}\mathrm{SSNs}}{\overbrace{p_{\left|{\mathcal{N}}_{IT}\right|+1}^C\left(t\right),\cdots ,p_{\left|{\mathcal{N}}_{IT}\right|+\left|{\mathcal{N}}_{MT}\right|}^C\left(t\right)}}\right\}$$ (35)

Define the notation ${\Psi }_m=P_m^{\max }{H}_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right)$ for simplicity in the following. Given the cooperative optimal solution $p_m^C\left(t\right)$, by plugging ${\mathcal{W}}_m^C\left(p_m,E_m\right)$ and $p_m^C\left(t\right)$ into Equation (32), the function ${\mathcal{W}}_m^C\left(p_m^C\left(t\right),E_m^C\left(t\right)\right)$ for the m-th SSN is further given by:

$$\begin{array}{ll}{\mathcal{W}}_m^C\left(p_m^C\left(t\right),E_m^C\left(t\right)\right)& =\frac{{\tau }_{rp}^2}{4r{\left(r+1\right)}^2}\left({\displaystyle \sum _{m\in \mathcal{N}}\left({\Psi }_m{\left({\displaystyle \sum _{m\in \mathcal{N}}{\wp }_m^C}\right)}^2\right)}-2{\Psi }_m{\left({\displaystyle \sum _{m\in \mathcal{N}}{\wp }_m^C}\right)}^2\right)\\ & +\frac{\left(E_m^C\left(t\right)-E_A\left(t\right)\right){\displaystyle \sum _{m\in \mathcal{N}}{\wp }_m^C}}{r\left(r+1\right)}+\frac{{\tau }_{rp}{P}_m^{\max }{\displaystyle \sum _{m\in \mathcal{N}}{\wp }_m^C}}{r}-\frac{1}{r}{\displaystyle \sum _{m\in \mathcal{N}}\left({\wp }_m^C{E}_m^C\left(t\right)\right)}\end{array}$$ (36)

where $E_m^C\left(t\right)$ is the cooperative optimal energy consumption value of the m-th SSN at time $t$.

By using $P^{NC}$ of the NoCoPC problem and $P^C$ of the CoPC problem, we then design a distributed optimal transmit power adjustment (OTPA) algorithm as stated in Algorithm 2. Recall that the pricing factor ${\wp }_m^R$ is employed to characterize the efficiency-to-stability ratio for the m-th SSN, for $m\in \mathcal{N}$, aiming to describe the power reduction efficiency under the stability of local spectrum sensing. Thus, in order to ensure the convergence of optimal transmit power adjustment, we design the average efficiency-to-stability ratio ${\overline{\varpi }}_m$ as a scaling coefficient in Algorithm 2 as follows:

$${\overline{\varpi }}_m=\{\begin{array}{ll}\frac{{\wp }_m^R}{{\displaystyle {\sum }_{m=1}^{n_{IT}}{\wp }_m^R}},& m\in {\mathcal{N}}_{IT}\\ \frac{{\wp }_m^R}{{\displaystyle {\sum }_{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\wp }_m^R}},& m\in {\mathcal{N}}_{MT}\end{array}$$ (37)

It is worth remarking that the adjusted optimal transmit power will be updated with respect ro the scaling coefficient ${\overline{\varpi }}_m$. In particular, the adjusted optimal transmit power for all IT and MT SSNs via OTPA algorithm can be made locally while guaranteeing that the condition of the average power constraint as given in Equation (3) is satisfied.

Algorithm 2: Distributed Optimal Transmit Power Adjust Algorithm
1:	Input: $P^{NC}$ and $P^C$.
2:	Output: $\left\{\underset{1\to n_{IT}}{\underbrace{{\tilde{p}}_1^{NC}\left(t\right),\cdots ,{\tilde{p}}_{n_{IT}}^{NC}\left(t\right)}},\stackrel{n_{IT}+1\to n_{IT}+n_{MT}}{\overbrace{{\tilde{p}}_{n_{IT}+1}^{NC}\left(t\right),\cdots ,{\tilde{p}}_{n_{IT}+n_{MT}}^{NC}\left(t\right)}}\right\}$ and $\left\{\underset{1\to n_{IT}}{\underbrace{{\tilde{p}}_1^C\left(t\right),\cdots ,{\tilde{p}}_{n_{IT}}^C\left(t\right)}},\stackrel{n_{IT}+1\to n_{IT}+n_{MT}}{\overbrace{{\tilde{p}}_{n_{IT}+1}^C\left(t\right),\cdots ,{\tilde{p}}_{n_{IT}+n_{MT}}^C\left(t\right)}}\right\}$.
3:	Initialization: $\left\{P_1^{\max },\cdots ,P_m^{\max },\cdots ,P_{\left|\mathcal{N}\right|}^{\max }\right\}$, $\left\{{\wp }_1^R,\cdots ,{\wp }_m^R,\cdots ,{\wp }_{\left|\mathcal{N}\right|}^R\right\}$, $\left\{{\wp }_1^C,\cdots ,{\wp }_m^C,\cdots ,{\wp }_{\left|\mathcal{N}\right|}^C\right\}$.
4:	for $m=1\to \left|\mathcal{N}\right|$ do
5:	${\tilde{p}}_m^{NC}\left(t\right)\leftarrow p_m^{NC}\left(t\right)$ using Equation (28), and ${\tilde{p}}_m^C\left(t\right)\leftarrow p_m^C\left(t\right)$ using Equation (34).
6:	end for
7:	if ${\displaystyle {\sum }_{m=1}^{n_{IT}}{\tilde{p}}_m^{NC}\left(t\right)}>n_{IT}{P}_{av}^{IT}$ or ${\displaystyle {\sum }_{m=1}^{n_{IT}}{\tilde{p}}_m^C\left(t\right)}>n_{IT}{P}_{av}^{IT}$ then
8:	repeat
9:	for $m=1\to n_{IT}$ do
10:	${\tilde{p}}_m^{NC}\left(t\right)\leftarrow \left({\wp }_m^R/{\displaystyle {\sum }_{m=1}^{n_{IT}}{\wp }_m^R}\right)\times {\tilde{p}}_m^{NC}\left(t\right)$ when ${\displaystyle {\sum }_{m=1}^{n_{IT}}{\tilde{p}}_m^{NC}\left(t\right)}>n_{IT}{P}_{av}^{IT}$.
11:	${\tilde{p}}_m^C\left(t\right)\leftarrow \left({\wp }_m^R/{\displaystyle {\sum }_{m=1}^{n_{IT}}{\wp }_m^R}\right)\times {\tilde{p}}_m^C\left(t\right)$ when ${\displaystyle {\sum }_{m=1}^{n_{IT}}{\tilde{p}}_m^C\left(t\right)}>n_{IT}{P}_{av}^{IT}$.
12:	end
13:	until ${\displaystyle {\sum }_{m=1}^{n_{IT}}{\tilde{p}}_m^{NC}\left(t\right)}>n_{IT}{P}_{av}^{IT}$ and ${\displaystyle {\sum }_{m=1}^{n_{IT}}{\tilde{p}}_m^C\left(t\right)}>n_{IT}{P}_{av}^{IT}$
14:	end if
15:	if ${\displaystyle {\sum }_{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\tilde{p}}_m^{NC}\left(t\right)}>n_{MT}{P}_{av}^{MT}$ or ${\displaystyle {\sum }_{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\tilde{p}}_m^C\left(t\right)}>n_{MT}{P}_{av}^{MT}$ then
16:	repeat
17:	for $m=n_{IT}+1\to n_{IT}+n_{MT}$ do
18:	${\tilde{p}}_m^{NC}\left(t\right)\leftarrow \left({\wp }_m^R/{\displaystyle {\sum }_{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\wp }_m^R}\right)\times {\tilde{p}}_m^{NC}\left(t\right)$ when ${\displaystyle {\sum }_{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\tilde{p}}_m^{NC}\left(t\right)}>n_{MT}{P}_{av}^{MT}$.
19:	${\tilde{p}}_m^C\left(t\right)\leftarrow \left({\wp }_m^R/{\displaystyle {\sum }_{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\wp }_m^R}\right)\times {\tilde{p}}_m^C\left(t\right)$ when ${\displaystyle {\sum }_{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\tilde{p}}_m^C\left(t\right)}>n_{MT}{P}_{av}^{MT}$.
20:	end
21:	until ${\displaystyle {\sum }_{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\tilde{p}}_m^{NC}\left(t\right)}>n_{MT}{P}_{av}^{MT}$ and ${\displaystyle {\sum }_{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\tilde{p}}_m^C\left(t\right)}>n_{MT}{P}_{av}^{MT}$
22:	end if

5. Congestion Mitigation for Bottlenecked Secondary User

We have shown that the bottlenecked SU is a little more inclined to be a congested SU node during the slot ${\tau }_{rp}$ due to its buffer overflow under the scenario of the hop-by-hop fluid flows of the data sequence of the SSR injected from $\left|{\mathcal{N}}_{IT}\right|+\left|{\mathcal{N}}_{ET}\right|$ IT and MT SSNs to this bottlenecked SU. We use the buffer size $B_b$ to denote the buffer saturation value of bottlenecked SU b. Recall that the amount of data traffic ${\vartheta }_b\left(t_0^{\prime }\right)$ in buffer of SU b at time $t_0^{\prime }$ evolves according to Equation (2) for the given slot ${\tau }_{rp}$. As depicted in Figure 3a, the buffer of bottlenecked SU b is composed of two buffer segments, i.e., the forward buffer that holds data traffic injected from upstream SUs denoted by ${\vartheta }_b^F\left({\tau }_{rp}\right)$, and the internal buffer that stores the data sequence of SSR transmitted from IT and MT SSNs denoted by ${\vartheta }_b^I\left({\tau }_{rp}\right)$. Normally, there exist idle segments within the forward buffer or the internal buffer of bottlenecked SU b when ${\vartheta }_b\left(t_0^{\prime }+{\tau }_{rp}\right)<B_b$. This phenomenon is referred to the normal status where the offered data load does not exceed available buffer capacity of bottleneck SU b because of its higher data rate to deliver the amount of data traffic to other SUs and remove the amount of the data sequence of SSR during the slot ${\tau }_{rp}$.

Figure 3.

Illustration of buffer based congestion detection at bottlenecked secondary user (SU) b. (a) Normal status without buffer overflow; (b) Congestion occurrence due to buffer overflow.

However, congestion in bottlenecked SU b will occur when the offered data load exceeds its available buffer capacity due to buffer overflow imposed by the data load involving the data sequence of the SSR injected from IT and MT SSNs together with the data traffic from upstream SUs. We assume that the congestion detection information of bottlenecked SU b can be transferred back to the m-th SSN by means of a hop-by-hop backpressure signal via the CCC form bottlenecked SU b to the m-th SSN. As a result, bottlenecked SU b is said to be a congested SU if and only if the amount of its current data traffic ${\vartheta }_b\left(t_0^{\prime }+{\tau }_{rp}\right)\ge B_b$ during the slot ${\tau }_{rp}$, as shown in Figure 3b. We further assume that the m-th SSN has the data sequence of the SSR with ${\mathsf{\Xi }}_m$ bits to transmit at time $t\in \left[t_0^{\prime },t_0^{\prime }+{\tau }_{rp}\right]$. According to [24], the data sequence of SSR with ${\mathsf{\Xi }}_m$ bits to transmit from the m-th SSN to bottlenecked SU b during the slot ${\tau }_{rp}$ under the transmit power $p_m\left(t\right)$ can be approximately expressed as follows based on channel capacity $C_{m\to b}$ given by Equation (4):

$${\mathsf{\Xi }}_m={\tau }_{rp}W{\log }_2\left(1+\frac{p_m\left(t\right)K{\left|h_{m\to b}\right|}^2}{N_{0}W}\right)$$ (38)

Clearly, ${\mathsf{\Xi }}_m$ depends on channel capacity $C_{m\to b}$ and is also a function of the transmit power $p_m\left(t\right)$. For ease of exposition, we use ${\mathsf{\Xi }}_m\left(p_m\left(t\right)\right)$ to denote the amount of bits of the data sequence of SSR transmitted from the m-th SSN to bottlenecked SU b under transmit power $p_m\left(t\right)$. Therefore, the aggregated amount of bits of the data sequence of SSR from all $\left|{\mathcal{N}}_{IT}\right|+\left|{\mathcal{N}}_{ET}\right|$ IT and MT SSNs to bottleneck SU b during the slot ${\tau }_{rp}$ under transmit power $p_m\left(t\right)$ is given as ${\displaystyle {\sum }_{m\in \mathcal{N}}{\mathsf{\Xi }}_m\left(p_m\left(t\right)\right)}$, for $\mathcal{N}={\mathcal{N}}_{IT}\cup {\mathcal{N}}_{MT}$. Recall that the internal buffer of bottlenecked SU b is used to store the amount of the data sequence of SSR transmitted from SSNs denoted by ${\vartheta }_b^I\left({\tau }_{rp}\right)$ during the slot ${\tau }_{rp}$ by using Equation (2). Hence, within the time interval ${\tau }_{rp}$, ${\vartheta }_b^I\left({\tau }_{rp}\right)$ of bottlenecked SU b can be further formulated by:

$${\vartheta }_b^I\left({\tau }_{rp}\right)={\displaystyle {\sum }_{m\in \mathcal{N}}{\mathsf{\Xi }}_m\left(p_m\left(t\right)\right)}-{\chi }_b\left({\tau }_{rp}\right)$$ (39)

where ${\chi }_b\left({\tau }_{rp}\right)$ is the amount of the data sequence of SSR removed by bottlenecked SU b within a time interval ${\tau }_{rp}$.

So far, we have mathematically derived the amount of the data sequence of SSR transmitted from SSNs to bottlenecked SU b under the condition that the m-th SSN employs the transmit power $p_m\left(t\right)$, for $m\in \mathcal{N}$. Next, we are concerned with how to mitigate the congestion of bottlenecked SU b by the aid of the proposed distributed power control framework for IT and MT SSNs. The basic idea of congestion mitigation for bottlenecked SU b is to alleviate its buffer load because of the accumulated amount of the data sequence of the SSR transmitted from IT and MT SSNs. It is uncovered that the transmit power of the m-th SSN may have a bearing on ${\mathsf{\Xi }}_m$ from Equation (38). Our objective is to leverage the distributed power control to reduce the amount of bits of the data sequence of the SSR transmitted from the m-th SSN to bottlenecked SU b, in order to lower the amount of the data sequence of the SSR in the internal buffer of bottlenecked SU b. This operation will further ensure that offered data load does not exceed available buffer capacity of bottlenecked SU b, i.e., ${\vartheta }_b\left(t_0^{\prime }+{\tau }_{rp}\right)<B_b$. The key point to achieve congestion mitigation for bottlenecked SU b is established with a block diagram shown in Figure 4. It is worth remarking that the amount of the data sequence of SSR in the internal buffer of bottlenecked SU b can be effectively reduced by the proposed distributed power control framework for IT and MT SSNs under the noncooperation and cooperation scenarios. Conceptually, the reduction of the amount of the data sequence of SSR will release the capacity of the internal buffer for bottlenecked SU b, which naturally further attains the congestion mitigation for bottlenecked SU b. In the following, we analyze the impact of optimal transmit power for the m-th SSN on the reduction of the amount of the data sequence of SSR in the internal buffer of bottleneck SU b rigorously from the noncooperation and cooperation cases.

Figure 4.

The block diagram of the key point to achieve congestion mitigation for bottleneck secondary user.

(1) Noncooperative Optimal Transmit Power Case: In this case, suppose, without loss of generality, that ${\tau }_{rp}W\triangleq 1$. By replacing $p_m\left(t\right)$ in Equation (38) with $p_m^{NC}\left(t\right)$ in Equation (28), ${\mathsf{\Xi }}_m\left(p_m^{NC}\left(t\right)\right)$ for the m-th SSN within a time interval ${\tau }_{rp}$ can be expressed as:

$$\begin{array}{ll}{\mathsf{\Xi }}_m\left(p_m^{NC}\left(t\right)\right)& ={\tau }_{rp}W{\log }_2\left(1+\frac{p_m^{NC}\left(t\right)K{\left|h_{m\to b}\right|}^2}{N_{0}W}\right)\\ & \approx {\log }_2\left(\frac{p_m^{NC}\left(t\right)K{\left|h_{m\to b}\right|}^2}{N_{0}W}\right)\\ & \approx {\log }_2\left(P_m^{\max }\left(1-\frac{H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2\left(r+1\right)e^{{\delta }_i\cdot D_m\left({\mathcal{P}}_d\Vert {\mathcal{P}}_f\right)}}\right)\frac{K{\left|h_{m\to b}\right|}^2}{N_{0}W}\right)\end{array}$$ (40)

By plugging ${\mathsf{\Xi }}_m\left(p_m^{NC}\left(t\right)\right)$ into Equation (39), we have:

$${\vartheta }_b^I\left({\tau }_{rp}\right)=\underset{\mathrm{for}{n}_{IT}\mathrm{IT}\mathrm{SSNs}}{\underbrace{{\displaystyle \sum _{m=1}^{n_{IT}}{\mathsf{\Xi }}_m\left(p_m^{NC}\left(t\right)\right)}}}+\underset{\mathrm{for}{n}_{MT}\mathrm{MT}\mathrm{SSNs}}{\underbrace{{\displaystyle \sum _{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\mathsf{\Xi }}_m\left(p_m^{NC}\left(t\right)\right)}}}-{\chi }_b\left({\tau }_{rp}\right)$$ (41)

Thus, the reduction of the amount of the data sequence of SSR in the internal buffer of bottlenecked SU b, denoted by $\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)$, is equivalent to:

$$\begin{array}{ll}\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)& ={\displaystyle \sum _{m=1}^{n_{IT}}\left({\mathsf{\Xi }}_m\left(p_m\left(t\right)\right)-{\mathsf{\Xi }}_m\left(p_m^{NC}\left(t\right)\right)\right)}+{\displaystyle \sum _{m=n_{IT}+1}^{n_{IT}+n_{MT}}\left({\mathsf{\Xi }}_m\left(p_m\left(t\right)\right)-{\mathsf{\Xi }}_m\left(p_m^{NC}\left(t\right)\right)\right)}\\ & \approx {\displaystyle \sum _{m\in \mathcal{N}}{\log }_2\left(\frac{p_m\left(t\right)}{p_m^{NC}\left(t\right)}\right)}\end{array}$$ (42)

Proposition 5.

Under the condition of the transmit power $p_m\left(t\right)=P_m^{\max }$ for the m-th SSN, $\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)$ satisfies the following upper bound:

$$\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)\le {\displaystyle \sum _{m\in \mathcal{N}}{\log }_2\left(1/\left(1-\frac{H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2\left(r+1\right)e^{{\delta }_i\cdot D_m\left({\mathcal{P}}_d\Vert {\mathcal{P}}_f\right)}}\right)\right)}$$ (43)

Proof. 

Since $p_m\left(t\right)=P_m^{\max }$. Under this condition, using Equation (28), we have:

$$\begin{array}{ll}\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)& \approx {\displaystyle \sum _{m\in \mathcal{N}}{\log }_2\left(\frac{p_m\left(t\right)}{p_m^{NC}\left(t\right)}\right)}\\ & \le {\displaystyle \sum _{m\in \mathcal{N}}{\log }_2\left(\frac{P_m^{\max }}{p_m^{NC}\left(t\right)}\right)}\\ & ={\displaystyle \sum _{m\in \mathcal{N}}{\log }_2\left(1/\left(1-\frac{H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2\left(r+1\right)e^{{\delta }_i\cdot D_m\left({\mathcal{P}}_d\Vert {\mathcal{P}}_f\right)}}\right)\right)}\end{array}$$ (44)

Then we obtain the upper bound in Equation (43), thus completing the proof. ☐

(2) Cooperative Optimal Transmit Power Case: In this case, with the assumption ${\tau }_{rp}W\triangleq 1$, by replacing $p_m\left(t\right)$ in Equation (38) with $p_m^C\left(t\right)$ in Equation (34), ${\mathsf{\Xi }}_m\left(p_m^C\left(t\right)\right)$ for the m-th SSN within a time interval ${\tau }_{rp}$ can be given by:

$$\begin{array}{ll}{\mathsf{\Xi }}_m\left(p_m^C\left(t\right)\right)& ={\tau }_{rp}W{\log }_2\left(1+\frac{p_m^C\left(t\right)K{\left|h_{m\to b}\right|}^2}{N_{0}W}\right)\\ & \approx {\log }_2\left(\frac{p_m^C\left(t\right)K{\left|h_{m\to b}\right|}^2}{N_{0}W}\right)\\ & \approx {\log }_2\left(P_m^{\max }\left(1-\frac{H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2\left(r+1\right){\displaystyle \sum _{m\in \mathcal{N}}{e}^{{\delta }_i\cdot D_m({\mathcal{P}}_d\Vert {\mathcal{P}}_f)}}}\right)\frac{K{\left|h_{m\to b}\right|}^2}{N_{0}W}\right)\end{array}$$ (45)

By substituting ${\mathsf{\Xi }}_m\left(p_m^C\left(t\right)\right)$ into Equation (39), we obtain:

$${\vartheta }_b^I\left({\tau }_{rp}\right)=\underset{\mathrm{for}{n}_{IT}\mathrm{IT}\mathrm{SSNs}}{\underbrace{{\displaystyle \sum _{m=1}^{n_{IT}}{\mathsf{\Xi }}_m\left(p_m^C\left(t\right)\right)}}}+\underset{\mathrm{for}{n}_{MT}\mathrm{MT}\mathrm{SSNs}}{\underbrace{{\displaystyle \sum _{m=n_{IT}+1}^{n_{IT}+n_{MT}}{\mathsf{\Xi }}_m\left(p_m^C\left(t\right)\right)}}}-{\chi }_b\left({\tau }_{rp}\right)$$ (46)

Therefore, similar to Equation (42), $\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)$ is also approximately formulated as:

$$\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)\approx {\displaystyle \sum _{m\in \mathcal{N}}{\log }_2\left(\frac{p_m\left(t\right)}{p_m^C\left(t\right)}\right)}$$ (47)

Proposition 6.

Under the condition of the transmit power $p_m\left(t\right)=P_m^{\max }$ for the m-th SSN, $\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)$ satisfies the following upper bound:

$$\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)\le {\displaystyle \sum _{m\in \mathcal{N}}{\log }_2\left(1/\left(1-\frac{H_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2\left(r+1\right){\displaystyle \sum _{m\in \mathcal{N}}{e}^{{\delta }_i\cdot D_m({\mathcal{P}}_d\Vert {\mathcal{P}}_f)}}}\right)\right)}$$ (48)

Proof. 

Similar to the proof of Proposition 5, it is easy to identify the upper bound in Equation (48), thus completing the proof. ☐

6. Simulation Results

In this section, we provide simulation results to evaluate the performance of the proposed congestion mitigation approach by using the distributed power control framework for IT and MT SSNs in the rectangular grid based SN-CRN, and investigate the impact of key system parameters on the performance. Particularly, our simulations pay more attention to evaluate the effect of the proposed distributed power control framework for IT and MT SSNs on the reduction of the amount of the data sequence of SSR in the internal buffer of bottlenecked SU b. Technically, the reduction of the amount of the data sequence of SSR in the internal buffer will free the buffer capacity of bottlenecked SU b, which can bring about the congestion mitigation for bottlenecked SU b. As illustrated in Figure 5, all the simulations are carried out on a rectangular grid topology within a torus area of 100 m × 100 m where one bottlenecked SU is randomly placed within the area of IT over three-tier structure. It suffices to mention that the horizontal or vertical distance between any SSNs is initialized to be $d$ = 20 m. For convenience, the IT SSN and the MT SSN are marked by IT-m and MT-n in sequence for $m\in {\mathcal{N}}_{IT}$ and $n\in {\mathcal{N}}_{MT}$, respectively, as shown in Figure 5. We assume that there are $N_c$ = 12 licensed uplink channels allocated to PUs. The SOP usage probability ${\delta }_b$ is set to 0.65 for the bottlenecked SU. Also, the total average power constraint assigned to IT and MT SSNs are assumed to $P_{av}^{IT}$ = 18 dBm and $P_{av}^{MT}$ = 8 dBm, respectively, to mitigate the interference among IT and MT SSNs. As for the AWGN multiple-access channel, the path-loss exponent $\kappa$ has been set to 8, and the channel bandwidth used for SSNs is assumed to be 2 MHz according to IEEE 802.15.4a channel model. The constant $K$ under the channel capacity formulation given in Equation (4) is defined as 0.005. In addition, the noise power spectral density $N_0$ under this channel model is initialized as −3 dBm. We also adopt the receiving reference power by bottleneck SU b $P_0$ = 20 dBm under the reference distance $d_0$ = 20 m.

Figure 5.

Simulation scenario: the rectangular grid topology with three-tier structure within a torus area of 100 m × 100 m. The dotted lines correspond to the transmissions of the spectrum sensing results from interior tier (IT) and middle tier (MT) spectrum sensor nodes to bottlenecked secondary user. SU: secondary user; PU: primary user; ET: exterior tier.

In all the simulations, the detection and false alarm probabilities for IT and MT SSNs over $N_c=12$ licensed uplink channels have been initialized as in Figure 6. We also set the number of subintervals in Algorithm 1 to be the same value for all SSNs, i.e., $\xi =\varsigma =8$. We also assume that the minimum value $X_d$ and the maximum value $Y_d$ in derivation of the detection probability distribution are set to 0.6 and 1, respectively. Likewise, the minimum value $X_f$ and the maximum value $Y_f$ in derivation of the false alarm probability distribution are set at 0 and 0.3, respectively.

Figure 6.

The detection and false alarm probabilities for spectrum sensor nodes from interior tier and middle tier over 12 uplink channels. IT: interior tier; MT: middle tier; SSN: spectrum sensor node.

For simplicity, we deliberately choose some of the IT and MT SSNs in the simulation including IT-1, MT-2, MT-5, and MT-7, to evaluate the performance of our developed approach. Specifically, the detection probability and false alarm probability distributions of the selected SSNs from IT and MT generated from Algorithm 1 are assumed to comply with the corresponding distributions as given by Figure 7.

Figure 7.

The detection probability and false alarm probability distributions of the selected spectrum sensor nodes involving IT-1, MT-2, MT-5, and MT-7. (a) Detection probability distribution; (b) False alarm probability distribution. IT: interior tier; MT: middle tier; SSN: spectrum sensor node.

The proposed OTPA algorithm for both the NoCoPC problem and the CoPC problem under the distributed power control framework for IT and MT SSNs is compared with the well-known power balancing (PB) algorithm in [35]. The PB algorithm is also a SNR balancing constrained power control iterative method which iteratively searches for decentralized transmit power level updated from the l-th iteration to the (l + 1)-th iteration. Let ${\gamma }_m^{tar}$ denote the target SNR for the m-th SSN to maintain a certain QoS requirement, for $m\in \mathcal{N}$. In this simulation, the target SNR can be set to ${\gamma }_m^{tar}=7$ dB. Therefore, the PB algorithm iteratively updates the transmit power for the m-th SSN according to [35]:

$$p_m^{\left(l+1\right)}=\min \left\{P_m^{\max },\left(\frac{{\gamma }_m^{tar}}{{\gamma }_m^{\left(l\right)}}\right)\cdot p_m^{\left(l\right)}\right\}$$ (49)

First the optimal transmit power is compared between PB algorithm with 250 iterations and our proposed distributed power control framework via the noncooperation and cooperation scenarios under the slot ${\tau }_{rp}$ = 12 ms with the evolution of discount factor r, as exhibited in Figure 8. It is shown that an increased discount factor from 0.1 to 0.9 will enhance the optimal transmit power of selected SSNs (i.e., IT-1, MT-2, MT-5, and MT-7) under the proposed OTPA algorithm for both the NoCoPC problem and the CoPC problem within the slot ${\tau }_{rp}$ = 12 ms. This is due to the fact that the discount factor r has an affect on the optimal transmit power of selected SSNs via Equations (28) and (34). Specifically, the optimal transmit power of selected SSNs grows in proportion to discount factor r. It is also revealed that the optimal transmit power of selected SSNs under a given discount factor satisfy the average power constraint of $P_{av}^{IT}$ = 18 dBm and $P_{av}^{MT}$ = 8 dBm. Thus, the optimal transmit power of selected SSNs will not be adjusted through OTPA algorithm. Comparing the performance of the NoCoPC problem and CoPC problem with the slot ${\tau }_{rp}$ = 12 ms, we can also observe that the optimal transmit power of selected SSNs by using PB algorithm presents a fixed constant value. The reason for this is that the optimal transmit power of the selected SSN by using PB algorithm converges to an expected equilibrium point after 250 iterations. It is also interesting that Proposition 2 has turned out that the optimal transmit power of selected SSNs derived by NoCoPC problem converges to a fixed Nash equilibrium point. Moreover, Proposition 4 has also guaranteed that the optimal transmit power of selected SSNs under the CoPC problem will reach to a fixed value. From Figure 8, it is implicitly revealed that the optimal transmit power of selected SSNs by the OTPA algorithm is obviously lower than that of PB algorithm. This observation is reasonable since PB algorithm generates more power consumption to maintain the target SNR. However, the optimal transmit power of selected SSNs via the proposed OTPA algorithm fully depends on the maximum transmit power of the selected SSNs and the pricing factors in differential game model.

Figure 8.

Optimal transmit power vs. discount factor r (${\tau }_{rp}=12$ ms, l = 250 iterations). NoCoPC: noncooperative power control; CoPC: cooperative power control; PB: power balancing.

Figure 9 displays the optimal transmit power comparison between PB algorithm with 250 iterations and our proposed distributed power control framework under discount factor $r$ = 0.5 with the evolution of the slot ${\tau }_{rp}$ from 10 ms to 15 ms. As can be seen, the proposed OTPA algorithm for both the NoCoPC problem and CoPC problem under the condition of discount factor $r$ = 0.5 outperforms PB algorithm with 250 iterations significantly in terms of the optimal transmit power of selected SSNs with the growth of the slot ${\tau }_{rp}$. This result further validates PB algorithm will result in much more power consumption aiming to maintain the target SNR. However, the optimal transmit power of selected SSNs by using the proposed OTPA algorithm entirely depends upon maximum transmit power of selected SSNs and pricing factors in differential game model. We can also observe that the optimal transmit power of selected SSNs of the proposed OTPA algorithm for the CoPC problem under discount factor $r$ = 0.5 is considerably lower than that of the NoCoPC problem. That is, the proposed OTPA algorithm for the CoPC problem outperforms that of the NoCoPC problem. This can be intuitively explained by the fact that there exists an operation of summing with respect to $e^{{\delta }_i\cdot D_m({\mathcal{P}}_d\Vert {\mathcal{P}}_f)}$ in the denominator of the analytical cooperative optimal transmit power $p_m^C\left(t\right)$. Moreover, the increase of the slot ${\tau }_{rp}$ will generate lower transmit power for both the NoCoPC problem and CoPC problem with discount factor $r$ = 0.5. This is because based on Equations (28) and (34), the optimal transmit power for both the NoCoPC problem and CoPC problem is inversely proportional to the slot ${\tau }_{rp}$. This observation emphasizes the importance of selecting the proper time interval of slot ${\tau }_{rp}$ on the optimal transmit power.

Figure 9.

Optimal transmit power vs. the slot ${\tau }_{rp}$ (r = 0.5, l = 250 iterations). NoCoPC: noncooperative power control; CoPC: cooperative power control; PB: power balancing.

In Figure 10, we look at the performance of the transmitted data sequence of SSR from selected SSNs to bottlenecked SU b during the slot ${\tau }_{rp}$ = 12 ms with the evolution of discount factor r from 0.1 to 0.9. From the results, we can see the transmitted data sequence of SSR for selected SSNs gradually increase with the growth of discount factor r. Meanwhile, the transmitted data sequence of SSR for PB algorithm with 250 iterations and our proposed distributed power control framework under the condition of the slot ${\tau }_{rp}$ = 12 ms tends to be close when discount factor r = 0.9. The reason for this is that the transmitted data sequence of SSR can be approximately expressed as a function of channel capacity according to Equation (38), which is in direct proportion to the optimal transmit power of selected SSNs. It has also been shown that an increased discount factor from 0.1 to 0.9 will increase the optimal transmit power of selected SSNs under the proposed OTPA algorithm for both the NoCoPC problem and CoPC problem with the slot ${\tau }_{rp}$ = 12 ms. Consequently, higher optimal transmit power yields more transmitted data sequence of SSR.

Figure 10.

Transmitted data sequence of SSR vs. discount factor r (${\tau }_{rp}=12$ ms, l = 250 iterations). NoCoPC: noncooperative power control; CoPC: cooperative power control; PB: power balancing.

In Figure 11, we examine the impact of discount factor r from 0.1 to 0.9 on the reduction of data sequence of SSR with respect to selected SSNs in the internal buffer of bottleneck SU b. It can be observed from the figure that the reduction of data sequence of SSR by our proposed congestion mitigation approach under the slot ${\tau }_{rp}$ = 12 ms will gradually decrease with the growth of discount factor r, except that the reduction result by PB algorithm with 250 iterations appears to a fixed constant value. This trend is the result of the inverse relationship between the reduction of the data sequence of SSR $\Delta {\vartheta }_b^I\left({\tau }_{rp}\right)$ and the optimal transmit power of selected SSNs $p_m^{NC}\left(t\right)$ for the NoCoPC problem or $p_m^C\left(t\right)$ for the CoPC problem according to Equations (42) and (47). Additionally, in Figure 11, we can also observe that the proposed congestion mitigation approach by using the OTPA algorithm for the CoPC problem with the slot ${\tau }_{rp}$ = 12 ms achieves the higher reduction of the data sequence of SSR compared with those of the NoCoPC problem and PB algorithm with 250 iterations. In other words, through cooperation among all IT and MT SSNs, the reduction of the data sequence of SSR in the internal buffer of bottlenecked SU b can be further improved. This observation has verified the analytical derivation of the proposed congestion mitigation approach. The explanation is twofold: (i) The proposed OTPA algorithm for the CoPC problem outperforms that of the NoCoPC problem owing to the lower cooperative optimal transmit power $p_m^C\left(t\right)$ obtained by the CoPC problem compared with the noncooperative optimal transmit power $p_m^{NC}\left(t\right)$ by the NoCoPC problem; (ii) The reduction of the data sequence of SSR is inversely proportional to the optimal transmit power of selected SSNs.

Figure 11.

Reduction of data sequence of spectrum sensing results for of selected spectrum sensor nodes vs. discount factor r (${\tau }_{rp}=12$ ms, l = 250 iterations). NoCoPC: noncooperative power control; CoPC: cooperative power control; PB: power balancing.

Figure 12 shows the comparison of the reduction of data sequence of SSR with respect to selected SSNs in the internal buffer of bottlenecked SU b, versus the slot ${\tau }_{rp}$ under the condition of discount factor r = 0.3, for the proposed congestion mitigation approach and PB algorithm with 250 iterations. As seen from Figure 11, the reduction of data sequence of SSR through PB algorithm has converged to the same constant value after the 250 iterations. This is the direct influence that the optimal transmit power of selected SSN will converge to an expected equilibrium point by using PB algorithm. As expected, the reduction of data sequence of SSR by the proposed congestion mitigation approach with discount factor r = 0.3 is larger than that of PB algorithm with 250 iterations. This is because our proposed approach can obtain smaller optimal transmit power than PB algorithm. According to Equations (42) and (47), the smaller optimal transmit power will lead to the more reduction of data sequence of SSR with respect to selected SSNs. It is also interesting that with the growth of the slot ${\tau }_{rp}$ the reduction of data sequence of SSR by our proposed approach will gradually increase, and the reduction by the CoPC problem is much higher than that of the NoCoPC problem. This result can be interpreted by the fact that the optimal transmit power of selected SSNs by the CoPC problem under discount factor $r$ = 0.3 is clearly lower than that of the NoCoPC problem with the growth of the slot ${\tau }_{rp}$. This result further gives rise to the larger reduction of data sequence of SSR.

Figure 12.

Reduction of data sequence of spectrum sensing resultsfor of selected spectrum sensor nodes vs. the slot ${\tau }_{rp}$ (r = 0.3, l = 250 iterations). NoCoPC: noncooperative power control; CoPC: cooperative power control; PB: power balancing.

7. Conclusions and Future Work

In this paper, we have developed a congestion mitigation approach by employing the distributed power control framework for SSNs in the rectangular grid based SN-CRN. Particularly, we defined the relative divergence between the detection probability and false alarm probability for a SSN under any uplink channel by adopting a Kullback-Leibler divergence framework. After deriving the detection probability and false alarm probability distributions for SSN according to mathematical statistics, we characterized the stability metric of local spectrum sensing based on entropy modeling framework. Aiming to gain the tradeoff between channel capacity and energy consumption, the distributed power control framework for IT and MT SSNs was proposed, and the power control problem was formulated as differential game model by taking into account the utility function maximization with linear differential equation constraint in regard to energy consumption. Further, we derived the theoretic optimal solutions to this game model under the scenario of cooperation or noncooperation via dynamic programming. Based on the obtained optimal transmit power of SSNs, we devised the congestion mitigation approach for bottleneck SU by alleviating buffer load over its internal buffer, and validated its performance with simulations.

What we have discussed in this paper is the portion of the foundation for SN-CRN. Possible directions for future work within this research involve examining the effect of imperfect CSI and outage constraint on distributed power control for SSNs by formulating the uncertain relation between the wireless channel conditions and the corresponding estimates. As another future work, we will try to investigate congestion mitigation approaches in future 5G mobile systems with the novel network architecture and networking technologies [36], e.g., fog computing-based radio access networks and network slicing-based mobile networks in currently practical applications.

Appendix A

Proof of Proposition 1.

For mathematical tractability, the problem $\mathcal{P}1$ can be converted into an finite horizon differential game model by relaxing the terminal time $t_0^{\prime }+{\tau }_{rp}$ to explore when $t_0^{\prime }+{\tau }_{rp}$ approaches $\infty$ (i.e., $t_0^{\prime }+{\tau }_{rp}\to \infty$) as infinite time horizon. Under this conversion, we then perform the maximization operation of the right hand side of Equation (25) with respect to $p_m\left(t\right)$. After some simplifications, we have the following expression of the noncooperative optimal solution:

$$p_m^{NC}\left(t\right)=P_m^{\max }+\frac{\partial {\mathcal{V}}_m^{NC}\left(p_m,E_m\right)}{\partial E_m}\frac{P_m^{\max }{H}_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2}$$ (A1)

By substituting $p_m^{NC}\left(t\right)$ into Equation (25), after some algebraic manipulations, we have:

$$\begin{array}{l}r{\mathcal{V}}_m^{NC}\left(p_m,E_m\right)-\frac{P_m^{\max }{H}_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}^2}{4}{\left(\frac{\partial {\mathcal{V}}_m^{NC}\left(p_m,E_m\right)}{\partial E_m}\right)}^2+{\wp }_m^C{E}_m\left(t\right)\\ =\frac{\partial {\mathcal{V}}_m^{NC}\left(p_m,E_m\right)}{\partial E_m}\left(E_A\left(t\right)-{\tau }_{rp}\left(P_m^{\max }+\frac{\partial {\mathcal{V}}_m^{NC}\left(p_m,E_m\right)}{\partial E_m}\frac{P_m^{\max }{H}_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2}\right)-E_m\left(t\right)\right)\end{array}$$ (A2)

To proceed, we derive the derivative of ${\mathcal{V}}_m^{NC}\left(p_m,E_m\right)$ with respect to $E_m\left(t\right)$ in Equation (A2). Upon solving the partial differential equation, after some simplifications, we now can express the function ${\mathcal{V}}_m^{NC}\left(p_m,E_m\right)$ as the partial differential equation constraint given by:

$$\frac{\partial {\mathcal{V}}_m^{NC}\left(p_m,E_m\right)}{\partial E_m}=-\frac{{\wp }_m^C}{r+1}$$ (A3)

This completes the proof. ☐

Appendix B

Proof of Proposition 3.

The proof is similar to Proposition 1. The only difference is that the objective function of the cooperative power control problem is to maximize the sum of the utility functions of all players. For convenience of derivation, we also relax the time interval of the game and discuss the infinite-horizon differential game (i.e., ${\tau }_{rp}\to \infty$), and we also set $t_0^{\prime }=0$. By performing the maximization operation of the right hand side of Equation (31) with respect to the transmit power of all players $p_1\left(t\right),p_2\left(t\right),\cdots ,p_{\left|\mathcal{N}\right|}\left(t\right)$, after some simplifications, we have:

$$p_m^C\left(t\right)=P_m^{\max }+\frac{\partial {\mathcal{W}}_m^C\left(p_m,E_m\right)}{\partial E_m}\frac{P_m^{\max }{H}_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2}$$ (A4)

By substituting $p_m^C\left(t\right)$ into Equation (31), after some algebraic manipulations, we obtain:

$$\begin{array}{l}{\mathcal{W}}_m^C\left(p_m,E_m\right)-{\displaystyle \sum _{m\in \mathcal{N}}\left(\frac{P_m^{\max }{H}_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}^2}{4r}{\left(\frac{\partial {\mathcal{W}}_m^C\left(p_m,E_m\right)}{\partial E_m}\right)}^2-\frac{{\wp }_m^C{E}_m\left(t\right)}{r}\right)}\\ =\frac{\partial {\mathcal{W}}_m^C\left(p_m,E_m\right)}{\partial E_m}\left(\frac{E_A\left(t\right)}{r}-\frac{{\tau }_{rp}}{r}\left(P_m^{\max }+\frac{\partial {\mathcal{W}}_m^C\left(p_m,E_m\right)}{\partial E_m}\frac{P_m^{\max }{H}_m\left({\Upsilon }_d\left(m\right)\Vert {\Upsilon }_f\left(m\right)\right){\tau }_{rp}}{2}\right)-\frac{E_m\left(t\right)}{r}\right)\end{array}$$ (A5)

Next, we try to derive the derivative of ${\mathcal{W}}_m^C\left(p_m,E_m\right)$ with respect to $E_m\left(t\right)$ in Equation (A5). Upon solving the partial differential equation, after some basic mathematical manipulations, we obtain the final result in Equation (32), thus completing the proof. ☐

Author Contributions

Long Zhang conceived the idea of this paper and determined the structure of research work; Haitao Xu provided valuable insights for the scheme and revised the manuscript; Long Zhang and Fan Zhuo performed the simulations and wrote the paper; Hucai Duan contributed to data analysis.

Conflicts of Interest

The authors declare no conflict of interest.