Performance Evaluation of Proximity Services and Wi-Fi for Public Safety Mission Critical Voice Application

Abstract

Proximity Services (ProSe) and Wi-Fi are two promising technologies that may provide support for Mission Critical Voice (MCV) applications in remote and rural areas by enabling Device-to-Device (D2D) communication. In this paper, several performance metrics of ProSe and Wi-Fi are evaluated and compared side-by-side under various configurations. The ns-3 simulation results show that ProSe outperforms Wi-Fi in terms of coverage range and access time with a medium traffic load, while Wi-Fi has a shorter access time under a light traffic load. In addition, with various user densities, ProSe offers better coverage range and access time a majority of the time. The evaluation in this paper provides insights to first responders on what to expect with either technology and how to improve the performance by adjusting different system parameters.

1. Introduction

Communication is vital for first responders in mission critical situations as it may be the difference between life and death. While there may exist a vast network infrastructure intended to provide first responders with broadband wireless access, it may not always be available or the most viable option at any given point in time. Regardless, the Mission Critical Voice (MCV) service should be resilient, available, and provide its required function when necessary, even in rural or remote areas. In such a situation, first responders may use Device-to-Device (D2D) communication to stay connected. With D2D, first responders are able to maintain connectivity over a direct-link between members of a group, allowing them to communicate with peers in their local vicinity without the intervention of a centralized entity. How reliable that link is will depend heavily on the underlying technology at hand.

Proximity Services (ProSe) is a feature of Long Term Evolution (LTE) that was first introduced by the 3^rd Generation Partnership Project (3GPP) in 2014, and is used to provide D2D communication. This technology provides synchronization, discovery, and communication services for local User Equipment (UE). Different modes of operation have been defined allowing public safety devices to communicate when both in-coverage and out-of-coverage of an LTE network. When out of coverage, UEs rely on pre-configured information to determine which resources to use [22]. This mode of operation is referred to as mode 2. While ProSe is deemed to be promising to support first responders in rural and remote areas, this LTE feature has yet to be widely adopted or implemented like its counterpart, Wi-Fi. Wi-Fi is an Institute of Electrical and Electronics Engineers (IEEE) standard that can be used in ad hoc mode to also provide D2D communication. While there are many different flavors of Wi-Fi, in this paper the term Wi-Fi is used to refer to the commercially available IEEE 802.11g standard that was ratified in 2003 [24]. Wi-Fi is implemented in many devices today, and is able to provide high throughput and short range wireless communication. As is described in [1] and other literature, LTE and Wi-Fi are both promising candidates for first responders, capable of providing communication during incidents in rural and remote areas where network coverage may be limited. Even though quite some effort has been put into investigating the performance of ProSe and Wi-Fi, only a few works compare them with another technology, while even less consider them for use by public safety.

Works [3]-[7] focus their efforts on characterizing the performance of ProSe alone. In [3], a thorough investigation is conducted that studies Block Error Rate (BLER) performance for the various ProSe channels at the physical layer. This work is later extended in [4] to determine the link capacity and coverage range of ProSe. While [4] considers many different deployment scenarios and the impact of various system parameters, it does not take into account the different applications that will use ProSe, e.g. mission critical applications. A mathematical model is presented in [6] that characterizes the likelihood of a UE receiving a message when multiple UEs are sharing the same channel. While [6] is thorough, it assumes any collisions that occur result in the loss of a message and does not consider the Signal-to-Interference-Plus-Noise Ratio (SINR) of the received message that could potentially be recovered, making it a very conservative approach. In [7], an investigation of Mission Critical Push-to-Talk (MCPTT) Access Time is performed to analyze how soon a first responder should expect to gain access to the channel after a Push-to-Talk (PTT) request is made. Again, while [7] is extensive and provides an analytical model to give a general idea of MCPTT performance over ProSe, the analysis only takes into account collisions that occur from application control messages which are negligible when compared to the amount of data messages that can be produced by the MCPTT application. A software-based implementation of ProSe, developed in ns-3, is presented in [5]. This is the same model that is used in [6] and [7] that will also be used in this paper.

Among the numerous research papers that provide a performance evaluation of Wi-Fi, only a small portion of papers dealt with public safety use cases and/or voice application related performance metrics. While investigating the performance of mesh networks to be used for public safety, [8] also highlights the shortcomings of Wi-Fi. Although the paper suggests the need and the prominent issues, such as capacity and interference, for deploying wireless networks in small towns and rural areas, the study itself is very limited since it only considers the throughput of unknown sources. In [9], an analytical model is developed that characterizes the Voice Over IP (VoIP) call capacity of single-hop and multi-hop Wi-Fi networks using various codecs. An overview of Bluetooth and Wi-Fi is provided in [13] , which describes how the protocols operate, their main features, and how they compare in terms of spectrum, power requirements, security, etc. In [12], a log-distance model is created to predict bit rate versus distance for Wi-Fi using physical devices in a remote parking lot. While deriving this model, it is also determined that Wi-Fi can deliver close to 5 Mbits/s at 100 m. The Wi-Fi error model in ns-3 is validated in [14], which describes the SINR of the Wi-Fi model in ns-3 being within 1 dB of the Wi-Fi devices used with a wireless channel emulator. In [10], an extensive analysis is performed using simulations to investigate the performance of 802.11b ad hoc networks, and describes the presence of “hidden stations”, which we will refer to in Section 5.2.

While the above and others analyze the performance metrics (e.g., range, link capacity, access time) of Wi-Fi and ProSe separately, we do not know of any published papers that compare the two under similar assumptions and in a comparable environment. Thus, the question that still remains is: How does ProSe compare to Wi-Fi with respect to public safety needs when it comes to providing a reliable link for voice communication without the network infrastructure? To address this issue, we study the impact that each technology has on various important performance metrics. First, in Section 2, the simulation tools and channel models used to perform the investigation are introduced. Then, in Section 3, we study the impact that the system load has on the 3GPP defined MCPTT Access Time of an MCV application, which indicates how quickly a user may access the channel. In Section 4, MCV Coverage Probability is defined and is used to measure the range that is achievable by both technologies using different configurations with regards to the MCV application. In Section 5, the user density analysis will be conducted for a more complete and comprehensive study of the technologies. Lastly, we will provide insights and conclusions to the analyses that were performed.

2. Model Overview

In this paper we consider various scenarios in which at least two UEs of the same group are communicating after being deployed in a disk-shaped area. The analysis starts with one pair of UEs in the system as shown in Figure 1. In Section 3 and Section 5 more users are considered to study the impact of the traffic load and/or user density. These additional UEs are deployed simply to introduce more traffic, so even as these additional UEs are introduced, the statistics generated for each sample are still taken only from a designated pair of UEs.

Figure 1:

Components in the model.

The MCPTT application model we use in the simulation assumes voice packets are generated by a high quality Adaptive Multi-Rate Wideband (AMR-WB) encoder, G.722.2, as described in [17]. The vocoder generates 60 byte voice packets every 20 ms when the MCV user is actively talking. This means that for voice communication the demand is 24 kbits/s at the application level, but at the link layer, the source data rate is even higher after taking into account the various headers that have to be added to the 60 byte voice packets depending on the wireless communication technology. Note that for each analysis, it is assumed that enough resources are made available by both technologies so that the bandwidth of the link is large enough to accommodate all possible traffic with no fragmentation. We also assume for both ProSe and Wi-Fi that all devices use the same configuration. The headers that are considered for both technologies are shown in Table 1 and Figure 1. Consequently, a wireless D2D link should provide at least 44 kbits/s or 42.4 kbits/s data rate when using Wi-Fi or ProSe, respectively, in order to support the MCPTT voice application. In addition to voice, the MCPTT Floor Control [20] and MCPTT Call Control protocols [19] are implemented to support MCPTT operations. The control messages generated by these protocols also require resources even though the amount of resources that are needed is not as significant as the voice traffic.

Table 1:

The sizes of the protocol headers.

Header	Size (bytes)	Technology
Real Time Transport Protocol (RTP)	14	Both
User Datagram Protocol (UDP)	8	Both
Internet Protocol (IP)	20	Both
Packet Data Convergence Protocol (PDCP)	2	ProSe
Radio Link Control (RLC)	2	ProSe
Logical Link Control (LLC)	8	Wi-Fi

The propagation models used in our simulations for both ProSe and Wifi are the simple Friis model as specified in [2] and the more comprehensive models from Wang and Rouil in [3] and [4]. In Section 3, the Friis model is used when analyzing the impact of the traffic load on access time. This enables us to focus solely on the transmission errors caused by collisions and the half duplex effect which are identified in [7] as this is the main consequence of increased traffic load. When evaluating the impact of the range in Section 4 and user density in Section 5, Wang and Rouil’s model is used, which takes path loss, shadowing, and interference into account. It is worth mentioning that we use Wang and Rouil’s model instead of the default Log-Distance model [14] for Wi-Fi, simply to make it possible for an apples-to-apples comparison of ProSe and Wi-Fi. It is also worth mentioning that the appropriate frequency values are configured as the input parameter for both propagation models: 793 MHz (Band 14 uplink frequency) for ProSe and 2.407 GHz (802.11 b/g/n) for Wi-Fi.

To support off-network mode MCPTT functionalities, D2D links need to be established between UEs. Therefore, ProSe UEs are operated in mode 2 [21], while Wi-Fi UEs are operated in ad hoc mode [23]. With these modes of operation, both technologies provide the necessary functions for UEs to communicate directly with one another so neither relies on an underlying infrastructure of base stations or access points. In each scenario simulated, all UEs in the system use the same technology to communicate. That is, the system is composed of all ProSe UEs or all Wi-Fi UEs, not both simultaneously.

Even though the results presented in Sections 3 and 5 focus on one set of ProSe parameters and one set of Wi-Fi parameters, we have studied other configuration values already and observed similar trends. A variety of system configurations are discussed in Section 4 to study the effect that the system parameters have on coverage range.

3. Impact of traffic load on access time

In this section, MCPTT access time is selected as the first performance metric to compare ProSe with Wi-Fi. This is because access time is a very important Key Performance Indicator (KPI) for first responders [7]. In the corresponding 3GPP standards [18], the MCPTT Access Time is defined as follows:

Definition 1 (MCPTT Access Time). The amount of time from when the user requests to talk until the time the user gets an indication that he/she may talk.

In the remainder of this section, the major assumptions for the corresponding analyses are listed in Section 3.1, the MCPTT access time obtained over ProSe and Wi-Fi is compared for a single pair of UEs in Section 3.2, and the MCPTT access time obtained over ProSe and Wi-Fi with additional users is compared in Section 3.3.

3.1. Assumptions

The following scenarios assume that first responders have already joined an active call, which means that once a first responder has been granted the floor (i.e. has obtained permission to speak), he/she may start speaking and voice packets (RTP) can be sent. In addition, to focus on the impact of the traffic load on access time, we assume a floor participant UE, UE_A, sends a preemptive floor request and thus the floor arbitrator UE, UE_B, has to grant the floor to UE_A without further delay. A more detailed explanation of the procedure can be found in Section 7 of [20] or a more general description can be found in procedure F2.1 of [7]. Note that voice traffic and system load are not considered in [7] but their significance will become clear in Sections 3.3 and 5.2. In addition, any collisions that occur result in the loss of both packets, which is the same assumption as in [6] and [7].

The MCV application on a UE will be generating voice packets at 24 kbits/s, and control messages from the call control and floor control protocols. In the simulation, we assume a ProSe sidelink period to be 40 ms. Thus, the value of T₂₀₁ (the Floor Request re-transmission timer) is set to 40 ms and the value of T₂₀₅ (the Floor Granted re-transmission timer) is 80 ms based on the 3GPP recommended default values [20]. Note that no T₂₀₁ or T₂₀₅ values are given for Wi-Fi since the MCPTT protocol is specified to work over ProSe and does not consider Wi-Fi. To address this, we simply use the same values for both ProSe and Wi-Fi. In addition, the 6 Mbits/s data rate mode is used for Wi-Fi, while the Modulation and Coding Scheme (MCS) = 18, number of Physical Resource Blocks (PRBs) N_PRB = 4, and k_trp = 2, which denotes the number of transmissions opportunities in a Time Resource Pattern (TRP) for ProSe. More details are given in Section 4 that will explain why the above system parameters are selected for ProSe and Wi-Fi to support the applications’ demand.

3.2. Single-pair Analysis

The analysis starts with the basic scenario of a single-pair case as shown in Figure 2. The simulation results shown in Figure 3 compare the access time statistics of ProSe vs Wi-Fi based on 100 000 samples. The MCPTT Access Time (in milliseconds) is on the y-axis, with the recorded statistics (minimum, average, and maximum) on the x-axis. Even though the confidence interval at 95 % is included in every figure with the average access time, those error bars are not visible in Figure 3 and Figure 6a since the confidence interval for both technologies is less than 1 ms. As can be seen in Figure 3, the access times for Wi-Fi are much smaller than they are for ProSe, with Wi-Fi providing a minimum, maximum, and average access time less than 5 ms. This difference in access time is attributed to the differences in each technology’s medium access control schemes. For ProSe the sidelink period behaves like a time window that comes around to all UEs periodically, so there is a restriction on the time frame in which a UE may transmit even if the channel is idle. This results in a scheduling delay that puts a lower bound on how soon after a packet is ready to be sent that it is actually transmitted. At the same time, this window also puts an upper bound on how long a packet will be queued for transmission, assuming the traffic demand does not exceed available radio resources. This is because as long as the packet is ready to be transmitted before the scheduling cutoff time of the next sidelink period, it will be transmitted in the next sidelink period. This behavior is described with great detail in Section 2 of [7]. On the contrary, Wi-Fi UEs can transmit almost instantly, as a UE may access the channel as soon as a packet is ready to be transmitted as long as no other UE is transmitting. This is made possible by Wi-Fi’s Distributed Control Function (DCF), which utilizes Carrier-Sense Multiple Access with Collision Avoidance (CSMA/CA) [24] to mitigate collisions.

Figure 2:

Preemptive Floor Request scenario with voice traffic.

Figure 3:

Baseline MCPTT Access Time with error bars that represent the 95 % confidence interval.

Figure 6:

Performance metrics for additional users in a single call.

3.3. Multi-pair Analysis

In this section we will compare how additional UEs may impact the access time performance of MCPTT over ProSe vs that of MCPTT over Wi-Fi. Two sets of scenarios are considered: first, we will evaluate the impact of more UEs on a single call (Figure 4), then we will evaluate the impact of additional UEs on separate calls (Figure 5). For each call we assume that only one UE is actively talking, meaning that in each call there is a single UE with a single application that has a demand of 24 kbits/s.

Figure 4:

Multiple users on a single call.

Figure 5:

Multiple users on separate calls.

As can be seen in Figure 6a, when adding more UEs to a single call, the impact on access time seems to be negligible for both technologies. This is because the amount of additional traffic introduced in this scenario is minimal due to the fact that only a single UE, i.e. the floor arbitrator, is ever transmitting voice packets at a time. The only additional traffic that is observed in this scenario is the periodic transmission of a “Group Call Announcement” message that should only be sent by a single UE about every 10 seconds. We also see a similar trend in Figure 6b, the ratio of undesirable outcomes. Coined in [7], this ratio is tightly coupled with the access time to illuminate the number of instances in which the user receives an indication that he/she may begin speaking and the UE is able to transmit voice packets, but the floor granted message was never received by UE_A. This can happen if the floor arbitrator never receives the floor request from UE_A, or if UE_A never receives the floor granted message from the floor arbitrator. In either case, it is possible that members of the group will hear only part or none of the requesting user’s transmission.

From Figure 7a we can see that the additional UEs on separate calls impact the access time performance for both technologies more noticeably. As more separate calls are introduced the access time increases from a few milliseconds to several hundred milliseconds, with maximums (not shown in graphs) exceeding 1 second. This is due primarily to the fact that in each additional call one UE is generating voice packets at 24 kbits/s. Observe that once 19 separate calls are active in the system, there is a cross-over point where Wi-Fi no longer outperforms ProSe and begins to perform worse. The trend of the relative performance of each technology is a consequence of the medium access control methods mentioned earlier. Although the DCF made it possible for Wi-Fi UEs to instantly access the channel when there was less traffic, this method does not scale well when more UEs in the system are actively transmitting. This is because, as more traffic is introduced, it is more likely that a signal will occupy the Wi-Fi channel. Thus, the back-off algorithm will delay the transmission of the control messages sent by UE_A and UE_B that must be exchanged in order for access to be requested and then granted. On the contrary, ProSe randomizes the selection of radio resources for transmission to minimize potential collisions, instead of sensing the channel. As explained in Section 3.2 already, this scheduling method puts an upper bound on how long a packet will be queued for transmission, assuming the traffic demand does not exceed available radio resources. Therefore, the access time of MCPTT over ProSe scales better when more UEs are introduced into the system: as more sidelink resources are occupied by more UEs, the access time increases gradually, instead of rapidly as for Wi-Fi. Once the channel is so saturated, i.e., there are around 25 group calls with an active talker, we see that the access time begins to decrease for Wi-Fi and the rate at which the access time is increasing for ProSe begin to fall. This does not mean that the performance is stabilizing or improving, but rather that the occurrence of undesirable outcomes is increasing. Figure 7b displays this increase. Therefore we can see that Wi-Fi outperforms ProSe when the number of active calls (i.e. amount of traffic) is small, but as more calls exist, eventually ProSe out-performs Wi-Fi. When the number of separate calls is so large, although the performance of both technologies is similar, neither is satisfactory due to the high ratio of undesirable outcomes.

Figure 7:

Performance metrics for additional calls.

In order to reduce the ratio of undesirable outcomes when there are a large number of active calls, we studied the pros and cons of adjusting T₂₀₁. For the 2 UE scenario of MCPTT over ProSe, [7] already showed that the larger the T₂₀₁ value, the less likely a floor participant will assume the floor arbitrator role prematurely, and thus the ratio of undesirable outcome occurrence may be smaller. For the comparison of Wi-Fi vs ProSe and for the multiple UE scenario, the impact of the T₂₀₁ configuration is shown in Figures 8a and 8b. The number of separate calls is still on the x-axis and either the average access time or undesirable outcome ratio is on the y-axis. There are 3 curves for each technology and each of them corresponds to a different T₂₀₁ value. Consistent with the findings in [7], we observe in Figure 8b that by increasing the configuration value of timer T₂₀₁, we can mitigate the occurrence of undesirable outcomes but not without incurring an increase in access time for both technologies, which can be seen in Figure 8a.

Figure 8:

Performance metrics for additional calls using various T₂₀₁ values.

4. Impact of range on coverage probability

In this section we define and analyze a coverage probability that applies to the MCV application. Moreover, this analysis compares the range of ProSe and Wi-Fi based on a set of constraints that are imposed on the loss, delay, and data rate of our MCV application model.

4.1. Assumptions

In this analysis we simulate a scenario with two UEs that are positioned x meters apart from each other with a wireless link provided by either ProSe or Wi-Fi. An MCV application transmits voice packets from one UE to the other for a total of 20 seconds. The loss, average delay, and throughput are derived from the resulting traces, and are later used to determine the coverage probability. 2000 simulation runs are conducted for each distance and system configuration.

According to [15] the requirements for MCPTT speech are a packet delay of 75 ms or less, and a packet loss ratio of 1 % or less. In [17] the data rate for the G.722.2 AMR-WB encoder and decoder is 23.85 kbits/s so we assume that the data rate must be at least this much. Thus, for any simulation at distance x, if all three metrics (delay, throughput, and loss ratio) meet the minimum requirements, the UE is determined to be covered, otherwise the UE is not covered. Therefore, the MCV Coverage Probability is defined as:

Definition 2 (MCV Coverage Probability). The ratio of samples for which the packet loss is no more than 1 %, the average packet delay is no more than 75 ms, and the resulting data rate is no less than 23.85 kbits/s at the conclusion of the transmission period for a pair of UEs.

The system parameter values used for the simulations in this section are selected such that the demand of the MCV application (24 kbits/s) can be met. For Wi-Fi, the following data rates are simulated because they can support the MCV application demand: 6, 24, and 54 Mbits/s. For ProSe, Table 2 summarizes the various sets of system parameter values that are simulated in this section.

Table 2:

List of ProSe configurations that are considered.

		Low			Medium			High
kTRP	PTB	MCS	NPRB	NTB	MCS	NPRB	NTB	MCS	NPRB	NTB
2	848	0	32	872	10	5	872	20	3	1384
8	212	0	9	224	7	2	224	14	1	256

To be more specific, for a typical ProSe configuration with a 40 ms sidelink period and N_TRP = 8, the valid k_TRP values are 1, 2, 4, and 8, where N_TRP is the number of ms that are available for transmission and k_TRP is how many of those N_TRP ms will be used for a transmission. The remaining 40 — N_TRP ms are a repetition of the first N_TRP ms. Thus, for a given source traffic rate that requires b bits to be transported every sidelink period, which in our case, b = 1696 (two 106 byte voice packets), we can select an MCS and number of PRBs, N_PRB, so that the size of a single Transport Block (TB), N_TB, can transport b bits. Put another way, the MCS and N_PRB are selected so that N_TB ≥ $\frac{b}{k_{TRP}}$ The range of feasible N_TB values are key for determining MCS and N_PRB combinations. Table 7.1.7.2.1-1 in [16] maps an MCS and N_PRB selection to an N_TB, which ultimately determines the maximum data rate that can be supported by ProSe. But since there exists multiple combinations of MCS and N_PRB that can support N_TB, we vary the combinations by selecting a “low”, “medium”, and “high” MCS since, as will be shown later, this combination affects the MCV coverage probability.

4.2. Analysis

In Figure 9 we have the distance, x, on the x-axis and the measured MCV coverage probability on the y-axis. Each line in Figure 9 corresponds to a technology with a specific configuration. Thus, “Wi-Fi Rate=6 Mbits/s” corresponds to the MCV coverage probability obtained with a Wi-Fi link using the 6 Mbits/s data rate, and “ProSe KTRP=2 MCS=10 PRBS=5” corresponds to the MCV coverage probability obtained with a ProSe link using k_TRP = 2, MCS = 10, and N_PRB = 5. This tells us that with a 6 Mbits/s Wi-Fi link, a 90 % MCV coverage probability can be be maintained up until about 55 m, while a ProSe link with k_TRP = 8, MCS = 7, and N_PRB = 2 can achieve a 90 % coverage probability up until about 530 m. It is clear from Figure 9 that ProSe provides much greater coverage than Wi-Fi, and this difference could play a significant role in the success of first responder operations since natural disasters in rural areas can span tens of km² [1].

Figure 9:

The baseline MCV Coverage Probability.

As we mentioned before, different ProSe configurations also lead to different MCV coverage probabilities. For example, the higher the MCS, the lower the MCV coverage probability. This is because the data transmission with a higher MCS requires a higher SINR ratio at the receiver to decode the message successfully. For a similar reason of SINR, the larger N_PRB is, the lower the MCV coverage probability. This is due to the fact that N_PRB determines how many symbols are available and thus directly affects the distribution of the transmit power. For instance, if N_PRB = 1 is used, then all the transmit power, P_TX, will be concentrated into a single PRB, whereas if N_PRB > 1, then the transmit power is distributed equally among N_PRB as $\frac{P_{TX}}{N_{PRBS}}$. This means that as you increase N_PRB, you are effectively decreasing the received power of the TB which, consequently, leads to a lower SINR. In addition, a larger k_TRP may improve the MCV coverage probability. To be more specific, if we need to transmit a total of b bits and the k_TRP = 1, then all b bits need to fit in a single TB. But if k_TRP = 2, then each TB only needs to transport $\frac{b}{2}$ bits and thus a smaller N_TB is required. This is significant because a smaller N_TB can be achieved with a lower MCS and/or fewer PRBs, which translates to a lower SINR threshold to decode the message and/or an increase in the received power of the TB. The downside of using a larger k_TRP is that it may increase the likeliness of a collision, but in this analysis only one user is ever transmitting at a time.

For Wi-Fi, it can be seen that the selected data rate plays a large role in the MCV coverage probability as the 6 Mbits/s data rate can achieve 90 % beyond 50 m while the distance for 54 Mbits/s is just past 20 m. This is because the selected data rate determines the key parameter MCS for transmission over the air. Similar to ProSe, a higher data rate is achieved with a higher MCS, but this ultimately sacrifices coverage.

5. Impact of user density on access time and coverage probability

5.1. Assumptions

In this section we combine the various aspects of Sections 3 and 4 for a complete study that extends beyond an idealistic channel or a single pair of UEs. The need to combine these aspects comes from the fact that in Section 3 we assume that all devices can communicate regardless of distance, while in Section 4 only a single voice application transmits data packets. Therefore, the analysis in this section considers the impact of user density which gives insight on how many active calls can be supported in a given area before the performance of each technology degrades.

The user density topology is depicted in Figure 10. Simulation scenarios and system configurations in this section are very similar to those in Section 3.3 as shown in Figure 5. However, in this analysis, packets are dropped based on the BLER models used in Section 4 since these models take noise into account that is introduced by interfering UEs. In this topology, R is the radius of the area in which UEs are deployed, and d is the maximum distance that can be between a pair of UEs on a call in the same group. Each dot in the circle represents a UE, and the color of the UE represents the call that the UE is a part of. R is selected based on the criteria that devices at distance R have a coverage probability less than 1 %, which means that devices at this distance will likely not interfere with each other. d will be varied using a subset of values between the distances at which a 95 % coverage probability is no longer achievable for either technology. Given the results from Section 4, we will use R = 800 m with d ϵ {40,120,200,280} m for both technologies.

Figure 10:

The user density topology.

5.2. Analysis

The results of the user density analysis can be seen in Figures 11a, 11b, and 11c. On the x-axis we have d, and the y-axis is for coverage probability in Figure 11a, the average access time in Figure 11b, and the ratio of undesirable outcomes in Figure 11c, respectively. Different curves in the figure reflect a different number of UE pairs with an active talker in the system. For example, the series “ProSe (5 calls)” corresponds to a ProSe system with 5 additional groups and a maximum distance of d between any pair of UEs on the same call.

Figure 11:

Performance metrics for additional calls using various user densities.

Figure 11a shows that both technologies can maintain a coverage probability greater than 90 % when d = 40 m, even when there are 45 additional calls. As expected, the Wi-Fi performance is very similar to ProSe at first but eventually falls slightly below that of ProSe as more traffic is introduced. In Section 4, when there was no additional traffic, the coverage probability for a pair of UEs using Wi-Fi or ProSe was very close to 100 %. Therefore, when d = 40 m, we can see from the slight decrease in the curves that the predominant factor affecting both ProSe and Wi-Fi is interference. However, as we increase d we can see that ProSe performs better at first and then falls to a performance similar to that of Wi-Fi. We also notice that Wi-Fi is almost always worse than ProSe for a given d and does not appear to change much as more traffic is introduced. This is because Wi-Fi is impacted more by the distance between devices while ProSe is impacted more by the total number of devices when 40m < d ≤ 280m. Observe that as d increases from 40 m to 280 m the coverage probability for Wi-Fi, with just 5 additional calls, decreases from 99 % to 29 %, but as the number of additional calls increases from 5 to 45 and d = 280 m the coverage probability only decreases from 29 % to 23 %. On the other hand, for ProSe, as d increases from 40 m to 280 m with 5 additional calls the coverage probability only decreases from 99 % to 82 %, but as the number of calls increases from 5 to 45 when d = 280 m the coverage probability drops from 82 % to 22 %. We can also see that the coverage probability decreases much faster for ProSe than for Wi-Fi as d increases with the number of additional calls. This is because more calls lead to more UEs, and a larger d means a more sparse deployment of UEs. Thus, the amount of interference increases because it is more likely for a UE to be closer in proximity to the member of another call.

As we can see from the curves in Figure 11b, the average access time for Wi-Fi increases much sooner than for ProSe. For example, when d = 40 m, the average access time increases from 131.20 ms with 5 additional calls to 624.57 ms when there are 45 additional calls, while for ProSe the average access time only increases from 55.61 ms to 55.64 ms when the additional calls change from 5 calls to 45 calls.

Figure 11c shows that ProSe outperforms Wi-Fi in terms of the ratio of undesirable outcomes, too. For Wi-Fi, the ratio of undesirable outcomes increases from 0.04 % to 1.1 % at d = 40 m when increasing the number of additional calls from 5 to 45, and when d = 280 m and there are 45 additional calls the ratio reaches 71 %. For ProSe, however, the ratio of undesirable outcomes is less than 1 % when d = 40 m and there are 45 additional calls, and only ever produces a 5.8 % ratio when d = 280 m with 45 additional calls.

One notable finding from the above analysis is that we observe very large access time values when using Wi-Fi as shown in Figure 12, which captures all of the recorded MCPTT Access times using a Cumulative Distribution Function (CDF) in the worst cases. This situation can arise when there are two devices that cannot detect each other’s transmissions, as depicted in Figure 13. These “hidden nodes” in combination with the operation of the MCPTT protocol make it possible for a floor request to go unanswered indefinitely. Basically, the interaction between the following protocol behaviors made the error scenario possible, assuming UE_B is the current floor arbitrator: (1) UE_B is within transmission range of both UE_A and UE_C, but UE_A and UE_C cannot hear each other due to the hidden node problem. The transmission of UE_A and UE_C always overlaps so that UE_B cannot receive UE_A’s Floor Request message successfully; – this is possible because UE_A transmits a Floor Request every 40 ms and UE_C might transmit an RTP packet every 20 ms even if at the application layer it is not intended for UE_B; (2) UE_B continues sending RTP packets every 20 ms to UE_A since it has not received the floor request from UE_A yet; (3) UE_A continues transmitting Floor Request messages and resetting the re-transmission timer, T₂₀₁ every 40 ms. UE_A’s re-transmission counter C₂₀₁ is also reset every time an RTP packet is received from UE_B, and thus the upper limit of the re-transmission counter C₂₀₁ is not reached.

Figure 12:

CDF of MCPTT Access Times in the worst cases

Figure 13:

Preemptive Floor Request scenario with RTP and hidden node.

Therefore, with the combination of (1), (2), and (3), UE_A will continue requesting the floor indefinitely and drive the access time to a very large value. This extreme case is more likely to occur over Wi-Fi than over ProSe because (1) is more likely to occur with Wi-Fi because of the different medium access control protocols. When both UE_A and UE_C try to access the channel at almost the same time, both of them might initiate transmission immediately if sensing that the channel is idle, which leads to a collision and the continued attempts of almost synchronized transmissions, while the hidden node situation keeps the back-offs from taking effect. At this point only some outside factor such as the user releasing the PTT button (which occurs in our simulation after 20 s), the call ending, or a back-off being triggered due to another UE’s signal could break the livelock. For ProSe though, the radio resources used for UE_A and UE_C will be randomized even if they have the packets ready at almost the same time. Therefore, the chances that (1) is true for a ProSe UE is much smaller than for Wi-Fi.

The above observation suggests that if the MCPTT protocol is to be used over Wi-Fi, then it needs to be modified to prevent such a livelock. The solution could be as simple as adding a limit to how long a request can be ongoing, or making it possible for the values for the re-transmission timers to be variable/randomized. Therefore, the average access time shown for Wi-Fi in Figure 11b is an underestimate of its true access time, if the Floor Request message can be re-transmitted indefinitely. On the other hand, for ProSe, we can see that in the worst case, when d = 280 m and there are 45 additional calls, the average access time is 67.32 ms, the ratio of undesirable outcomes is 6.3 %, and the maximum access time experienced is 1.12 s, which may still be reasonable.

6. Conclusion

We compare several performance metrics for MCPTT over ProSe vs MCPTT over Wi-Fi, since ProSe and Wi-Fi are the potential broadband wireless access technologies that first responders may utilize in rural and remote areas when the traditional in-network coverage is not available or desired. Results in Section 3 showed that Wi-Fi may provide faster access times than ProSe when a user wishes to speak when the traffic is relatively light (i.e. less active calls). However, once the traffic was heavier, ProSe outperformed Wi-Fi until the channel was too saturated, at which point the two technologies offered very similar performance. Section 4 showed that with ProSe, better coverage can be achieved at greater distances than Wi-Fi, which was reflected through a higher MCV coverage probability. Under our simulation assumptions, a 90 % MCV coverage probability can be achieved anywhere from 305 m to 530 m with ProSe, while with Wi-Fi the range can be anywhere from 20 m to 55 m. The variation in range within each technology is attributed to system’s configuration. The analysis in Section 5 sought to combine the aspects of Section 3 and Section 4 to provide a more comprehensive study. We could see that for a given area, both ProSe and Wi-Fi suffer as the number of transmitting UEs increases and becomes more dense, while ProSe is more susceptible to interference and Wi-Fi lacks coverage range. We also observed very large access times due to the operation of the MCPTT protocol and that modifications may be needed to address this, especially when deploying MCPTT over Wi-Fi.

7. Data Availability

The data used to support the findings of this study are available from the corresponding author upon request.