Ergodic secrecy rate and outage probability in NOMA IRS massive MIMO networks with jamming

Abstract

This research paper considers a wireless system that assumes massive multiple-input multiple-output (MIMO) technical solutions for the Base Station (BS), while both near and far users have just one antenna. This study primarily aims to examine the security implications of the physical layer in this system using Non-Orthogonal Multiple Access (NOMA) along with an Intelligent Reconfigurable Surface (IRS). It is assumed that the BS broadcasts signals to pairs of NOMA users, and both categories of users also receive signals from the IRS. Additionally, the far user is regarded as untrusted, while the jammer is present. Moreover, to assess the confidentiality performance of the studied system, the Ergodic Secrecy Rate (ESR) and Secrecy Outage Probability (SOP) are considered, and their closed-form expressions are provided. Our findings indicate that the integration of IRS leads to an exponential enhancement in the ESR, consequently reducing the SOP. Additionally, our observations highlight the superior efficiency of networks featuring a jamming mechanism. Furthermore, the employment of a massive multi-antenna setup amplifies the transfer Signal-to-Noise Ratio (SNR), leading to heightened ESR and a simultaneous reduction in the SOP.

Introduction

In contemporary society, the exponential proliferation of Internet-connected devices, projected to reach approximately 50 billion, is fundamentally transforming various industries through the Industrial Internet of Things (IoT). Within this paradigm, Industrial IoT leverages advanced technologies such as sensors, Cloud/Fog/Edge computing, and artificial intelligence to optimize manufacturing efficiency, particularly in smart manufacturing, logistics, and operational technology, all within the broader IoT ecosystem¹. Moreover, the escalating demand for wireless communication in fifth-generation (5G) networks underscores the pressing need to enhance network security. In response to these challenges, Intelligent Reflecting Surface (IRS) technology has emerged as a promising solution, significantly improving both the capacity and security of wireless communication systems^2,3.

The current state of wireless networks exhibits a lack of controllability, thereby raising significant security concerns. According to the fundamental principles of wireless communication, the signal transmission from a base station (BS) intended for a specific user is inherently broadcast to all users within the network. Consequently, this indiscriminate transmission poses a risk of unauthorized interception, potentially exposing sensitive information to eavesdropping. This security vulnerability becomes even more critical with the increasing demand for wireless communication, particularly in the context of 5G networks, where the proliferation of connected devices exacerbates potential threats⁴. In this regard, one of the key methodologies that facilitates connectivity for massive IoT devices in wireless networks is Non-Orthogonal Multiple Access (NOMA)^4–6.

NOMA technology has the potential to achieve an optimal balance between spectrum efficiency and user fairness, surpassing the performance of the traditional orthogonal multiple access (OMA) scheme². Furthermore, the implementation of NOMA involves transmitting signals from the source to multiple users with varying transmission power levels, thereby enhancing overall system capacity. However, the selection of a non-orthogonal power range in this study introduces interference among users, consequently leading to multiple access interference challenges. Unlike conventional OMA approaches, which allocate orthogonal resources to users, NOMA operates by assigning a single resource block (RB) in either the time or frequency domain to multiple users. It employs power-domain superposition coding to facilitate efficient resource sharing^7,8. Specifically, in a power-domain NOMA system, signals are transmitted simultaneously to users experiencing strong (near users) and weak (far users) channel conditions⁹. At the receiver end, the system employs successive interference cancellation (SIC) to mitigate inter-user interference arising from the shared resource channel. To enhance fairness in internal secrecy within NOMA, it is imperative to align the reordering of SIC with secrecy requirements. Unlike conventional NOMA, which predominantly relies on channel gains, this approach is essential for improving security. Moreover, active beamforming using multiple antennas enables the dynamic adjustment of the decoding sequence. Ensuring data confidentiality necessitates the application of maximum ratio combination (MRC) precoding, which further strengthens the security of transmitted signals^10,11.

Intelligent Reflecting Surfaces (IRS) represent a groundbreaking technology that enhances signal strength, reduces antenna requirements, and optimizes system performance, particularly within Orthogonal Multiple Access (OMA) configurations. The integration of passive reflecting elements with active signal manipulation substantially improves data transmission rates, power efficiency, and user fairness in Non-Orthogonal Multiple Access (NOMA) systems. In addition, IRS deployment offers a range of additional benefits, including enhanced signal management, targeted signal propagation, cost-effectiveness, improved connectivity, and increased data security.

Structurally, an IRS unit consists of an array of passive reflecting elements, each capable of adjusting phase shifts to steer incident signals towards intended users. Consequently, IRS has the ability to dynamically reshape electromagnetic environments. This functionality closely resembles that of a remote base station (BS), which can be deployed alongside existing infrastructure to enhance the quality of service (QoS) in communication networks^8,12,13. Furthermore, IRS is distinguished by an arrangement of “nearly-passive” elements that enable the advantageous manipulation of reflected signals through the programmable adjustment of their reflection coefficients. When strategically placed - such as on building facades - to establish a line-of-sight (LoS) path between a BS and its users, IRS can facilitate robust communication, effectively bridging connectivity gaps where direct links may be unavailable^14–16.

Eavesdropping poses a significant threat to wireless networks, making secure communication a critical concern. In response, physical layer security has emerged as a novel strategy to enhance the confidentiality of wireless transmissions. Unlike conventional physical layer security approaches, such as relay-aided or jammer-aided secure communication, ISR-aideded secure transmission eliminates the need for an additional transmitter to generate artificial signals or interference. Consequently, this approach eliminates the need for additional power consumption^17,18. In addition, ensuring secure communication is particularly crucial when transmitting sensitive data, such as banking credentials, pricing information, and mobile messages. Traditionally, network security has relied on cryptographic techniques, which often face challenges related to encryption key management and distribution among communicating devices. These limitations have motivated the adoption of physical layer security, which is grounded in information theory and introduces the concept of secrecy capacity, a metric derived from the inherent randomness of wireless channels.

Recent research has increasingly focused on optimizing resource allocation strategies to improve secrecy capacity. Furthermore, efforts have been made to integrate physical layer security into multi-antenna systems, exploring advanced resource management and signal processing techniques to strengthen secure transmissions. In this context, minimizing the transmit power is of paramount importance. Maintaining low power consumption at the base station is essential not only to meet secrecy requirements, but also to ensure efficient transmission while mitigating security vulnerabilities^1,19–21.

Massive multiple input and multiple output (MIMO) technology, using large antenna arrays, offers several advantages in wireless networks. Essentially, massive MIMO, utilizing numerous antennas, can significantly enhance security as a physical layer security application. It improves resilience against signal fading and interference by leveraging spatial diversity. This enables higher data rates and accommodates more devices. Despite its promise, traditional implementations face challenges such as signal blockage and high hardware costs due to the use of numerous antennas requiring dedicated RF chains. Addressing these limitations is crucial for more efficient and cost-effective deployment of massive MIMO systems in the future of wireless communication^14,22,23.

Related works

The study in²⁴ investigates the role of Intelligent Reflecting Surfaces (IRSs) in enhancing wireless coverage. The analysis quantitatively evaluates the benefits of IRS-assisted communication systems, particularly in terms of coverage expansion, Signal-to-Noise Ratio (SNR) improvement over direct links, and potential latency reduction. The findings underscore the substantial advantages of IRS integration, including significant improvements in coverage, SNR, and latency reduction.

Similarly, the work in²⁵ explores the optimization of wireless channels using IRSs, wherein passive elements are leveraged to enhance hybrid network performance in a cost-effective manner. However, several challenges remain, particularly in optimizing reflection coefficients, accurate channel estimation, and practical deployment strategies.

The study in²⁶ provides a comprehensive overview of IRS applications in wireless communications, encompassing theoretical models, hardware implementations, inherent system constraints, and prospective research directions. Moreover, security enhancements in IoT wireless networks are examined by employing advanced physical layer security techniques. Notably, the use of fading channels in conjunction with IRSs is emphasized to improve security, and various applications are reviewed to optimize user secrecy rates while outlining future research avenues.

The work in²⁷ specifically addresses secrecy enhancement in IRS-assisted MIMO wiretap channels. Secrecy-optimized IRS and transmitter configurations are proposed for both known and unknown eavesdropper scenarios. The results indicate that, in cases where eavesdropper information is available, secrecy rates improve significantly. Conversely, when the eavesdropper’s presence is uncertain, a trade-off emerges between maintaining service quality for legitimate users and achieving optimal secrecy rates.

Furthermore, the study in²⁸ investigates the use of intelligent surfaces to secure indoor wireless communication. By dynamically adjusting IRS reflection elements, the approach aims to improve legitimate user signal reception while simultaneously mitigating the risk of interception by eavesdroppers. The findings demonstrate the effectiveness of IRSs in reinforcing secrecy between communicating parties.

The work in²⁹ extends this line of research by exploring IRS-based secrecy rate maximization in 5G/6G networks. An optimization framework integrating deep learning techniques is proposed to significantly reduce computational complexity, thereby enhancing both security and system efficiency.

Similarly, the study in³⁰ investigates the role of IRSs in improving Non-Orthogonal Multiple Access (NOMA) systems. The findings highlight that IRS-assisted NOMA effectively customizes wireless channels to meet user requirements while simultaneously enhancing secrecy. By optimizing power allocation, beamforming strategies, and IRS parameters, the proposed approach outperforms conventional NOMA and orthogonal access schemes, regardless of IRS deployment.

The research in³¹ further explores IRS contributions to wireless network security, focusing on mitigating eavesdropping threats. The results confirm that increasing the number of IRSs leads to enhanced signal quality and stronger security mechanisms, influencing secrecy probability and diversity order. These insights are validated through extensive simulations.

Additionally, the study in³² examines IRS applications in NOMA-based wireless communication with an emphasis on transmit power reduction while preserving signal integrity. The approach demonstrates substantial power savings compared to existing methodologies. Similarly, the work in³³ analyzes the impact of IRSs in multi-user networks by comparing their performance across different access schemes, including OMA and NOMA. The study underscores the advantages of dynamically adjustable IRS configurations in enhancing capacity within NOMA networks while proposing strategic placement techniques tailored to distinct access schemes.

The research in³⁴ extends the discussion to IRS-assisted wireless networks, demonstrating how intelligent surfaces can enhance data rates for multiple users, including those with weaker signal conditions. An advanced optimization framework is introduced to effectively manage power allocation, beam direction, and surface reflections, with simulation results verifying the efficiency of the approach. : In³⁵, the authors investigate secure NOMA transmission using transmit antenna selection, improving physical layer security by optimizing antenna selection strategies. The focus in³⁶ is on a NOMA system with an untrusted near user, where power allocation and secrecy beamforming techniques are used to enhance security. Authors in³⁷, analyzes physical layer security in random NOMA networks using stochastic geometry.

A synthesized summary of the most recent works in this domain is provided in Table 1.

Table 1.

: comparison of related works.

Reference	Investigation focus	Methodology	Key differences from our work
24	Coverage and SNR gain in IRS-aided systems	IRS-assisted communication	No focus on NOMA or security
25	IRS-based wireless channel optimization	IRS reflection coefficient optimization	No security analysis or NOMA integration
26	Security enhancements in IoT networks	IRS-based secrecy rate optimization	No NOMA or jamming considerations
27	Secure MIMO transmission using IRS	IRS-assisted secure beamforming	No NOMA or secrecy outage analysis
28	Secure transmission in IRS-SWIPT systems	Deep learning for IRS optimization	No focus on NOMA or massive MIMO
29	Physical layer security in RIS-aided networks	RIS-based secrecy rate enhancement	No integration of NOMA or jamming
30	IRS-assisted NOMA with secrecy constraints	IRS-assisted beamforming & NOMA	No jamming effect considered
31	IRS-assisted NOMA security	IRS deployment for secrecy enhancement	No analysis of ergodic secrecy rate (ESR) under jamming
35	Secure NOMA with transmit antenna selection	Antenna selection for physical layer security	Does not consider IRS or massive MIMO
36	Security in NOMA with untrusted near user	Power allocation and secrecy beamforming	Focuses on an untrusted near user rather than a far user as in our work
37	Physical layer security in random NOMA networks	Stochastic geometry for secrecy analysis	No IRS integration or jamming mechanism
Our work	Secure NOMA with IRS and jamming	IRS-assisted secrecy enhancement, massive MIMO, jamming for untrusted far user	Introduces jamming to protect against an untrusted far user while leveraging IRS for secrecy enhancement, considers both ergodic secrecy rate (ESR) and secrecy outage probability (SOP), and analyzes the impact of IRS placement and power allocation strategies

Motivation and contribution

and :This paper investigates the security performance of IRS-assisted NOMA networks in the presence of jamming and massive MIMO, introducing several novel contributions.

• We analyze the impact of an untrusted far user in an IRS-assisted NOMA system, where the far user acts as an internal eavesdropper. Unlike conventional studies that assume a trusted far user, we introduce a jamming-based strategy to counteract potential security threats while leveraging the IRS to enhance secrecy performance. Second, we derive closed-form expressions for the ESR and SOP, providing a quantitative analysis of how IRS deployment and jamming techniques contribute to improved security. Our findings demonstrate that an increase in the number of IRS elements and BS antennas significantly enhances secrecy, particularly in scenarios where the untrusted far user actively attempts to intercept signals.

• Furthermore, we optimize the placement of the IRS to improve secure communication. While prior studies assume fixed IRS locations, our work evaluates the impact of IRS-user distances on secrecy performance. The results show that placing the IRS closer to the near user improves secrecy, whereas strategic placement can further minimize the risk of eavesdropping. In addition, we introduce the concept of secrecy energy efficiency (SEE) as a novel metric, analyzing the trade-off between power consumption and security performance. Our results reveal that an optimal IRS-assisted power allocation and jamming strategy significantly enhances SEE, ensuring secure communication with efficient energy utilization.

• We propose a jamming-based secure transmission strategy, distinct from existing studies that primarily focus on beamforming or passive IRS enhancements. Unlike existing studies that primarily focus on beamforming or passive IRS enhancements, we integrate an active jamming mechanism specifically designed for IRS-assisted NOMA systems. By directing jamming power selectively towards the untrusted far user, our approach maximizes the secrecy rate of the near user without excessive power consumption.

• The numerical results demonstrate that increasing the number of BS antennas improves the secrecy rate for both near and far users while simultaneously decreasing the SOP. Moreover, integrating a jammer into the network significantly enhances both SOP and ESR. Additionally, the investigation reveals that as the distance from the IRS to the near user increases, the average secrecy rate decreases while the SOP increases. Conversely, an increased distance from the IRS to the far user improves the average secrecy rate but reduces the SOP.

Paper order and notation

This paper is organized as follows: “The model of channel and system” describes the system model for secure communication utilizing IRS-assisted NOMA, including the setup of a base station with multiple antennas and an analysis of channel statistics. “Performance analysis” explores the secrecy outage probability and ergodic secrecy rate under the influence of a jammer. “Numerical results” provides numerical results and an analysis based on the proposed system model. Finally, “Conclusion” concludes the research.

Notation: In the following sections, vectors are represented by lowercase letters, and matrices are represented by bold uppercase letters. The notation denotes an matrix, while signifies the identity matrix. The superscript signifies the conjugate-transpose (Hermitian) operation.

We utilize the zero-mean circularly symmetric complex Gaussian distribution with variance represented as . The Euclidean distance is indicated by . shows the CDF of a random variable (RV) X, while its PDF is represented as . represents the likelihood of an event, and signifies the statistical expectation.

The model of channel and system

A downlink NOMA transmission system consists of a BS equipped with M massive MIMO antennas, serving two types of users: near users () and far users (), each with a single antenna. In this setup, the far user is considered untrusted and acts as an internal eavesdropper. It is assumed that all users, including the eavesdropper, operate in half-duplex (HD) mode.

As shown in Fig. 1, there are direct links between the BS and near- and far-reach users. The system’s efficiency and performance are enhanced by integrating an IRS with N passive reflecting elements. In addition, a jammer is introduced to transmit interference signals to the far user, which improves the average secrecy rate. This system configuration highlights the interaction between the BS, IRS and users, offering an effective approach to improving wireless security and communication efficiency in IRS-assisted NOMA networks.

Fig. 1.

IRS-assisted NOMA massive MIMO system illustration.

To further enhance security and QoS, the IRS is not only used to passively reflect signals, but also actively assists in relaying transmission to both near- and far-remote users. By functioning as a relay, the IRS helps to strengthen connections, improve signal coverage, and ensure secure and reliable communication. In addition, the IRS receives interference signals from the jammer and directs them toward the far user, increasing secrecy by reducing the risk of unauthorized intercept. This strategic IRS deployment offers a promising solution to mitigate eavesdropping threats and optimize wireless network security. The parameters used in this study are summarized in Table 2.

Table 2.

Notations.

Parameter	Definition
	BS and respective Channel coefficient
	BS and respective Channel coefficient
	Channel gain, between IRS and
	Channel gain, between IRS and
	Channel gain from the BS to the IRS
	The NLoS part of
	The LoS element of
	Channel gain from the jammer to the IRS
	Diagonal matrix representing the effective elements
	of the IRS phase shifts
	BS and associated Small-scale fading
	BS and associated Small-scale fading
	BS and RIS associated Small-scale fading
	IRS and associated Small-scale fading
	IRS and associated Small-scale fading
	IRS and the jammer associated Small-scale fading
P	BS transmit power
	Transmit SNR
	Jammer’s transmit SNR
	Rician factor
M	Number of antennas at the MBS’s
N	Quantity of elements in the IRS

Signal model

The proposed system model employs a two-phase approach for data transmission. In the initial phase, the BS broadcasts a signal as the combination of and which are consecutively denoted data signals from near and far users, with the transmission powered by P. Furthermore, while represents the expected value, . Therefore, BS transmits the signal.

1

where denote the power distribution gains for near- and far-range users, respectively. Based on the core principle of NOMA, , with the additional condition that . Moreover, the signal received at IRS in the initial phase is defined as follows.

2

where represents the channel between BS and IRS, where denotes the space of complex-valued matrices . In this context, M corresponds to the number of transmit antennas in the BS, while N represents the number of reflecting elements at the IRS.

Afterward, the IRS transmits the received signal from the BS to both pairs of NOMA users in the next phase, aiming to enhance secrecy in the presence of the received signal from the jammer at the far user.

The combination of the signals received from the first and second phases at the near-user is defined as

3

where is the channel between BS and , represents the channel between BS and IRS, indicates the channel between IRS and . Furthermore, represent the diagonal matrix, where the magnitude of the reflection gain and the phase adjustment of the elements of N the IRS th are denoted by and , respectively. In addition, represents the additive white Gaussian noise (AWGN) in the near user, which is modeled as a complex circularly symmetric Gaussian random variable with zero mean with variance .

In this study, the far user is regarded as an untrusted entity. Based on the principles of NOMA, both users receive signals broadcasted by the base station (BS), allowing simultaneous communication. However, given the potential security risks associated with the far user, it is imperative to implement robust countermeasures to safeguard confidential information.

To this end, a pivotal strategy involves the deployment of jamming techniques, which serve to obscure and distort the signal received by the far user. The primary objective of this approach is to mitigate the risk of unauthorized interception, particularly by potential eavesdroppers, thereby preserving the integrity of transmitted data.

More specifically, the jamming signal is mathematically expressed as , where denotes the channel between the jammer and the IRS, represents the jamming signal and corresponds to the transmission power of the jammer. The jamming signal is initially received by the IRS and subsequently retransmitted in a strategic manner to interfere with the far user’s reception.

Furthermore, the optimal placement of the IRS within the network plays a crucial role in directing the jamming signal efficiently toward the far user. This deliberate positioning significantly improves the effectiveness of the security mechanism, thus reinforcing the overall secrecy of the communication system.

Consequently, the signal received from the far user is given by

4

where is the channel between BS and , denotes the channel between IRS and . In addition, is AWGN at with variance .

The deployment of IRS in massive MIMO systems has been extensively studied to optimize network performance. Recent findings in¹⁴ suggest that placing the IRS in close proximity to users yields significant performance gains compared to conventional approaches in which the IRS is positioned near the BS. This insight challenges traditional design principles and underscores the critical role of strategic IRS placement in maximizing system efficiency. Empirical evidence further validates that placing the IRS closer to end-users enhances signal quality and improves spectral efficiency in massive MIMO networks.

Motivated by these observations, we consider a scenario in which the propagation environment is characterized by rich scattering, with scatterers distributed on the ground, and users positioned at a considerable distance from the BS. Under these conditions, the direct BS-user channel is modeled as a Rayleigh fading channel. Specifically, the channel between the BS and the user, , is expressed as , where represents the large-scale fading coefficient, while denotes the small-scale fading component, following an independent and identically distributed complex Gaussian distribution (i.i.d.) with zero mean and unit variance. In scenarios with a high density of scatterers, the BS-user channel is well modeled by a Rayleigh fading process dominated by NLoS components. Conversely, in the absence of rich scattering, the channel exhibits a purely LoS nature.

Furthermore, the channel between the IRS and the user is given by , where denotes the large-scale fading coefficient. Given that the IRS is located close to the user and considering the relatively low density of scatterers in this region, the IRS-user channel predominantly consists of a LoS component, denoted by . Moreover, the BS-IRS channel encompasses both LoS and NLoS components, given that the IRS is deployed at an elevated location (e.g., mounted on the facade of a tall building near the users). This strategic placement ensures an improved line-of-sight connection with the BS while simultaneously optimizing reflected signal paths toward users. Therefore, the channel model between BS and IRS, , is considered as

5

where represents the large-scale fading factor, and is the Rician factor, which quantifies the power ratio between the Line-of-Sight (LoS) component, , and the Non-Line-of-Sight (NLoS) component, . The Rician factor determines the relative dominance of LoS and NLoS contributions to the channel. .

: Moreover, when , the power of the LoS component completely decreases, which means that the channel consists purely of scattered multipath components without a direct path. In this case, the RIS-BS channel follows a Rayleigh fading model, which is well suited for scenarios dominated by rich scattering and multipath propagation, where the received signal strength varies randomly due to constructive and destructive interference of multiple independent paths.

In contrast, as , the power of the LoS component vastly outweighs scattered multipath contributions, leading to a quasi-deterministic channel that closely resembles free space propagation with minimal fading effects. In such cases, the RIS-BS channel primarily exhibits characteristics of LoS propagation.

Furthermore, is the channel between BS and , where large-scale NLoS is indicated by and small-scale NLoS channel power gain is . denotes the channel between IRS and , while and show the large- and small-scale LoS channel coefficients for the IRS- link, respectively. In addition, and the large-scale coefficient is , and the power gain of the small-scale fading channel between the jammer and the IRS is represented by .

Channel statistics

Based on the NOMA principle along with the underlying considerations, employing a radio resource block results in the signal from the far user, received by the nearby user, showing a higher SNR compared to the signal captured by the far end user. Consequently, leveraging SIC, the signal of the far user can be decoded by the near user and eliminate it, enabling the decoding of the intended signal without interference. In contrast, the signal received by the far user from the nearby user is insignificant due to its lower transmission power compared to the signal directed to the far user. Consequently, the far user decodes the desired signal without the need to employ SIC. Moreover, in a massive MIMO system, zero-force is nearly optimal for wireless data transmission. On the other hand, to achieve interference-free communication using Zero-Forcing Beamforming (ZFBF), it is crucial to have highly accurate Channel State Information (CSI) at BS. This requires employing advanced channel estimation techniques and gathering sufficient training data to ensure a precise CSI at the BS. During the training phase, individual users send specific pilot sequences to the BS without contamination, allowing for accurate CSI estimation. This accurate CSI is crucial for both the BS and users. However, practical challenges such as estimation errors, limited feedback accuracy, and mobile channels make achieving perfect CSI difficult. Imperfect CSI affects the effectiveness of ZFBF designs, leading to interference between beams³⁸. When , where represents the ZF precoder associated with the nearby , the instantaneous SNR for can be expressed as :

6

where represents the normalization of the transmit SNR, and . Consequently, the SNR of is expressed as

7

The far user, considered untrustworthy, consistently attempts signal interception from the nearby user. This involves treating the nearby user’s signal as noise, subtracting it from the received transmission, and then analyzing the resulting signal, defining this process as strategic interception. In the following, the definition of these signals is as follows.

8

where , represents the ZF beamforming vector, and . Additionally, when we have

9

At present, it is crucial to calculate the PDF and CDF of the SNR for ongoing analysis. In Theorem 1, we delve into unraveling the CDF of , exploring its statistical landscape in concise detail.

Theorem 1

The CDF of is written as follows

10

where .

Proof

To compute the CDF of , the following steps should be followed:

11

in this step, by substituting from equation (7) into the definition of the CDF, we obtain it. It should be noted that and . Based on definition of channel between BS and IRS in (5) and ,

. It is evident that and i.i.d. Therefore, comprises i.i.d. Gaussian variables, i.e., . Consequently, is a Gaussian vector where, .

In the second step, we introduce two auxiliary variables defined as and based on the assumption that we have a rich scattering environment . Then, the CDF of Y is given by

12

while the PDF is obtained by differentiating the CDF:

13

Furthermore, the CDF of Z is defined as

14

and while PDF is given by

15

Currently, by plugging two auxiliary variables into (11) we have

16

Finally, substituting Eqs. (12) and (15), we obtain Eq. (10).

.

Corollary 1

When the CDF of is simplified as

17

Proof

If we assume , then can be simplified by introducing a new auxiliary variable . Therefore, by calculating the CDF of W, (17) is obtained.

Theorem 2

The cdf of can be expressed as

18

let and where . Furthermore, .

Proof

By using instead of into equation (11), we have:

19

at first we assume three auxiliary variables, , , and . Therefore, we have

20

The CDF of the Gaussian variable Y is denoted as (12), obtained by substituting with . Similarly, the PDF of Y resembles (13), utilizing in place of .

Furthermore, the CDF and PDF of the variable Z are described in (14) and (15), respectively, achieved by applying instead of , and replacing with .

Here, the pdf of exponential variable J with parameter is defined as

21

Finally, following several mathematical manipulations and substituting Eqs. (12), (15), and (21) into Eq. (20), we derive the Eq. (18).

Unfortunately, it is evident that the integrals in Eq. (18) lack a closed-form expression. Consequently, in the subsequent analysis, we make an assumption to facilitate simplification.

Corollary 2

When , can be simplified as

22

where .

Proof

We assume , then we should calculate this probability . Therefore,

23

While , invoking both and into Eq. (23) and subsequent mathematical manipulation leads to the equation presented in (22).

Performance analysis

This article is focused on assessing the performance of a complex communication network, specifically the massive MIMO NOMA-aided IRS system, with the goal of understanding how effectively it ensures secure communication. To achieve this, the article thoroughly examines two key measures: SOP and ESR. This section of the study is dedicated to deriving precise mathematical expressions for ESR and SOP. These formulas aim to provide a clear quantitative understanding of how effectively the network maintains data secrecy against eavesdropping.

Average secrecy rate

The secrecy rate defines the difference among the communication and maximum rates an eavesdropper can achieve. It’s a crucial measure in secure communication, highlighting the gap between intended information and potential interception, emphasizing the importance of minimizing data accessible to unauthorized entities for confidentiality and integrity. Therefore, according to³¹ the rate of the secrecy is determined as

24

here, represents the secrecy rate of , and denotes the secrecy rate of , where . Additionally, is expressed as follows

25

where and indicate the CDF and PDF of , consecutively.

Proposition 1

The rate of the secrecy for is derived as

26

where .

Proof

For calculating the secrecy rate of following steps should be passed. Initially, by plugging (10) into (25) it is evident that

27

To solve the complex integral presented in Eq. (27), we employ the partial integration method, expressed as . Thus, we proceed by assuming and . Upon solving this complex integral, we derive the asymptotic equation as stated in Eq. (26).

Proposition 2

The secrecy rate of is defined as

28

It should be noted that, unfortunately, closed-form expression can not be written for because of a complex integral. Therefore, in the subsequent numerical analysis, we resolve this integral numerically to obtain the result.

Proof

By invoking in (25) we have

29

by substituting from Eq. (22) and simplifying it, we obtained the equation in (28).

Probability of secrecy outage

The SOP represents the likelihood that the trusted user’s secure communication capacity-distinct from potential eavesdroppers-exceeds a specific threshold. This metric helps assess how reliably the user maintains an advantage in secure communication and provides valuable insights into the effectiveness of data protection mechanisms. The outage probability or an average user is provided as

30

In (30), represents the average secrecy rate in this context, while indicates the target secrecy rate.

Lemma 1

The SOP is defined as

31

Where , and the substitution of is provided in (17). Meanwhile, the PDF of , derived in (22), is also available. Therefore, by plugging (22) into (31),

32

It’s important to highlight that the SOP closed-form expression in this specific scenario remains elusive, potentially necessitating numerical methods for resolution. The section dedicated to numerical results provides an opportunity to observe and analyze the characteristics of the SOP in detail.

Remark 1

: The SOP decreases as the transmit SNR at the BS increases, due to the improved received signal quality at the legitimate user, which enhances the secrecy rate. At low SNR, the SOP remains high as the legitimate user struggles to maintain a secrecy advantage over the untrusted far user. However, with increasing SNR, the SOP declines, indicating improved security. Moreover, a higher number of IRS elements further amplifies the received SNR, accelerating the reduction in SOP and reinforcing the role of IRS in secure NOMA transmissions.

Secrecy energy efficiency

: secrecy energy efficiency (SEE) is defined as the amount of secrecy capacity achieved per unit of total power consumption in a communication system. It quantifies how efficiently power is utilized to enhance secure transmission against eavesdroppers while maintaining energy constraints. Therefore, it can be calculated as

33

where represents the power consumed by the base station (BS). Here, denotes the power consumption of the BS circuit, and is the amplifier efficiency parameter.

Furthermore, the total power consumed by users is given by

34

where and denote the power consumption of the near and far users, respectively.

The power consumed by the jammer is expressed as

35

which accounts for both the transmit power and the jammer circuit power consumption, where represents the jammer’s amplifier efficiency parameter.

Additionally, the power consumption of the intelligent reflecting surface (IRS) is modeled as

36

where represents the power consumed by each individual IRS element, and denotes the power consumption of the IRS controller.

Therefore, secrecy energy efficiency can be calculated by

37

Numerical results

Here, the assessment of the performance of the IRS-assisted NOMA network is conducted via Monte Carlo simulations using MATLAB. Specifically, the analysis focuses on assessing the ESR (Ergodic Secrecy Rate) and SOP (Secrecy Outage Probability) with respect to varying parameters, including the number of IRS elements and the number of antennas at the BS, transmit SNR, and the distance between the IRS and the far user and the near user. The fixed parameter values are explicitly defined in Table 3, while details regarding the variable parameters are provided in conjunction with the respective figures. It’s imperative to highlight that the simulation results stem from an averaging process encompassing realizations in MATLAB.

Table 3.

Value of the parameters.

Parameter	Value
	30 dB
	20 dB
M	64
N	50
	0.25

Figure 2 provides the ESR in a NOMA system increases exponentially as the transmit SNR rises for both near and far users. This improvement occurs because higher transmit SNR leads to enhanced received SNR at paired NOMA users. Transitioning from 64 to 128 BS antennas boosts the ESR for near users due to the amplified transmit SNR. Similarly, for far users, an increase in the number of antennas results in a comparable improvement in ESR. These observations emphasize the distinct effect of SNR and the quantity of antennas on user-specific ESR within NOMA system.

Fig. 2.

Ergodic secrecy rate of near user and far user.

The ESR is illustrated in Fig. 3, which indicates the competition for both nearby and far users’ secrecy rates compared to the transmit SNR for two different values of BS antennas. It also depicts a specific situation when equals (, and ). Similar to Fig. 2, it is evident that increasing causes an enhancement in the average secrecy rate due to the enhanced received SNR at both users. Additionally, when s are equal, the secrecy rate is better than when they are not equal. Moreover, an increase in the count of antennas at the BS leads to an enhancement in the average secrecy rate.

Fig. 3.

Ergodic secrecy rate in terms of transmit SNR of BS.

The ESR in terms of the transmit power of the jammer is depicted in Fig. 4 for various values of the number of IRS components. By increasing the transmit SNR of the jammer, the ESR exhibits an exponential increase. This phenomenon occurs because increase in enhances the secrecy rate for the far user, thereby resulting in an improvement in the average secrecy rate. Furthermore, this rate shows an increase with a higher number of IRS components, as it boosts the secrecy rates for both users.

Fig. 4.

Ergodic secrecy rate versus transmit SNR of jammer.

The ESR concerning the location of the IRS is demonstrated in Fig. 5. In Fig. 5a, the effect of the distance between the IRS and on the ESR is presented. Obviously, as the distance between the IRS and increases, the ESR decreases due to higher path loss, which weakens the received signal and reduces the secrecy rate..

Fig. 5.

The IRS location impact on the ergodic secrecy rate.

Conversely, Fig. 5b demonstrates the effect of the distance between the IRS and on the ESR. It’s observable that as the distance of the IRS and enlarges, the secrecy rate for the far user decreases. However, the ESR increases in line with the definition of ESR (i.e., ).

: Fig. 6 illustrates the relationship between the SOP and “The BS’s transmit SNR across various numbers of IRS components shows that as the transmit SNR at the BS rises, a clear trend emerges: increasing the parameter improves the mean secrecy rate. This improvement leads to a reduction in the SOP, suggesting stronger security performance. Additionally, a higher number of IRS components boosts the ESR for both NOMA users and the overall average secrecy rate. Consequently, this simultaneous improvement results in a lower probability of secrecy outage, indicating a positive trend in strengthening the system’s security against potential vulnerabilities.

Fig. 6.

Compare secrecy rate with and without a jammer, in relation to the jammer’s transmit SNR.

In Fig. 7, we observe how the SOP changes as the distance between the IRS and the far user () varies across different transmit SNR levels at the BS. This decline occurs because the secrecy rate experienced by the far user reduces, resulting in an increase in the ESR, and in turn lowers the overall SOP. Moreover, when the transmit SNR at the BS decreases, there is an improvement in the SOP. This enhancement arises due to an increase in the secrecy rates observed at both the near and far users, collectively contributing to an overall betterment in the secrecy performance of the system.

Fig. 7.

Compare secrecy rate with and without a jammer, versus IRS and distance to the far user.

Figure 8 illustrates the relationship between SOP and the distance between the IRS and the near user (), considering different transmit SNR values from the BS. It is clear that as the distance between the IRS and increases, the SOP rises. This is because, as the IRS moves further from increases, this communication link’s large-scale fading also increases, subsequently reducing ESR for the nearby user. Consequently, based on the SOP (denoted as in (30)), it experiences an exponential increase. Furthermore, reducing the transmit SNR for the BS improves the probability of the secrecy outage .

Fig. 8.

SOP compared to transmit SNR in the presence of a jammer.

Figure 9 presents the SOP compared to the jammer’s transmit SNR for different numbers of BS transmit antennas. Clearly, the probability of secrecy outage is decreased by increasing the jammer’s SNR. This reduction occurs because as the jammer’s transmit SNR () increases, the secrecy rate of the far user decreases, consequently resulting in an enhancement in the ESR and a subsequent reduction in the SOP.

Fig. 9.

SOP compared to transmit SNR in the absence of a jammer.

Furthermore, an notable trend emerges regarding the impact of the number of BS antennas on the probability of the secrecy outage. It becomes apparent that as the number of antennas at the BS increases, the probability of secrecy outage decreases. This decline is attributed leading to an enhancement in the ESR associated with the increasing the quantity of BS antennas. Therefore, as the ESR improves because of the enhanced array at the BS, the SOP decreases correspondingly.

Conclusion

This paper conducts a comprehensive performance analysis showcasing NOMA supported by IRS within large-scale MIMO systems, focusing on the secrecy rate for pairs of NOMA users. Additionally, the analysis considers SOP for these pairs. It characterizes the secrecy rate and SOP for both nearby and far users, highlighting the impact of crucial system parameters such as the number of BS antennas, IRS elements, transmit SNR, jammer’s transmit SNR, and the distance between the IRS and users. The numerical results demonstrate that increasing the number of BS antennas improves the secrecy rate for both nearby and far users, while simultaneously decreasing the SOP. Moreover, integrating a jammer into the network significantly improves both SOP and ESR. Furthermore, the investigation reveals that as the distance from the IRS to the nearby user increases, the average secrecy rate decreases while the SOP enhances. Conversely, an increased the distance from the IRS to the far user improves the average secrecy rate but reduces the SOP.

Pdf of

To derive the PDF of , defined as , it is evident that

1

Where and and .

Author contributions

Min Zhu: Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Resources, Data Curation, Writing - Original Draft.Hamid Gholizadeh Fard: Conceptualization, Validation, Formal analysis, Investigation, Resources, Data Curation, Writing - Original Draft. Bahar Hazrati: Conceptualization, Validation, Formal analysis, Investigation, Resources, Data Curation, Writing - Original Draft.Behrooz Mosallaei: Conceptualization, Formal analysis, Investigation, Resources, Data Curation, Writing - Original Draft. Alhussein G. Alkhayer: Formal analysis, Investigation, Resources, Data Curation, Writing - Original Draft.Kai Jin: Resources, Data Curation, Writing - Original Draft. Jingyu Zhang: Conceptualization, Validation, Curation, Writing - Original Draft.

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Competing interests

The authors declare no competing interests.