IRS Assisted Dual-Mode Relay Based Adaptive Transmission

1. Introduction

The rapid advancement of wireless communication technologies has led to a significant increase in both the number of users and the traffic load within communication systems. To provide high-quality data services to mobile users, ensuring adequate network coverage is essential. Relay technology, which employs relay nodes to receive and forward signals, can extend communication range and enhance users’ signal reception quality [1]. Decode-and-Forward (DF) and Amplify-and-Forward (AF) are two of the most prevalent relaying schemes in this domain. However, these conventional active relay technologies fundamentally rely on active components, including carrier signal generators, analog-to-digital converters, and power amplifiers [2], resulting in substantial hardware complexity and significant power consumption. Intelligent Reflecting Surfaces (IRS) consist of numerous passive reflecting elements, each capable of dynamically adjusting its reflection coefficient under the control of an IRS controller to modify both the amplitude and phase of the incident signal. As a low-cost, flexibly deployable, and fully passive device, IRS can be applied in various domains such as physical-layer security [3,4,5,6] and enhancement of wireless data transmission [7,8]. It is regarded as a revolutionary technology in the field of wireless communications, offering new design paradigms for signal forwarding and relaying.

Although there has been considerable theoretical and practical research on IRS, their limited signal processing capabilities and passive nature make them more susceptible to channel fading. Moreover, practical implementations of IRS encounter constraints such as limited operating frequency bands [9] and insufficient dynamic adjustment capabilities [10]. Consequently, some research efforts have shifted toward integrated designs that combine IRS with relay techniques. The authors of [11] investigated an IRS-assisted two-way Amplify-and-Forward (AF) relaying system, where one IRS is deployed between the transmitter and the relay node, and another is positioned between the relay and the receiver to enhance signal transmission. By optimizing the phase shift matrices of both IRSs, the signals forwarded by the relay and those transmitted from the source are constructively superimposed in phase at the destination receiver, thereby reducing the bit-error rate (BER). In [12], the authors proposed a UAV-aided communication scheme that jointly employs an active relay and an IRS. In this study, the UAV is equipped with a multi-antenna AF relay and an IRS consisting of multiple passive reflecting elements, which operate both independently and collaboratively to forward signals. Through joint optimization of the UAV’s trajectory, the IRS reflection coefficients, and the relay beamforming parameters, the achievable data rate at the target receiver is maximized.

Despite the studies in [11,12] that integrate IRS with relays, they primarily treat the passive IRS as a supplementary tool to enhance the performance of active relays, without considering the flexible replacement of active relays by IRS (i.e., the adaptive selection between relay and IRS). In fact, both relay and IRS have their own advantages and disadvantages regarding energy efficiency, hardware and computational complexity, and coverage. The ability to adaptively replace active relays with passive IRS based on dynamic wireless channel conditions—which can either reduce power consumption or enhance data transmission under fixed power constraints—holds significant research importance for improving communication performance from the source to the destination node. Therefore, this paper proposes a dual-mode relay (DMR) capable of dynamically switching between an active relaying mode and a passive IRS reflection mode. The DMR consists of multiple dual-mode relay units (DMRU), each of which can flexibly switch between active relaying and passive IRS reflection based on wireless channel conditions. Multiple DMRUs cooperatively process the incident signal in a composite manner to relay data information. Furthermore, under a transmit power constraint, we design a DMR-based Adaptive Transmission (DMRAT) scheme. This scheme evaluates all possible DMR operational modes and, for each mode, jointly optimizes the beamforming matrix at the transmitter and the operational parameters of the DMRUs operating in either active or passive mode using an Alternating Optimization (AO) algorithm, with the goal of improving the data rate at the receiver.

The rest of this paper is organized as follows. Section 2 describes the system model, while Section 3 presents the design of DMRAT. Section 4 evaluates the performance of the proposed method. Finally, we conclude this paper in Section 5.

Throughout this paper, we will use the following notations. The set of complex numbers is denoted as $\mathbb{C}$, while vectors and matrices are represented by lower-case and upper-case bold letters. ${(·)}^T$ and ${(·)}^H$ represent matrix transpose and conjugate transpose, respectively. $|·|$ denotes the absolute value of a complex number.

2. System Model

We consider a communication system consisting of one transmitter (Tx), one receiver (Rx), and a dual-mode relay (DMR), as illustrated in Figure 1. The Tx is equipped with $N_T$ antennas and transmits with power $P_T$, while the Rx has $N_R$ antennas. Due to obstacles or severe channel fading between the Tx and Rx, no direct link satisfying the transmission quality requirement is available [13]. The DMR consists of $N_{DM}$ DMRUs, each capable of dynamically switching between active relaying and passive IRS reflection modes for signal forwarding, under the control of a control unit (CU) co-located with the Tx. When a DMRU operates in active relaying mode, it functions as a full-duplex DF relay. The total transmit power consumed by the DMR is denoted as $P_{DM}$. Assuming that the signal processing delays for both DF relaying and passive IRS reflection are negligible, the signals actively forwarded and passively reflected by all DMRUs can arrive at the Rx simultaneously.

Prior to communication, the Tx broadcasts a pilot signal with all DMRUs of the DMR operating in passive IRS reflection mode. The Rx estimates the Channel State Information (CSI) [14] between itself and both the IRS and the Tx, denoted as $\mathbf{H}$ and $\mathbf{G}$ in Figure 1, and feeds this information back to the Tx. The Tx then transmits the data $\mathbf{x}={[x_1{x}_2\cdots x_{N_I}]}^T$. The Tx precodes $\mathbf{x}$ using the beamforming matrix ${\mathbf{W}}_{Tx}\in {\mathbb{C}}^{N_T\times N_I}$ and then map the pre-processed output to the $N_T$ transmit antennas for transmission. The channels between the Tx and the DMR, and between the DMR and the Rx, are represented by $\mathbf{H}\in {\mathbb{C}}^{N_{DM}\times N_T}$ and $\mathbf{G}\in {\mathbb{C}}^{N_R\times N_{DM}}$, respectively. We employ a spatially uncorrelated Rayleigh flat fading channel model to characterize these channels, where the elements are independent and identically distributed complex Gaussian random variables with zero mean and unit variance. Both $\mathbf{H}$ and $\mathbf{G}$ exhibit quasi-static flat fading [15] characteristics.

3. Design of DMRAT

This section presents the design of the DMRAT under the constraint of the total transmit power at both the Tx and the DMR. For simplicity, we assume $N_T=N_R=N_{DM}=N_I=M$, where M is a positive integer.

3.1. Basic Signal Processing in the DMR

In practical applications, when the channel quality from the Tx to the Rx via the IRS is sufficiently good, the DF relay may not be necessary. In such cases, using only IRS reflection can provide the Rx with adequate signal reception quality. Conversely, when the channel condition is poor, using IRS reflection alone may not be sufficient to ensure good reception quality at the Rx. Therefore, it is necessary to set some DRMUs to active relaying mode to compensate for channel fading, albeit at the cost of some transmit power. To reduce the power consumption of the DMR, the Tx can configure a subset of the DMRUs to operate in IRS mode. As a result, some DMRUs operate in DF relaying mode, while others function in passive IRS reflection mode. The Rx then receives a composite signal constructed from multiple signal components originating from the DMRUs operating in these two distinct modes. To control the operating modes of the M DMRUs in the DMR, we define a mode selection matrix $\mathbf{S}=\mathrm{diag}(s_1,s_2,\cdots ,s_M)$, where each element $s_m\in \{0,1\}$ ($m\in \{1,2,\cdots ,M\}$) indicates whether the m-th DMRU operates in DF relaying mode or IRS reflection mode.

Let $U_m$ denote the m-th DMRU. We define ${\Omega }_{DF}$ as the set of DMRUs operating in DF relaying mode and ${\Omega }_{IRS}$ as the set of DMRUs operating in IRS reflection mode. ${\Omega }_{DF}\cup {\Omega }_{IRS}=\Omega$ holds, where $\Omega$ represents the entire set of DMRUs. For each $U_m\in {\Omega }_{DF}$, the mode indicator is set as $s_m=1$; conversely, for $U_m\in {\Omega }_{IRS}$, $s_m=0$. Under the control of the mode selection matrix $\mathbf{S}$, the received signal at the Rx can be expressed as:

$${\mathbf{y}}_{DM}=\sqrt{P_{DM}}\mathbf{G}\mathbf{S}{\mathbf{W}}_{DF}\mathbf{x}+\sqrt{P_T}\mathbf{G}(\mathbf{S}\oplus \mathbf{I})\mathbf{\Phi }\mathbf{H}{\mathbf{W}}_{Tx}\mathbf{x}+\mathbf{n}.$$

(1)

The first term on the right-hand side (RHS) of Equation (1) represents the signal forwarded by the DMRUs operating in the DF relaying mode, while the second term corresponds to the signal reflected by the DMRUs operating in the IRS reflection mode. ${\mathbf{W}}_{DF}=[{\mathbf{w}}_{DF,1}{\mathbf{w}}_{DF,2}\cdots {\mathbf{w}}_{DF,M}]$ denotes the beamforming matrix for the DMRUs in ${\Omega }_{DF}$, where ${\mathbf{w}}_{DF,m}\in {\mathbb{C}}^{M\times 1}$ is the m-th column vector of ${\mathbf{W}}_{DF}$. $\mathbf{n}$ represents the AWGN vector, whose elements have zero mean and variance ${\sigma }_n^2$. The reflection coefficient matrix of the DMRUs in ${\Omega }_{IRS}$ is represented as $\mathbf{\Phi }=\mathrm{diag}({\beta }_1{e}^{j{\alpha }_1},\cdots ,{\beta }_m{e}^{j{\alpha }_m},\cdots ,{\beta }_M{e}^{j{\alpha }_M})$, where ${\alpha }_m$ and ${\beta }_m$ denote the phase coefficient and amplitude coefficient, respectively, of the m-th DMRU operating in IRS mode. ${\mathbf{W}}_{Tx}=[{\mathbf{w}}_{Tx,1}{\mathbf{w}}_{Tx,2}\cdots {\mathbf{w}}_{Tx,M}]$ represents the beamforming matrix at the Tx. The matrices $\mathbf{S}$ and $\mathbf{S}\oplus \mathbf{I}$ (where $\mathbf{I}$ is the identity matrix and ⊕ denotes element-wise modulo-2 addition) are applied to ${\mathbf{W}}_{DF}$ and $\mathbf{\Phi }$, respectively, to select the operating mode of each DMRU. When $s_m=0$, the matrix $\mathbf{S}$ sets the m-th row elements of ${\mathbf{W}}_{DF}$ to zero; when $s_m=1$, $\mathbf{S}\oplus \mathbf{I}$ sets the m-th reflection coefficient in $\mathbf{\Phi }$ to zero, i.e., ${\beta }_m{e}^{j{\alpha }_m}=0$.

Under the total transmit power constraint $P_{total}=P_T+P_{DM}$, we let $P_T=P_{DM}=P_{total}/2$ for example. The subsequent analysis can be applied to other power allocations. The Rx employs a filtering matrix $\mathbf{F}=[{\mathbf{f}}_1\cdots {\mathbf{f}}_m\cdots {\mathbf{f}}_M]$ for post-processing the received signal ${\mathbf{y}}_{DM}$, where ${\mathbf{f}}_m\in {\mathbb{C}}^{M\times 1}$ is the filter vector corresponding to the desired data $x_m$. The estimated signal can be expressed as ${\widehat{\mathbf{y}}}_{DM}={\mathbf{F}}^H{\mathbf{y}}_{DM}$. The data rate at the Rx is calculated as:

$$R_{DM}=\sum _{m=1}^M{log}_2\left\{1+{\displaystyle \frac{P_{total}{\left|{\mathbf{f}}_m^H\left({\mathbf{h}}_{eq,m}^{DF}{\mathbf{w}}_{DF,m}+{\mathbf{h}}_{eq,m}^{IRS}{\mathbf{w}}_{Tx,m}\right)\right|}^2}{2{\sigma }_n^2}}\right\},$$

(2)

where ${\mathbf{h}}_{eq,m}^{DF}$ and ${\mathbf{h}}_{eq,m}^{IRS}$ are the m-th row vectors of the equivalent channel matrices ${\mathbf{H}}_{eq}^{DF}$ and ${\mathbf{H}}_{eq}^{IRS}$, respectively. The matrix ${\mathbf{H}}_{eq}^{DF}=\mathbf{G}\mathbf{S}$ represents the channel between the Rx and the DMRUs operating in DF relaying mode, while ${\mathbf{H}}_{eq}^{IRS}=\mathbf{G}(\mathbf{S}\oplus \mathbf{I})\mathbf{\Phi }\mathbf{H}$ represents the channel associated with the DMRUs operating in IRS mode.

3.2. Design of DMR’s Operating Parameters

To maximize the data rate $R_{DM}$ under the total power constraint $P_{total}$, it is essential to optimize the mode selection matrix $\mathbf{S}$ and jointly design the beamforming matrices ${\mathbf{W}}_{Tx}$, ${\mathbf{W}}_{DF}$, and the reflection coefficient matrix $\mathbf{\Phi }$. To achieve this object, we first derive the optimal ${\mathbf{W}}_{Tx}$, ${\mathbf{W}}_{DF}$, and $\mathbf{\Phi }$ that maximize $R_{DM}$ for a fixed $\mathbf{S}$, leading to the optimization problem expressed in Equation (3):

$$\begin{array}{cccc}\hfill & \underset{{\mathbf{W}}_{Tx},{\mathbf{W}}_{DF},\mathbf{\Phi }}{max}\hfill & \hfill & R_{DM}\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.\hfill & \hfill & {\beta }_m\in [0,1],{\alpha }_m\in [0,2\pi ),m\in \{1,\cdots ,M\};\hfill \\ \hfill & \hfill & & {\mathbf{W}}_{DF}^H{\mathbf{W}}_{DF}=\mathbf{I},{\mathbf{W}}_{Tx}^H{\mathbf{W}}_{Tx}=\mathbf{I}.\hfill \end{array}$$

(3)

We assume that both $P_T$ and $P_{DM}$ are entirely allocated to the data transmission of the target Tx–Rx pair, and that the processing at the Tx and the DF relaying mode DMRUs does not introduce additional gain to the transmitted signal; i.e., ${\mathbf{W}}_{Tx}^H{\mathbf{W}}_{Tx}=\mathbf{I}$ and ${\mathbf{W}}_{DF}^H{\mathbf{W}}_{DF}=\mathbf{I}$ hold. Subsequently, for a given total transmit power $P_{total}$, we configure different mode selection matrices $\mathbf{S}$ and employ the AO algorithm to solve the optimization problem given in Equation (3). The main idea of the algorithm is as follows. First, initialize $\mathbf{\Phi }=\mathbf{I}$ and design both ${\mathbf{W}}_{Tx}$ and ${\mathbf{W}}_{DF}$ accordingly. Then, based on the determined ${\mathbf{W}}_{Tx}$ and ${\mathbf{W}}_{DF}$, optimize $\mathbf{\Phi }$. Subsequently, with the newly obtained $\mathbf{\Phi }$, redesign ${\mathbf{W}}_{Tx}$ and ${\mathbf{W}}_{DF}$. This process iterates until the improvement in $R_{DM}$ falls below a predefined threshold, or the number of iterations exceeds a preset maximum value, at which point the algorithm terminates. Thus, for a given $\mathbf{S}$, we can compute a set of ${\mathbf{W}}_{Tx}$, ${\mathbf{W}}_{DF}$, and $\mathbf{\Phi }$ that maximizes $R_{DM}$. In what follows, we will elaborate on the method for computing the DMR’s operating parameters using AO under a fixed $\mathbf{S}$.

3.2.1. Design of Beamforming at the Tx and DMR

First, we initialize $\mathbf{\Phi }=\mathbf{I}$ (i.e., set ${\beta }_m=1$ and ${\alpha }_m=0$), and apply Zero-Forcing Beamforming (ZFBF) [16] to design ${\mathbf{W}}_{Tx}$ and ${\mathbf{W}}_{DF}$. This method effectively eliminates mutual interference among multiple concurrent signals, thereby improving data transmission performance. Under this initialization, Equation (3) can be rewritten as:

$$\begin{array}{cccc}\hfill & \underset{{\mathbf{W}}_{Tx},{\mathbf{W}}_{DF}}{max}\hfill & \hfill & R_{DM}\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.\hfill & \hfill & {\beta }_m=1,{\alpha }_m=0,m\in \{1,\cdots ,M\};\hfill \\ \hfill & \hfill & & {\mathbf{W}}_{DF}^H{\mathbf{W}}_{DF}=\mathbf{I},{\mathbf{W}}_{Tx}^H{\mathbf{W}}_{Tx}=\mathbf{I}.\hfill \end{array}$$

(4)

We use ${\mathbf{v}}_{DF,m}$ to denote the m-th column vector of the matrix ${\mathbf{V}}_{DF}={\left({\mathbf{H}}_{eq}^{DF}\right)}^H{\left[{\mathbf{H}}_{eq}^{DF}{\left({\mathbf{H}}_{eq}^{DF}\right)}^H\right]}^{-1}$, and ${\mathbf{v}}_{Tx,m}$ to represent the m-th column vector of ${\mathbf{V}}_{Tx}={\left({\mathbf{H}}_{eq}^{IRS}\right)}^H{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}$. Then, the m-th column of the precoding matrix ${\mathbf{W}}_{DF}$, denoted as ${\mathbf{w}}_{DF,m}$, can be obtained as ${\mathbf{w}}_{DF,m}={\mathbf{v}}_{DF,m}{\parallel {\mathbf{v}}_{DF,m}\parallel }_F^{-1}$, where ${\parallel ·\parallel }_F$ represents the Frobenius norm of a vector or matrix. Similarly, the m-th column of ${\mathbf{W}}_{Tx}$ can be calculated as ${\mathbf{w}}_{Tx,m}={\mathbf{v}}_{Tx,m}{\parallel {\mathbf{v}}_{Tx,m}\parallel }_F^{-1}$.

3.2.2. Design of the IRS’s Reflection Coefficient Matrix

After computing ${\mathbf{W}}_{Tx}$ and ${\mathbf{W}}_{DF}$, Equation (4) becomes:

$$\begin{array}{cccc}\hfill & \underset{\mathbf{\Phi }}{max}\hfill & \hfill & R_{DM}\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.\hfill & \hfill & {\beta }_m\in [0,1],{\alpha }_m\in [0,2\pi ),m\in \{1,\cdots ,M\}.\hfill \end{array}$$

(5)

Based on the definition of $s_m$ and Equation (2), when $s_m=0$, the selection matrix $\mathbf{S}$ sets the m-th row of ${\mathbf{W}}_{DF}$ to zero, resulting in ${\mathbf{h}}_{eq,m}^{DF}{\mathbf{w}}_{DF,m}=0$. Conversely, when $s_m=1$, the operation $\mathbf{S}\oplus \mathbf{I}$ sets the m-th main diagonal element of $\mathbf{\Phi }$ to zero (i.e., ${\beta }_m{e}^{j{\alpha }_m}=0$), which implies ${\mathbf{h}}_{eq,m}^{IRS}{\mathbf{w}}_{Tx,m}=0$. Substituting ${\mathbf{w}}_{DF,m}={\mathbf{v}}_{DF,m}{\parallel {\mathbf{v}}_{DF,m}\parallel }_F^{-1}$ and ${\mathbf{w}}_{Tx,m}={\mathbf{v}}_{Tx,m}{\parallel {\mathbf{v}}_{Tx,m}\parallel }_F^{-1}$ into Equation (2) leads to:

$$\begin{array}{cc}\hfill R_{DM}& =R_{DF}+R_{IRS}\hfill \\ \hfill & =\sum _{U_i\in {\Omega }_{DF}}{log}_2\left\{1+{\displaystyle \frac{P_{total}{\mathbf{f}}_m^H\left({\mathbf{h}}_{eq,m}^{DF}{\mathbf{v}}_{DF,m}\right){\left({\mathbf{h}}_{eq,m}^{DF}{\mathbf{v}}_{DF,m}\right)}^H{\mathbf{f}}_m}{2{\sigma }_n^2{\parallel {\mathbf{v}}_{DF,m}\parallel }_F^2}}\right\}\hfill \\ \hfill & +\sum _{U_j\in {\Omega }_{IRS}}{log}_2\left\{1+{\displaystyle \frac{P_{total}{\mathbf{f}}_m^H\left({\mathbf{h}}_{eq,m}^{IRS}{\mathbf{v}}_{Tx,m}\right){\left({\mathbf{h}}_{eq,m}^{IRS}{\mathbf{v}}_{Tx,m}\right)}^H{\mathbf{f}}_m}{2{\sigma }_n^2{\parallel {\mathbf{v}}_{Tx,m}\parallel }_F^2}}\right\},\hfill \end{array}$$

(6)

where $i,j\in \{1,\cdots ,M\}$. Note that the first term on the RHS of Equation (6) does not contain the unknown parameter $\mathbf{\Phi }$. Therefore, only the second term on the RHS of Equation (6) needs to be optimized. Consequently, Equation (5) can be equivalently expressed as:

$$\begin{array}{cccc}\hfill & \underset{\mathbf{\Phi }}{max}\hfill & \hfill & R_{IRS}\hfill \\ \hfill & \mathrm{s}.\mathrm{t}.\hfill & \hfill & {\beta }_m\in [0,1],{\alpha }_m\in [0,2\pi ),m\in \{1,\dots ,M\}.\hfill \end{array}$$

(7)

We can derive ${\mathbf{h}}_{eq,m}^{IRS}{\mathbf{v}}_{Tx,m}={\left\{{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}_{m,m}=1$, where ${(·)}_{m,m}$ denotes the element at the m-th row and m-th column of a matrix. Given that $\left|\right|{\mathbf{v}}_{Tx,m}{\left|\right|}_F^2={\mathbf{v}}_{Tx,m}^H{\mathbf{v}}_{Tx,m}$ and ${\mathbf{v}}_{Tx,m}^H{\mathbf{v}}_{Tx,m}={\left({\mathbf{V}}_{Tx}^H{\mathbf{V}}_{Tx}\right)}_{m,m}$, we can have:

$${\left({\mathbf{V}}_{Tx}^H{\mathbf{V}}_{Tx}\right)}_{m,m}\&=\left\{{\left\{{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}^H\right.{\left.\times {\left({\mathbf{H}}_{eq}^{IRS}\right)}^H{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}_{m,m}.$$

(8)

Substituting ${\left\{{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}^H={\left\{{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}^H{\mathbf{H}}_{eq}^{IRS}$ into Equation (8) and simplifying, we obtain ${\left({\mathbf{V}}_{Tx}^H{\mathbf{V}}_{Tx}\right)}_{m,m}={\left\{{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}_{m,m}^H$. Since ${\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}$ is a symmetric matrix, ${\left\{{\left\{{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}^H\right\}}_{m,m}={\left\{{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}_{m,m}$ holds; therefore, $\left|\right|{\mathbf{v}}_{Tx,m}{\left|\right|}_F^2={\left\{{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}_{m,m}$. As a result, $R_{IRS}$ can be simplified as:

$$R_{IRS}=\sum _{U_m\in {\Omega }_{IRS}}{log}_2\left\{1+{\displaystyle \frac{P_{total}{\mathbf{f}}_m^H{\mathbf{f}}_m}{2{\sigma }_n^2{\left\{{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}_{m,m}}}\right\}.$$

(9)

According to Jensen’s inequality, we have:

$$R_{IRS}\le {log}_2\left\{1+\sum _{U_m\in {\Omega }_{IRS}}{\displaystyle \frac{P_{total}{\mathbf{f}}_m^H{\mathbf{f}}_m}{2{\sigma }_n^2{\left\{{\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}\right\}}_{m,m}}}\right\}.$$

(10)

Note that $R_{IRS}$ reaches its maximum value if and only if all non-zero main diagonal elements of ${\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}$ in Equation (10) are equal; this condition can also maximize $R_{DM}$. Suppose that after optimizing $\mathbf{\Phi }$, all non-zero main diagonal elements of ${\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}$ become $\gamma$, achieving the maximum $R_{DM}$. It is observed that when ${\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}=\gamma (\mathbf{S}\oplus \mathbf{I})$, all non-zero main diagonal elements of ${\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}$ are exactly $\gamma$; therefore, the condition ${\left[{\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H\right]}^{-1}=\gamma (\mathbf{S}\oplus \mathbf{I})$ can be used as a constraint for optimizing $\mathbf{\Phi }$. Furthermore, using the property of diagonal matrices, we have ${\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H=1/\left[\gamma (\mathbf{S}\oplus \mathbf{I})\right]$. Thus, to maximize $R_{DM}$, it is necessary to solve for $\mathbf{\Phi }$ such that ${\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H=1/\left[\gamma (\mathbf{S}\oplus \mathbf{I})\right]$ holds. The computation procedure is detailed as follows.

By expanding $\mathbf{\Phi }\mathbf{H}$ in ${\mathbf{H}}_{eq}^{IRS}=\mathbf{G}(\mathbf{S}\oplus \mathbf{I})\mathbf{\Phi }\mathbf{H}$, we can rewrite ${\mathbf{H}}_{eq}^{IRS}$ as:

$${\mathbf{H}}_{eq}^{IRS}=\mathbf{G}(\mathbf{S}\oplus \mathbf{I})\left[\begin{array}{cccc}{\mathbf{h}}_1& & & \\ & {\mathbf{h}}_2& & \\ & & \ddots & \\ & & & {\mathbf{h}}_M\end{array}\right]\left[\begin{array}{c}{\mathit{\varphi }}_1\\ {\mathit{\varphi }}_2\\ ⋮\\ {\mathit{\varphi }}_M\end{array}\right].$$

(11)

Here, ${\mathbf{h}}_m\in {\mathbb{C}}^{1\times M}$ is the m-th row vector of $\mathbf{H}$, and ${\mathit{\varphi }}_m=\mathrm{diag}({\beta }_m{e}^{j{\alpha }_m},\cdots ,{\beta }_m{e}^{j{\alpha }_m})\in {\mathbb{C}}^{M\times M}$. According to Equation (11), we get:

$${\mathbf{H}}_{eq}^{IRS}{\left({\mathbf{H}}_{eq}^{IRS}\right)}^H=\mathbf{D}\left[\begin{array}{c}{\mathit{\varphi }}_1\\ ⋮\\ {\mathit{\varphi }}_M\end{array}\right]\left[\begin{array}{ccc}{\mathit{\varphi }}_1^H& \cdots & {\mathit{\varphi }}_M^H\end{array}\right]{\mathbf{D}}^H={\displaystyle \frac{1}{\gamma }}{(\mathbf{S}\oplus \mathbf{I})}^{-1},$$

(12)

where $\mathbf{D}\in {\mathbb{C}}^{M\times M}$ is defined as $\mathbf{D}=\mathbf{G}(\mathbf{S}\oplus \mathbf{I})\left[\begin{array}{cccc}{\mathbf{h}}_1& & & \\ & {\mathbf{h}}_2& & \\ & & \ddots & \\ & & & {\mathbf{h}}_M\end{array}\right]$.

Left-multiplying both sides of Equation (12) by the left pseudo-inverse of $\mathbf{D}$, defined as ${\left({\mathbf{D}}^H\mathbf{D}\right)}^{-1}{\mathbf{D}}^H$, and then right-multiplying by the right pseudo-inverse of ${\mathbf{D}}^H$, expressed as $\mathbf{D}{\left({\mathbf{D}}^H\mathbf{D}\right)}^{-1}$, we obtain:

$$\left[\begin{array}{c}{\mathit{\varphi }}_1\\ ⋮\\ {\mathit{\varphi }}_M\end{array}\right]\left[\begin{array}{ccc}{\mathit{\varphi }}_1^H& \cdots & {\mathit{\varphi }}_M^H\end{array}\right]={\displaystyle \frac{1}{\gamma }}{\left({\mathbf{D}}^H\mathbf{D}\right)}^{-1}\left[{\mathbf{D}}^H{(\mathbf{S}\oplus \mathbf{I})}^{-1}\mathbf{D}\right]{\left({\mathbf{D}}^H\mathbf{D}\right)}^{-1}.$$

(13)

Expanding the term on the left-hand side (LHS) of Equation (13), and noting that all matrices on the RHS are known without containing any unknown parameters, we can calculate the RHS of Equation (13) and denote the result as $\mathbf{A}\in {\mathbb{C}}^{M\times M}$. Thus, we have:

$$\left[\begin{array}{ccc}{\mathit{\varphi }}_1{\mathit{\varphi }}_1^H& \cdots & {\mathit{\varphi }}_1{\mathit{\varphi }}_M^H\\ ⋮& \ddots & ⋮\\ {\mathit{\varphi }}_M{\mathit{\varphi }}_1^H& \cdots & {\mathit{\varphi }}_M{\mathit{\varphi }}_M^H\end{array}\right]={\displaystyle \frac{1}{\gamma }}\mathbf{A}={\displaystyle \frac{1}{\gamma }}\left[\begin{array}{ccc}{\mathbf{a}}_{11}& \cdots & {\mathbf{a}}_{1M}\\ ⋮& \ddots & ⋮\\ {\mathbf{a}}_{M1}& \cdots & {\mathbf{a}}_{MM}\end{array}\right],$$

(14)

where ${\mathbf{a}}_{pq}\in {\mathbb{C}}^{M\times M}$ ($p,q\in \{1,\cdots ,M\}$ indicate the relative position of ${\mathbf{a}}_{pq}$ within $\mathbf{A}$) is a diagonal matrix whose non-zero main diagonal elements are equal (denoted as ${\xi }_{pq}$). Since ${\beta }_p,{\beta }_q\in [0,1]$, the main diagonal elements of ${\mathit{\varphi }}_p{\mathit{\varphi }}_q^H$ satisfy $\left|{\beta }_p{e}^{j{\alpha }_p}{\beta }_q{e}^{-j{\alpha }_q}\right|\le 1$. Therefore, the corresponding ${\xi }_{pq}$ of ${\mathbf{a}}_{pq}$ must also satisfy ${\xi }_{pq}/\gamma \le 1$. By defining $\gamma ={\sum }_{p=1}^M{\sum }_{q=1}^M{\xi }_{pq}$, the condition ${\xi }_{pq}/\gamma \le 1$ is satisfied. From Equation (14), we can derive ${\mathit{\varphi }}_p{\mathit{\varphi }}_q^H={\mathbf{a}}_{pq}/\gamma$, which allows us to establish a set of equations for the unknown variables ${\alpha }_1,{\alpha }_2,\dots ,{\alpha }_M$ and ${\beta }_1,{\beta }_2,\cdots ,{\beta }_M$ as follows:

$$\left\{\begin{array}{c}{\beta }_m^2={\displaystyle \frac{1}{\gamma }}{\xi }_{mm}\hfill \\ {\beta }_1{e}^{j{\alpha }_1}{\beta }_m{e}^{-j{\alpha }_m}={\displaystyle \frac{1}{\gamma }}{\xi }_{1m}\hfill \end{array}\right.,$$

(15)

where $m\in \{1,\cdots ,M\}$. When $p=q$, ${\xi }_{pq}$ is a real number; otherwise, ${\xi }_{pq}$ is complex. Equation (15) comprises a total of $2M-1$ independent equations with $2M$ unknowns, thus leading to infinitely many solutions. The general solution of Equation (15) for the unknown variables ${\beta }_1{e}^{{\alpha }_1},{\beta }_2{e}^{{\alpha }_2},\cdots ,{\beta }_M{e}^{{\alpha }_M}$ can be expressed as:

$$\left\{\begin{array}{c}{\beta }_1{e}^{j{\alpha }_1}=\sqrt{{\displaystyle \frac{{\xi }_{11}}{\gamma }}}{e}^{j{\alpha }_1}\hfill \\ {\beta }_{m^{\prime }}{e}^{j{\alpha }_{m^{\prime }}}={\displaystyle \frac{\sqrt{\gamma }{\xi }_{1m^{\prime }}}{\sqrt{{\xi }_{11}}}}{e}^{j{\alpha }_1}\hfill \end{array}\right.,$$

(16)

where $m^{\prime }\in \{2,\cdots ,M\}$. To obtain a particular solution, we can assign an arbitrary value to any one of ${\beta }_1{e}^{{\alpha }_1},\cdots ,{\beta }_M{e}^{{\alpha }_M}$ (e.g., ${\beta }_m{e}^{{\alpha }_m}$); afterwards, the values of the remaining ${\beta }_{m^{\prime }}{e}^{{\alpha }_{m^{\prime }}}$ for $m^{\prime }\in \{1,\cdots ,M\}-\left\{m\right\}$ can be computed using Equation (16).

Subsequently, ${\mathbf{W}}_{TX}$ and ${\mathbf{W}}_{DF}$ are recalculated under the current optimized $\mathbf{\Phi }$. Using these updated beamforming matrices, $\mathbf{\Phi }$ is then re-optimized. This process continues iteratively until the improvement in $R_{DM}$—defined as the difference between $R_{DM}^{\left(l\right)}$ and $R_{DM}^{(l-1)}$ obtained in the l-th and $(l-1)$-th iterations—falls below a predefined threshold $\epsilon$, or until the iteration count l reaches the maximum value $l_{max}$. The selection of $\epsilon$ influences the algorithm’s performance and efficiency: a value too large may cause premature termination, resulting in suboptimal data rate performance, while a value too small can yield better optimization at the expense of prolonged convergence. In this work, $\epsilon ={10}^{-5}$ is chosen based on simulation studies. To summarize, we present the AO procedure for jointly determining ${\mathbf{W}}_{Tx}$, ${\mathbf{W}}_{DF}$, and $\mathbf{\Phi }$ in Algorithm 1.

Algorithm 1 AO-based Joint Optimization of ${\mathbf{W}}_{Tx}$, ${\mathbf{W}}_{DF}$, and $\mathbf{\Phi }$

1: Initialize: Set iteration counter $l=0$, max iterations $l_{max}=50$, threshold $\epsilon ={10}^{-5}$, and ${\mathbf{\Phi }}_0=\mathbf{I}$. 2: Compute ${\mathbf{W}}_{DF,0}$ and ${\mathbf{W}}_{Tx,0}$ using Equation (3), and calculate $R_{DM,0}$ using Equation (2). 3: Update $l\leftarrow l+1$. 4: Solve for ${\mathbf{\Phi }}_l$ based on ${\mathbf{W}}_{DF,l-1}$ and ${\mathbf{W}}_{Tx,l-1}$ using Equation (16). 5: Calculate ${\mathbf{W}}_{DF,l}$ and ${\mathbf{W}}_{Tx,l}$ based om ${\mathbf{\Phi }}_l$ using using Equation (3). 6: Calculate $R_{DM,l}$ based on ${\mathbf{W}}_{DF,l}$, ${\mathbf{W}}_{Tx,l}$, and ${\mathbf{\Phi }}_l$ using Equation (2). 7: If $R_{DM,l}-R_{DM,l-1}>\epsilon$ and $l\le l_{max}$, repeat Steps 3–7; otherwise, proceed to Step 8. 8: Output: ${\mathbf{W}}_{DF}={\mathbf{W}}_{DF,l}$, ${\mathbf{W}}_{Tx}={\mathbf{W}}_{Tx,l}$, and $\mathbf{\Phi }={\mathbf{\Phi }}_l$. The algorithm ends.

3.2.3. Selection of DMR Operating Mode

For the DMR operating mode selection matrix $\mathbf{S}$, since the DMR contains M DMRUs and has a total of $2^M$ distinct operating modes, it is necessary to perform joint optimization of ${\mathbf{W}}_{Tx}$, ${\mathbf{W}}_{DF}$, and $\mathbf{\Phi }$ using the AO technique for each possible $\mathbf{S}$ to maximize $R_{DM}$. Then, by comparing $R_{DM}$ across different $\mathbf{S}$ configurations, the configuration that yields the highest $R_{DM}$ is selected.

4. Evaluation

This section employs MATLAB simulation to evaluate the performance of the DMRAT scheme. The total transmit power is set to $P_{total}=P_T+P_{DM}$, with $P_T=P_{DM}=P_{total}/2$. Note that when all DMRUs operate in IRS passive reflection mode, $P_{DM}=0$, and accordingly we configure $P_T=P_{total}$. We define the transmit power normalized by noise power as $\zeta =10lg(P_{total}/{\sigma }_n^2)$ dB, and set $\zeta \in [0,20]$ dB for the simulation. We use Monte Carlo simulation with 1500 independent trials. In each trial, the channels $\mathbf{H}$ and $\mathbf{G}$ are generated randomly.

Figure 2 illustrates the variation of the data rate performance at the Rx along with $\zeta$ for different values of M. We denote the proposed DMRAT using AO as DMRAT-AO, and compare it with three other schemes: DMR-IRS (where all DMRUs adopt passive IRS reflection mode), DMR-DF (where all DMRUs adopt DF relaying mode), and DMR-R (where all DMRUs randomly adopt either passive IRS reflection mode or DF relaying mode). As the figure shows, the data rates of all methods increase with the growth of M. This is due to the improved signal processing gain provided by multiple antennas. For a fixed M, DMRAT-AO outperforms the other three methods in the achievable data rate. This is because DMRAT-AO utilizes AO to jointly optimize the operating parameters of both the Tx and the DMR, selecting the optimal operating mode for the DMR to maximize the data rate at the Rx. DMR-DF employs the DF relaying mode to decode and forward the signal transmitted by the Tx, effectively compensating for channel fading through the consumption of transmit power. Moreover, in this study we assume that the DF relay can decode the signal from the Tx without errors; hence, the rate performance of DMR-DF is influenced solely by the channel between the DMR and the Rx. In other words, under the total transmit power constraint $P_{total}$, DMR-DF allocates half of $P_{total}$ to the Tx, ensuring reliable communication from the Tx to the DMR. In contrast, DMR-IRS allocates all transmit power to the Tx, and its data rate performance at the Rx is cooperatively affected by the channel fading between the Tx and DMR, as well as the DMR and Rx. Poor quality in either of these two links will degrade the data rate of DMR-IRS. Therefore, given the same $\zeta$, DMR-DF significantly outperforms DMR-IRS, which exhibits the poorest rate performance. DMR-R randomly selects the operating mode of each DMRU. When channel conditions are poor, inappropriate selection of the IRS mode may lead to reduced data rate at the Rx. As a result, the rate performance of DMR-R is inferior to that of DMR-DF.

Figure 3 shows the data rate at the Rx versus the number of iterations under $M=2$ for different DMR operating modes. When all DMRUs operate in DF relaying mode, only ZFBF is required to design ${\mathbf{W}}_{Tx}$ and ${\mathbf{W}}_{DF}$, without needing to compute $\mathbf{\Phi }$ or apply AO. Therefore, the data rate achieved by DMR-DF remains unchanged as the number of iterations increases. In contrast, the data rate of DMRAT-AO gradually increases during the first five iterations, consistently outperforming the other methods, and converges after approximately five iterations. The data rate of DMR-IRS stabilizes after five iterations, while that of DMR-R ceases to improve after four iterations. This is because DMRAT-AO uses AO to select the DMR operating mode that maximizes the data rate at the Rx, resulting in superior performance. Additionally, at the initial iteration (i.e., $l=0$), since DMRAT-AO sets initial operating state of the DMR to DMR-DF, the data rates of DMRAT-AO and DMR-DF are identical when $l=0$.

5. Conclusion

To address the challenges of high power consumption in active relays and the limited fading resistance of passive IRS, this paper proposes a dual-mode relay (DMR) capable of dynamically switching between active relaying and passive IRS reflection modes according to channel conditions. Under a total transmit power constraint, we design a DMR-based Adaptive Transmission (DMRAT) method that utilizes AO. DMRAT iterates over all possible DMR operating modes and, for each mode, applies AO to jointly optimize the beamforming matrices at the Tx and the DMR, as well as the IRS reflection coefficient matrix, thereby maximizing the data rate for the target communication pair. The optimal DMR configuration is determined by comparing the maximum achievable rate across all operating modes. Simulation results demonstrate that DMRAT significantly enhances the data rate performance at the intended receiver.