Spectrum Sharing Design for Integrated Aeronautical Communication and Radar System

1. Introduction

To achieve a safer and more efficient air traffic management (ATM), it is crucial to modernize the aeronautical communications system (ACS)[1]. A new promising ACS for cruise flight safety has emerged, named L-band digital aeronautical communication system (L-DACS), which can significantly improve the operation efficiency of ATM. Additionally, the L-DACS candidate system based on the orthogonal frequency division multiplexing (OFDM) physical-layer technique must be able to operate in the presence of interference from other equipment [2], and also cause the minimum possible interference to legacy systems (e.g., distance measuring equipment (DME) [3] in the same frequency band. This is particularly important since the current ATM system relies on the foundation infrastructure, communication, navigation and surveillance (CNS), which provides critical functions for the safety of aircraft (AC). However, the availability of dedicated frequency spectrum for CNS is struggling to keep up with the increasing saturation of the licensed aviation spectrum band especially in congested airspace. Radar systems, which are used for navigation or surveillance function, have occupied a major portion of the spectrum for civilian aviation operations, leaving the usable spectrum for ACS scarce. Therefore, efficient spectrum utilization strategies are of paramount importance to enhance the performance of integrated CNS system. A cost-efficient strategy for achieving high-quality CNS system performance is to leverage already existing and available technologies such as multiple-input multiple-output (MIMO). It can possess obvious advantages in terms of throughput and reliability by exploiting the spatial-domain channel correlation [4]. Despite the evident merits of MIMO, the effective deployment of CNS is confronted with another challenge, which is the vulnerability of the air-to-ground (A2G) data link. This situation originates from the mutual interference induced by the presence of the various avionics equipment. To address this issue, spectrum sharing between aeronautical communication and radar systems presents a promising solution[5], which has the potential to enable the utilization of additional spectrum resources for communication systems. Consequently, it remains crucial to continue enhancing the spectral efficiency (SE) in order to satisfy the stringent demands of future ACS. In this context, non-orthogonal multiple access (NOMA) has been recognized as a potentially effective technique of improving the SE, as it allows simultaneous multi-user transmission sharing the same resource block [6,7]. Specifically, the core principle of NOMA is to enable the access of multiple users within the same resource blocks while differentiating them in the power domain. This is accomplished by utilizing superposition coding (SC) at the transmitter and successive interference cancellation (SIC) at the receiver [8,9]. In contrast to the design of orthogonal multiple access (OMA), NOMA-based framework can achieve higher SE as serving multiple users in the same resource block, thereby significantly enhancing communication performance.

1.1. Related Works

In recent decades, research interest in communication and radar integration has increased, driven by the goal of suppressing mutual interference and enabling functional collaboration. Generally speaking, existing research on integration can be divided into two main categories: radar and communication coexistence (RCC) [10], and integrated sensing and communications (ISAC) [11]. Although the connotation of terminologies may vary, the sensing functionality primarily refers to radar sensing, which has been a mainstream focus in research. The major distinction between the two categories lies in whether the communication and radar functions are implemented through two separate systems or integrated within a unified framework. The first approach RCC represents a loose form of integration, yielding limited benefits such as reduced signaling overhead and interference suppression [12]. While ISAC delivers both functions using the same waveform, designed to optimize the joint performance of both communication and radar systems [11]. Ultimately, the objective of integration is to surpass the separate functionalities of communication and radar, fostering effective cooperation between them to achieve mutual benefits.

1.1.1. Studies on RCC

The objective of RCC is to provide effective management of the mutual interference between the communication and radar systems, allowing both to operate on the same spectrum and achieve their individual tasks. The study on RCC initially focused on single-antenna techniques, the potential of the OFDM was explored for both communication and radar functions in [13]. The aeronautical spectral coexistence of OFDM communication system and navigation system in L-band was studied in [14], where the pulse blanking technique was employed to mitigate the interference in time domain. Exploiting multiple antennas, the closed-form expression for the optimal transmission waveform was derived to minimize the multi-user interference under different radar sensing criteria in [15]. As an innovative contribution, [16] developed branch-and-bound-based algorithms to investigate the optimal waveform design for RCC within a MIMO framework. Moreover, a precoder for MIMO-radar spectrally-coexistent with a MIMO cellular system is designed [17], where spectrum sharing with zero or minimal interference is achieved by using the proposed space projection. The research in [18] addresses the joint design of the communication and radar systems with co-located antennas focusing on spectrum sharing, aiming to maximize the radar signal-to-interference-plus-noise ratio (SINR) or the communication rate. As a further advance, [19] utilized federated transfer learning for beneficially radar-aided beam prediction in MIMO communication systems to preserve users’ location privacy, where guaranteeing beam prediction accuracy.

1.1.2. Studies on ISAC

By leveraging a joint platform to promote the communication and radar functions, ISAC significantly reduces hardware costs compared to RCC. As a result, ISAC has emerged as a key area of focus in the research on communication and radar integration. The research [20] proposed two sophisticated strategies for implementing ISAC: separated deployment and shared deployment. Both strategies aim to generate high-quality radar beam patterns while meeting communication requirements. Based on the aforementioned antenna deployment strategy, low-complexity beamformer (BF) design algorithms were devised in [21]. To explore the inherent performance boundaries of ISAC, [22] introduced a Pareto optimization framework that characterizes the performance region based on the trade-off between the radar’s peak-to-sidelobe ratio and the SINR attained by the communication user. Additionally, [23] presented a novel joint design of the transmitter and radar receiver, focusing on maximizing the radar’s received SINR for the first time within the ISAC. In [24], a blind beam tracking approach was exploited for UAV-satellite communication system considering the unstable beam pointing due to the UAV navigation. The author in [25] dealt with waveform design of ISAC to improve target detect ability in clutter environment considering the service quality of communication users, while maximizing the output signal-to-clutter-plus-noise ratio (SCNR) of MIMO radar. Furthermore, [26] proposed two types of receiver structures for ISAC systems designed to cancel interference from a priori known dedicated radar signals. Based on these structures, the optimal BF was derived to minimize radar beampattern errors. In [27], a dual-functional base station employing NOMA was designed to serve multiple communication users while simultaneously exploiting the superimposed communication signals for target sensing. Further, [28] investigated a MIMO ISAC base station that detects radar-centric users while transmitting mixed multicast-unicast messages to both radar-centric and communication-centric users within the same spectrum resource.

Note that most of the aforementioned research contributions assume that radar targets are non-communicative and only need to be detected, which differs from scenarios involving aircraft in flight. This assumption essentially treats communication and radar as two isolated systems. Considering the diverse future applications of ACS for ATM, there is a need to develop more sophisticated integration schemes for communication and radar to support a variety of co-located avionics. Although previous works have established a study foundation for spectrum sharing in ground-based wireless communication, the exploration of adopting ISAC in integrated aeronautical communication and radar system (IACRS) remains largely unexplored.

1.2. Motivation and Contributions

As unveiled by the recent works [27,28], the employment of NOMA can offer flexible resource allocation and diverse information transmission options for the communication and radar integration. However, to the best of our knowledge, the interplay between NOMA and ISAC, as well as the potential performance enhancements they may provide, has not been explored in civil aeronautical applications. The primary challenges can be described as follows: 1) The non-convex optimization problem is challenging to solve because of the highly-intertwined variables, including the transceiver BF and the power allocation coefficients; 2) The integration of MIMO introduces additional complexity in the form of A2G channel, leading to intricate interplay between the subproblems. Therefore, the deployment of ISAC in the IACRS to achieve efficient performance needs to be further investigated. In view of this, we propose a novel concept of NOMA-motivated spectrum sharing design for IACRS. For the sake of brevity, the primary contributions of this paper are summarized as follows with explicit comparisons to existing state-of-the-art in Table 1.

• We in investigate an integrated transmission framework for aeronautical communication and radar system, in which MIMO is utilized at the GBS and ACs to facilitate flexible multiple access scheme, and NOMA is deployed to realize dual-spectrum sharing between the communication and the navigation avionic devices. Given the proposed framework, we formulate a weighted achievable sum rate and the sensing SCNR maximization problem. Aiming for the joint optimization of the GBS transmit BF, the airborne receivers BFs, as well as the power allocation, the minimum operation requirements of both the communication and radar functions is guaranteed.
• A practical A2G MIMO channel model accounting for the AC dynamics is proposed. This model considers both the AC position and attitude to characterize the steering directions. A rotation matrix is constructed with the AC attitude represented by Euler angles (i.e., heading angle, pitch angle, and roll angle) to derive an equivalent position (EP) given the AC’s current position. Consequently, the realistic angle-of-arrivals (AoAs)/angle-of-departures (AoDs) can be obtained by leveraging the geometrical information.
• We develop an alternating optimization (AO) algorithm, where the original problem is decomposed into two subproblems that are solved alternatively. By combining the GBS transmit BF and the power allocation subproblems, we construct the auxiliary variables which incorporate the optimization variables, so as to simplify the optimization process. Afterwards, the penalty-based method is invoked to handle the non-convex constraint for the covariance matrix of BF. For the airborne receivers BFs design, we effectively solve them by utilizing the sequential rank-one constraint relaxation (SROCR) while fixing the other optimization variables.
• Our numerical results indicate that the proposed algorithm achieves superior sum rate performance and sensing performance compared to the benchmark schemes for the dual-function of communication and radar. It is demonstrated that the proposed NOMA-motivated MIMO IACRS schemes are capable of significantly boosting the performance of A2G datalink to support ACs receive messages, while simultaneously constructing a high-quality radar detection capability. Furthermore, the system performance gain becomes more significant, when the AC’s attitude is considered.

1.3. Organization and Notation

The rest of this paper is organized as follows. In Section 2, we introduce the model of IACRS, and formulate the joint sum rate and sensing SCNR problem. In Section 3, an AO-based iterative algorithm is developed to solve the formulated maximizing problem. Numerical results are presented in Section 4 to verify the effectiveness of the proposed algorithms compared to other benchmarks. Finally, conclusions are drawn in Section 5.

Notation: Matrices, vectors and scalars are denoted by bold-face upper-case, bold-face lower-case, and italic letters, respectively. ${\mathbb{C}}^{N\times 1}$ denotes the space of $N\times 1$ complex-valued vectors. For a vector $\mathbf{a}$, ${\mathbf{a}}^H$ denotes its (Hermitian) conjugate transpose, and $∥\mathbf{a}∥$ denotes its Euclidean norm. ; $\mathbf{X}⪰0$ indicates that matrix $\mathbf{X}$ is positive semidefinite; $\mathrm{rank}\left(\mathbf{X}\right)$ and $\mathrm{Tr}\left(\mathbf{X}\right)$ denote the rank and the trace of $\mathbf{X}$, respectively. ${∥\mathbf{X}∥}_{*}$, ${∥\mathbf{X}∥}_2$ are the nuclear norm and spectral norm of matrix $\mathbf{X}$, respectively. $\mathcal{CN}(\mu ,{\sigma }^2)$ denotes the distribution of a circularly symmetric complex Gaussian (CSCG) random variable with mean $\mu$ and variance ${\sigma }^2$. The Kronecker product operator is denoted by ⊗.

2. System Model and Problem Formulation

In this section, we begin by introducing the NOMA-motivated MIMO IACRS model along with the corresponding channel model. Then, we present the coordinate transformation mechanism to illustrate the relationship between the AC’s attitude and the beam pointing. Lastly, we provide the problem formulation.

2.1. System Description

As illustrated in Figure 1, an A2G MIMO integrated communication and radar system is considered. It has a dual-functional ground base station (GBS) and K ACs indexed by $k\in \mathcal{K}=\left\{1,...,K\right\}$. We assume that the communication system and the radar function are allocated on the same frequency band, which is similar to the case of the dedicated bands allocated to L-DACS and DME system. The dual-functional GBS is installed with a uniform planar array (UPA) comprising $M_G=M_{G,x}\times M_{G,y}$ transmit antennas. The k-th AC is also equipped with a UPA consisted of $M_A=M_{A,x}\times M_{A,y}$ antennas. Given the A2G multipath components, the digital BFs are employed at both GBS and ACs, so that the transmission of data stream for the k-th AC can be realized.

The dedicated radar sensing signal is assumed to be independent of the information symbols. Let $s_{c,k}$ and $s_{r,k}$ denote the transmitted communication signal and radar sensing signal intended for the k-th AC, respectively. Since the signals transmitted by the GBS in various time blocks have the same form, we focus on the t-th time block, which is consisted of T consecutive time slots, and the time-block index t is omitted in the following. The BF constructed for the k-th AC denoted by the transmit beamforming matrix ${\mathbf{w}}_k\in {\mathbb{C}}^{M_G\times 1}$. Therefore, the transmitted dual-functional signal $x\in {\mathbb{C}}^{M_G\times 1}$ consists of the data signal and sensing signal for all ACs, i.e.,

$$\begin{array}{cc}\hfill \mathbf{x}& =\sum _{k\in \mathcal{K}}{\mathbf{w}}_k(\sqrt{{\alpha }_{c,k}}{s}_{c,k}+\sqrt{{\alpha }_{r,k}}{s}_{r,k}),\forall k\in \mathcal{K},\hfill \end{array}$$

(1)

where ${\alpha }_{c,k}\ge 0$ and ${\alpha }_{r,k}\ge 0$ denote the power allocation factor of the communication and radar sensing signals of the k-th AC, respectively. Without loss of generality, we have ${\alpha }_{c,k}+{\alpha }_{r,k}=1$. Thus, the transmit power of the dual-functional GBS can be calculated as

$$\mathbb{E}\left(\mathbf{x}{\mathbf{x}}^H\right)=\sum _{k\in \mathcal{K}}\mathrm{Tr}\left({\mathbf{w}}_k{\mathbf{w}}_k^H\right)\le P_G,$$

(2)

where $P_G$ is the transmit power constraint for GBS. Different from existing research contributions that multiple communication users served by NOMA [7], NOMA-motivated technology is employed at the GBS to support both communication and sensing function for the same AC in this work. Considering the high-power signal characteristic of DME system with information-embedded, we set the fixed SIC order, as known that the SIC ordering has no influence on the high Signal-to-Noise Ratio (SNR) slope of the ergodic communication rate and the sensing rate [29]. Specifically, the AC first deals with the radar signal by treating the communication signal as interference. The estimated radar signal is then subtracted from received signal with the remaining portion utilized for demodulating and decoding the communication signal. In the following, the radar sensing model and communication model of the proposed system will be introduced.

1) MIMO Radar Sensing Model: Unlike primary radar systems that measure the orientation and distance of targets using the detected reflections of radio signals [30], the DME system relies on targets equipped with a radar transponder. It responds to each interrogation signal by transmitting encoded data containing the requested information. Hence, the transmit beam pattern used for typical radar detection is not a feasible metric considering the operating mode of DME. For the radar sensing function with interrogation mode, we define the SCNR as the key metric. The SCNR represents the radar system’s detection range and sensitivity, which directly influence the probability of detection. It is determined by the gain of the line-of-sight (LoS) path and the situation of clutter. The AC received the superimposed radar sensing signal is given by

$$\begin{array}{c}\hfill {\mathbf{y}}_{c,k}=\underset{\mathrm{AC}\mathrm{target}}{\underbrace{{\mathbf{v}}_k^H{\beta }_0{\mathbf{a}}_{\mathrm{A}}\left({\theta }_{\mathrm{A},\mathrm{LoS}},{\varphi }_{\mathrm{A},\mathrm{LoS}}\right){\mathbf{a}}_{\mathrm{G}}^H\left({\theta }_{\mathrm{G},\mathrm{LoS}},{\varphi }_{\mathrm{G},\mathrm{LoS}}\right)\mathbf{x}}}+\underset{\mathrm{clutter}}{\underbrace{{\mathbf{v}}_k^H\mathbf{c}}}+\underset{\mathrm{noise}}{\underbrace{{\mathbf{v}}_k^H{\mathbf{z}}_{r,k}}},\end{array}$$

(3)

where ${\mathbf{v}}_k\in {\mathcal{C}}^{M_A\times 1}$ represents the receive beamforming matrix of the k-th AC adhering to the power constraint $\mathrm{Tr}\left({\mathbf{v}}_k{\mathbf{v}}_k^H\right)\le P_{A,k},\forall k\in \mathcal{K}$, ${\beta }_0$ denotes the complex amplitude of the LoS, and ${\mathbf{z}}_{r,k}\in {\mathcal{C}}^{M_A\times 1}$ stands for the additive white Gaussian noise (AWGN) satisfying $\mathcal{CN}(0,{\mathbf{I}}_{r,k})$. The clutter, represented by $\mathbf{c}\in {\mathcal{C}}^{M_A\times 1}$, follows $\mathbf{c}\sim \mathcal{CN}(0,{\mathbf{R}}_{\mathbf{c}})$ including the non-line-of-sight (NLoS). It can generally be modeled as either signal-independent or signal-dependent. In the case of signal-independent clutter, the covariance matrix ${\mathbf{R}}_c$ is assumed to remain constant. Conversely, for the signal-dependent clutter, it can be can be modeled as $\mathbf{c}={\sum }_{l=1}^L{\beta }_l{\mathbf{a}}_{\mathrm{A}}\left({\theta }_{\mathrm{A},l},{\varphi }_{\mathrm{A},l}\right){\mathbf{a}}_{\mathrm{G}}^H\left({\theta }_{\mathrm{G},l},{\varphi }_{\mathrm{G},l}\right)\mathbf{x}={\sum }_{l=1}^L{\beta }_l{\mathbf{A}}_{\mathrm{GA},l}\mathbf{x}$. The steering vectors at the GBS and AC are represented by ${\mathbf{a}}_{\mathrm{G}}\left({\theta }_{\mathrm{G},·},{\varphi }_{\mathrm{G},·}\right)\in {\mathbb{C}}^{M_{\mathrm{G}}\times 1}$ and ${\mathbf{a}}_{\mathrm{A}}\left({\theta }_{\mathrm{A},·},{\varphi }_{\mathrm{A},·}\right)\in {\mathbb{C}}^{M_{\mathrm{A}}\times 1}$, respectively. Detailed expressions are provided in Section 2.3. The variables ${\theta }_{\mathrm{G},·}$ and ${\varphi }_{\mathrm{G},·}$ (${\theta }_{\mathrm{A},·}$ and ${\varphi }_{\mathrm{A},·}$) denote the azimuth and elevation components of the AoD at GBS (AoA at AC) for the LoS and NLoS paths, respectively. Given $\mathbf{x}$ in (1), the corresponding covariance matrix for the clutter can be calculated as

$${\mathbf{R}}_{\mathbf{c}}=\sum _{k=1}^K\sum _{l=1}^L\left({\beta }_l{\mathbf{A}}_{\mathrm{GA},l}{\mathbf{w}}_k\right){\left({\beta }_l{\mathbf{A}}_{\mathrm{GA},l}{\mathbf{w}}_k\right)}^H=\sum _{k=1}^K\sum _{l=1}^L{\beta }_l^2{\mathbf{A}}_{\mathrm{GA},l}{\mathbf{w}}_k{\mathbf{w}}_k^H{\mathbf{A}}_{\mathrm{GA},l}^H.$$

(4)

By employing classical iterative methods [31], ${\mathbf{R}}_c$ can be considered constant when the BF matrix keeps fixed from the previous iteration. Consequently, in the subsequent discussion, we will assume that the clutter covariance matrix remains constant during the single iteration. From (3), we set ${\mathbf{A}}_{\mathrm{GA},0}={\mathbf{a}}_{\mathrm{A}}\left({\theta }_{\mathrm{A},\mathrm{LoS}},{\varphi }_{\mathrm{A},\mathrm{LoS}}\right){\mathbf{a}}_{\mathrm{G}}^H\left({\theta }_{\mathrm{G},\mathrm{LoS}},{\varphi }_{\mathrm{G},\mathrm{LoS}}\right)$ as LoS component, the sensing SCNR for the radar signal at the k-th AC is given as

$$\begin{array}{c}\hfill {\tau }_k^r={\displaystyle \frac{{\alpha }_{r,k}{\left|{\mathbf{v}}_k^H{\beta }_0{\mathbf{A}}_{\mathrm{GA},0}{\mathbf{w}}_k\right|}^2}{{\alpha }_{c,k}{\left|{\mathbf{v}}_k^H{\beta }_0{\mathbf{A}}_{\mathrm{GA},0}{\mathbf{w}}_k\right|}^2+{\left|{\mathbf{v}}_k^H(\mathbf{c}+{\mathbf{z}}_{r,k})\right|}^2}}.\end{array}$$

(5)

2) MIMO Communication Model: Given the transmit signal $\mathbf{x}$ in (1), the received communication signal of the k-th AC is expressed as

$$\begin{array}{cc}\hfill {\mathbf{y}}_{c,k}& =\underset{\mathrm{desired}\mathrm{communication}\mathrm{signal}}{\underbrace{{\mathbf{v}}_k^H{\mathbf{h}}_{\mathrm{GA},k}{\mathbf{w}}_k\sqrt{{\alpha }_{c,k}}{s}_{c,k}}}+\underset{\mathrm{inter}-\mathrm{function}\mathrm{sensing}\mathrm{interference}\mathrm{removed}\mathrm{by}\mathrm{SIC}}{\underbrace{{\mathbf{v}}_k^H{\mathbf{h}}_{\mathrm{GA},k}{\mathbf{w}}_k\sqrt{{\alpha }_{r,k}}{s}_{r,k}}}\hfill \\ \hfill & +\underset{\mathrm{inter}-\mathrm{aircraft}\mathrm{interference}\mathrm{not}\mathrm{eliminate}}{\underbrace{{\mathbf{v}}_k^H{\mathbf{h}}_{\mathrm{GA},k}\sum _{i\ne k}^K{\mathbf{w}}_k(\sqrt{{\alpha }_{c,i}}{s}_{c,i}+\sqrt{{\alpha }_{r,i}}{s}_{r,i})}}+{\mathbf{n}}_{c,k},\hfill \end{array}$$

(6)

where ${\mathbf{h}}_{\mathrm{GA},k}\in {\mathbb{C}}^{K_{\mathrm{A}}\times K_{\mathrm{G}}}$ denotes the spatial-frequency domain GBS-AC channel matrix of the k-th AC, ${\mathbf{n}}_{c,k}\sim \mathcal{CN}(0,{\sigma }_c^2{\mathbf{I}}_{c,k})$. Inspired by NOMA technology, the inter-function interference will be eliminated by exploiting SIC, while treating the signal from other ACs as interference. As a result, the achievable rate of communication message at the k-th AC after SIC is given as

$$\begin{array}{c}\hfill R_k^c={log}_2(1+{\displaystyle \frac{{\alpha }_{c,k}{\left|{\mathbf{v}}_k^H{\mathbf{h}}_{\mathrm{GA},k}{\mathbf{w}}_k\right|}^2}{{\sum }_{i\ne k}^K{\left|{\mathbf{v}}_k^H{\mathbf{h}}_{\mathrm{GA},k}{\mathbf{w}}_i\right|}^2+{\sigma }_c^2{\left|{\mathbf{v}}_k\right|}^2}}).\end{array}$$

(7)

Therefore, the communication throughput for all of the ACs is represented by achievable sum rate. In this study, it is assumed that the channel state information (CSI) can be perfectly estimated to evaluate the maximum performance gain of the proposed IACRS framework. The Doppler shift is a significant factor in aeronautical scenarios, while developing a robust design that accounts for imperfect CSI is beyond the scope of the current work and will be addressed in future research.

2.2. Channel Model

As there are very limited dominant scatterers near to both the GBS and AC in the high-altitude A2G propagation surroundings, we adopt a geometric channel model with one LoS path and L NLoS paths. Assuming that the CSI remains unchanged within each time block1, ${\mathbf{h}}_{\mathrm{GA},k}$ in the t-th time block can be expressed as Rician model,

$$\begin{array}{c}\hfill {\mathbf{h}}_{\mathrm{GA},k}=\sqrt{{\displaystyle \frac{\kappa }{1+\kappa }}}{\mathbf{h}}_{\mathrm{GA},k}^{\mathrm{LoS}}+\sqrt{{\displaystyle \frac{1}{1+\kappa }}}{\mathbf{h}}_{\mathrm{GA},k}^{\mathrm{NLoS}},\forall k\in \mathcal{K},\end{array}$$

(8)

where $\kappa$ denotes the Rician factor, which quantifies the energy ratio between the LoS and NLoS paths. ${\mathbf{h}}_{\mathrm{GA},k}^{\mathrm{LoS}}$ is LoS component of k-th AC and it can be expressed as

$$\begin{array}{c}\hfill {\mathbf{h}}_{\mathrm{GA},k}^{\mathrm{LoS}}={\beta }_{\mathrm{LoS}}{\mathbf{a}}_{\mathrm{A}}\left({\theta }_{\mathrm{A},\mathrm{LoS}},{\varphi }_{\mathrm{A},\mathrm{LoS}}\right){\mathbf{a}}_{\mathrm{G}}^H\left({\theta }_{\mathrm{G},\mathrm{LoS}},{\varphi }_{\mathrm{G},\mathrm{LoS}}\right).\end{array}$$

(9)

Similar to the channel state of the LoS path ${\mathbf{h}}_{\mathrm{GA},k}^{\mathrm{LoS}}$, ${\mathbf{h}}_{\mathrm{GA},k}^{\mathrm{NLoS}}$ is the set as NLoS component,

$$\begin{array}{c}\hfill {\mathbf{h}}_{\mathrm{GA},k}^{\mathrm{NLoS}}=\sqrt{{\displaystyle \frac{1}{L}}}\sum _{l=1}^L{\beta }_l{\mathbf{a}}_{\mathrm{A}}\left({\theta }_{\mathrm{A},l},{\varphi }_{\mathrm{A},l}\right){\mathbf{a}}_{\mathrm{G}}^H\left({\theta }_{\mathrm{G},l},{\varphi }_{\mathrm{G},l}\right),\end{array}$$

(10)

where L is the number of NLoS rays. Additionally, within channel matrix (9) and (10), ${\beta }_{\mathrm{LoS}}$ and ${\beta }_l$ represent the complex-valued gain of the LoS path and NLoS path, respectively.

2.3. Coordinate Transformation

The steering directions at the GBS hinge on the AC’s position, while those at the AC side are also influenced by its attitude. To simplify the mathematical notations, we focus on deriving the AoD (denoted by ${\theta }_{\mathrm{G},\mathrm{LoS}}$ and ${\varphi }_{\mathrm{G},\mathrm{LoS}}$) and AoA (denoted by ${\theta }_{\mathrm{A},\mathrm{LoS}}$ and ${\varphi }_{\mathrm{A},\mathrm{LoS}}$) of the LoS path, while the AoDs and AoAs of the NLoS paths can be calculated in the similar manner. To appropriately describe the relationship between the AoA (AoD) and the AC’s navigation information, we introduce the following reference frames, which are all illustrated in Figure 2.

• The GBS-geodetic coordinate frame (g-frame): Its origin is chosen as the center of gravity of the GBS-UPA, and its axis $x_{\mathrm{g}}$ and $y_{\mathrm{g}}$ are aligned with the directions of east and north, respectively. The axis $z_{\mathrm{g}}$ is perpendicular to the ground surface pointing upwards, thus completing a right-handed coordinate frame. We assume that the row and column of the GBS-UPA are aligned with the axes $x_{\mathrm{g}}$ and $y_{\mathrm{g}}$, respectively.
• The AC-body coordinate frame (b-frame): Its origin is the AC center of gravity (ACCG). The axes $x_{\mathrm{b}}$, $y_{\mathrm{b}}$ and $z_{\mathrm{b}}$ are aligned with its longitudinal (forward), lateral (right), and vertical (downward) direction respectively, which are parallel to $x_{\mathrm{g}}$, $y_{\mathrm{g}}$ and $z_{\mathrm{g}}$, respectively.
• The inertial reference frame (i-frame): Its origin coincides with the ACCG, and its axes $x_{\mathrm{i}}$ (roll axis) and $y_{\mathrm{i}}$ (pitch axis) point out along the direction of the AC’s head and starboard wing, respectively. And its axis $z_{\mathrm{i}}$ (yaw axis) points downward, completing a right-handed coordinate frame.
• The AC-UPA coordinate frame (u-frame): Its origin is chosen as the AC-UPA center of gravity. The axes $x_{\mathrm{u}}$ and $y_{\mathrm{u}}$ are aligned with the row and column of the AC-UPA, respectively. The axis $z_{\mathrm{u}}$ is perpendicular to the plane spanned by the axes $x_{\mathrm{u}}$ and $y_{\mathrm{u}}$. We assume that the axes $x_{\mathrm{u}}$, $y_{\mathrm{u}}$ and $z_{\mathrm{u}}$ are parallel to $x_{\mathrm{i}}$, $y_{\mathrm{i}}$ and $z_{\mathrm{i}}$, respectively. On the basis of the relationship of u-frame and i-frame, the u-frame would be consistent with the i-frame when the attitude of AC changes.

In the g-frame, the GBS’s position is fixed at coordinate origin, while the AC’s position in the t-th time slot is denoted by ${\mathbf{p}}_{\mathrm{g}}=\left(x_{\mathrm{g}},y_{\mathrm{g}},z_{\mathrm{g}}\right)$. On the other hand, the AC’s attitude in the t-th time slot is described by its Euler angles $\mathbf{E}=\left(\psi ,\theta ,\varphi \right)$ in the i-frame. The angles $\psi ,\theta$ and $\varphi$ are referred to as the roll, pitch, and yaw angles, respectively. Hence, the i-frame is consistent with the b-frame when the AC’s attitude is $\mathbf{E}=(0,0,0)$. Note that the Euler angles are given with a specific rotation order: first around the roll axis, then the pitch axis, and finally the yaw axis. The coordinate transformation matrix ${\mathbf{C}}_{\mathrm{b}}^{\mathrm{i}}$ expresses the relationship between the b-frame and the i-frame, which can be realized by three successive spatial rotation:

$${\mathbf{T}}_1\left(\psi \right)=\left[\begin{array}{ccc}cos\psi & sin\psi & 0\\ -sin\psi & cos\psi & 0\\ 0& 0& 1\end{array}\right],{\mathbf{T}}_2\left(\theta \right)=\left[\begin{array}{ccc}cos\theta & 0& -sin\theta \\ 0& 1& 0\\ sin\theta & 0& cos\theta \end{array}\right],{\mathbf{T}}_3\left(\varphi \right)=\left[\begin{array}{ccc}1& 0& 0\\ 0& cos\varphi & sin\varphi \\ 0& -sin\varphi & cos\varphi \end{array}\right].$$

(11)

The order of the rotations is critical to the orientation of the b-frame [24]. Thus, ${\mathbf{C}}_{\mathrm{b}}^{\mathrm{i}}$ can be mathematically expressed as

$${\mathbf{C}}_{\mathrm{b}}^{\mathrm{i}}={\mathbf{T}}_3\left(\varphi \right){\mathbf{T}}_2\left(\theta \right){\mathbf{T}}_1\left(\psi \right)=\left[\begin{array}{ccc}cos\theta cos\psi & cos\theta sin\psi & -sin\theta \\ sin\varphi sin\theta cos\psi -cos\varphi sin\psi & sin\varphi sin\theta sin\psi +cos\varphi cos\psi & sin\varphi cos\theta \\ cos\varphi sin\theta cos\psi +sin\varphi sin\psi & cos\varphi sin\theta sin\psi -sin\varphi cos\psi & cos\varphi cos\theta \end{array}\right].$$

(12)

Since the GBS and AC positions, as well as the AC attitude, are defined in different coordinate frames, we introduce the concept of AC’s EP in the g-frame to make it tractable for the derivation of the steering vectors. The definition of EP is detailed in Appendix A. Given the ${\tilde{\mathbf{p}}}_{\mathrm{g}}$ and the corresponding AoA, the steering vector of the LoS path at the AC is expressed as

$${\mathbf{a}}_{\mathrm{A}}\left({\theta }_{\mathrm{A},\mathrm{LoS}},{\varphi }_{\mathrm{A},\mathrm{LoS}}\right)={\mathbf{a}}_{\mathrm{A},x}\left({\mu }_{\mathrm{A},\mathrm{LoS}}\right)\otimes {\mathbf{a}}_{\mathrm{A},y}\left({\nu }_{\mathrm{A},\mathrm{LoS}}\right),$$

(13)

where ${\mu }_{\mathrm{A},\mathrm{LoS}}={\displaystyle \frac{2\pi d}{{\lambda }_{\mathrm{c}}}}cos\left({\theta }_{\mathrm{A},\mathrm{LoS}}\right)sin\left({\varphi }_{\mathrm{A},\mathrm{LoS}}\right)$ and ${\nu }_{\mathrm{A},\mathrm{LoS}}={\displaystyle \frac{2\pi d}{{\lambda }_{\mathrm{c}}}}sin\left({\theta }_{\mathrm{A},\mathrm{LoS}}\right)sin\left({\varphi }_{\mathrm{A},\mathrm{LoS}}\right)$ represent the spatial frequencies along the axes $x_{\mathrm{u}}$ and $y_{\mathrm{u}}$, respectively, with the antenna spacing of d and the carrier wavelength of ${\lambda }_{\mathrm{c}}$. The steering vectors ${\mathbf{a}}_{\mathrm{A},x}\left({\mu }_{\mathrm{A},\mathrm{LoS}}\right)$ and ${\mathbf{a}}_{\mathrm{A},y}\left({\nu }_{\mathrm{A},\mathrm{LoS}}\right)$, along the axes $x_{\mathrm{u}}$ and $y_{\mathrm{u}}$ respectively, are given by

$${\mathbf{a}}_{\mathrm{A},x}\left({\mu }_{\mathrm{A},\mathrm{LoS}}\right)=\left[\begin{array}{c}1,e^{j{\mu }_{\mathrm{A},\mathrm{LoS}}},...,e^{j(M_{\mathrm{A}}-1){\mu }_{\mathrm{A},\mathrm{LoS}}}\end{array}\right],$$

(14a)

$${\mathbf{a}}_{\mathrm{A},y}\left({\nu }_{\mathrm{A},\mathrm{LoS}}\right)=\left[\begin{array}{c}1,e^{j{\nu }_{\mathrm{A},\mathrm{LoS}}},...,e^{j(N_{\mathrm{A}}-1){\nu }_{\mathrm{A},\mathrm{LoS}}}\end{array}\right].$$

(14b)

Similarly, the steering vectors of the NLoS paths at the AC are given by ${\mathbf{a}}_{\mathrm{A}}\left({\theta }_{\mathrm{A},l},{\varphi }_{\mathrm{A},l}\right)={\mathbf{a}}_{\mathrm{A},x}\left({\mu }_{\mathrm{A},l}\right)\otimes {\mathbf{a}}_{\mathrm{A},y}\left({\nu }_{\mathrm{A},l}\right)$, where the AoAs are respectively replaced by the AoAs of the NLoS paths for the calculations of ${\mu }_{\mathrm{A},l}$ and ${\nu }_{\mathrm{A},l}$. Furthermore, the steering vectors of the LoS and NLoS paths at the GBS are given by

$${\mathbf{a}}_{\mathrm{G}}\left({\theta }_{\mathrm{G},\mathrm{LoS}},{\varphi }_{\mathrm{G},\mathrm{LoS}}\right)={\mathbf{a}}_{\mathrm{G},x}\left({\mu }_{\mathrm{G},\mathrm{LoS}}\right)\otimes {\mathbf{a}}_{\mathrm{G},y}\left({\nu }_{\mathrm{G},\mathrm{LoS}}\right),$$

(15a)

$${\mathbf{a}}_{\mathrm{G}}\left({\theta }_{\mathrm{G},l},{\varphi }_{\mathrm{G},l}\right)={\mathbf{a}}_{\mathrm{G},x}\left({\mu }_{\mathrm{G},l}\right)\otimes {\mathbf{a}}_{\mathrm{G},y}\left({\nu }_{\mathrm{G},l}\right),$$

(15b)

where ${\mathbf{a}}_{\mathrm{G},x}\left({\mu }_{\mathrm{G},\mathrm{LoS}}\right)$ and ${\mathbf{a}}_{\mathrm{G},x}\left({\mu }_{\mathrm{G},l}\right)$, as well as ${\mathbf{a}}_{\mathrm{G},y}\left({\mu }_{\mathrm{G},\mathrm{LoS}}\right)$ and ${\mathbf{a}}_{\mathrm{G},y}\left({\mu }_{\mathrm{G},l}\right)$ are the steering vectors along the axes $x_{\mathrm{g}}$ and $y_{\mathrm{g}}$, respectively.

2.4. Problem Formulation

As discussed in the previous subsection, two conflicting objectives must be addressed in the NOMA-motivated MIMO dual-function IACRS model: maximizing communication throughput and enhancing radar detection capability. To achieve a balance between communication and radar performance, an integrated waveform design is essential. This design should allocate spatial and power resources across the different propagation paths, ensuring that both the communication and radar functions meet their respective performance requirements. Specifically, the goal is to maximize the weighted sum of the achievable sum rate and the sensing SCNR for the system, subject to constraints that include the minimum communication rate for each AC, radar-specific requirements, and the total transmit power budget. The resultant optimization problem is formulated as

$$\underset{\{{\mathbf{w}}_k,{\mathbf{v}}_k,{\alpha }_{c,k},{\alpha }_{r,k}\}}{max}f(R_k^c,{\tau }_k^r)={\rho }_c\sum _{k=1}^K{R}_k^c+{\rho }_r\sum _{k=1}^K{\tau }_k^r,$$

(16a)

$$\mathrm{s}.\mathrm{t}.R_k^c\ge {\overline{R}}_k,\forall k\in \mathcal{K},$$

(16b)

$${\tau }_k^r\ge {\overline{\tau }}_k,\forall k\in \mathcal{K},$$

(16c)

$$\begin{array}{c}\sum _{k\in \mathcal{K}}\mathrm{Tr}\left({\mathbf{w}}_k{\mathbf{w}}_k^H\right)\le P_G,\forall k\in \mathcal{K},\end{array}$$

(16d)

$$\begin{array}{c}\mathrm{Tr}\left({\mathbf{v}}_k{\mathbf{v}}_k^H\right)=P_{A,k},\forall k\in \mathcal{K}\end{array}$$

(16e)

$${\alpha }_{c,k}+{\alpha }_{r,k}=1,\forall k\in \mathcal{K},$$

(16f)

$${\alpha }_{c,k}\ge 0,{\alpha }_{r,k}\ge 0,\forall k\in \mathcal{K},$$

(16g)

where ${\rho }_c\ge 0$ and ${\rho }_r\ge 0$ represent the regularization parameters, ${\overline{R}}_k$ denotes the minimum required communication data rate for the k-th AC. The constraint (16c) defines the minimum required sensing SCNR, with ${\overline{\tau }}_k$ representing the threshold value for the k-th AC. Additionally, (16d) and (16e) impose the maximum power constraints on the dual-functional GBS and each AC, respectively. The power allocation constraints, given by (16f) and (16g), are essential to ensure the implementation of SIC. The problem (16) is non-convex, arising from the non-convex objective function and the non-convex constraints in (16b) and (16c). These constraints, highly coupled with the transmit BFs of GBS $\left\{{\mathbf{w}}_k\right\}$, power allocation factors $\{{\alpha }_{c,k},{\alpha }_{r,k}\}$ and the receiver BFs of ACs, $\left\{{\mathbf{v}}_k\right\}$, present significant challenges. Consequently, finding the globally optimal solution is non-trivial. In the following, we propose an AO-based algorithm to find a high-quality near optimal solution.

3. AO-Based Alternative Optimization Algorithm

To tackle the above problems, we propose an AO-based algorithm, which decomposes the original optimization problem into two subproblems: the transmit BFs of GBS combined with power allocation and the receiver BFs of ACs. Each group of variables is iteratively optimized while keeping the others fixed, resulting in an iterative optimization process. The main challenge to solve problem (16) arises from the fact that both the objective function and the left-hand-side (LHS) are non-concave with respect to the optimization variables. To address the issue, the penalty-based approach and the SROCR method are applied for addressing each subproblem iteratively.

3.1. GBS Transmit BFs Design and Power Allocation

We first solve the optimization problem with joint optimizing ${\mathbf{w}}_k$, ${\alpha }_{c,k}$ and ${\alpha }_{r,k}$ for given ${\mathbf{v}}_k$. The auxiliary varibles are defined for the covariance matrix of transmit beamforming matrix ${\mathbf{W}}_k\triangleq {\mathbf{w}}_k{\mathbf{w}}_k^H,\forall k\in \mathcal{K}$, where ${\mathbf{W}}_k⪰0$, $\mathrm{rank}\left({\mathbf{W}}_k\right)=1$, and the one for the receiver beamforming matrix ${\mathbf{V}}_k\triangleq {\mathbf{v}}_k{\mathbf{v}}_k^H,\forall k\in \mathcal{K}$, where ${\mathbf{V}}_k⪰0$, $\mathrm{rank}\left({\mathbf{V}}_k\right)=1$. Further, we have equivalent channel ${\tilde{\mathbf{h}}}_{\mathrm{GA},k}={\mathbf{v}}_k^H{\mathbf{h}}_{\mathrm{GA},k}$. Then, the porblem (16) can be reformulated as

$$\underset{\{{\mathbf{W}}_k,{\alpha }_{c,k},{\alpha }_{r,k}\}}{max}{\rho }_c\sum _{k=1}^K{R}_k^c+{\rho }_r\sum _{k=1}^K{\tau }_k^r$$

(17a)

$$\mathrm{s}.\mathrm{t}.{\alpha }_{c,k}\mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_k\right)-{\Gamma }_{c,k}{N}_k\ge 0,\forall k\in \mathcal{K},$$

(17b)

$${\alpha }_{r,k}-{\overline{\tau }}_k{\displaystyle \frac{{\mathbf{v}}_k^H{\tilde{\mathbf{R}}}_{\mathbf{c}}{\mathbf{v}}_k}{\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)}}-{\overline{\tau }}_k{\alpha }_{c,k}\ge 0,\forall k\in \mathcal{K},$$

(17c)

$$\mathrm{rank}\left({\mathbf{W}}_k\right)=1,{\mathbf{W}}_k⪰0,\forall k\in \mathcal{K},$$

(17d)

$$\left(16\mathrm{d}\right)\sim \left(16\mathrm{g}\right),$$

(17e)

where ${\tilde{\mathbf{H}}}_{\mathrm{GA},k}\triangleq {\tilde{\mathbf{h}}}_{\mathrm{GA},k}{\tilde{\mathbf{h}}}_{\mathrm{GA},k}^H$, ${\Gamma }_{c,k}=2^{{\overline{R}}_k}-1$, $N_k={\sum }_{i\ne k}^K{\left|{\mathbf{v}}_k^H{\mathbf{h}}_{\mathrm{GA},k}{\mathbf{w}}_i\right|}^2+{\sigma }_c^2{\left|{\mathbf{v}}_k\right|}^2$, ${\tilde{\mathbf{R}}}_{\mathbf{c}}={\mathbf{R}}_{\mathbf{c}}+\mathbf{I}$ and ${\Lambda }_{\mathbf{v},k}={\beta }_0^2{\mathbf{A}}_{\mathrm{GA},0}^H{\mathbf{V}}_k{\mathbf{A}}_{\mathrm{GA},0}$. Now, the reformulated problem (17) is still non-convex due to the objective function and the non-convex constraints (17c) and (17d). In the following, we first address the non-convex objective function and the constraint (17c). To facilitate this, three auxiliary variables are introduced such that

$${\varpi }_{r,k}^2={\mathbf{v}}_k^H{\tilde{\mathbf{R}}}_{\mathbf{c}}{\mathbf{v}}_k=\mathrm{Tr}\left({\mathbf{V}}_k{\tilde{\mathbf{R}}}_{\mathbf{c}}\right),\forall k\in \mathcal{K},$$

(18)

$${\mathcal{J}}_{c,k}={\displaystyle \frac{{\alpha }_{c,k}\mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_k\right)}{{\sum }_{i\ne k}^K\mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_i\right)+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k\right)}},\forall k\in \mathcal{K},$$

(19)

$${\mathcal{L}}_{r,k}={\displaystyle \frac{{\alpha }_{r,k}\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)}{{\alpha }_{c,k}\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)+{\varpi }_k^2}}.$$

(20)

Subsequently, problem (17) can be equivalently reformulated as

$$\underset{\{{\mathbf{W}}_k,{\alpha }_{c,k},{\alpha }_{r,k}\}}{max}{\rho }_c\sum _{k=1}^K{log}_2(1+{\mathcal{J}}_{c,k})+{\rho }_r\sum _{k=1}^K{\mathcal{L}}_{r,k}$$

(21a)

$$\mathrm{s}.\mathrm{t}.{\alpha }_{r,k}-{\displaystyle \frac{{\overline{\tau }}_k{\varpi }_k^2}{\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)}}-{\overline{\tau }}_k{\alpha }_{c,k}\ge 0,\forall k\in \mathcal{K},$$

(21b)

$${\varpi }_{r,k}^2\ge {\mathbf{v}}_k^H{\tilde{\mathbf{R}}}_{\mathbf{c}}{\mathbf{v}}_k,\forall k\in \mathcal{K},$$

(21c)

$${\mathcal{J}}_{c,k}\le {\displaystyle \frac{{\alpha }_{c,k}\mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_k\right)}{{\sum }_{i\ne k}^K\mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_i\right)+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k\right)}},\forall k\in \mathcal{K},$$

(21d)

$${\mathcal{L}}_{r,k}\le {\displaystyle \frac{{\alpha }_{r,k}\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)}{{\alpha }_{c,k}\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)+{\varpi }_{r,k}^2}},$$

(21e)

$$\left(16\mathrm{d}\right)\sim \left(16\mathrm{g}\right),\left(17\mathrm{b}\right),\left(17\mathrm{d}\right).$$

(21f)

This is because, at the optimal solution of (21), it can be obviously verified that constraints (21c), (21d) and (21e) are always satisfied with equality. To illustrate, consider that at the optimal solution of (21), the value of $\mathcal{J}c,k$ can be increased to ensure that constraint (21d) is met with strict equality if it was initially satisfied with a strict inequality. This also increases the value of the objective function. We can see the same result when ${\mathcal{L}}_{r,k}$ increased for the constraint (21d). Furthermore, if the constraint (21c) is strictly satisfied, ${\varpi }_k$ can be decreased for guaranteeing the strict equality, while simultaneously increasing the value of the objective function. Hence, problem (21) is equivalent to problem (17).

The objective function in (21) is concave with respect to $\mathcal{J}c,k$, and the second term in the LHS of (21b) is concave jointly with respect to ${\varpi }_k^2$ and $\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)$. Moreover, the constraints (21c), (21d) and (21e) are non-convex regarding their respective optimization variables. To handle these, we introduce another two auxiliary variables ${\mathcal{F}}_{c,k}$ and ${\mathcal{G}}_{r,k}$ as

$${\mathcal{J}}_{c,k}\begin{array}{c}(\sum _{i\ne k}^K\mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_i\right)+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k\right))\end{array}\le {\mathcal{F}}_{c,k}^2\le {\alpha }_{c,k}\mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_k\right),$$

(22)

$${\mathcal{L}}_{r,k}({\alpha }_{c,k}\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)+{\varpi }_k^2)\le {\mathcal{G}}_{r,k}^2\le {\alpha }_{r,k}\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right).$$

(23)

Next, we deal with the constraint (22), which can be equivalently transformed into the following two constraints:

$$\begin{array}{c}\hfill \begin{array}{c}\sum _{i\ne k}^K\mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_i\right)+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k\right)\end{array}\le {\displaystyle \frac{{\mathcal{F}}_{c,k}^2}{{\mathcal{J}}_{c,k}}},\end{array}$$

(24a)

$${\displaystyle \frac{{\mathcal{F}}_{c,k}^2}{{\alpha }_{c,k}}}\le \mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_k\right).$$

(24b)

It is evident that the constraint (24a) is non-convex due to the non-concavity of right-hand-side (RHS), whereas the constraint (24b) is convex. However, the RHS of (24a) is convex function joint with respect to ${\mathcal{F}}_{c,k}$ and ${\mathcal{J}}_{c,k}$. A lower bound of the RHS of (24a) at the $n_1$-th iteration of SCA can be derived by applying the first-order Taylor expression for any given feasible points, $\{{\mathcal{F}}_{c,k}^{\left(n_1\right)},{\mathcal{J}}_{c,k}^{\left(n_1\right)}\}$, shown as

$$\begin{array}{cc}\hfill {\displaystyle \frac{{\mathcal{F}}_{c,k}^2}{{\mathcal{J}}_{c,k}}}& \ge {\displaystyle \frac{{\left({\mathcal{F}}_{c,k}^{\left(n_1\right)}\right)}^2}{{\mathcal{J}}_{c,k}^{\left(n_1\right)}}}+{\displaystyle \frac{2{\mathcal{F}}_{c,k}^{\left(n_1\right)}}{{\mathcal{J}}_{c,k}^{\left(n_1\right)}}}({\mathcal{F}}_{c,k}-{\mathcal{F}}_{c,k}^{\left(n_1\right)})-{\displaystyle \frac{{\left({\mathcal{F}}_{c,k}^{\left(n_1\right)}\right)}^2}{{\left({\mathcal{J}}_{c,k}^{\left(n_1\right)}\right)}^2}}({\mathcal{J}}_{c,k}-{\mathcal{J}}_{c,k}^{\left(n_1\right)})\hfill \\ \hfill & ={\displaystyle \frac{2{\mathcal{F}}_{c,k}^{\left(n_1\right)}}{{\mathcal{J}}_{c,k}^{\left(n_1\right)}}}{\mathcal{F}}_{c,k}-{\displaystyle \frac{{\left({\mathcal{F}}_{c,k}^{\left(n_1\right)}\right)}^2}{{\left({\mathcal{J}}_{c,k}^{\left(n_1\right)}\right)}^2}}{\mathcal{J}}_{c,k}\triangleq {\rm Y}({\mathcal{F}}_{c,k},{\mathcal{J}}_{c,k}),\forall k\in \mathcal{K}.\hfill \end{array}$$

(25)

In a similar way, the constraint (23) also can be equivalently transformed as

$${\alpha }_{c,k}\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)+{\varpi }_k^2\le {\displaystyle \frac{{\mathcal{G}}_{r,k}^2}{{\mathcal{L}}_{r,k}}},$$

(26a)

$${\displaystyle \frac{{\mathcal{G}}_{r,k}^2}{{\alpha }_{r,k}}}\le \mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right).$$

(26b)

The constraint (26b) is obviously convex. Based on the define of ${\mathbf{R}}_{\mathbf{c}}$ in (4), the constraint (26a) can be obtained as

$$\begin{array}{c}\hfill {\alpha }_{c,k}\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)+\mathrm{Tr}\left({\mathbf{V}}_k{\mathbf{R}}_{\mathbf{c}}\right)+\mathrm{Tr}\left({\mathbf{V}}_k\right)\le {\displaystyle \frac{{\mathcal{G}}_{r,k}^2}{{\mathcal{L}}_{r,k}}}.\end{array}$$

(27)

The (26a) is non-convex as its RHS is not concave, while the RHS is convex function joint with respect to ${\mathcal{G}}_{r,k}$ and ${\mathcal{L}}_{r,k}$. Then, the approximation for the RHS of (26a) can be bounded using the first-order Taylor expansion at feasible points $\{{\mathcal{G}}_{r,k}^{\left(n_1\right)},{\mathcal{L}}_{r,k}^{\left(n_1\right)}\}$ as

$$\begin{array}{cc}\hfill {\displaystyle \frac{{\mathcal{G}}_{r,k}^2}{{\mathcal{L}}_{r,k}}}& \ge {\displaystyle \frac{2{\mathcal{G}}_{r,k}^{\left(n_1\right)}}{{\mathcal{L}}_{r,k}^{\left(n_1\right)}}}{\mathcal{G}}_{r,k}-{\displaystyle \frac{{\left({\mathcal{G}}_{r,k}^{\left(n_1\right)}\right)}^2}{{\left({\mathcal{L}}_{r,k}^{\left(n_1\right)}\right)}^2}}{\mathcal{L}}_{r,k}\triangleq {\rm Y}({\mathcal{G}}_{r,k},{\mathcal{L}}_{r,k}),\forall k\in \mathcal{K}.\hfill \end{array}$$

(28)

For the non-convex constraint (21c), a lower bound can be obtained by the first-order Taylor expression as

$$\begin{array}{c}\hfill {\varpi }_{r,k}^2\ge {\left({\varpi }_{r,k}^{\left(n_1\right)}\right)}^2+2{\varpi }_{r,k}^{\left(n_1\right)}({\varpi }_{r,k}-{\varpi }_{r,k}^{\left(n_1\right)})\triangleq \mathsf{\Omega }\left({\varpi }_{r,k}\right),\end{array}$$

(29)

where ${\varpi }_{r,k}^{\left(n_1\right)}$ is the feasible point at the $n_1$-th iteration. The LHS of (29) is a convex function with respect to ${\varpi }_{r,k}$. Therefore, we have the following reformulated optimization problem:

$$\underset{\mathcal{X}}{max}f({\mathcal{J}}_{c,k},{\mathcal{L}}_{r,k})$$

(30a)

$$\mathrm{s}.\mathrm{t}.\mathsf{\Omega }\left({\varpi }_{r,k}\right)\ge {\mathbf{v}}_k^H{\tilde{\mathbf{R}}}_{\mathbf{c}}{\mathbf{v}}_k,\forall k\in \mathcal{K},$$

(30b)

$$\begin{array}{c}\hfill \begin{array}{c}\sum _{i\ne k}^K\mathrm{Tr}\left({\tilde{\mathbf{H}}}_{\mathrm{GA},k}{\mathbf{W}}_i\right)+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k\right)\end{array}\le {\rm Y}({\mathcal{F}}_{c,k},{\mathcal{J}}_{c,k}),\forall k\in \mathcal{K},\end{array}$$

(30c)

$${\alpha }_{c,k}\mathrm{Tr}\left({\mathbf{W}}_k{\Lambda }_{\mathbf{v},k}\right)+{\varpi }_k^2\le {\rm Y}({\mathcal{G}}_{r,k},{\mathcal{L}}_{r,k}),\forall k\in \mathcal{K},$$

(30d)

$$\left(16\mathrm{d}\right)\sim \left(16\mathrm{g}\right),\left(17\mathrm{b}\right),\left(17\mathrm{d}\right),\left(21\mathrm{b}\right),\left(24\mathrm{b}\right),\left(26\mathrm{b}\right),$$

(30e)

where $\mathcal{X}\triangleq \{{\mathbf{W}}_k,{\alpha }_{c,k},{\alpha }_{r,k},{\varpi }_{r,k},{\mathcal{J}}_{c,k},{\mathcal{L}}_{r,k},{\mathcal{F}}_{c,k},{\mathcal{G}}_{r,k}\}$ and $f({\mathcal{J}}_{c,k},{\mathcal{L}}_{r,k})$ replaces the original objective function. The non-convexity of this optimization problem arises primarily from the rank-one constraint (17d). A widely adopted approach to tackle this problem is to obtain the semidefinite relaxation (SDR) of the reformulated problem which omits the rank-one constraints By removing the rank-one constraints, SDR could could efficiently obtain a global optimum through convex solvers like CVX. If the obtained solution does not satisfy the rank-one condition, the rank-one solution can be reconstructed through the Gaussian randomization method. The main advantage of SDR is its relatively low computational complexity, as the relaxed problem typically needs to be solved only once. However, the solution derived from the relaxed problem is not guaranteed to be the global optimum of the original problem  (16). One drawback of SDR is that the significant performance degradation may occur due to the reconstruction. The process of reconstructing a rank-one solution can lead to significant performance degradation. Moreover, there is no assurance that the reconstructed rank-one solution will always satisfy all of the constraints for the original problem, potentially resulting in an infeasible solution.

To avoid the drawback, we endeavour to transform the rank-one constraints into penalty terms incorporated within the objective function. This reformulated result can be effectively addressed using SCA considering both solution optimality and feasibility. Building on this concept, the non-convex rank-one constraint (17d) is equivalent to the equality constraint

$$\parallel {\mathbf{W}}_k{\parallel }_{*}-{\parallel {\mathbf{W}}_k\parallel }_2=0,\forall k\in \mathcal{K},$$

(31)

where ${\Vert ·\Vert }_{*}$ and ${\Vert ·\Vert }_2$ denote the nuclear norm and spectral norm of the matrix, respectively. It can be readily verified that, if the covariance matrix is of rank-one, the above equality constraint (31) is certainly satisfied for any ${\mathbf{W}}_k$ satisfying the constrain (17d). Otherwise, the inequality $\parallel {\mathbf{W}}_k{\parallel }_{*}-\parallel {\mathbf{W}}_k{\parallel }_2>0$ must be hold if the matrices are not rank-one. Thus, the penalty term for the rank-one equality constraint is introduced to the objective function of problem (16), yielding the following optimization problem:

$$\underset{\mathcal{X}}{max}f({\mathcal{J}}_{c,k},{\mathcal{L}}_{r,k})-{\displaystyle \frac{1}{\eta }}\sum _{k\in \mathcal{K}}(\parallel {\mathbf{W}}_k{\parallel }_{*}-\parallel {\mathbf{W}}_k{\parallel }_2),$$

(32a)

$$\mathrm{s}.\mathrm{t}.\left(16\mathrm{d}\right)\sim \left(16\mathrm{g}\right),\left(17\mathrm{b}\right),\left(17\mathrm{d}\right),\left(21\mathrm{b}\right),\left(24\mathrm{b}\right),\left(26\mathrm{b}\right),$$

(32b)

where $\eta$ is the regularization parameter as penalty factor. The violation of the (31) is penalized by the factor ${\displaystyle \frac{1}{\eta }}$, when ${\mathbf{W}}_k$ are not of rank-one. It is important to recognize that the selection of $\eta$ has a significant impact on the objective function. As $\eta$ tends towards 0, causing ${\displaystyle \frac{1}{\eta }}$ to approach ∞, the rank of matrix ${\mathbf{W}}_k$ will be definitely one. Nevertheless, in this case, the objective function will be primarily dominated by the penalty term, leading to a bad solution in terms of maximizing both communication throughput and radar detection capability. To tackle this, we can initialize $\eta$ with a sufficiently large value to obtain a good starting point for the optimization problem. Subsequently, $\eta$ can be gradually reduced to a sufficiently small value with the update rule $\eta =\epsilon \eta ,0<\epsilon <1$. As a result, feasible rank-one matrix solutions associated with a near-optimal performance can eventually be obtained. However, the spectral norm as the second term in the penalty introduces non-convexity to the objective function. To tackle this, we apply the first-order Taylor expansion around the feasible point ${\mathbf{W}}_k^{\left(n_1\right)}$, we could obtain its upper bound $-\parallel {\mathbf{W}}_k{\parallel }_2\le {\widehat{\mathbf{W}}}_k^{\left(n_1\right)}$ of is given by

$${\widehat{\mathbf{W}}}_k^{\left(n_1\right)}\triangleq -{\parallel {\mathbf{W}}_k^{\left(n_1\right)}\parallel }_2-\mathrm{Tr}\left[{\mathbf{u}}_{max,k}^{\left(n_1\right)}{\left({\mathbf{u}}_{max,k}^{\left(n_1\right)}\right)}^H({\mathbf{W}}_k-{\mathbf{W}}_k^{\left(n_1\right)})\right],$$

(33)

where ${\mathbf{u}}_{max,k}^{\left(n_1\right)}$ is the eigenvector corresponding to the largest eigenvalue of ${\mathbf{W}}_k^{\left(n_1\right)}$. Thus, the problrm (32) can be approximated as

$$\underset{\mathcal{X}}{min}-f({\mathcal{J}}_{c,k},{\mathcal{L}}_{r,k})+{\displaystyle \frac{1}{\eta }}\sum _{k\in \mathcal{K}}\left(\right(\parallel {\mathbf{W}}_k{\parallel }_{*}+{\widehat{\mathbf{W}}}_k^{\left(n_1\right)}),$$

(34a)

$$\mathrm{s}.\mathrm{t}.\left(16\mathrm{d}\right)\sim \left(16\mathrm{g}\right),\left(17\mathrm{b}\right),\left(17\mathrm{d}\right),\left(21\mathrm{b}\right),\left(24\mathrm{b}\right),\left(26\mathrm{b}\right).$$

(34b)

The problem (34) is a quadratic semidefinite program (QSDP), which can be efficiently addressed using CVX. The iterative procedure will terminate when the penalty term becomes sufficiently small within a predefined accuracy, thereby satisfying the condition ${\sum }_{k\in \mathcal{K}}(\parallel {\mathbf{W}}_k{\parallel }_{*}-\parallel {\mathbf{W}}_k{\parallel }_2)\le {ϵ}_0$. Algorithm 1 provides a comprehensive outline of the penalty-based procedure developed to address problem (17).

Algorithm 1 Penalty-based approach to solve the joint transmit BF design and power allocation subproblem (34)

1: Initialize feasible points $\{{\mathbf{W}}_k^0,{\alpha }_{c,k}^0,{\alpha }_{r,k}^0\}$ and the penalty parameter $\eta$. 2: repeat: outer loop 3:     Set iteration index $n=0$ for inner loop. 4:     repeat: inner loop 5:         Use  (18), (19), (20), (22) and (23) to calculate the current value $\{{\varpi }_{r,k}^{\left(n_1\right)},{\mathcal{J}}_{c,k}^{\left(n_1\right)},{\mathcal{L}}_{r,k}^{\left(n_1\right)},{\mathcal{F}}_{c,k}^{\left(n_1\right)},{\mathcal{G}}_{r,k}^{\left(n_1\right)}\}$. 6:         Solve the convex problem (34) with given feasible points, and the obtained optimal varibles are denoted by $\{{\mathbf{W}}_k^{*},{\alpha }_{c,k}^{*},{\alpha }_{r,k}^{*}\}$. 7:         $\{{\mathbf{W}}_k^{(n_1+1)},{\alpha }_{c,k}^{(n_1+1)},{\alpha }_{r,k}^{(n_1+1)}\}$ by the obtained optimal solutions and $n_1=n_1+1$. 8:     until the fractional for the objective function variation value lowers below a predefined threshold ${ϵ}_{\mathrm{inner}}>0$ 9:     Update feasible points $\{{\mathbf{W}}_k^0,{\alpha }_{c,k}^0,{\alpha }_{r,k}^0\}$ by the current optimal solutions. 10:     Update $\eta =\epsilon \eta$. 11: until the norm difference constraint is lower than the maximum tolerable threshold ${ϵ}_{\mathrm{outer}}>0$.

3.2. ACs Receiver BFs Design

To optimize ${\mathbf{v}}_k$ for given ${\mathbf{w}}_k$, ${\alpha }_{c,k}$ and ${\alpha }_{r,k}$, the auxiliary variable ${\tilde{\mathbf{g}}}_{\mathrm{GA},k}={\mathbf{h}}_{\mathrm{GA},k}{\mathbf{w}}_k$ is introduced, which is the equivalent channel when ${\mathbf{w}}_k$ is fixed. It can be observed that the objective function is the addition of a concave function and an affine function. In the following, we also deal with the non-convex objective function and tackle the rank-one constraint. First, we deal with the term for the communication throughput, which can be expressed as

$$\begin{array}{cc}\hfill R_k^c=& {log}_2(1+{\displaystyle \frac{{\alpha }_{c,k}\mathrm{Tr}\left({\mathbf{V}}_k{\tilde{\mathbf{G}}}_{\mathrm{GA},k}\right)}{{\sum }_{i\ne k}^K\mathrm{Tr}\left({\mathbf{V}}_k{\tilde{\mathbf{G}}}_{\mathrm{GA},i}\right)+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k\right)}})\hfill \\ \hfill =& {log}_2({\alpha }_{c,k}\mathrm{Tr}\left({\mathbf{V}}_k{\tilde{\mathbf{G}}}_{\mathrm{GA},k}\right)+\begin{array}{c}\sum _{i\ne k}^K\mathrm{Tr}\left({\mathbf{V}}_k{\tilde{\mathbf{G}}}_{\mathrm{GA},i}\right)\end{array}\hfill \\ \hfill & +{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k\right))\underset{J_k}{\underbrace{\begin{array}{c}-{log}_2({\sum }_{i\ne k}^K\mathrm{Tr}\left({\mathbf{V}}_k{\tilde{\mathbf{G}}}_{\mathrm{GA},i}\right)\end{array}+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k\right))}},\hfill \end{array}$$

(35)

where ${\tilde{\mathbf{G}}}_{\mathrm{GA},k}\triangleq {\tilde{\mathbf{g}}}_{\mathrm{GA},k}{\tilde{\mathbf{g}}}_{\mathrm{GA},k}^H$. The non-convexity of the objective function originates from the second term $J_k$. However, since the objective can be expressed as a difference of two concave functions, a concave lower bound can be established by applying the first-order Taylor expansion at specified feasible points $({\mathbf{V}}_1^{\left(n_2\right)},...,{\mathbf{V}}_K^{\left(n_2\right)})$ in the $n_2$-th iteration of the algorithm as follows:

$$\begin{array}{cc}\hfill J_k& \ge {\widehat{J}}_k\triangleq -{log}_2(\sum _{i\ne k}^K\mathrm{Tr}\left({\mathbf{V}}_k^{\left(n_2\right)}{\tilde{\mathbf{G}}}_{\mathrm{GA},i}\right)+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k^{\left(n_2\right)}\right))\hfill \\ \hfill & -{\displaystyle \frac{{\sum }_{i\ne k}^K\mathrm{Tr}\left(({\mathbf{V}}_k-{\mathbf{V}}_k^{\left(n_2\right)}){\tilde{\mathbf{G}}}_{\mathrm{GA},i}\right)+{\sigma }_c^2\mathrm{Tr}({\mathbf{V}}_k-{\mathbf{V}}_k^{\left(n_2\right)})}{({\sum }_{i\ne k}^K\mathrm{Tr}\left({\mathbf{V}}_k^{\left(n_2\right)}{\tilde{\mathbf{G}}}_{\mathrm{GA},i}\right)+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k^{\left(n_2\right)}\right))ln2}}.\hfill \end{array}$$

(36)

Then, we define

$${\widehat{R}}_k^c\triangleq {log}_2({\alpha }_{c,k}\mathrm{Tr}\left({\mathbf{V}}_k{\tilde{\mathbf{G}}}_{\mathrm{GA},k}\right)+\begin{array}{c}\sum _{i\ne k}^K\mathrm{Tr}\end{array}\left({\mathbf{V}}_k{\tilde{\mathbf{G}}}_{\mathrm{GA},i}\right)+{\sigma }_c^2\mathrm{Tr}\left({\mathbf{V}}_k\right))+{\widehat{J}}_k,$$

(37)

which establishes a lower bound for $R_k^c$. By leveraging it, the constraint (16b) can be transformed into a conservative approximation with ${\widehat{R}}_k^c\ge {\overline{R}}_k$. Next, we tackle the second term of the objective function, a lower bound is given by

$$\begin{array}{cc}\hfill {\tau }_k^r\ge {\widehat{\tau }}_k^r=& {log}_{10}\left({\alpha }_{c,k}{\Lambda }_{\mathbf{w},k}\mathrm{Tr}\left({\mathbf{V}}_k\right)\right)-{log}_{10}({\alpha }_{c,k}{\Lambda }_{\mathbf{w},k}\mathrm{Tr}\left({\mathbf{V}}_k^n\right)+\mathrm{Tr}\left({\mathbf{V}}_k^n{\tilde{\mathbf{R}}}_{\mathbf{c}}\right))\hfill \\ \hfill & -{\displaystyle \frac{{\alpha }_{c,k}{\Lambda }_{\mathbf{w},k}\mathrm{Tr}({\mathbf{V}}_k-{\mathbf{V}}_k^{\left(n_2\right)})+\mathrm{Tr}\left(({\mathbf{V}}_k-{\mathbf{V}}_k^{\left(n_2\right)}){\tilde{\mathbf{R}}}_{\mathbf{c}}\right)}{({\alpha }_{c,k}{\Lambda }_{\mathbf{w},k}\mathrm{Tr}\left({\mathbf{V}}_k^{\left(n_2\right)}\right)+\mathrm{Tr}\left({\mathbf{V}}_k^{\left(n_2\right)}{\tilde{\mathbf{R}}}_{\mathbf{c}}\right))ln10}},\hfill \end{array}$$

(38)

where ${\Lambda }_{\mathbf{w},k}={\beta }_0^2{\mathbf{A}}_{\mathrm{GA},0}{\mathbf{W}}_k{\mathbf{A}}_{\mathrm{GA},0}^H$. We can observe that ${\widehat{\tau }}_k^r$ is a convex function with respect to ${\mathbf{V}}_k$. Hence, problem (16) can be rewritten as follows:

$$\underset{{\mathbf{V}}_k}{max}f\left({\mathbf{V}}_k\right)={\rho }_c\sum _{k=1}^K{\widehat{R}}_k^c+{\rho }_r\sum _{k=1}^K{\widehat{\tau }}_k^r,$$

(39a)

$$\mathrm{s}.\mathrm{t}.{\widehat{R}}_k^c\ge {\overline{R}}_k,\forall k\in \mathcal{K},$$

(39b)

$$({\alpha }_{r,k}-{\Gamma }_{r,k}{\alpha }_{c,k}){\beta }_0^2\mathrm{Tr}\left({\Lambda }_{\mathbf{w},k}{\mathbf{V}}_k\right)-{\Gamma }_{r,k}\mathrm{Tr}\left({\mathbf{V}}_k{\tilde{\mathbf{R}}}_{\mathbf{c}}\right)\ge 0,$$

(39c)

$$\mathrm{rank}\left({\mathbf{V}}_k\right)=1,{\mathbf{V}}_k⪰0,\forall k\in \mathcal{K},$$

(39d)

$$\left(16\mathbf{d}\right)\sim \left(16\mathbf{g}\right),$$

(39e)

where ${\Gamma }_{r,k}={10}^{{\displaystyle \frac{{\overline{\tau }}_k}{10}}}$. To address the rank-one issue, we invoke the sequential rank-one constraint relaxation (SROCR) method while considering the computing power of avionic computers. The basic framework of the SROCR method is detailed in [32], demonstrating its ability to successively obtain a feasible rank-one solution with favorable complexity. To facilitate this method, the non-convex rank-one constraint (39d) is replaced with the following relaxed convex constraint:

$${\lambda }_{max}\left({\mathbf{V}}_k\right)\ge {\delta }^{\left(n_2\right)}\mathrm{Tr}\left({\mathbf{V}}_k\right),$$

(40)

where ${\lambda }_{max}\left({\mathbf{V}}_k\right)$ is the largest eigenvalue of ${\mathbf{V}}_k$ and ${\delta }^{\left(n_2\right)}\in [0,1]$ denotes a relaxation parameter which controls the largest eigenvalue to trace ratio of ${\mathbf{V}}_k$ in the $n_2$-th iteration of the SROCR algorithm, as the solution ${{\mathbf{V}}_k}^{*}$ is guaranteed to satisfy the following condition:

$${\delta }^{\left(n_2\right)}\le {\displaystyle \frac{{\lambda }_{max}\left({{\mathbf{V}}_k}^{*}\right)}{\mathrm{Tr}\left({{\mathbf{V}}_k}^{*}\right)}}\le 1.$$

(41)

Thus,  (40) will be equivalent to the SDR when ${\delta }^{\left(n_2\right)}=0$, which omits the non-convex rank-one constraint. This motivates us to incrementally increase ${\delta }^{\left(n_2\right)}$ from 0 in iterations, allowing the transformed constraint to gradually converge to the actual rank-one constraint set as ${\delta }^{\left(n_2\right)}$ approaches 1. As a result, problem (39) can be rewritten as the following relaxed optimization problem:

$$\underset{{\mathbf{V}}_k}{max}f\left({\mathbf{V}}_k\right),$$

(42a)

$$\mathrm{s}.\mathrm{t}.{\left({\mathbf{u}}_{max,k}^{\left(n_2\right)}\right)}^H{\mathbf{V}}_k{\mathbf{u}}_{max,k}^{\left(n_2\right)}\ge {\delta }^{\left(n_2\right)}\mathrm{Tr}\left({\mathbf{V}}_k\right),\forall k\in \mathcal{K},$$

(42b)

$$\left(16\mathrm{d}\right)\sim \left(16\mathrm{g}\right),\left(39\mathrm{b}\right)\sim \left(39\mathrm{c}\right),$$

(42c)

where ${\mathbf{u}}_{max,k}^{\left(n_2\right)}$ is the eigenvector corresponding to the largest eigenvalue of ${\mathbf{V}}_k^{\left(n_2\right)}$, which is the obtained solution feasible with ${\delta }^{\left(n_2\right)}$. Now, the problem (42) has been reformulated into a convex problem, which can be solved using optimization tools such as CVX [33]. The optimal value obtained from (42) serves as a lower bound for the original problem. This is because the non-convex components have been replaced with their respective lower bound approximations, ensuring that the solution reflects a conservative result of the original problem. After each iteration, relaxation parameter ${\delta }^{\left(n_2\right)}$ is updated as

$${\delta }^{\left(n_2\right)}=min(1,{\displaystyle \frac{{\lambda }_{max}\left({\mathbf{V}}_k^{\left(n_2\right)}\right)}{\mathrm{Tr}\left({\mathbf{V}}_k^{\left(n_2\right)}\right)}}+{\sigma }^{\left(n_2\right)}),$$

(43)

where ${\sigma }^{\left(n_2\right)}$ is a predefined step size. If the problem (42) is infeasible at the current step ${\sigma }^{\left(n_2\right)}$, we will reduce the step size as ${\sigma }^{(n_2+1)}={\sigma }^{\left(n_2\right)}/J$, where $J\ne 2$ in order to speed up making (42) solvable. Setting $f\left({\mathbf{V}}_k^{\left(n_2\right)}\right)$ as the objective function value achieved by solution ${\mathbf{V}}_k^{\left(n_2\right)}$, the algorithm will be terminated when $\left|f\left({\mathbf{V}}_k^{\left(n_2\right)}\right)-f\left({\mathbf{V}}_k^{(n_2-1)}\right)\right|\le {ϵ}_1$ and $\left|1-{\delta }^{(n_2-1)}\right|\le {ϵ}_2$ are simultaneously satisfied, where ${ϵ}_1$ and ${ϵ}_2$ are convergence thresholds. Hence, the proposed method can be assured to converge to a locally optimal solution with rank-one constraint [34] by iteratively updating parameter ${\delta }^{\left(n_2\right)}$ and solving problem (42). The details to solve problem (39) with the SROCR method are summarized in Algorithm 2.

Algorithm 2 SROCR method for solving the receiver BF design subproblem (42)

1: Initialize convergence thresholds ${ϵ}_1$ and ${ϵ}_2$, step size ${\sigma }^{\left(n_2\right)}$, relaxation parameter ${\delta }^{\left(n_2\right)}$, and $n_2=0$. 2: repeat 3:     For given $\{{\delta }^{\left(n_2\right)},{\mathbf{V}}_k^{\left(n_2\right)}\}$, solve the problem (42). 4:     if problem (42) is feasible then 5:         ${\mathbf{V}}_k^{(n_2+1)}={\mathbf{V}}_k$, ${\sigma }^{(n_2+1)}={\sigma }^{\left(0\right)}$. 6:     else 7:         ${\mathbf{V}}_k^{(n_2+1)}={\mathbf{V}}_k^{\left(n_2\right)}$, ${\sigma }^{(n_2+1)}={\sigma }^{\left(n_2\right)}/J$. 8:     end if 9:     $n_2=n_2+1$, 10:     update ${\delta }^{\left(n_2\right)}=min(1,{\displaystyle \frac{{\lambda }_{max}\left({\mathbf{V}}_k^{\left(n_2\right)}\right)}{\mathrm{Tr}\left({\mathbf{V}}_k^{\left(n_2\right)}\right)}}+{\sigma }^{\left(n_2\right)})$. 11: until$\left|f\left({\mathbf{V}}_k^{\left(n_2\right)}\right)-f\left({\mathbf{V}}_k^{(n_2-1)}\right)\right|\le {ϵ}_1$ and $\left|1-{\delta }^{(n_2-1)}\right|\le {ϵ}_2$. ${\mathbf{V}}_k^{*}={\mathbf{V}}_k^{\left(n_2\right)}$.

3.3. Complexity Analysis

Following the above discussions, the proposed AO-based alternative algorithm is utilized to address the original problem (16), which starts by initializing a feasible solution, including the aforementioned two subproblems. The main computational burden of proposed AO-based algorithm is primarily determined by the complexities of Algorithm 1 and Algorithm 2. Given the solution accurancy $\mathcal{E}$, the complexity Algorithm 1 is $\mathcal{O}(I_o{I}_i(K{M}_G^{3.5}+{\left(7K\right)}^{3.5})log(1/\mathcal{E}))$, where $I_i$, $I_o$ and $7K$ denote the number of inner and outer iterations required for convergence, and the number of scalar optimization variables, respectively. Considering the relaxed problem (42) being a standard SDP, Algorithm 2 exhibits a complexity of $\mathcal{O}(\left(K{M}_A^3\right)log(1/\mathcal{E}))$. Thus, the total computational complexity of proposed AO-based algorithm given by is $\mathcal{O}((I_o{I}_i(K{M}_G^{3.5}+{\left(7K\right)}^{3.5})+K{M}_A^3)log(1/\mathcal{E}))$.

4. Numerical Results

To evaluate the performance of the proposed IACRS, we present numerical results obtained from Monte Carlo simulations in this section. In particular, we assume that the GBS and ACs are all equipped with UPA with same number of elements $M_G=M_A=4\times 4$, which is of half-wavelength spacing between adjacent antennas. The number of aircraft is 2 in default scenario. The channels between GBS and ACs are assumed to experience Rician fading with the path loss of $L_k=L_0+20{log}_{10}\left(d_k\right)\left(\mathrm{dB}\right)$, where $L_0$ is the path loss at the reference distance $d=1$ meter (m), and $d_k$ represents the distance from the GBS [20]. The parameter settings are setting as ${\overline{R}}_k=$ 0.5 bit/s/Hz, ${\overline{\tau }}_k=$ 1 dB and $P_G=P_A\left(k\right)=$ 30 dBm, $\forall k\in \mathcal{K}$ unless specified otherwise. The initial penalty factor of Algorithm 1 is set to $\eta ={10}^5$, and the convergence thresholds of inner and outer loops are set to ${ϵ}_{\mathrm{inner}}=1.001$ and ${ϵ}_{\mathrm{outer}}={10}^{-5}$. The initial step size of Algorithm 2 is set to ${\sigma }^{\left(0\right)}=0.1$, and the convergence thresholds for difference in objective function and relaxation parameter is set to ${ϵ}_1={10}^{-3}$ and ${ϵ}_2={10}^{-5}$, respectively. All numerical results are obtained using 300 Monte-Carlo simulations with random channel realizations. For performance comparison, we consider the following benchmark schemes.

• TDMA-based dual-function scheme: In this scheme, the GBS with multi-antenna successively transmits dedicated messages and interrogation detecting singal to ACs over $2K$ time slots employing one common BF. Accordingly, for TDMA-based scheme, the sensing SCNR at the k-th AC is given by

  $$\begin{array}{c}\hfill {\tau }_k^{\mathrm{TDMA}}={\displaystyle \frac{1}{2K}}{\displaystyle \frac{{\left|{\mathbf{v}}_k^H{\beta }_0{\mathbf{A}}_{\mathrm{GA},0}{\mathbf{w}}_k\right|}^2}{{\left|{\mathbf{v}}_k^H(\mathbf{c}+{\mathbf{z}}_{r,k})\right|}^2}},\end{array}$$

  (44)

  the corresponding achievable rate for the k-th AC is

  $$\begin{array}{c}\hfill R_k^{\mathrm{TDMA}}={\displaystyle \frac{1}{2K}}{log}_2(1+{\displaystyle \frac{{\left|{\mathbf{v}}_k^H{\mathbf{h}}_{\mathrm{GA},k}{\mathbf{w}}_k\right|}^2}{{\sigma }_c^2{\left|{\mathbf{v}}_k\right|}^2}}).\end{array}$$

  (45)
  The problem of maximizing sum of ${\tau }_k^{\mathrm{TDMA}}$ and $R_k^{\mathrm{TDMA}}$ can be solved by using SCA algorithm, as there are no inter-aircraft or inter-function interference terms involved.
• MIMO-based dual-function scheme: In this scheme, the MIMO-only GBS transmits communication data to the ACs employing distinct BFs, which are also simultaneously utilized for the detection of the ACs. Notably, each AC directly receives its intended signal while treating the signals for other ACs as interference without the assistance of SIC, which means a low level of integration. Therefore, the sensing SCNR and the sum rate for the transmitted signal at the AC are similar with (3) and (6), respectively.
• MRT/MRC-based dual-function scheme: In this scheme, the system model is same as the proposed integrated framework, while using a linear transmitter and receiver, maximum-ratio transmission/maximum-ratio combining (MRT/MRC)[35]. Hence, the approximate closed-form expressions of the transmit BFs of GBS and the receiver BFs of ACs can be obtained.

In Figure 3, the convergence behavior of the proposed algorithm is studied with varying values of the reduction factor $ϵ$. The numerical results indicate that the objective function rapidly converges to a stable value for any values of the reduction factor, while the rank-one term approaches nearly zero after a few iterations. This behavior confirms the convergence of the proposed algorithm. It demonstrates that the algorithm effectively identifies a feasible rank-one solution with high performance, applicable to both GBS transmitter and airborne receiver. Furthermore, the results reveal that as the parameter $ϵ$ decreases, the algorithm achieves faster convergence speed but results in lower objective values, which corresponds to reduced system performance. This highlights a trade-off between convergence speed and system performance.

In Figure 4, we use the equivalent norm difference, $\parallel {\mathbf{W}}_k{\parallel }_{*}-\parallel {\mathbf{W}}_k{\parallel }_2,\parallel {\mathbf{V}}_k{\parallel }_{*}-{\parallel {\mathbf{V}}_k\parallel }_2,\forall k\in \mathcal{K}$, as evaluation metric to assess results of both the penalty-based approach and the SROCR method for the rank-one constraint. It is also observed that the value of $\sigma$ has little or no effect on the rank-one term, so we only focus on selecting $ϵ$ in Figure 3. Additionally, as $ϵ$ decreases, the proposed algorithm exhibits faster convergence but leads to a lower objective value, i.e., worse system performance, reflecting a trade-off. Therefore, in the following simulations, we set $ϵ=0.1$, which provides a balance between convergence speed and system performance.

We evaluate the impact of the AC’s number on the proposed IACRS framework in Figure 5. It is shown that the weighted sum of the achievable sum rate and sensing SCNR in the proposed scheme increases consistently during the iterations, regardless of the number of ACs. Specifically, the number of iterations increases as the number of ACs grows. For instance, when $K=2$, the objective value gradually rises over approximately 7 iterations before converging to a stable value. However, when the number of ACs is increased to $K=5$, the number of iterations rises to around 12, due to the expanded search space resulting from the increased number of ACs.

In Figure 6, we first present the obtained achievable sum rate versus the transmit power $P_G$ in Figure 6a. It can be observed that the achievable sum rate of all schemes increases with increasing transmit power. Furthermore, the performance enhancement attained by NOMA-motivated framework upon increasing $P_G$ is more significant than that of TDMA-based scheme, since NOMA benefits from a flexible resource allocation scheme enabling the double spectrum sharing. In Figure 6b, we also present the sensing SCNR obtained by the four schemes. It can be observed that the proposed scheme outperforms the other baseline scheme especially at high transmit power. The SCNR gain achieved with effective attitude information, compared to position-only information, becomes more pronounced as transmit power increases. However, in contrast to TDMA-based and proposed scheme, the sensing SCNR achieved by the MIMO-based scheme seems to be bounded by a certain value, with its enhancement becoming negligible. This is because, in the absence of interference mitigation, the MIMO-based scheme becomes interference-limited. These results demonstrate that the proposed NOMA-motivated MIMO IACRS scheme is particularly suitable for aeronautical A2G communication in environments with saturated dedicated spectrum resources, while providing high-quality sensing SCNR for multiple aircraft detection in secondary radar applications.

Further investigation of the achievable sum rate and average SCNR versus the number of ACs is given in Figure 7. We consider different scenrios for AC’s number as $K=\{2,3,4,5,6\}$, with their azimuth positions given by the sets $[0,{\displaystyle \frac{\pi }{2}}]$, $[0,{\displaystyle \frac{\pi }{2}},\pi ]$, $[0,{\displaystyle \frac{\pi }{2}},\pi ,{\displaystyle \frac{3\pi }{2}}]$, $[0,{\displaystyle \frac{\pi }{2}},\pi ,{\displaystyle \frac{3\pi }{2}}]$, $[0,{\displaystyle \frac{\pi }{3}},{\displaystyle \frac{2\pi }{2}},\pi ,{\displaystyle \frac{4\pi }{3}}]$,$[0,{\displaystyle \frac{\pi }{3}},{\displaystyle \frac{2\pi }{2}},\pi ,{\displaystyle \frac{4\pi }{3}},{\displaystyle \frac{5\pi }{3}}]$, respectively. All ACs remain at an elevation angle of ${\displaystyle \frac{\pi }{3}}$, which is consistent with typical cruise situation. It can be observed that the sum rate of all schemes, except the TDMA-based scheme, first increases and then converges as K adds. This trend can be explained as follows: for small values of K, increasing the number of ACs provides more spatial degrees of freedom (DoFs), which can be exploited to maximize the sum rate. However, as K becomes larger, the azimuth angles between the ACs become smaller leading to increased channel correlation, which limits system performance.

It is also seen that the average SCNR of the MIMO-based scheme first increases (from 2 to 3 ACs) and then decreases (from 3 to 6 ACs) as K increases. The proposed scheme, utilizing the AO-based algorithm, is the only one for which the average SCNR consistently increases with the number of ACs. This result highlights that the MRT/MRC scheme does not benefit from the NOMA-motivated framework, thus leading to a degradation in radar sensing performance. Additionally, it is observed that TDMA-based scheme has a significantly lower average SCNR, which is due to the reduction in available time slots allocated to the sensing function of a single AC as the number of ACs increases.

In Figure 8, the simulation results highlight the trade-off between sensing SCNR and sum rate in proposed system under varying NLoS conditions. As the number of NLoS paths increases, SCNR initially improves but eventually degrades due to the growing clutter from excessive multipath components. In contrast, the sum rate generally benefits from the presence of NLoS paths, reflecting the system’s ability to exploit spatial diversity. However, this improvement saturates as the system approaches its capacity to effectively handle multipath propagation. Additionally, it is clear that increasing the number of UPA elements results in a noticeable performance gain. In summary, there exists a trade-off between achieving higher data rates and maintaining sensing signal quality, as more NLoS components increase capacity but also introduce greater interference. To optimize system performance, careful management of multipath effects is essential to mitigate clutter while maximizing the benefits of spatial diversity.

5. Conclusion and Discussion

This paper proposes a novel NOMA-motivated MIMO dual-function scheme for IACRS, where a MIMO GBS transmits NOMA-structured superimposed radar sensing signals and communication messages to the ACs achieving double spectrum sharing. For the proposed framework, corresponding BF optimization and power allocation problems are formulated for enhancing both communication and radar performance, while fulfilling the transmitted data rate and the sensing SCNR requirements. An AO-based algorithm is developed to find a near-optimal solution to solve the resultant non-convex optimization problem. The main findings of the research are summarized as follows:

• A new A2G MIMO channel model that accounts for AC dynamics, including both position and attitude, is introduced to accurately characterize the steering direction. Since the AoDs and AoAs depend on the time-varying position and attitude, the proposed channel model significantly improves system performance, both in the proposed scheme and the baseline.
• To validate the effectiveness of the proposed scheme, it is compared with three baselines. The numerical results demonstrate that the proposed scheme achieves superior dual-function system performance compared to the benchmark schemes. In contrast, the MRT/MRC scheme does not benefit from the NOMA-motivated framework, as it relies only on valid channel information, resulting in inferior communication performance.
• One limitation of our framework is that obtaining accurate CSI becomes challenging due to the time-varying nature of A2G wireless channels and AC mobility, which directly impacts system performance. Addressing imperfect CSI could be a direction for future work. Additionally, the SIC process requires significant computational resources, and in low SNR environments, its performance may degrade substantially, leading to erroneous signal decoding and severely affecting system stability and data transmission quality.