Service Provisioning and Path Planning with Obstacle Avoidance for Low-Altitude Wireless Networks Senning Wan, Bin Li, Hongbin Chen, Lei Liu Abstract This paper investigates the three-dimensional (3D) deployment of uncrewed aerial vehicles (UAVs) as aerial base stations in heterogeneous communication networks under constraints imposed by diverse ground obstacles. Given the diverse data demands of user devices (UDs), a user satisfaction model is developed to provide personalized services. In particular, when a UD is located within a ground obstacle, the UAV must approach the obstacle boundary to ensure reliable service quality. Considering constraints such as UAV failures due to battery depletion, heterogeneous UDs, and obstacles, we aim to maximize overall user satisfaction by jointly optimizing the 3D trajectories of UAVs, transmit beamforming vectors, and binary association indicators between UAVs and UDs. To address the complexity and dynamics of the problem, a block coordinate descent method is adopted to decompose it into two subproblems. The beamforming subproblem is efficiently addressed via a bisection-based water-filling algorithm. For the trajectory and association subproblem, we design a deep reinforcement learning algorithm based on proximal policy optimization to learn an adaptive control policy. Simulation results demonstrate that the proposed scheme outperforms baseline schemes in terms of convergence speed and overall system performance. Moreover, it achieves efficient association and accurate obstacle avoidance. I Introduction The upcoming sixth-generation (6G) wireless networks aim to provide massive connectivity and ultra-high-speed transmission for millions of interconnected devices [ 1 ] . However, the deployment of terrestrial infrastructure in sparsely populated areas, such as deserts, mountains, and forests, faces significant challenges due to environmental and economic constraints. Furthermore, terrestrial base stations (BSs) are often vulnerable to unexpected events caused by environmental factors, such as earthquakes, floods, and tsunamis. Therefore, establishing a reliable emergency communication system via temporary aerial networks has become a key objective in the evolution toward 6G [ 2 ] . Uncrewed aerial vehicles (UAVs), due to their high mobility and rapid deployment capabilities, have been extensively adopted. When functioning as aerial BSs, UAVs can dynamically adjust their trajectories according to actual environmental conditions and terrain features, thereby enhancing system robustness and communication efficiency [ 3 , 4 , 5 ] . In practical scenarios, UAV flight is frequently constrained by other UAVs and ground obstacles, which not only increases the complexity of collision avoidance but also imposes critical challenges on communication quality [ 6 , 7 ] . In this work, we consider the three-dimensional (3D) trajectory optimization of UAVs. Specifically, for ground obstacles, UAVs are allowed to either bypass them laterally or fly over them to ensure safe flight. Moreover, considering that user devices (UDs) have heterogeneous requirements for data rates, we develop a user satisfaction model to characterize the performance of communication services. Given the aforementioned considerations, the UAV flight trajectories and communication decisions must be carefully designed to maximize user satisfaction. I-A Previous Work According to Shannon’s capacity formula, proper trajectory planning can significantly reduce the communication distance between UDs and UAVs, thereby enabling the fulfillment of high data rate requirements [ 8 , 9 ] . For example, Lu et al. [ 8 ] studied the joint optimization of trajectory planning and communication design for multiple UAV-BSs, aiming to maximize the fairness of throughput among the ground users (GUs) requiring service. Shang et al. [ 9 ] employed a double deep Q-network (DDQN) to derive reward parameters based on fairness and throughput, which interact with the dynamic channel environment to continuously update UAV positions and maximize the accumulated reward. In addition, proper resource allocation, such as UAV transmit power control, significantly affects system performance and user service quality [ 10 , 11 ] . For instance, Li et al. [ 10 ] investigated a secure communication system assisted by reconfigurable intelligent surfaces (RIS), where the UAV trajectory, RIS passive beamforming, and transmit power of legitimate transmitters were jointly optimized to maximize the average worst-case secrecy rate. In [ 11 ] , Liu et al. proposed a joint design of transmit beamforming and UAV trajectory for enhanced communications and radar sensing tasks. During UAV flight, although trajectory design considers practical factors such as UAV speed, initial/final positions, and energy efficiency, collision avoidance is crucial for ensuring UAV survivability and operational safety in low-altitude environments with mixed obstacles. Therefore, careful trajectory planning is necessary [ 12 , 13 , 14 , 15 , 16 , 17 ] . Singla et al. [ 12 ] utilized onboard image collection devices and neural networks to extract visual features and estimate distances between UAVs and obstacles, enabling effective obstacle avoidance. Zheng et al. [ 13 ] investigated the 3D UAV trajectory planning problem in data collection systems, aiming to collect data in minimal time while avoiding spatial obstacles. Similarly, Gao et al. [ 14 ] considered ground terminal mobility, 3D building avoidance, and the practical assumption that UAVs can fly between buildings to minimize the total time cost. To achieve this objective, a multi-step dueling DDQN method was adopted, allowing the agent to interact with the environment and iteratively refine its motion policy. However, the aforementioned works mainly focus on single-UAV scenarios, while obstacle avoidance in multi-UAV environments remains underexplored. Gao et al. [ 15 ] proposed an energy-efficient velocity control algorithm for massive UAV networks based on mean field game theory, jointly addressing energy consumption, channel capacity, and obstacle avoidance. In [ 16 ] , Wang et al. investigated efficient and accurate solutions for path-following, obstacle sensing, and avoidance subtasks, as well as their conflict-free fusion and scheduling. Additionally, Gao et al. [ 17 ] proposed a potential game-based multi-agent deep deterministic policy gradient (DDPG) method to optimize obstacle-avoidance trajectories for multiple UAVs. In summary, research on 3D obstacle avoidance for multi-UAV scenarios remains limited and warrants further investigation. TABLE I: Differences in Our Scheme Compared to the Most Relevant Schemes Schemes Heterogeneous User Multiple obstacles Energy limitation 3D trajectory User satisfaction Learning Methodology [ 9 ] – – ✓ – – DRL [ 13 ] – – ✓ – – BCD [ 14 ] – – – ✓ – DRL [ 18 ] ✓ – ✓ – ✓ BCD [ 19 ] – – – ✓ ✓ Interference-Aware Deployment [ 20 ] – – ✓ – ✓ BCD Our Scheme ✓ ✓ ✓ ✓ ✓ BCD+DRL In conventional UAV-assisted communication systems, the primary objective is typically to maximize system throughput. However, next-generation UAV communication networks are expected to be user-centric. Under the premise of heterogeneous and personalized user demands, another crucial performance metric is the user demand satisfaction gain, commonly referred to as user satisfaction. This metric is determined by the extent to which diverse throughput requirements are fulfilled. Depending on the type of requested data and individual user preferences, users may perceive different levels of satisfaction even under identical throughput conditions [ 18 , 19 , 20 , 21 ] . In [ 18 ] , Tian et al. jointly considered task processing delay and energy efficiency, constructed a novel user satisfaction model, and formulated a total satisfaction maximization problem to optimize both offloading decisions and UAV scheduling strategies. Similarly, Lai et al. [ 19 ] proposed an interference-aware deployment algorithm to reduce co-channel interference among UAV-BSs, thereby improving user satisfaction and ensuring that most GUs meet their minimum data rate requirements. In [ 20 ] , Barick et al. utilized the cooperative capabilities of UAVs and ground edge servers to maximize user satisfaction, ultimately enhancing the profit of the service provider. Moreover, Zhang et al. [ 21 ] developed a deep reinforcement learning (DRL)-based self-regulating approach to maximize the cumulative user satisfaction score during dynamic UAV network operations, such as UAV arrivals and departures. The distinctions between our proposed scheme and the relevant existing schemes are summarized in Table I . I-B Motivations and Contributions Based on the above studies, several critical challenges remain unsolved, including meeting personalized service demands of UDs, accurately analyzing and modeling ground obstacles of diverse shapes, achieving efficient 3D obstacle avoidance for UAVs, and effectively handling UAV battery depletion. In light of the aforementioned challenges, the main contributions of this study are summarized as follows: • We consider a heterogeneous communication scenario in a mixed-obstacle environment, where UAVs serve as aerial BSs to assist ground wireless communications. UAV trajectory planning is influenced by ground obstacles with diverse geometries. When UDs are located within obstacle-covered regions, UAVs must approach the obstacles to ensure stable service. To enhance system robustness, UAV failures due to battery depletion are also considered in the design of subsequent association decisions. Furthermore, UDs are classified into multiple urgency levels based on their service requirements, and a corresponding user satisfaction model is constructed. To optimize overall system performance, we jointly consider 3D trajectory planning, transmit beamforming, and association decisions between UAVs and UDs, with the objective of maximizing overall user satisfaction. • To address the resulting high-dimensional and non-convex optimization problem, a block coordinate descent (BCD) framework is adopted to decouple it into two tractable subproblems. For the beamforming subproblem, a closed-form solution is derived based on a bisection-based water-filling algorithm. Given the complexity of the subproblem involving trajectory planning and UAV-UD association strategies, a DRL approach based on the proximal policy optimization (PPO) algorithm is proposed to efficiently learn the optimal control policy. • We examine the complexity of the proposed algorithm and conduct extensive simulation experiments. The results show that, in all considered scenarios, the proposed scheme outperforms the benchmark scheme in terms of user satisfaction, robustness, and computational complexity. I-C Organization and Notation The remainder of this paper is organized as follows. Section II introduces the proposed UAV-assisted communication system model and formulates the optimization problem. Section III presents the proposed solution approach in detail. Section IV provides simulation results to validate the performance of the proposed algorithm. Finally, Section V concludes the paper. Notation: Scalars are denoted by italic letters x , vectors by bold lowercase letters \mathbf{x} , and matrices by bold uppercase letters \mathbf{X} . The set of all complex-valued matrices of size N\times M is denoted by \mathbb{C}^{N\times M} . For a complex vector \mathbf{a} , \|\mathbf{a}\| denotes its Euclidean norm, \mathbf{a}^{\mathrm{T}} denotes the transpose, and \mathbf{a}^{\mathrm{H}} denotes the Hermitian (conjugate transpose). The notation \succeq denotes the matrix inequality under the positive semidefinite cone. The expectation operator is denoted by \mathbb{E}[\cdot] . The operator \otimes represents the Kronecker product. The notation [\cdot]_{k,k} refers to the k -th diagonal element of a matrix. The operator [\cdot]^{+}=\max(\cdot,0) denotes the non-negative projection. II System Model In this section, we provide an overview of a low-altitude UAV-assisted communication scenario with mixed ground obstacles. We then present a detailed description of the scenario description, obstacle model, communication model, and user satisfaction model. II-A Scenario Description Fig. 1: System model. As shown in Fig. 1 , we consider a 3D UAV-assisted communication system operating in an environment with ground obstacles of various shapes. In this system, a set of N UAVs, each equipped with N_{\mathrm{r}} antennas, act as aerial BSs to provide wireless services for K single-antenna UDs. The sets of UAV and UD indices are denoted by \mathcal{N}=\{1,\dots,n,\dots N\} and \mathcal{K}=\{1,\dots,k,\dots K\} , respectively. The total mission duration T is uniformly divided into L non-overlapping intervals, each of duration \delta_{t}=\frac{T}{L} , to enable tractable modeling and optimization. The set of time slots is defined as \mathcal{T}=\{1,\dots,t,\dots,T\} . A 3D Cartesian coordinate system is adopted for system modeling. Specifically, the location of UAV n at time slot t is denoted by \mathbf{q}_{n}[t]=\left(x_{n}[t],y_{n}[t],z_{n}[t]\right)^{T} , while the location of UD k is fixed, and given by \mathbf{u}_{k}=\left(x_{k},y_{k},0\right)^{T} . The UAV operates within the feasible region \mathcal{B}=\{(x,y,z)\in\mathbb{R}^{3}\;\mid\;0\leq x\leq x_{\max},\;0\leq y\leq y_{\max},\;H_{\min}\leq z\leq H_{\max}\} , where x_{\max} and y_{\max} represent the horizontal flight boundaries, and H_{\min} and H_{\max} denote the minimum and maximum allowable UAV altitudes, respectively. Accordingly, the UAV flight positions must satisfy the following boundary constraints: \displaystyle\mathbf{q}_{n}[t]\in\mathcal{B},\forall n\in\mathcal{N}. (1) To ensure collision avoidance during communication tasks, a minimum safety distance d_{\min} is enforced between any two UAVs. Moreover, to guarantee trajectory feasibility and control accuracy, each UAV is subject to a maximum velocity v_{\max} and a maximum acceleration a_{\max} . Consequently, in each time slot, the UAV trajectories must satisfy the following constraints: \displaystyle\mathbf{q}_{n}[t+1] \displaystyle=\mathbf{q}_{n}[t]+\mathbf{v}_{n}[t]\,\delta_{t}+\frac{1}{2}\mathbf{a}_{n}[t]\,\delta_{t}^{2}, (2) \displaystyle\left\|\mathbf{q}_{n}[t]-\mathbf{q}_{n^{\prime}}[t]\right\|^{2} \displaystyle\geq d_{\min}^{2},\quad\forall n,n^{\prime}\in\mathcal{N},\;n\neq n^{\prime}, (3) \displaystyle\left\|\mathbf{v}_{n}[t]\right\| \displaystyle\leq v_{\max},\quad\forall t\in\mathcal{T}, (4) \displaystyle\left\|\mathbf{a}_{n}[t]\right\| \displaystyle\leq a_{\max},\quad\forall t\in\mathcal{T}. (5) The propulsion power consumption of UAV n during flight at time slot t is given by \displaystyle P_{n}^{\text{fly}}[t]= \displaystyle\frac{1}{2}d_{0}\rho_{0}g_{0}A_{1}\|\mathbf{v}_{n}[t]\|^{3}+P_{0}\left(1+\frac{3\|\mathbf{v}_{n}[t]\|^{2}}{U_{\text{tip}}^{2}}\right) \displaystyle+P_{1}\left(1+\frac{\|\mathbf{v}_{n}[t]\|^{4}}{4v_{0}^{4}}-\frac{\|\mathbf{v}_{n}[t]\|^{2}}{2v_{0}^{2}}\right)^{\frac{1}{2}}, (6) where P_{0} denotes the blade profile power, P_{1} is the induced power during hovering, and d_{0} is the drag coefficient. The parameters \rho_{0} and g represent the air density and rotor solidity, respectively. U_{\text{tip}} denotes the rotor blade tip speed, A_{1} is the rotor disk area, and v_{0} represents the mean rotor speed. Accordingly, the flight energy consumption of UAV n at time slot t is calculated by E_{n}^{\text{fly}}[t]=P_{n}^{\text{fly}}[t]\cdot\delta_{t}. (7) II-B Obstacle Model (a) Cylindrical obstacles. (b) Rectangular obstacle. Fig. 2: The obstacle shape is projected on the x-y axis. In low-altitude environments, UAVs are prohibited from entering obstacle regions due to safety considerations. In other words, UAV trajectories must be constrained to avoid collisions with obstacles. In this work, we optimize the 3D trajectory of UAVs in the presence of ground obstacles with various shapes. Let the obstacle heights be denoted by a constant H_{\text{obs}} , which satisfies the constraint H_{\min}H_{\text{obs}} , it is permitted to fly over the obstacle, and the horizontal position constraints can be ignored. Conversely, if z_{n}[t]\leq H_{\text{obs}} , the UAV must satisfy the following horizontal constraints: 1) First, we consider a simplified scenario where cylindrical obstacles are projected onto the x - y plane, as illustrated in Fig. 2 (a). Assume that there are I cylindrical obstacles, where the center and radius of obstacle i are denoted by \mathbf{q}_{\text{obs}}^{i}=\left(x_{\text{obs}}^{i},y_{\text{obs}}^{i}\right)^{\mathrm{T}} and R_{\text{obs}}^{i} , respectively. The UAV trajectory constraint to avoid collision with these cylindrical obstacles is formulated as \left\|\tilde{\mathbf{q}}_{n}[t]-\mathbf{q}_{\text{obs}}^{i}\right\|^{2}\geq\left(R_{\text{obs}}^{i}\right)^{2},\forall n\in\mathcal{N},\;i\in\mathcal{I}, (8) where \tilde{\mathbf{q}}_{n}[t] denotes the UAV’s horizontal position, defined as \tilde{\mathbf{q}}_{n}[t]=\left(x_{n}[t],y_{n}[t]\right)^{T} . 2) To generalize the model, we also consider rectangular obstacles that are more common in practical environments, as shown in Fig. 2 (b). Suppose there are J rectangular obstacles, with the center and side lengths of obstacle j denoted by \mathbf{q}_{\text{obs}}^{j}=\left(x_{\text{obs}}^{j},y_{\text{obs}}^{j}\right)^{T} and \mathbf{d}_{\text{obs}}^{j}=\left(w_{\text{obs}}^{j},h_{\text{obs}}^{j}\right)^{T} , respectively. Then, the UAV’s trajectory constraint to avoid rectangular obstacles is given by \left|\tilde{\mathbf{q}}_{n}[t]-\mathbf{q}_{\text{obs}}^{j}\right|\succeq{\mathbf{d}_{\text{obs}}^{j}},\forall n\in\mathcal{N},\;j\in\mathcal{J}. (9) II-C Communication Model To enable UAVs to better serve UDs, it is necessary to assign a reasonable UAV to each UD. We define the binary variable \alpha_{n,k}[t]\in\{0,1\} to indicate the association between UAV n and UD k . The UAV-UD association must satisfy the following constraints: \sum_{n=1}^{N}\alpha_{n,k}[t]=1,\quad\forall k\in\mathcal{K},\;t\in\mathcal{T}, (10) \sum_{k=1}^{K}\alpha_{n,k}[t]\leq K_{n},\quad\forall n\in\mathcal{N},\;t\in\mathcal{T}, (11) constraint ( 10 ) ensures that each UD k is associated with exactly one UAV, while constraint ( 11 ) ensures that each UAV n is associated with at most K_{n} UDs. Specifically, when E_{n}^{\text{res}}[t]=E_{n}^{\text{UAV}}-\sum_{s=1}^{t}E_{n}^{\text{fly}}[s]\leq 0 , the UAV is considered to be in a failure state, and we set \beta_{n}=0 ; otherwise, \beta_{n}=1 . If UAV n is available (i.e., \beta_{n}=1 ) and provides communication service to UD k at time slot t , then \alpha_{n,k}[t]=1 ; in all other cases, \alpha_{n,k}[t]=0 . In particular, if UAV n is initially assigned to UD k but becomes unavailable due to energy depletion (i.e., \beta_{n}=0 ), UD k will be reassigned to another candidate UAV n^{\prime} to maintain communication and avoid service interruption. In this case, we set \alpha_{n,k}[t]=0 and \alpha_{n^{\prime},k}[t]=1 . The candidate UAV n^{\prime} must satisfy two conditions: it must be energy-sufficient, i.e., E_{n^{\prime}}^{\text{res}}[t]>0 , and not overloaded, i.e., \sum_{k=1}^{K}\alpha_{n^{\prime},k}[t]\leq K_{n^{\prime}} . Due to the complexity of the environment, the communication link between UAV n and UD k is often obstructed by trees and buildings, leading to Rician fading. Therefore, the channel between UAV n and UD k at time slot t consists of a combination of line-of-sight (LoS) and non-line-of-sight (NLoS) components, and is expressed as \mathbf{h}_{n,k}[t]=\sqrt{\rho d_{n,k}^{-\beta}[t]}\left(\sqrt{\frac{\varsigma}{1+\varsigma}}\overline{\mathbf{h}}_{n,k}^{\mathrm{LoS}}[t]+\sqrt{\frac{1}{1+\varsigma}}\tilde{\mathbf{h}}_{n,k}^{\mathrm{NLoS}}[t]\right), (12) where d_{n,k}[t]=\sqrt{\left\|\mathbf{q}_{n}[t]-\mathbf{u}_{k}\right\|^{2}} denotes the Euclidean distance between UAV n and UD k at time slot t , \beta is the path loss exponent, and \varsigma represents the Rician factor. The LoS component \overline{\mathbf{h}}_{n,k}^{\mathrm{LoS}}[t]\in\mathbb{C}^{N_{\mathrm{r}}\times 1} is given by \displaystyle\overline{\mathbf{h}}_{n,k}^{\mathrm{LoS}}[t]=\ \displaystyle\left(1,\,\ldots,\,e^{-j\frac{2\pi bf_{c}}{c}\sin\varpi_{n,k}[t](a_{x}-1)\cos\theta_{n,k}[t]},\right. \displaystyle\left.\ldots,\,e^{-j\frac{2\pi bf_{c}}{c}\sin\varpi_{n,k}[t](A_{x}-1)\cos\theta_{n,k}[t]}\right) \displaystyle\otimes\left(1,\,\ldots,\,e^{-j\frac{2\pi bf_{c}}{c}\sin\varpi_{n,k}[t](a_{y}-1)\cos\theta_{n,k}[t]},\right. \displaystyle\left.\ldots,\,e^{-j\frac{2\pi bf_{c}}{c}\sin\varpi_{n,k}[t](A_{y}-1)\cos\theta_{n,k}[t]}\right), (13) where b denotes the antenna spacing factor, and c=\mathbf{v}_{n}[t] is the UAV flying speed. The parameter f_{c} represents the carrier frequency of the information signal. Let a_{x} and a_{y} denote the row and column indices of the uniform planar array, respectively. The horizontal and vertical angles of departure (AoDs) from UD k are defined as \theta_{n,k}[t] and \varpi_{n,k}[t] , respectively. Specifically, the AoDs are given by \displaystyle\theta_{n,k}[t] \displaystyle=\arccos\left(\frac{y_{n}[t]-y_{k}}{\left\|\mathbf{q}_{n}[t]-\mathbf{u}_{k}]\right\|}\right), (14) \displaystyle\varpi_{n,k}[t] \displaystyle=\arcsin\left(\frac{z_{n}[t]}{\sqrt{\left\|\mathbf{q}_{n}[t]-\mathbf{u}_{k}\right\|^{2}}}\right). (15) Besides, the \tilde{\mathbf{h}}_{n,k}^{\mathrm{NLoS}}[t]\in\mathbb{C}^{N_{\mathrm{r}}\times 1} denotes the NLoS component of the channel at time slot t . It is modeled as a zero-mean complex Gaussian random vector with independent and identically distributed elements and unit variance, i.e., \tilde{\mathbf{h}}_{n,k}^{\mathrm{NLoS}}[t]\sim\mathcal{CN}(\mathbf{0},\mathbf{I}) . Accordingly, the received signal at UD k from UAV n at time slot t can be expressed as \displaystyle y_{n,k}[t] \displaystyle=\mathbf{w}_{n,k}^{H}[t]\mathbf{h}_{n,k}[t]s_{n,k}[t] \displaystyle\quad+\sum_{j=1}^{N}\sum_{\begin{subarray}{c}i=1\\ i\neq k\end{subarray}}^{K}\alpha_{j,i}[t]\mathbf{w}_{n,k}^{H}[t]\mathbf{h}_{j,i}[t]s_{j,i}[t] \displaystyle\quad+\mathbf{w}_{n,k}^{H}[t]\mathbf{n}, (16) where \mathbf{w}_{n,k}[t]\in\mathbb{C}^{N_{\mathrm{r}}\times 1} is the unit-norm beamforming vector between UAV n and UD k at time slot t , satisfying \mathbf{w}_{n,k}^{\mathrm{H}}[t]\mathbf{w}_{n,k}[t]=1 . The symbol s_{n,k}[t] denotes the signal transmitted from UAV n to UD k , with \mathbb{E}\left[|s_{n,k}[t]|^{2}\right]=1 . In addition, \mathbf{n}\sim\mathcal{CN}(0,\sigma^{2}\mathbf{I}) denotes the additive white Gaussian noise vector with zero mean and covariance \sigma^{2}\mathbf{I} . Hence, the signal-to-interference-plus-noise ratio is given by \displaystyle\Upsilon_{n,k}[t]=\frac{\left|\mathbf{w}_{n,k}^{H}[t]\mathbf{h}_{n,k}[t]\right|^{2}}{\sum_{j=1}^{N}\sum_{\begin{subarray}{c}i=1\\ i\neq k\end{subarray}}^{K}\alpha_{j,i}[t]\left|\mathbf{w}_{n,k}^{H}[t]\mathbf{h}_{j,i}[t]\right|^{2}+\sigma^{2}}. (17) According to Shannon’s capacity formula, the achievable data rate from UAV n to UD k is \displaystyle R_{n,k}[t]=B_{0}\log_{2}\left(1+\Upsilon_{n,k}[t]\right), (18) where B_{0} denotes the channel bandwidth. II-D Satisfaction Model Let S_{n,k}[t]\in[0,1] denote the satisfaction level of UD k with respect to the communication service provided by UAV n at time slot t . To capture the personalized service demands of UDs, the communication requirement of UD k is modeled using a generalized satisfaction mapping function, as described in [ 22 ] f_{n,k}[t]=\frac{1}{1+\exp\left(-\lambda_{n,k}[t]\left(R_{n,k}[t]-R_{n,k}^{\text{target}}+\frac{C}{\lambda_{n,k}[t]}\right)\right)}, (19) where \lambda_{n,k}[t] represents the dynamic urgency level of the transmission task between UAV n and UD k . A higher value of \lambda_{n,k}[t] indicates a more urgent task, which requires a higher transmission rate. The term R_{n,k}^{\text{target}} denotes the corresponding required transmission rate, and C>7 is a predefined shaping constant. When the actual data rate exceeds the target rate, i.e., R_{n,k}[t]>R_{n,k}^{\text{target}} , the output of the satisfaction function satisfies f_{n,k}>\frac{1}{1+\exp(-C)}\approx 1 . The satisfaction indicator S_{n,k}[t] is then defined as S_{n,k}[t]=\begin{cases}f_{n,k}[t],&\text{if }\alpha_{n,k}[t]=1,\\ 0,&\text{if }\alpha_{n,k}[t]=0,\end{cases} (20) when there is no association between UD k and UAV n , that is, \alpha_{n,k}[t]=0 , then the satisfaction level becomes S_{n,k}[t]=0 . II-E Problem Formulation Our objective is to maximize the overall user satisfaction by jointly optimizing the UAV flying trajectory \mathbf{Q}\triangleq\{\mathbf{q}_{n}[t],\forall n\in\mathcal{N},t\in\mathcal{T}\} , transmit beamforming \mathbf{W}\triangleq\{\mathbf{w}_{n,k}[t],\forall n\in\mathcal{N},k\in\mathcal{K},t\in\mathcal{T}\} , and the binary association variable between UDs and UAVs \bm{\alpha}\triangleq\{\alpha_{n,k}[t],\forall n\in\mathcal{N},k\in\mathcal{K},t\in\mathcal{T}\} . The corresponding optimization problem is formulated as \displaystyle\max_{\mathbf{Q},\mathbf{W},\bm{\alpha}}\quad \displaystyle S_{\text{total}}=\sum_{t=1}^{T}\sum_{k=1}^{K}S_{n}[t]=\sum_{t=1}^{T}\sum_{n=1}^{N}\sum_{k=1}^{K}S_{n,k}[t] (21a) s.t. \displaystyle\quad\eqref{eq:x_constraint},\eqref{eq:collision_avoidance},\eqref{eq:max_velocity},\eqref{eq:max_acceleration},\eqref{eq:obs_circle},\eqref{eq:obs_rect},\eqref{eq:alpha1},\eqref{eq:alpha2}, (21b) \displaystyle\quad\text{Tr}\left(\mathbf{W}_{n}^{H}[t]\mathbf{W}_{n}[t]\right)\leq P_{n}^{\max},n\in\mathcal{N}, (21c) \displaystyle\quad S_{n}[t]\leq K_{n},S_{\text{total}}\leq K,\forall n\in\mathcal{N}, (21d) where K_{n} denotes the maximum number of UDs associated with UAV n . Constraint ( 1 ) defines the UAV flight boundary limits in the 3D space. Constraint ( 3 ) ensures a minimum safety distance between any two UAVs. Constraints ( 4 ) and ( 5 ) impose limits on UAV velocity and acceleration, respectively. Constraints ( 8 ) and ( 9 ) restrict the UAV trajectories to avoid cylindrical and rectangular obstacles, respectively. Constraints ( 10 ) and ( 11 ) govern the binary association decisions between UAVs and UDs. Constraint ( 21c ) ensures that the transmit power of UAV n does not exceed its maximum allowable value, where \mathbf{W}_{n}[t] represents the beamforming vector at time slot t . Constraint ( 21d ) enforces the user satisfaction requirement. III Problem Solution It is noteworthy that problem ( 21 ) is a non-convex optimization problem involving disjunctive planning. In general, such non-convex problems are difficult to solve directly due to their inherent computational complexity. Our analysis reveals that the main challenges arise from the non-convexity of both the objective function and several constraints, particularly the trajectory-related restrictions imposed by obstacles, as described in constraints ( 8 ) and ( 9 ). To address these challenges, we propose a low-complexity heuristic algorithm based on the BCD framework. The original problem is decomposed into two subproblems: a beamforming subproblem and a joint trajectory and association subproblem. For the beamforming subproblem, given the UAV trajectory \mathbf{Q} and UAV-UD association variables \bm{\alpha} , we design a bisection-based water-filling algorithm that yields a closed-form solution for the beamforming vectors. For the joint trajectory and association subproblem, given the beamforming \mathbf{W} , we formulate the problem as a Markov decision process (MDP) and solve it using the PPO algorithm. Through alternating optimization of these two algorithms, the system is guided to converge more rapidly and robustly toward a better solution. Furthermore, the proposed framework remains applicable even when faced with various real-world disruptions. For example, GPS drift can cause trajectory deviations, which may be mitigated using robust localization or sensor fusion techniques. Additionally, the energy constraints of UAVs, which are influenced by wind disturbances, can be alleviated through adaptive energy management strategies or robust trajectory re-planning. These modifications primarily expand the state/action spaces and constraints but do not alter the core structure of the proposed framework. The details of the proposed solution framework are presented in the following section. III-A Beamforming Optimization In our model, the wireless channel is characterized by a Rician fading distribution, where the presence or absence of a LoS path is implicitly determined by the surrounding environment. Since the beamforming vectors are optimized based on channel state information (CSI), which inherently reflects the impact of obstacles, the optimization process naturally avoids directions with severe attenuation. Consequently, we do not explicitly incorporate geometric obstacle constraints in the beamforming optimization. Considering that the beamforming matrices of UAVs are independent across different time slots, we omit the time index for simplicity in the beamforming subproblem. Given the UAV positions \mathbf{Q} and UAV-UD association indicators \bm{\alpha} , the beamforming optimization can be reformulated as follows: \displaystyle\max_{\mathbf{W}}\quad \displaystyle S_{\text{total}} (22a) s.t. \displaystyle\text{Tr}\left(\mathbf{W}_{n}^{H}\mathbf{W}_{n}\right)\leq P_{n}^{\max},\forall n\in\mathcal{N}. (22b) This problem is equivalent to the following sum-rate maximization problem: \displaystyle\max_{\mathbf{W}}\quad \displaystyle\sum_{n=1}^{N}\sum_{k=1}^{K}R_{n,k}[t] (23a) s.t. \displaystyle\text{Tr}\left(\mathbf{W}_{n}^{H}\mathbf{W}_{n}\right)\leq P_{n}^{\max},\forall n\in\mathcal{N}. (23b) To eliminate the interference among UDs served by the same UAV in communication system, the zero-forcing (ZF) beamforming strategy is adopted. Specifically, for each UAV n , the beamforming vector \mathbf{w}_{n,k}[t] is designed such that \mathbf{w}_{n,k}^{H}[t]\mathbf{h}_{n,j}[t]=0,\forall j\neq k , which ensures that the signal intended for UD k is orthogonal to the channels of other UDs served by the same UAV. The ZF strategy is feasible under the condition N_{\mathrm{r}}\geq K_{n} , under which it effectively suppresses inter-UD interference with tractable complexity and near-optimal performance. The received signal of UAV n , as defined in ( 16 ), can be rewritten as \mathbf{y}_{n}=\mathbf{W}_{n}^{H}\mathbf{H}_{n}\mathbf{s}_{n}+\mathbf{W}_{n}^{H}\mathbf{n} , where \mathbf{y}_{n}=[y_{n,1},\ldots,y_{n,K}]^{T} , \mathbf{s_{n}}=[s_{n,1},\ldots,s_{n,K}]^{T} , and \mathbf{H}_{n}\in\mathbb{C}^{K\times N_{\mathrm{r}}} is the channel matrix from UAV n to the associated UDs, with the k -th row denoted by \mathbf{h}_{n,k}^{T} . The ZF beamforming matrix is defined as \mathbf{W}_{n}=\mathbf{H}_{n}^{H}\left(\mathbf{H}_{n}\mathbf{H}_{n}^{H}\right)^{-1}\mathbf{P}_{n}^{\frac{1}{2}}=\widetilde{\mathbf{H}}_{n}\mathbf{P}_{n}^{\frac{1}{2}}, (24) where \widetilde{\mathbf{H}}_{n}=\mathbf{H}_{n}^{H}\left(\mathbf{H}_{n}\mathbf{H}_{n}^{H}\right)^{-1} , and the power allocation matrix \mathbf{P}_{n}=\text{diag}(p_{n,1},\ldots,p_{n,K}) . According to the water-filling algorithm, the optimal power allocation is given by p_{n,k}=\left[\frac{1}{\mu_{n}}-\sigma^{2}\cdot\left[\left(\widetilde{\mathbf{H}}_{n}^{H}\widetilde{\mathbf{H}}_{n}\right)^{-1}\right]_{k,k}\right]^{+}, (25) where \mu_{n} is a normalization factor to ensure the power constraint \sum_{k=1}^{K}p_{n,k}-P_{n}^{\max}\leq\varepsilon . The parameter \varepsilon denotes the desired accuracy threshold. The algorithm can be summarized in Algorithm 1 . Algorithm 1 Water-Filling and Bisection-Based Algorithm for Solving ( 24 ) 1: Input: \mathbf{h}_{n,k} , \mathbf{H}_{n} , \sigma^{2} , \varepsilon . 2: Output: Optimal beamforming based on ( 25 ). 3: Calculate matrix \widetilde{\mathbf{H}}_{n}^{H}\widetilde{\mathbf{H}}_{n} , and obtain \mathbf{h}_{n,k} . 4: Initialize \mu_{\max}=\mu_{\min}=\mu_{\text{mid}} . 5: for n\leq N do 6: Find upper and lower bounds for \mu_{n} : 7: for k\leq K do 8: if \mathbf{h}_{n,k}\sigma^{2}\leq 1/\mu_{\max} then set \mu_{\max}=1/(\mathbf{h}_{n,k}\sigma^{2}) , 9: else set \mu_{\min}=1/(\mathbf{h}_{n,k}\sigma^{2}) . 10: end for 11: Find the optimal \mu_{n} using bisection method: 12: repeat 13: Compute midpoint \mu_{\text{mid}}=(\mu_{\max}+\mu_{\min})/2 . 14: Calculate the sum of power allocations P_{\text{sum}}=\sum_{k=1}^{K}\left[\dfrac{1}{\mu_{n}}-\sigma^{2}\cdot\left[\left(\widetilde{\mathbf{H}}_{n}^{H}\widetilde{\mathbf{H}}_{n}\right)^{-1}\right]_{k,k}\right]^{+} . 15: if P_{\text{sum}}>P_{n}^{\max} then set \mu_{\min}=\mu_{\text{mid}} , 16: else set \mu_{\max}=\mu_{\text{mid}} . 17: until |P_{\text{sum}}-P_{n}^{\max}|\leq\varepsilon 18: end for III-B Trajectory and Association Optimization After obtaining the beamforming matrix \mathbf{W} from Algorithm 1 , we proceed to optimize the UAV trajectory \mathbf{Q} and UAV-UD association decisions \bm{\alpha} . Given \mathbf{W} , the original problem ( 21 ) can be simplified as \displaystyle\max_{\mathbf{Q},\bm{\alpha}}\quad \displaystyle S_{\text{total}} (26a) s.t. \displaystyle\quad\eqref{eq:x_constraint},\eqref{eq:collision_avoidance},\eqref{eq:max_velocity},\eqref{eq:max_acceleration},\eqref{eq:obs_circle},\eqref{eq:obs_rect},\eqref{eq:alpha1},\eqref{eq:alpha2}, (26b) \displaystyle\quad S_{n}[t]\leq K_{n},S_{\text{total}}\leq K,\forall n\in\mathcal{N}. (26c) Since problem ( 26 ) is a non-convex integer programming problem, it presents significant computational challenges when approached directly. In addition, the inherent uncertainty and variability introduced by time-varying channel conditions and heterogeneous task characteristics further hinder the applicability of traditional offline optimization techniques, which often assume static or fully known environments. To overcome these limitations and enable real-time, adaptive decision-making for heterogeneous resource allocation, we introduce a DRL framework to derive the optimal joint configuration strategy. III-B 1 Modeling of MDP To facilitate the application of DRL techniques, we reformulate the original optimization problem ( 26 ) as an MDP. In the context of DRL, an MDP is commonly characterized by a tuple (\mathcal{S},\mathcal{A},\mathcal{R},\gamma) , where \mathcal{S} denotes the state space, \mathcal{A} represents the action space, \mathcal{R} specifies the reward function, and \gamma\in[0,1] is the discount. The discount factor \gamma reflects the agent’s preference for long-term returns, with values closer to 1 indicating a greater emphasis on future rewards. The definitions of the state space, action space, and reward function are provided as follows. State: The state is the basis of decision-making. We define the state space within each time slot t as s_{t}=\left\{\mathbf{q}_{n}[t],E_{n}^{\text{res}}[t],\mathbf{v}_{n}[t],\lambda_{n,k}[t],\forall n,k\right\}, (27) where \mathbf{q}_{n}[t] denotes the position of UAV n , E_{n}^{\text{res}}[t] represents its residual battery level, and \mathbf{v}_{n}[t]=[\|\mathbf{v}_{n}[t]\|,\varphi_{n}[t]] characterizes the UAV’s flight velocity, including its magnitude and direction. In addition, \lambda_{n,k}[t] represents the urgency level of UAV n associated with UD k . Action: Since the optimal beamforming \mathbf{W} solution has already been determined, the action space thus includes a_{t}=\left\{\mathbf{a}_{n}[t],\alpha_{n,k}[t],\forall n,k\right\}, (28) where \mathbf{a}_{n}[t]=[\|\mathbf{a}_{n}[t]\|,\phi_{n}[t]] denotes the magnitude and direction of the UAV acceleration, and \alpha_{n,k}[t] represents the association indicator between UAV n and UD k . Reward: The reward function considered in this paper consists of two components: reward and penalty. The definition of the penalty term is given as follows: \displaystyle r_{t}=\begin{cases}S_{\text{total}}-\kappa_{1}\sum_{n=1}^{N}d^{2}(\mathbf{q}_{n}[t],\mathcal{B}),\\ \qquad\text{if }\mathbf{q}_{n}[t]\notin\mathcal{B},\\[6.0pt] S_{\text{total}}-\kappa_{2}\sum_{i=1}^{I}\sum_{n=1}^{N}(D_{ni}^{c})^{2},\\ \qquad\text{if }\|\tilde{\mathbf{q}}_{n}[t]-\mathbf{q}_{\text{obs}}^{i}\|\leq R_{\text{obs}}^{i},~z_{n}[t]\leq H_{\text{obs}},\\[6.0pt] S_{\text{total}}-\kappa_{3}\sum_{j=1}^{J}\sum_{n=1}^{N}(D_{nj}^{r})^{2},\\ \qquad\text{if }\left|\tilde{\mathbf{q}}_{n}[t]-\mathbf{q}_{\text{obs}}^{j}\right|\preceq\mathbf{d}_{\text{obs}}^{j},~z_{n}[t]\leq H_{\text{obs}},\\[6.0pt] S_{\text{total}}-\kappa_{4}\sum_{n