嫩模为农机站台 商家为吸引人气利用美女作秀

Intelligent transportation and autonomous driving systems have emerged as key components of modern wireless communication research [1]. The increasing need for hyper-reliable, low-latency communication (HRLLC) in vehicular networks requires the development of advanced communication frameworks that ensure efficient data exchange between vehicles, infrastructure, and cloud/edge nodes. With the advent of sixth-generation (6G) networks, these systems are expected to support massive connectivity, ultra-low latency, and high data rates[2]. However, several challenges remain, particularly in environments with limited or unreliable terrestrial infrastructure, such as remote areas, disaster-stricken zones, and ultradense urban environments [3]. For instance, while platoon-based networks require robust intra- and inter-platoon communications to maintain synchronized movement and dynamic adaptation to road conditions, keeping data fresh in such a system is challenging due to varying network conditions, mobility patterns, and channel uncertainties.

In this regard, a critical metric in vehicular network research is Age of Information (AoI), which quantifies the timeliness or “freshness” of data received by a target node [4]. Unlike traditional network performance metrics such as throughput and latency, AoI provides direct insight into how current the received information is, making it particularly relevant for applications such as autonomous vehicle control, collision avoidance, and real-time traffic management[5]. Ensuring low AoI is essential for maintaining situational awareness in highly dynamic vehicular environments, where even minor delays in information updates can lead to significant safety risks.

Integrating HAPS into vehicular communication networks offers a promising solution to overcome coverage limitations and challenges related to AoI[6]. Operating at altitudes of approximately 20?km, HAPS provides wide-area connectivity, strong line-of-sight (LoS) links, and low-latency communication, effectively complementing terrestrial and satellite-based networks[7, 1]. By acting as aerial base stations or relays, HAPS can enhance the reliability and freshness of data in vehicle-to-everything (V2X) networks, particularly in infrastructure-constrained areas [8].

To further improve resource allocation and minimize AoI in HAPS-assisted V2X networks, deep reinforcement learning (DRL) techniques have shown remarkable potential. DRL enables autonomous and dynamic decision-making, allowing vehicular agents to optimize real-time communication strategies [9]. Among various DRL approaches, deep deterministic policy gradient (DDPG) and its multi-agent extension, multi-agent DDPG (MADDPG), have demonstrated effectiveness in handling complex high-dimensional control problems [10]. While DDPG provides a straightforward, independent learning mechanism, its limited adaptation to external interference affects its effectiveness. Fully decentralized MADDPG (FD-MADDPG) on the other hand, offers a more scalable and adaptable solution, allowing multiple agents to learn in parallel without centralized dependency. The decentralized framework, combined with HAPS integration, positions FD-MADDPG as a promising approach for optimizing AoI-aware communication in next-generation vehicular networks.

This study focuses on optimizing AoI-aware resource allocation in HAPS-enabled V2X networks. In particular, we propose a DRL-based framework using DDPG to optimize AoI, improving communication efficiency and autonomous decision-making. By integrating platoon-based vehicular coordination, we develop a resource allocation model that enhances intra- and inter-platoon data exchange, ensuring low AoI and stable connectivity. Our model leverages HAPS as an aerial relay to extend network coverage, enhance reliability, and provide seamless connectivity in infrastructure-limited scenarios. The provided simulation results validate the effectiveness of our approach, demonstrating significant improvements in AoI reduction and network reliability.

The rest of this paper is structured as follows. Section?II presents the system model and problem formulation, detailing the role of HAPS in the V2X framework. Section?III describes the DRL-based resource allocation methods, while Section?IV provides the simulation results. Finally, Section?V concludes the study and outlines future research directions.

All symbols used throughout this paper are summarized in Table I and they are also defined in first appearance for clarity and consistency.

The HAPS-enabled V2X architecture considered in this study is represented in Fig.?1 at the top of the next page. This vehicular network consists of multiple autonomous vehicle platoons, each led by a platoon leader (PL) responsible for both intra- and inter-platoon communications. The system supports three primary communication modes, namely vehicle-to-infrastructure (V2I), vehicle-to-vehicle (V2V), and vehicle-to-HAPS (V2H). In V2I communication, the PL connects to roadside units (RSU) or cellular base stations to exchange critical data [11]. V2V communication facilitates direct short-range data sharing within a platoon, ensuring coordinated movement and operational efficiency. Lastly, V2H communication leverages HAPS, which operates at approximately 20 km altitude, to extend coverage beyond terrestrial infrastructure and enhance network reliability, particularly in remote or infrastructure-limited areas, by providing real-time updates and connectivity resilience.

This section explores two reinforcement learning approaches to optimize AoI in HAPS-assisted V2X networks. The first, DDPG, follows a single-agent paradigm, where each PL independently optimizes its AoI based on local observations. While this allows decentralized decision-making, it does not inherently account for inter-agent dependencies, leading to suboptimal resource allocation in congested networks. Due to the lack of inter-agent coordination, DDPG struggles to adapt to dynamic interference patterns, resulting in performance degradation and slower convergence.

The second approach, FD-MADDPG, extends DDPG to a multi-agent reinforcement learning framework, allowing multiple PL to learn concurrently without explicit coordination. Unlike traditional MADDPG, which employs a centralized critic, FD-MADDPG eliminates the need for centralized training. This enables real-time decisions based solely on local observations, improving scalability and robustness in large-scale V2X networks. Using independent learning strategies, FD-MADDPG is known to achieve faster convergence and lower AoI, particularly in high-mobility and dense-vehicle scenarios. In addition, it enables dynamically adjusting transmission and resource allocation strategies based on environmental variations. This makes FD-MADDPG more efficient in handling network congestion and spectral efficiency.

In both approaches, each PL acts as an agent that interacts with the vehicle environment by observing situations and taking the necessary actions according to its predefined policy. Therefore, at any time $t$, each PL $j$ observes the state space, which is given by

According to (7), PLs observe not only the instant channel states but also the AoI and the amount of interference caused by other platoons in the previous step. In addition, $\zeta^{r}_{j}$ and $T^{r}_{j}$ in (7) represent the remaining intra-platoon message load and the remaining time budget, respectively. It should be noted here that each PL maintains an independent experience replay buffer $D_{j}=\{s_{j}^{t},a_{j}^{t},r_{j}^{t},s_{j}^{t+1}\}$, ensuring that training remains decentralized.

Next, the local reward function for both DDPG and FD-MADDPG is defined as follows

where $F(\cdot)$ penalizes high transmission power while restricting it to the same range as the other components, and the stepwise function $G(\cdot)$ ensures that minimum capacity constraints are met [4]. However, a key distinction between DDPG and FD-MADDPG lies in how they optimize their reward function. In particular, DDPG utilizes a centralized critic that considers the overall interference in the environment. In contrast, FD-MADDPG operates in a fully decentralized manner, where each agent optimizes its reward independently. Considering that, Algorithm 1 provides a structured approach to training both DDPG and FD-MADDPG. While DDPG benefits from a global interference-aware reward function as outlined in Algorithm-1, FD-MADDPG relies solely on local observations and independent reward updates.

Both DDPG and FD-MADDPG employ a single-critic approach per agent that can be described by:

1. 1.

   Each agent maintains an independent critic network $Q_{\phi_{j}}$, updated using its own local rewards $r_{j}^{t}$.

2. 2.

   Each agent maintains an actor network $\pi_{\theta_{j}}$, which determines optimal actions based on local state observations.

Furthermore, to maximize the reward for each agent, the policy gradient is updated for both methods as

in which $\pi_{j}(a_{j}|s_{j})$ denotes the policy function for $j$th agent, which defines the probability of selecting action $a_{j}$ given the state $s_{j}$. The term $Q_{\phi_{j}}(s_{j},a_{j})$ refers to the action-value function approximated by the critic network parameterized by $\phi_{j}$, which estimates the expected cumulative discounted reward obtained by executing action $a_{j}$ in state $s_{j}$, and subsequently following the agent’s policy $\pi_{j}$. This function serves as a baseline for the policy gradient and facilitates the evaluation of action quality within continuous action domains. Besides, $\mathbb{E}_{s,a\sim D}$ in (9) indicates the expected value over state-action pairs sampled from the experience replay buffer.

Finally, it should be noted that the critic network is updated by minimizing the following loss function:

where $y_{j}^{t}=r_{j}^{t}+\gamma Q_{\phi_{j}^{\prime}}(s_{j}^{t+1},\pi_{\theta_{j}^{\prime}}(s_{j}^{t+1}))$ with reward discount factor $\gamma$.

This section presents simulation results for the proposed HAPS-V2X network consisting of a HAPS, an RSU, and five PLs with six followers each in an urban area. Specifically, two DRL-based approaches are utilized in this scenario to optimize resource allocation with minimum AoI and are compared in terms of average AoI value and reward function convergence. The values of the other parameters used for the simulations are determined as shown in Table II.

Fig.?2 presents a comparison of the reward function convergence for the utilized methods. According to this figure, FD-MADDPG clearly converges faster to a higher and more precise reward value compared to the DDPG approach. This is due to the fact that each PL in the FD-MADDPG learns independently using only its local observations, which accelerates convergence in multi-agent environments. Accordingly, considering also the long-term fluctuations, it can be concluded from this figure that the learning process of DDPG is longer and more challenging than that of FD-MADDPG for the considered HAPS-V2X scenario.

In Fig.?3, FD-MADDPG is shown to be providing much lower AoI values in V2X networks compared to DDPG under similar training conditions, which can be further reduced with HAPS support. In addition, it is observed that DDPG is heavily affected by the increase in inter-platoon spacing. For example, in the HAPS-V2X scenario, the average AoI for DDPG is approximately 13 ms while for FD-MADDPG it is only 6 ms when the gap between platoons is 5 m. It is noteworthy that when the spacing between platoons is increased to 35 m, the AoI increases by only about 3 ms for FD-MADDPG, whereas it increases by almost 20 ms for DDPG. On the one hand, this shows that the decentralized solution handles interference and channel variations more effectively, keeping information fresher across the network and improving overall network reliability in HAPS-supported V2X scenarios. On the other hand, it indicates that the information update rate of the DDPG algorithm remains relatively slow even with HAPS support.

This paper investigates the contribution of HAPS integration and the effectiveness of two different DRL approaches in resource allocation prioritizing information freshness in V2X networks. Numerical evaluations have proved significant improvements in network reliability, showing that lower AoI, faster convergence, and thus, better spectrum utilization can be achieved with FD-MADDPG compared to the conventional DDPG model. These findings highlight that HAPS can play a critical role in providing uninterrupted connectivity in time-critical scenarios, especially in environments with limited infrastructure. It should be noted here that the widespread use of solar-powered HAPS will also add a sustainable and environmentally friendly dimension to 6G networks, supporting greener communications.

Future research can be focused on exploring energy-efficient learning strategies, adaptive reward mechanisms, and real-world deployments in large-scale vehicular networks. Additionally, investigating HAPS mobility and hybrid artificial intelligence-driven optimization can be considered to further enhance the adaptability of these networks. Comparing the performance of the proposed scheme with other recently used optimization methods, such as attention-based DRL, federated learning, or game-theoretic approaches, can be another stimulating direction.