可重構智能超表面輔助的非地面網(wǎng)絡安全傳輸與軌跡優(yōu)化

徐可馨; 隆克平; 陸陽; 張海君

doi:10.11999/JEIT240981

可重構智能超表面輔助的非地面網(wǎng)絡安全傳輸與軌跡優(yōu)化

doi: 10.11999/JEIT240981 cstr: 32379.14.JEIT240981

1.
北京科技大學計算機與通信工程學院北京 100083
2.
中國電力科學研究院北京 102209

基金項目: 國家自然科學基金(U2441227, U22B2003)，國防基礎科研計劃(JCKY2022110C010)，中央高?；究蒲袠I(yè)務費專項資金(FRF-TP-22-002C2)，通信抗干擾全國重點實驗室開放課題(IFN20230201)

詳細信息

作者簡介:
徐可馨：女，博士生，研究方向為非地面網(wǎng)絡、6G移動通信以及人工智能技術

隆克平：男，教授，研究方向為6G移動通信、光無線通信

陸陽：男，高級工程師(教授級)，研究方向為電力通信技術

張海君：男，教授，研究方向為6G移動通信、數(shù)字孿生、通感一體化技術以及衛(wèi)星通信

通訊作者:
隆克平　longkeping@ustb.edu.cn

中圖分類號: TN929.5
計量
- 文章訪問數(shù): 323
- HTML全文瀏覽量: 144
- PDF下載量: 77
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2024-11-01
- 修回日期: 2025-02-24
- 網(wǎng)絡出版日期: 2025-02-25
- 刊出日期: 2025-02-28

Joint Secure Transmission and Trajectory Optimization for Reconfigurable Intelligent Surface-aided Non-Terrestrial Networks

1.
School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China
2.
China Electric Power Research Institute Co. Ltd. , Beijing 102209, China

Funds: The National Natural Science Foundation of China (U2441227, U22B2003), The Defense Industrial Technology Development Program (JCKY2022110C010), The Fundamental Research Funds for the Central Universities (FRF-TP-22-002C2), The National Key Laboratory of Wireless Communications Foundation (IFN20230201)

摘要

摘要: 由于衛(wèi)星與地面用戶之間的直連受限于覆蓋范圍和鏈路質(zhì)量以及非地面網(wǎng)絡存在竊聽威脅等問題，該文考慮一個無人機中繼的非地面網(wǎng)絡安全傳輸系統(tǒng)，引入可重構智能超表面(RIS)，提高合法用戶信號質(zhì)量。同時為了兼顧系統(tǒng)高傳輸速率和高安全需求，該文設計衛(wèi)星到無人機的傳輸速率與地面合法用戶的安全速率的加權和作為系統(tǒng)效用，并以此作為優(yōu)化目標，進而提出一種基于雙層雙延遲深度確定性策略梯度(TTD3)的聯(lián)合衛(wèi)星與無人機波束成形、RIS相移矩陣以及無人機軌跡優(yōu)化方法，通過采用雙層深度強化學習結構解耦波束成形和軌跡優(yōu)化兩個子問題，實現(xiàn)系統(tǒng)效用最大化。仿真結果驗證了所提方法在動態(tài)非地面網(wǎng)絡環(huán)境下的有效性，同時在高安全需求下，通過對比不同算法、不同配置方案以及不同RIS元件數(shù)量下的仿真結果，證明了該文所提方法能夠提升系統(tǒng)安全傳輸性能。
- 可重構智能超表面 /
- 非地面網(wǎng)絡 /
- 深度強化學習 /
- 安全傳輸
Abstract: Objective The proliferation of technologies such as the Internet of Things, smart cities, and next-generation mobile communications has made Non-Terrestrial Networks (NTNs) increasingly important for global communication. Future communication systems are expected to rely heavily on NTNs to provide seamless global coverage and efficient data transmission. However, current NTNs face challenges, including limited coverage and link quality in direct satellite-to-ground user connections, as well as eavesdropping threats. To address these challenges, a system integrating Reconfigurable Intelligent Surfaces (RIS) with a twin-layer Deep Reinforcement Learning (DRL) algorithm is proposed. This approach aims to satisfy the system’s requirements for high transmission rates and enhanced security, improving the signal strength for legitimate users while facilitating real-time updates and optimization of channel state information in NTNs. Methods First, an RIS-aided downlink NTNs system using an Unmanned Aerial Vehicle (UAV) as a relay is established. To balance the system’s transmission rate and security requirements, the weighted sum of the satellite-to-UAV transmission rate and the secure rate of the legitimate ground user is designed as the system utility, which serves as the optimization objective. A joint optimization method based on the Twin-Twin Delayed Deep Deterministic Policy Gradient (TTD3) algorithm is then proposed. This method jointly optimizes satellite and UAV beamforming, the RIS phase shift matrix, and UAV trajectory. The algorithm divides the optimization problem into two layers for solution. The first-layer DRL optimizes satellite and UAV beamforming, as well as the RIS phase shift matrix. The second-layer DRL optimizes the UAV’s trajectory based on its position, user mobility, and channel state information. The twin DRL shares the same reward function, guiding the agents in each layer to adjust their actions and explore optimal strategies, ultimately enhancing the system’s utility. Results and Discussions (1) Compared to the Deep Deterministic Policy Gradient (DDPG), the proposed TTD3 algorithm exhibits smaller dynamic fluctuations, demonstrating greater stability and robustness (Fig. 2). (2) The UAV trajectory and user secrecy rate performance under four different schemes and algorithms show that the proposed method balances service for legitimate users. The UAV trajectory is smoother compared to that based on DDPG, and the overall user secrecy rate is also higher. This confirms that the proposed method can adapt to dynamically changing NTNs environments while improving user secrecy rates (Fig. 3, Fig. 4). (3) As the number of RIS reflecting elements increases, the degrees of freedom and precision of beamforming improve. Therefore, the overall user secrecy rates of different algorithms increase, resulting in enhanced system performance (Fig. 5). Conclusions This paper investigates an RIS-assisted downlink secure transmission system for NTNs, addressing the presence of eavesdropping threats. To meet the requirements of high transmission rates and security across different scenarios, the optimization objective is formulated as the weighted sum of the transmission rate from the satellite to the UAV and the secrecy rate of legitimate ground users. A TTD3-based joint optimization method for satellite and UAV beamforming, RIS phase shift matrix, and UAV trajectory is proposed. By adopting a twin-layer DRL structure, the beamforming and trajectory optimization subproblems are decoupled to maximize system utility. Simulation results validate the effectiveness of the proposed algorithm. Additionally, comparisons across different algorithms, RIS element counts, and schemes in high-security-demand scenarios demonstrate that the TTD3 algorithm is well-suited for dynamically changing NTNs environments and can significantly enhance system transmission performance. Future research will explore integrating emerging technologies, such as federated learning and meta-learning, to achieve distributed, low-latency policy optimization, thereby facilitating network resource optimization and interference analysis in large-scale, multi-satellite, and multi-UAV complex scenarios.
- Reconfigurable Intelligent Surface (RIS) /
- Non-Terrestrial Networks (NTNs) /
- Deep Reinforcement Learning (DRL) /
- Secure transmission

HTML全文

圖 1 RIS輔助的NTNs下行傳輸系統(tǒng)

下載: 全尺寸圖片幻燈片

圖 2 不同算法下的所提方案訓練過程中獎勵曲線變化情況

下載: 全尺寸圖片幻燈片

圖 3 不同方案下的UAV軌跡優(yōu)化情況

下載: 全尺寸圖片幻燈片

圖 4 不同方案下的用戶保密速率性能對比

下載: 全尺寸圖片幻燈片

圖 5 不同RIS反射元件個數(shù)下不同算法的用戶保密速率性能對比

下載: 全尺寸圖片幻燈片

1 基于TTD3算法的NTNs安全傳輸與軌跡優(yōu)化流程

初始化1：TTD3中的第1層TD3的6個神經(jīng)網(wǎng)絡參數(shù)以及第2層　TD3的6個神經(jīng)網(wǎng)絡參數(shù)；
初始化2：軟更新因子$\psi $，每次迭代步數(shù)${N_{{\mathrm{step}}}}$，迭代次數(shù) 　Eposide，經(jīng)驗存放空間${{B}}$，更新間隔$C$，批次大小$v$；
(1) for ${\mathrm{step}} = 1$ to Eposide do
(2) 　　初始化UAV的位置、用戶的位置以及信道狀態(tài)；
(3) 　for ${\mathrm{step}} = 1$ to ${N_{{\mathrm{step}}}}$ do
(4) 　　　獲得$ {{\boldsymbol{h}}_{\rm{S,U}}},{{\mathrm{SINR}}_{\rm{S,U}}},{{\boldsymbol{h}}_{{\rm U},i}} + {{\boldsymbol{h}}_{{\rm R},i}}{\boldsymbol{\varTheta}} {h_{{\mathrm{U,R}}}} $作為${{\boldsymbol{s}}_1}$, $ {\boldsymbol{q}} $作　　　　　為${{\boldsymbol{s}}_2}$；
(5) 　　根據(jù)式(21)產(chǎn)生動作${{\boldsymbol{a}}_1}$和${{\boldsymbol{a}}_2}$；
(6) 　　執(zhí)行相應的動作獲得相應的即時獎勵$ {r_1} $和${r_2}$，并觀察　　　　新狀態(tài)${\boldsymbol{s}}_1^{'}$和${\boldsymbol{s}}_2^{'}$；
(7)　　將狀態(tài)轉(zhuǎn)移元組$({{\boldsymbol{s}}_1},{{\boldsymbol{a}}_1},{r_1},{\boldsymbol{s}}_1^{'})$和　　　　 $ ({{\boldsymbol{s}}_2},{{\boldsymbol{a}}_2},{r_2},{{\boldsymbol{s}}_2}^{'}) $存儲在$B$中；
(8)　　隨機抽取$v$條經(jīng)驗進行訓練；
(9) 　　根據(jù)式(27)獲得目標值；
(10) 　根據(jù)策略延遲更新機制更新Actor網(wǎng)絡和Critic網(wǎng)絡參數(shù)；
(11) 　以式(26)對目標Actor網(wǎng)絡和Critic網(wǎng)絡參數(shù)更新；
(12) end for
(13) end for

下載: 導出CSV

參考文獻(16)

[1]	AZARI M M, SOLANKI S, CHATZINOTAS S, et al. Evolution of non-terrestrial networks from 5G to 6G: A survey[J]. IEEE Communications Surveys & Tutorials, 2022, 24(4): 2633–2672. doi: 10.1109/COMST.2022.3199901.
[2]	ZHOU Di, SHENG Min, LI Jiandong, et al. Aerospace integrated networks innovation for empowering 6G: A survey and future challenges[J]. IEEE Communications Surveys & Tutorials, 2023, 25(2): 975–1019. doi: 10.1109/COMST.2023.3245614.
[3]	JIANG Bin, YAN Yingchun, YOU Li, et al. Robust secure transmission for satellite communications[J]. IEEE Transactions on Aerospace and Electronic Systems, 2023, 59(2): 1598–1612. doi: 10.1109/TAES.2022.3203027.
[4]	LI Yabo, ZHANG Haijun, LONG Keping, et al. Exploring sum rate maximization in UAV-based Multi-IRS networks: IRS association, UAV altitude, and phase shift design[J]. IEEE Transactions on Communications, 2022, 70(11): 7764–7774. doi: 10.1109/TCOMM.2022.3206884.
[5]	KHAN W U, LAGUNAS E, MAHMOOD A, et al. RIS-assisted energy-efficient LEO satellite communications with NOMA[J]. IEEE Transactions on Green Communications and Networking, 2024, 8(2): 780–790. doi: 10.1109/TGCN.2023.3344102.
[6]	ZHANG Haijun, HUANG Miaolin, ZHOU Huan, et al. Capacity maximization in RIS-UAV networks: A DDQN-based trajectory and phase shift optimization approach[J]. IEEE Transactions on Wireless Communications, 2023, 22(4): 2583–2591. doi: 10.1109/TWC.2022.3212830.
[7]	KHAN W U, LAGUNAS E, ALI Z, et al. Opportunities for physical layer security in UAV communication enhanced with intelligent reflective surfaces[J]. IEEE Wireless Communications, 2022, 29(6): 22–28. doi: 10.1109/MWC.001.2200125.
[8]	LI Jingyi, XU Sai, LIU Jiajia, et al. Reconfigurable intelligent surface enhanced secure aerial-ground communication[J]. IEEE Transactions on Communications, 2021, 69(9): 6185–6197. doi: 10.1109/TCOMM.2021.3086517.
[9]	GUO Xufeng, CHEN Yuanbin, and WANG Ying. Learning-based robust and secure transmission for reconfigurable intelligent surface aided millimeter wave UAV communications[J]. IEEE Wireless Communications Letters, 2021, 10(8): 1795–1799. doi: 10.1109/LWC.2021.3081464.
[10]	YANG Helin, LIU Shuai, XIAO Liang, et al. Learning-based reliable and secure transmission for UAV-RIS-assisted communication systems[J]. IEEE Transactions on Wireless Communications, 2024, 23(7): 6954–6967. doi: 10.1109/TWC.2023.3336535.
[11]	趙柏, 林敏, 肖圣杰, 等. 基于速率分割的可重構智能表面輔助星地融合網(wǎng)絡魯棒安全傳輸方案[J]. 通信學報, 2023, 44(12): 50–60. doi: 10.11959/j.issn.1000-436x.2023221. ZHAO Bai, LIN Min, XIAO Shengjie, et al. Rate splitting based robust secure transmission scheme in RIS-assisted satellite-terrestrial integrated network[J]. Journal on Communications, 2023, 44(12): 50–60. doi: 10.11959/j.issn.1000-436x.2023221.
[12]	ZHAO Bai, LIN Min, CHENG Ming, et al. Robust downlink transmission design in IRS-assisted cognitive satellite and terrestrial networks[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(8): 2514–2529. doi: 10.1109/JSAC.2023.3288234.
[13]	NGO Q T, PHAN K T, MAHMOOD A, et al. Hybrid IRS-assisted secure satellite downlink communications: A fast deep reinforcement learning approach[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2024, 8(4): 2858–2869. doi: 10.1109/TETCI.2024.3378605.
[14]	LI Huifang, LI Jing, LIU Meng, et al. UAV-assisted secure communication for coordinated satellite-terrestrial networks[J]. IEEE Communications Letters, 2023, 27(7): 1709–1713. doi: 10.1109/LCOMM.2023.3267119.
[15]	ZHOU Gui, PAN Chunhua, REN Hong, et al. Stochastic learning-based robust beamforming design for RIS-aided millimeter-wave systems in the presence of random blockages[J]. IEEE Transactions on Vehicular Technology, 2021, 70(1): 1057–1061. doi: 10.1109/TVT.2021.3049257.
[16]	LIN Zhi, NIU Hehao, AN Kang, et al. Refracting RIS-aided hybrid satellite-terrestrial relay networks: Joint beamforming design and optimization[J]. IEEE Transactions on Aerospace and Electronic Systems, 2022, 58(4): 3717–3724. doi: 10.1109/TAES.2022.3155711.