面向物聯(lián)網(wǎng)的云邊端協(xié)同計算中任務(wù)卸載與資源分配算法研究

施建鋒; 陳忻陽; 李寶龍

doi:10.11999/JEIT240659

面向物聯(lián)網(wǎng)的云邊端協(xié)同計算中任務(wù)卸載與資源分配算法研究

doi: 10.11999/JEIT240659 cstr: 32379.14.JEIT240659

南京信息工程大學(xué)電子與信息工程學(xué)院南京 214442

基金項目: 國家自然科學(xué)基金(62201274, 62201275)，江蘇省自然科學(xué)基金(BK20210641)

詳細(xì)信息

作者簡介:
施建鋒：男，副教授，研究方向為空天地一體化網(wǎng)絡(luò)、用戶為中心網(wǎng)絡(luò)、B5G和6G網(wǎng)絡(luò)

陳忻陽：男，碩士生，研究方向為物聯(lián)網(wǎng)、衛(wèi)星邊緣計算、資源分配

李寶龍：男，副教授，研究方向為無線光通信、數(shù)據(jù)管理、物聯(lián)網(wǎng)

通訊作者:
施建鋒　jianfeng.shi@nuist.edu.cn

中圖分類號: TN927
計量
- 文章訪問數(shù): 696
- HTML全文瀏覽量: 357
- PDF下載量: 112
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2024-07-26
- 修回日期: 2024-12-10
- 網(wǎng)絡(luò)出版日期: 2024-12-17
- 刊出日期: 2025-02-28

Research on Task Offloading and Resource Allocation Algorithms in Cloud-edge-end Collaborative Computing for the Internet of Things

School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Nanjing 214442, China

Funds: The National Natural Science Foundation of China (62201274, 62201275), The Natural Science Foundation of Jiangsu Province (BK20210641)

摘要

摘要: 為滿足遠(yuǎn)郊和災(zāi)區(qū)物聯(lián)網(wǎng)(IoT)設(shè)備的時延與能耗需求，該文構(gòu)建了由IoT終端、低地球軌道(LEO)衛(wèi)星和云計算中心組成的新型動態(tài)衛(wèi)星物聯(lián)網(wǎng)模型。在時延、能耗等實際約束條件下，將系統(tǒng)時延與能耗加權(quán)和視為系統(tǒng)開銷，構(gòu)造了最小化系統(tǒng)開銷的任務(wù)卸載、功率和計算資源聯(lián)合分配問題。針對動態(tài)任務(wù)到達(dá)場景，提出一種模型輔助的自適應(yīng)深度強化學(xué)習(xí)(MADRL)算法，實現(xiàn)任務(wù)卸載決策、通信資源和計算資源的聯(lián)合配置。該算法將問題分為兩部分解決，第1部分通過模型輔助、二分搜索算法和梯度下降法優(yōu)化了通信資源與計算資源；第2部分通過自適應(yīng)深度強化學(xué)習(xí)算法訓(xùn)練出Q網(wǎng)絡(luò)以適應(yīng)隨機任務(wù)的到達(dá)，進行卸載決策優(yōu)化。該算法實現(xiàn)了有效的資源分配和可靠及時的任務(wù)卸載決策，且在降低系統(tǒng)開銷方面表現(xiàn)出優(yōu)異的效果。仿真結(jié)果表明，引入衛(wèi)星的移動性，使得系統(tǒng)開銷降低了41%。引入星間協(xié)作技術(shù)，使系統(tǒng)開銷降低了22.1%。此外，該文所提算法收斂性能好。與基準(zhǔn)算法相比，該算法的系統(tǒng)開銷降低了3%，在不同環(huán)境下的性能表現(xiàn)都是最優(yōu)。
- 云邊端協(xié)同計算 /
- 衛(wèi)星物聯(lián)網(wǎng) /
- 深度強化學(xué)習(xí) /
- 任務(wù)卸載 /
- 資源分配.
Abstract: Objective With the rapid pace of digital transformation and the smart upgrading of the economy and society, the Internet of Things (IoT) has become a critical element of new infrastructure. Current wide-area IoT networks primarily rely on 5G terrestrial infrastructure. While these networks continue to evolve, challenges persist, particularly in remote or disaster-affected areas. The high cost and vulnerability of base stations hinder deployment and maintenance in these locations. Satellite networks provide seamless coverage, flexibility, and reliability, making them compelling alternatives to terrestrial networks for achieving global connectivity. Satellite-assisted Internet of Things (SIoT) can deliver ubiquitous and reliable connectivity for IoT devices. Typically, IoT devices offload tasks to edge servers or cloud platforms due to their limited power, computing, and caching resources. Mobile Edge Computing (MEC) helps reduce latency by caching content and placing edge servers closer to IoT devices. Low Earth Orbit (LEO) satellites with integrated processing units can also serve as edge computing nodes. Although cloud platforms offer abundant computing resources and a reliable power supply, the long distance between IoT devices and the cloud results in higher communication latency. With the explosive growth of IoT devices and the diversification of application requirements driven by 5G, it is essential to design a collaborative architecture that integrates cloud, edge, and end devices. Recent research has extensively explored MEC-enhanced SIoT systems. However, many studies focus solely on edge or cloud computing, with little emphasis on their integration, satellite mobility, or resource constraints. Furthermore, LEO satellites providing edge services face challenges due to their limited onboard resources and the high mobility of the satellite constellation, complicating resource allocation and task offloading. Single-satellite solutions may not satisfy performance expectations during peak demand. Inter-Satellite Collaboration (ISC) technology, which utilizes visible light communications, can significantly increase system capacity, extend coverage, reduce individual satellite resource consumption, and prolong network operational life. Although some studies address three-tier architectures involving IoT devices, satellites, and clouds, proposing load balancing mechanisms through ISC for optimizing offloading and resource allocation, many rely on static assumptions about network topologies and user associations. In practice, LEO satellites require frequent switching and dynamic adjustments in offloading strategies to maintain service quality due to their high-speed mobility. Therefore, there is a need for a method of task offloading and resource allocation in a dynamic environment that considers satellite mobility and limited resources. To address these research gaps, this paper proposes a dynamic ISC-enhanced cloud-edge-end SIoT network model. By formulating the joint optimization problem of offloading decisions and resource allocation as a Mixed Integer Non-Linear Programming (MINLP) problem, a Model-assisted Adaptive Deep Reinforcement Learning (MADRL) algorithm is developed to achieve minimum system cost in a changing environment. Methods The LEO satellite mobility model and the SIoT network model with ISC are constructed to analyze end-to-end latency and system energy consumption. This evaluation considers three modes: local computing, edge computing, and cloud computing. A joint optimization MINLP problem is formulated, focusing on task offloading and resource allocation to minimize system costs. A MADRL algorithm is introduced, integrating traditional optimization techniques with deep reinforcement learning. The algorithm operates in two parts. The first part optimizes communication and computational resource allocation using a model-assisted binary search algorithm and gradient descent method. The second part trains a Q-network to adapt offloading decisions based on stochastic task arrivals through an adaptive deep reinforcement learning approach. Results and Discussions Simulation experiments were conducted under various dynamic scenarios. The MADRL algorithm exhibits strong convergence properties, as demonstrated in the analysis. Comparisons of different learning rates and exploration decay factors reveal optimal parameter values. Incorporating satellite mobility reduces system costs by 41% compared to static scenarios, enabling dynamic resource allocation and improved efficiency. Integrating ISC reduces system costs by 22.1%. This demonstrates the effectiveness of inter-satellite load balancing in improving resource utilization. Additionally, the MADRL algorithm achieves a 3% reduction in system costs compared to the Deep Q Learning (DQN) algorithm, highlighting its adaptability and efficiency in dynamic environments. System costs decrease as satellite speed increases, with the MADRL algorithm consistently outperforming other methods. Conclusions This paper presents an innovative dynamic SIoT model that integrates IoT devices, LEO satellites, and a cloud computing center. The model addresses the latency and energy consumption issues faced by IoT devices in remote and disaster-stricken areas. The task offloading and resource allocation problem that minimizes system cost is constructed by incorporating ISC techniques to enhance satellite edge performance and by taking satellite mobility into account. A MADRL algorithm that combines traditional optimization with deep reinforcement learning is proposed. This approach effectively optimizes task offloading decisions and resource allocation. Simulation results demonstrate that our model and algorithm significantly reduce system costs. Specifically, the incorporation of satellite mobility and ISC technology leads to cost reductions of 41% and 22.1%, respectively. Compared to benchmark algorithms, the MADRL shows superior performance across various test environments, highlighting its significant application advantages.
- Cloud-edge-end collaborative computing /
- Satellite-assited Internet of Things (SIoT) /
- Deep Reinforcement Learning (DRL) /
- Task offloading /
- Resource allocation.

HTML全文

圖 1 LEO衛(wèi)星s與設(shè)備m之間的幾何關(guān)系

下載: 全尺寸圖片幻燈片

圖 2 MADRL 算法流程圖

下載: 全尺寸圖片幻燈片

圖 3 不同學(xué)習(xí)率下的收斂性能

下載: 全尺寸圖片幻燈片

圖 4 不同學(xué)習(xí)率下的系統(tǒng)累積開銷

下載: 全尺寸圖片幻燈片

圖 5 不同衰減因子下的系統(tǒng)累積開銷

下載: 全尺寸圖片幻燈片

圖 6 衛(wèi)星移動性對DQN, MADRL算法性能的影響

下載: 全尺寸圖片幻燈片

圖 7 ISC對DQN, MADRL算法性能的影響

下載: 全尺寸圖片幻燈片

圖 8 不同算法下的累積系統(tǒng)開銷

下載: 全尺寸圖片幻燈片

圖 9 不同算法下系統(tǒng)開銷與LEO衛(wèi)星速率的關(guān)系

下載: 全尺寸圖片幻燈片

表 1 基本符號及其含義

符號	含義
$\mathcal{M}$	設(shè)備集合
$\mathcal{D}$	災(zāi)區(qū)設(shè)備集合
$\mathcal{R}$	遠(yuǎn)郊設(shè)備集合
$\mathcal{S}$	LEO衛(wèi)星集合
$d_m^n$	時隙n設(shè)備m生成任務(wù)的大小
$c_m^n$	時隙n設(shè)備m的工作負(fù)載
$w_m^n$	時隙n設(shè)備m處理任務(wù)所需CPU周期數(shù)
$T_m^{n,\max }$	時隙n設(shè)備m處理任務(wù)的最大容忍時延
$x_m^n$	時隙n設(shè)備m的任務(wù)卸載決策
$f_m^n$	時隙n設(shè)備m的CPU工作頻率
$p_m^n$	時隙n設(shè)備m的傳輸功率
$t_m^n$	時隙n設(shè)備m的系統(tǒng)時延
$e_m^n$	時隙n設(shè)備m的系統(tǒng)能耗
$c_m^n$	時隙n設(shè)備m的系統(tǒng)開銷

下載: 導(dǎo)出CSV

1 自適應(yīng)DRL算法

輸入：開銷矩陣
(1)初始化在線網(wǎng)絡(luò)Q和目標(biāo)網(wǎng)絡(luò)Q_hat
(2)初始化訓(xùn)練參數(shù)
(3) 　for episode =1 to n_ep do
(4) 　　初始化狀態(tài) s
(5) 　　for n=1 to N do
(6) 　　　根據(jù)$\varepsilon $貪婪策略選擇動作a
(7) 　　　更新狀態(tài)$ {\boldsymbol{s}}' $
(8) 　　end for
(9) 　end for
(10) if ${\mathcal{D}} $的大小≥ n_b：
(11) 　從D中隨機抽取最小批量轉(zhuǎn)移元組
(12) 　根據(jù)任務(wù)狀態(tài)選擇DQN或DDQN計算y值
(13) end if
(14)計算損失函數(shù)${\text{Loss}}(\theta )$
(15)更新在線網(wǎng)絡(luò)Q
(16)每隔X步，更新目標(biāo)網(wǎng)絡(luò)：Q_hat=Q
(17)更新狀態(tài)$ {\boldsymbol{s}} \leftarrow {\boldsymbol{s}}' $
(18)返回Q網(wǎng)絡(luò)

下載: 導(dǎo)出CSV

表 2 主要參數(shù)設(shè)置

參數(shù)	值
災(zāi)區(qū)設(shè)備數(shù)D	300
遠(yuǎn)郊設(shè)備數(shù)R	5
衛(wèi)星服務(wù)范圍半徑r	1 400 km
任務(wù)大小$d_m^n$	[1e2,1e3,1e4,1e5,1e6] bit
任務(wù)負(fù)載$c_m^n$	[1,1.5] kcycle/bit
最大容忍時延$T_m^{n,\max }$	[0.05,0.1] s
電氣系數(shù)${{\varepsilon }}$	10^–28
信道帶寬B	10 MHz
天線增益G	20 dBi
噪聲溫度T	290 K
IoT設(shè)備m的最大能耗$E_m^{\max }$	5 W
LEO衛(wèi)星s的最大能耗$E_s^{\max }$	2 000 W
云計算中心單核CPU工作頻率$f_{\text{c}}$	1.45 GHz
云計算中心核心數(shù)${N_{\text{c}}}$	256

下載: 導(dǎo)出CSV

參考文獻(xiàn)(33)

[1]	工業(yè)和信息化部. 物聯(lián)網(wǎng)新型基礎(chǔ)設(shè)施建設(shè)三年行動計劃(2021-2023年)[R]. 2021. Ministry of Industry and Information Technology of the People's Republic of China. A three-year action plan for the construction of new IoT infrastructure[R]. 2021.
[2]	CUI Gaofeng, DUAN Pengfei, XU Lexi, et al. Latency optimization for hybrid GEO–LEO satellite-assisted IoT networks[J]. IEEE Internet of Things Journal, 2023, 10(7): 6286–6297. doi: 10.1109/JIOT.2022.3222831.
[3]	DE COLA T and BISIO I. QoS optimisation of eMBB services in converged 5G-satellite networks[J]. IEEE Transactions on Vehicular Technology, 2020, 69(10): 12098–12110. doi: 10.1109/TVT.2020.3011963.
[4]	KANEKO K, NISHIYAMA H, KATO N, et al. Construction of a flexibility analysis model for flexible high-throughput satellite communication systems with a digital channelizer[J]. IEEE Transactions on Vehicular Technology, 2018, 67(3): 2097–2107. doi: 10.1109/TVT.2017.2736010.
[5]	BOERO L, BRUSCHI R, DAVOLI F, et al. Satellite networking integration in the 5G ecosystem: Research trends and open challenges[J]. IEEE Network, 2018, 32(5): 9–15. doi: 10.1109/MNET.2018.1800052.
[6]	CHIEN W C, LAI C F, HOSSAIN M S, et al. Heterogeneous space and terrestrial integrated networks for IoT: Architecture and challenges[J]. IEEE Network, 2019, 33(1): 15–21. doi: 10.1109/MNET.2018.1800182.
[7]	RANAWEERA P, JURCUT A D, and LIYANAGE M. Survey on multi-access edge computing security and privacy[J]. IEEE Communications Surveys & Tutorials, 2021, 23(2): 1078–1124. doi: 10.1109/COMST.2021.3062546.
[8]	LI Chengcheng, ZHANG Yasheng, XIE Renchao, et al. Integrating edge computing into low earth orbit satellite networks: Architecture and prototype[J]. IEEE Access, 2021, 9: 39126–39137. doi: 10.1109/ACCESS.2021.3064397.
[9]	WANG Dezhi, WANG Wei, KANG Yuhan, et al. Distributed data offloading in ultra-dense LEO satellite networks: A stackelberg mean-field game approach[J]. IEEE Journal of Selected Topics in Signal Processing, 2023, 17(1): 112–127. doi: 10.1109/JSTSP.2022.3226400.
[10]	TANG Zhixuan, YU Kai, YANG Guannan, et al. New bridge to cloud: An ultra-dense LEO assisted green computation offloading approach[J]. IEEE Transactions on Green Communications and Networking, 2023, 7(2): 552–564. doi: 10.1109/TGCN.2022.3208819.
[11]	YU Shuai, GONG Xiaowen, SHI Qian, et al. EC-SAGINs: Edge-computing-enhanced space–air–ground-integrated networks for internet of vehicles[J]. IEEE Internet of Things Journal, 2022, 9(8): 5742–5754. doi: 10.1109/JIOT.2021.3052542.
[12]	LIU Yi, JIANG Li, QI Qi, et al. Energy-efficient space–air–ground integrated edge computing for internet of remote things: A federated DRL approach[J]. IEEE Internet of Things Journal, 2023, 10(6): 4845–4856. doi: 10.1109/JIOT.2022.3220677.
[13]	TANG Qingqing, FEI Zesong, LI Bin, et al. Stochastic computation offloading for LEO satellite edge computing networks: A learning-based approach[J]. IEEE Internet of Things Journal, 2024, 11(4): 5638–5652. doi: 10.1109/JIOT.2023.3307707.
[14]	ZHU Xiangming and JIANG Chunxiao. Delay optimization for cooperative multi-tier computing in integrated satellite-terrestrial networks[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(2): 366–380. doi: 10.1109/JSAC.2022.3227083.
[15]	ZHANG Shanghong, CUI Gaofeng, LONG Yating, et al. Joint computing and communication resource allocation for satellite communication networks with edge computing[J]. China Communications, 2021, 18(7): 236–252. doi: 10.23919/JCC.2021.07.019.
[16]	TANG Qingqing, FEI Zesong, LI Bin, et al. Computation offloading in LEO satellite networks with hybrid cloud and edge computing[J]. IEEE Internet of Things Journal, 2021, 8(11): 9164–9176. doi: 10.1109/JIOT.2021.3056569.
[17]	CAO Bin, ZHANG Jintong, LIU Xin, et al. Edge–cloud resource scheduling in space–air–ground-integrated networks for internet of vehicles[J]. IEEE Internet of Things Journal, 2022, 9(8): 5765–5772. doi: 10.1109/JIOT.2021.3065583.
[18]	LI Zhipeng, LI Meng, and WANG Qian. Predator–prey model based asymmetry resource allocation in satellite–terrestrial network[J]. Symmetry, 2021, 13(11): 2113. doi: 10.3390/sym13112113.
[19]	LEE Y and CHOI J P. Connectivity analysis of mega-constellation satellite networks with optical intersatellite links[J]. IEEE Transactions on Aerospace and Electronic Systems, 2021, 57(6): 4213–4226. doi: 10.1109/TAES.2021.3090914.
[20]	ZHANG Hangyu, LIU Rongke, KAUSHIK A, et al. Satellite edge computing with collaborative computation offloading: An intelligent deep deterministic policy gradient approach[J]. IEEE Internet of Things Journal, 2023, 10(10): 9092–9107. doi: 10.1109/JIOT.2022.3233383.
[21]	GAO Xiangqiang, HU Yingmeng, SHAO Yingzhao, et al. Hierarchical dynamic resource allocation for computation offloading in LEO satellite networks[J]. IEEE Internet of Things Journal, 2024, 11(11): 19470–19484. doi: 10.1109/JIOT.2024.3367937.
[22]	易必杰. 面向空天地一體化網(wǎng)絡(luò)的計算卸載策略研究[D]. [博士/碩士論文], 西安電子科技大學(xué), 2023. doi: 10.27389/d.cnki.gxadu.2023.002544. YI Bijie. Research on computing offloading strategy for space-air-ground integrated network[D]. Xidian University, 2023. doi: 10.27389/d.cnki.gxadu.2023.002544.
[23]	FANG Hai, JIA Yangyang. WANG Yuanle, et al. Matching game based task offloading and resource allocation algorithm for satellite edge computing networks[C]. Proceedings of 2022 International Symposium on Networks, Computers and Communications (ISNCC), Shenzhen, China, 2022: 1–5, doi: 10.1109/ISNCC55209.2022.9851813.
[24]	郭子楨, 梁俊, 肖楠, 等. 軟件定義衛(wèi)星網(wǎng)絡(luò)多控制器可靠部署算法[J]. 西安交通大學(xué)學(xué)報, 2021, 55(2): 158–165. doi: 10.7652/xjtuxb202102019. GUO Zizhen, LIANG Jun, XIAO Nan, et al. Multi-controller reliable deployment algorithm for software defined satellite network[J]. Journal of Xi’an Jiaotong University, 2021, 55(2): 158–165. doi: 10.7652/xjtuxb202102019.
[25]	KUROSE J and ROSS K. Computer Networking: A Top-Down Approach[M]. 6th ed. Boston: Pearson, 2012.
[26]	謝希仁. 計算機網(wǎng)絡(luò)[M]. 4版. 大連: 大連理工大學(xué)出版社, 2003. XIE Xiren. Computer Networking[M]. 4th ed. Dalian: Dalian University of Technology Press, 2003.
[27]	QI Xiaoxin, ZHANG Bing, QIU Zhiliang, et al. Using inter-mesh links to reduce end-to-end delay in walker delta constellations[J]. IEEE Communications Letters, 2021, 25(9): 3070–3074. doi: 10.1109/LCOMM.2021.3095227.
[28]	YOU Changsheng, HUANG Kaibin, and CHAE H. Energy efficient mobile cloud computing powered by wireless energy transfer[J]. IEEE Journal on Selected Areas in Communications, 2016, 34(5): 1757–1771. doi: 10.1109/JSAC.2016.2545382.
[29]	NOWAK R. Generalized binary search[C]. 2008 46th Annual Allerton Conference on Communication, Control, and Computing, Monticello, USA, 2008: 568–574. doi: 10.1109/ALLERTON.2008.4797609.
[30]	AVRIEL M. Nonlinear Programming: Analysis and Methods[M]. Englewood Cliffs: Prentice-Hall, 1976.
[31]	MNIH V, KAVUKCUOGLU K, SILVER D, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540): 529–533. doi: 10.1038/nature14236.
[32]	VAN HASSELT H, GUEZ A, and SILVER D. Deep reinforcement learning with double Q-learning[C]. The Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, 2016: 2094–2100.
[33]	SHUAI Jiaqi, CUI Haixia, HE Yejun, et al. Dynamic satellite edge computing offloading algorithm based on distributed deep learning[J]. IEEE Internet of Things Journal, 2024, 11(16): 27790–27802. doi: 10.1109/JIOT.2024.3404830.