多接入邊緣計算賦能的AI質(zhì)檢系統(tǒng)任務(wù)實時調(diào)度策略

周曉天; 孫上; 張海霞; 鄧伊琴; 魯彬彬

doi:10.11999/JEIT230129

多接入邊緣計算賦能的AI質(zhì)檢系統(tǒng)任務(wù)實時調(diào)度策略

doi: 10.11999/JEIT230129 cstr: 32379.14.JEIT230129

1.
山東大學(xué)控制科學(xué)與工程學(xué)院濟南 250061
2.
山東省無線通信技術(shù)重點實驗室濟南 250100

基金項目: 國家自然科學(xué)基金(61860206005, U22A2003, 61971270)

詳細(xì)信息

作者簡介:
周曉天：男，副教授，研究方向為無線通信與網(wǎng)絡(luò)、邊緣計算與智能通信等

孫上：女，碩士生，研究方向為物聯(lián)網(wǎng)、邊緣計算等

張海霞：女，教授，研究方向為無線通信與網(wǎng)絡(luò)、無線資源管理、智能通信技術(shù)等

鄧伊琴：女，博士后，研究方向為邊緣計算、車聯(lián)網(wǎng)、無線資源優(yōu)化等

魯彬彬：男，碩士，研究方向為車聯(lián)網(wǎng)、計算卸載、資源優(yōu)化等

通訊作者:
張海霞　haixia.zhang@sdu.edu.cn

中圖分類號: TN929.5; TP18
計量
- 文章訪問數(shù): 827
- HTML全文瀏覽量: 606
- PDF下載量: 105
- 被引次數(shù): 0
出版歷程
- 收稿日期: 2023-03-03
- 修回日期: 2023-08-15
- 網(wǎng)絡(luò)出版日期: 2023-08-17
- 刊出日期: 2024-02-29

Real-time Task Scheduling for Multi-access Edge Computing-enabled AI Quality Inspection Systems

1.
School of Control Science and Engineering, Shandong University, Jinan 250061, China
2.
Shandong Key Laboratory of Wireless Communication Technologies, Shandong University, Jinan 250100, China

Funds: The National Natural Science Foundation of China (61860206005, U22A2003, 61971270)

摘要

摘要: AI質(zhì)檢是智能制造的重要環(huán)節(jié)，其設(shè)備在進(jìn)行產(chǎn)品質(zhì)量檢測時會產(chǎn)生大量計算密集型和時延敏感型任務(wù)。由于設(shè)備計算能力不足，執(zhí)行檢測任務(wù)時延較大，極大影響生產(chǎn)效率。多接入邊緣計算(MEC)通過將任務(wù)卸載至邊緣服務(wù)器為設(shè)備提供就近算力，提升任務(wù)執(zhí)行效率。然而，系統(tǒng)中存在信道變化和任務(wù)隨機到達(dá)等動態(tài)因素，極大影響卸載效率，給任務(wù)調(diào)度帶來了挑戰(zhàn)。該文面向多接入邊緣計算賦能的AI質(zhì)檢任務(wù)調(diào)度系統(tǒng)，研究了聯(lián)合任務(wù)調(diào)度與資源分配的長期時延最小化問題。由于該問題狀態(tài)空間大、動作空間包含連續(xù)變量，該文提出運用深度確定性策略梯度(DDPG)進(jìn)行實時任務(wù)調(diào)度算法設(shè)計。所設(shè)計算法可基于系統(tǒng)實時狀態(tài)信息給出最優(yōu)決策。仿真結(jié)果表明，與基準(zhǔn)算法相比，該文所提算法具有更好的性能表現(xiàn)和更小的任務(wù)執(zhí)行時延。
- 多接入邊緣計算 /
- 任務(wù)調(diào)度 /
- 資源分配 /
- 深度強化學(xué)習(xí) /
- AI質(zhì)檢系統(tǒng)
Abstract: AI-based quality inspection is an important part of intelligent manufacturing, where the devices produce a large amount of computation-intensive and time-sensitive tasks. Owing to the insufficient computation capability of end devices, the latency to execute these inspection tasks is large, which greatly affects manufacturing efficiency. To this end, Multi-access Edge Computing (MEC) is proposed to provide computation resources through offloading tasks to the edge servers deployed nearby. The execution efficiency is therefore improved. However, the dynamic channel state and random task arrival greatly impact the task offloading efficiency and consequently bring challenges to task scheduling. In this paper, the joint task scheduling and resource allocation problem with the purpose of minimizing the long-term delay of MEC-enabled system is studied. As the state space of the problem is large and the action space contains continuous variables, a Deep Deterministic Policy Gradient (DDPG) based real-time task scheduling algorithm is proposed. The proposed algorithm can make optimal decision with real-time system state information. Simulation results confirm the promising performance of the proposed algorithm, which achieves lower task execution latency than that of the benchmark algorithm.
- Multi-access Edge Computing (MEC) /
- Task scheduling /
- Resource allocation /
- Deep reinforcement learning /
- AI-based quality inspection system

HTML全文

圖 1 MEC賦能的AI質(zhì)檢系統(tǒng)任務(wù)調(diào)度結(jié)構(gòu)圖

下載: 全尺寸圖片幻燈片

圖 2 算法結(jié)構(gòu)圖

下載: 全尺寸圖片幻燈片

圖 3 不同學(xué)習(xí)率下的累積獎勵

下載: 全尺寸圖片幻燈片

圖 4 系統(tǒng)長期任務(wù)執(zhí)行時延方案對比

下載: 全尺寸圖片幻燈片

圖 5 系統(tǒng)平均任務(wù)執(zhí)行時延方案對比

下載: 全尺寸圖片幻燈片

算法1 基于DDPG的AI質(zhì)檢任務(wù)實時調(diào)度算法
輸入：估計網(wǎng)絡(luò)參數(shù)$ {\theta _0} $和目標(biāo)網(wǎng)絡(luò)參數(shù)$ \theta {'_0} $;
其他基本參數(shù)$ \gamma ,N,T,{f^l},{f^c},K,\lambda ,B,{N_0},\xi ,\varepsilon $;
輸出：訓(xùn)練完成的Actor網(wǎng)絡(luò)模型參數(shù)
(1) For ep = 1, 2, ···, K :
(2) 　　初始化AI質(zhì)檢系統(tǒng)環(huán)境，得到環(huán)境的初始狀態(tài) s(0);
(3) 　　For t = 1, 2, ···, T :
(4) 　　　　根據(jù)Actor網(wǎng)絡(luò)的輸出疊加 OU 噪聲后選擇一個動　　　　　　作a(t) 輸入環(huán)境;
(5) 　　　　觀測AI質(zhì)檢系統(tǒng)環(huán)境的輸出獎勵和下一時刻的狀態(tài);
(6) 　　　　將元組$\left( {{\boldsymbol{s}}(t),{\boldsymbol{a}}(t),r(t),{\boldsymbol{s}}(t + 1)} \right)$存儲到經(jīng)驗池中;
(7) 　　　　從經(jīng)驗池中選擇小批量數(shù)據(jù);
(8) 　　　　按式 (13) 更新估計網(wǎng)絡(luò)的Actor網(wǎng)絡(luò)參數(shù);
(9) 　　　　按式 (14) 更新估計網(wǎng)絡(luò)的Critic網(wǎng)絡(luò)參數(shù);
(10) 　　　按式 (15) 更新目標(biāo)網(wǎng)絡(luò)的參數(shù);
(11) 　 End
(12) End

下載: 導(dǎo)出CSV

表 1 仿真參數(shù)設(shè)置

參數(shù)名	參數(shù)值	參數(shù)名	參數(shù)值
AI質(zhì)檢設(shè)備數(shù)量 N	[6, 14]	MEC服務(wù)器CPU頻率 f^c	[4, 8] GHz
任務(wù)數(shù)據(jù)量大小 d_i	[30, 50] KB	設(shè)備的CPU頻率 f^l	0.5 GHz
任務(wù)所需計算資源大小 k_i	[1 800, 2 600] cycle/Byte	設(shè)備的傳輸功率 p_n	25 dBm
子信道帶寬 B	1 MHz	高斯噪聲功率譜密度 N₀	–174 dBm/Hz
時隙長度 T_s	100 ms	經(jīng)驗池大小	200 000
回合周期數(shù) T	1 000	溢出懲罰參數(shù) ξ	10
折扣因子$\gamma $	0.99	路徑損耗常數(shù)${\beta _1}$	10^–12.7
任務(wù)隊列最大長度	10	路徑損耗指數(shù)${\beta _2}$	3
設(shè)備任務(wù)平均到達(dá)率 λ	[4, 10] s^–1	設(shè)備到基站距離d	[0,500] m

下載: 導(dǎo)出CSV

表 2 算法參數(shù)設(shè)置

網(wǎng)絡(luò)	參數(shù)名	參數(shù)值
Actor	學(xué)習(xí)率	[10^–5, 10^–4]
	隱藏層個數(shù)	3
	隱藏層神經(jīng)元數(shù)量	64, 64, 64
	激活函數(shù)	LeakyReLU, Softmax, Sigmoid
	軟替換策略參數(shù) ε	0.01
Critic	學(xué)習(xí)率	[10^–5, 10^–4]
	隱藏層個數(shù)	3
	隱藏層神經(jīng)元數(shù)量	64, 64, 64
	激活函數(shù)	LeakyReLU, Tanh
	軟替換策略參數(shù) ε	0.01

下載: 導(dǎo)出CSV

參考文獻(xiàn)(22)

[1]	周華, 鄭榮, 肖榮. 工業(yè)場景下AI質(zhì)檢關(guān)鍵技術(shù)及平臺架構(gòu)研究[J]. 現(xiàn)代信息科技, 2022, 6(5): 149–151,156. doi: 10.19850/j.cnki.2096-4706.2022.05.039. ZHOU Hua, ZHENG Rong, and XIAO Rong. Research on key technology and platform architecture of AI quality inspection under industrial scene[J]. Modern Information Technology, 2022, 6(5): 149–151,156. doi: 10.19850/j.cnki.2096-4706.2022.05.039.
[2]	蔣音. 深度學(xué)習(xí)技術(shù)開啟工業(yè)AI質(zhì)檢新范式[J]. 大數(shù)據(jù)時代, 2022(11): 38–48. JIANG Yin. Deep learning offers a new paradigm of quality inspection supported by industrial AI[J]. Big Data Time, 2022(11): 38–48.
[3]	DAI Yueyue, ZHANG Ke, MAHARJAN S, et al. Deep reinforcement learning for stochastic computation offloading in digital twin networks[J]. IEEE Transactions on Industrial Informatics, 2021, 17(7): 4968–4977. doi: 10.1109/TII.2020.3016320.
[4]	胡致遠(yuǎn), 胡文前, 李香, 等. 面向業(yè)務(wù)可達(dá)性的廣域工業(yè)互聯(lián)網(wǎng)調(diào)度算法研究[J]. 電子與信息學(xué)報, 2021, 43(9): 2608–2616. doi: 10.11999/JEIT200583. HU Zhiyuan, HU Wenqian, LI Xiang, et al. Research on wide area industrial internet scheduling algorithm based on service reachability[J]. Journal of Electronics &Information Technology, 2021, 43(9): 2608–2616. doi: 10.11999/JEIT200583.
[5]	BAHRAMI M. Cloud computing for emerging mobile cloud apps[C]. 2015 3rd IEEE International Conference on Mobile Cloud Computing, Services, and Engineering, San Francisco, USA, 2015: 4–5.
[6]	ZHANG Fan, HAN Guanjie, LIU Li, et al. Deep reinforcement learning based cooperative partial task offloading and resource allocation for IIoT applications[J]. IEEE Transactions on Network Science and Engineering, 2022: 1.
[7]	MAO Yuyi, YOU Changsheng, ZHANG Jun, et al. A survey on mobile edge computing: The communication perspective[J]. IEEE Communications Surveys & Tutorials, 2017, 19(4): 2322–2358. doi: 10.1109/COMST.2017.2745201.
[8]	李一倩, 劉留, 李慧婷, 等. 工業(yè)物聯(lián)網(wǎng)無線信道特性研究[J]. 物聯(lián)網(wǎng)學(xué)報, 2019, 3(4): 34–47. doi: 10.11959/j.issn.2096-3750.2019.00130. LI Yiqian, LIU Liu, LI Huiting, et al. Research on characteristics of industrial IoT wireless channel[J]. Chinese Journal on Internet of Things, 2019, 3(4): 34–47. doi: 10.11959/j.issn.2096-3750.2019.00130.
[9]	張克, 劉留, 袁澤, 等. 工業(yè)物聯(lián)網(wǎng)無線信道與噪聲特性[J]. 電信科學(xué), 2018, 34(8): 87–97. doi: 10.11959/j.issn.1000-0801.2018217. ZHANG Ke, LIU Liu, YUAN Ze, et al. Wireless channel and noise characteristics in industrial internet of things[J]. Telecommunications Science, 2018, 34(8): 87–97. doi: 10.11959/j.issn.1000-0801.2018217.
[10]	GUO Kai, YANG Mingcong, ZHANG Yongbing, et al. Joint computation offloading and bandwidth assignment in cloud-assisted edge computing[J]. IEEE Transactions on Cloud Computing, 2022, 10(1): 451–460. doi: 10.1109/TCC.2019.2950395.
[11]	YANG Lei, LIU Bo, CAO Jiannong, et al. Joint computation partitioning and resource allocation for latency sensitive applications in mobile edge clouds[J]. IEEE Transactions on Services Computing, 2021, 14(5): 1439–1452. doi: 10.1109/TSC.2018.2890603.
[12]	劉斐, 曹鈺杰, 章國安. 車聯(lián)網(wǎng)場景下移動邊緣計算協(xié)作式資源分配策略[J]. 電訊技術(shù), 2021, 61(7): 858–864. doi: 10.3969/j.issn.1001-893x.2021.07.012. LIU Fei, CAO Yujie, and ZHANG Guoan. Collaborative resource allocation strategy for mobile edge computing in vehicular networks[J]. Telecommunication Engineering, 2021, 61(7): 858–864. doi: 10.3969/j.issn.1001-893x.2021.07.012.
[13]	周天清, 曾新亮, 胡海琴. 基于混合粒子群算法的計算卸載成本優(yōu)化[J]. 電子與信息學(xué)報, 2022, 44(9): 3065–3074. doi: 10.11999/JEIT211390. ZHOU Tianqing, ZENG Xinliang, and HU Haiqin. Computation offloading cost optimization based on hybrid particle swarm optimization algorithm[J]. Journal of Electronics &Information Technology, 2022, 44(9): 3065–3074. doi: 10.11999/JEIT211390.
[14]	周天清, 胡海琴, 曾新亮. NOMA-MEC系統(tǒng)中基于改進(jìn)遺傳算法的協(xié)作式計算卸載與資源管理[J]. 電子與信息學(xué)報, 2022, 44(9): 3014–3023. doi: 10.11999/JEIT220306. ZHOU Tianqing, HU Haiqin, and ZENG Xinliang. Cooperative computation offloading and resource management based on improved genetic algorithm in NOMA-MEC systems[J]. Journal of Electronics &Information Technology, 2022, 44(9): 3014–3023. doi: 10.11999/JEIT220306.
[15]	LUO Quyuan, LI Changle, LUAN T H, et al. Collaborative data scheduling for vehicular edge computing via deep reinforcement learning[J]. IEEE Internet of Things Journal, 2020, 7(10): 9637–9650. doi: 10.1109/JIOT.2020.2983660.
[16]	ZHANG Weiting, YANG Dong, PENG Haixia, et al. Deep reinforcement learning based resource management for DNN inference in industrial IoT[J]. IEEE Transactions on Vehicular Technology, 2021, 70(8): 7605–7618. doi: 10.1109/TVT.2021.3068255.
[17]	CHEN Ying, LIU Zhiyong, ZHANG Yongchao, et al. Deep reinforcement learning-based dynamic resource management for mobile edge computing in industrial internet of things[J]. IEEE Transactions on Industrial Informatics, 2021, 17(7): 4925–4934. doi: 10.1109/TII.2020.3028963.
[18]	YU Shuai, CHEN Xu, ZHOU Zhi, et al. When deep reinforcement learning meets federated learning: Intelligent multitimescale resource management for multiaccess edge computing in 5G ultradense network[J]. IEEE Internet of Things Journal, 2021, 8(4): 2238–2251. doi: 10.1109/JIOT.2020.3026589.
[19]	SUTTON R S and BARTO A G. Reinforcement Learning: An Introduction[M]. 2nd ed. Cambridge: Bradford Books, 2018: 62–64.
[20]	LIU Binghong, LIU Chenxi and PENG Mugen. Computation offloading and resource allocation in unmanned aerial vehicle networks[J]. IEEE Transactions on Vehicular Technology, 2023, 72(4): 4981–4995. doi: 10.1109/TVT.2022.3222907.
[21]	DAI Bin, NIU Jianwei, REN Tao, et al. Toward mobility-aware computation offloading and resource allocation in end–edge–cloud orchestrated computing[J]. IEEE Internet of Things Journal, 2022, 9(19): 19450–19462. doi: 10.1109/JIOT.2022.3168036.
[22]	QIAO Guanhua, LENG Supeng, MAHARJAN S, et al. Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks[J]. IEEE Internet of Things Journal, 2020, 7(1): 247–257. doi: 10.1109/JIOT.2019.2945640.