一種基于聯(lián)邦學(xué)習(xí)資源需求預(yù)測的虛擬網(wǎng)絡(luò)功能遷移算法
doi: 10.11999/JEIT210743 cstr: 32379.14.JEIT210743
-
1.
重慶郵電大學(xué)通信與信息工程學(xué)院 重慶 400065
-
2.
重慶郵電大學(xué)移動通信重點實驗室 重慶 400065
基金項目: 國家自然科學(xué)基金(62071078)，重慶市教委科學(xué)技術(shù)研究項目(KJZD-M201800601)
A Virtual Network Function Migration Algorithm Based on Federated Learning Prediction of Resource Requirements
-
1.
School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
-
2.
Key Laboratory of Mobile Communications Technology, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
Funds: The National Natural Science Foundation of China (62071078), The Science and Technology Research Program of Chongqing Municipal Education Commission (KJZD-M201800601)
-
摘要: 針對網(wǎng)絡(luò)切片場景下時變網(wǎng)絡(luò)流量引起的虛擬網(wǎng)絡(luò)功能(VNF)遷移問題，該文提出一種基于聯(lián)邦學(xué)習(xí)的雙向門控循環(huán)單元(FedBi-GRU)資源需求預(yù)測的VNF遷移算法。該算法首先建立系統(tǒng)能耗和負(fù)載均衡的VNF遷移模型，然后提出一種基于分布式聯(lián)邦學(xué)習(xí)框架協(xié)作訓(xùn)練預(yù)測模型，并在此框架的基礎(chǔ)上設(shè)計基于在線訓(xùn)練的雙向門控循環(huán)單元(Bi-GRU)算法預(yù)測VNF的資源需求?；谫Y源預(yù)測結(jié)果，聯(lián)合系統(tǒng)能耗優(yōu)化和負(fù)載均衡，提出一種分布式近端策略優(yōu)化(DPPO)的遷移算法提前制定VNF遷移策略。仿真結(jié)果表明，兩種算法的結(jié)合有效地降低了網(wǎng)絡(luò)系統(tǒng)能耗并保證負(fù)載均衡。
-
關(guān)鍵詞:
- 虛擬網(wǎng)絡(luò)功能 /
- 預(yù)測 /
- 遷移 /
- 深度強化學(xué)習(xí)
Abstract: In order to solve the problem of virtual network function migration caused by time-varying network traffic in network slicing, a Virtual Network Function (VNF) migration algorithm based on Federated learning with Bidirectional Gate Recurrent Units (FedBi-GRU) prediction of resource requirements is proposed. Firstly, a VNF migration model of system energy consumption and load balancing is established, and then a framework based on distributed federated learning is introduced to cooperatively train the predictive model. Secondly, considering predicting the resource requirements of VNF, an online training Bidirectional Gate Recurrent Unit (Bi-GRU) algorithm on the basis of the framework is designed. Finally, on the grounds of the resource prediction results, system energy consumption optimization and load balancing are combined, and a Distributed Proximal Policy Optimization (DPPO) migration algorithm is proposed to formulate a VNF migration strategy in advance. The simulation results show that the combination of the two algorithms reduces effectively the energy consumption of the network system and ensures the load balance.-
Key words:
- Virtual Network Function(VNF) /
- Prediction /
- Migration /
- Deep reinforcement learning
-
表 1 基于DDPO的VNF遷移算法
輸入：VNF的資源需求預(yù)測結(jié)果$ {r_{t + 1}} = \{ r_{t + 1}^{\text{C}},r_{t + 1}^{\text{M}},r_{t + 1}^{\text{B}}\} $，物理網(wǎng)絡(luò)圖$ {G^{\text{P}}} = ({N^{\text{P}}},{L^{\text{P}}}) $，SFC網(wǎng)絡(luò)圖$ G_i^{\text{V}} = (N_i^{\text{V}},L_i^{\text{V}}) $ 輸出：VNF映射策略$ \pi $ (1) 根據(jù)VNF的資源需求預(yù)測結(jié)果，計算各個物理節(jié)點的資源利用率${\eta _{\rm{R}}}$ (2) if ${\eta _{\rm{R} } } \le \eta _{\rm{R} }^{\textq7j3ldu95 }\& \& {\eta _{\rm{R} } } \ge \eta _{\rm{R}}^{ {\text{up} } }$ then (3) 初始化全局參數(shù)$ ({\theta _{\text{c}}},{\theta _{\text{a}}}) $，局部參數(shù)$ (\theta _{\text{c}}^n,\theta _{\text{a}}^n) $，全局PPO網(wǎng)絡(luò)最大迭代次數(shù)$ {K_{{\text{max}}}} $，局部PPO網(wǎng)絡(luò)最大迭代次數(shù)$ M $，線程數(shù)$ N $，學(xué)習(xí)率
$ ({\varepsilon _{\text{c}}},{\varepsilon _{\text{a}}}) $(4) for ${\text{thread} } = 1, 2,\cdots ,N$ do (5) for ${\text{episode} } = 1,2, \cdots ,M$ do (6) 從本地Actor網(wǎng)絡(luò)的策略$ \pi ({s_n}(t)\left| {{a_n}(t),\theta _{\text{a}}^n} \right.) $中選取映射動作$ a(t) $ (7) if $ {\eta _1} \in (\eta _1^{\textq7j3ldu95},\eta _1^{{\text{up}}})\& \& {\eta _2} \in (\eta _2^{\textq7j3ldu95},\eta _2^{{\text{up}}})\& \& {\eta _3} \in (\eta _3^{\textq7j3ldu95},\eta _3^{{\text{up}}})\& \& T \le {T_{{\text{tot}}}} $ then (8) 執(zhí)行動作$ a(t) $，根據(jù)式(16)得到瞬時獎勵$ r(t) $，并轉(zhuǎn)移到狀態(tài)$ s(t + 1) $ (9) 從本地Actor網(wǎng)絡(luò)獲得優(yōu)勢函數(shù)$ A({s_n}(t),{a_n}(t)) $ (10) else (11) 式(16)瞬時獎勵$ r(t) = - {1 \mathord{\left/ {\vphantom {1 \varepsilon }} \right. } \varepsilon } $，從本地${\rm{Actor}}$網(wǎng)絡(luò)重新選取動作$ a(t) $ (12) end if (13) end for (14) 根據(jù)式(24)更新全局PPO的Critic網(wǎng)絡(luò)累計梯度$ \Delta {\theta _{\text{c}}} $ (15) 根據(jù)式(26)更新全局PPO的Actor網(wǎng)絡(luò)累計梯度$ \Delta {\theta _{\text{a}}} $ (16) 將$ \Delta {\theta _{\text{c}}} $和$ \Delta {\theta _a} $推送至全局PPO網(wǎng)絡(luò)進行異步更新 (17) $ {\theta _{\text{c}}} \leftarrow {\theta _{\text{c}}} + {\varepsilon _{\text{c}}}\Delta {\theta _{\text{c}}} $,$ {\theta _{\text{a}}} \leftarrow {\theta _{\text{a}}} + {\varepsilon _{\text{a}}}\Delta {\theta _{\text{a}}} $ (18) end for (19) 同步全局PPO網(wǎng)絡(luò)參數(shù)至本地PPO網(wǎng)絡(luò)參數(shù)：$ \theta _{\text{c}}^{n'} = {\theta _{\text{c}}} $, $ \theta {_{\text{a}}^{n'}} = \theta _{\text{a}}^{} $ (20) 繼續(xù)執(zhí)行步驟4—步驟17 (21) until $ K \ge {K_{{\text{max}}}} $ (22) end if 
下載: 導(dǎo)出CSV
表 2 仿真參數(shù)
仿真參數(shù) 描述 取值 $ {N^{\text{P}}} $ 物理節(jié)點數(shù)量 22 $ P_n^{\text} $ 物理節(jié)點待機能耗 Uniform[100,150](W) $ P_n^{{\text{cpu}}} $ 物理節(jié)點CPU滿載能耗 Uniform[250,300](W) $ P_m^{\text{s}} $ 物理節(jié)點狀態(tài)切換能耗 Uniform[15,25](W) $ {C_n} $ 物理節(jié)點CPU資源容量 Uniform[200,300](units) $ {M_n} $ 物理節(jié)點存儲資源容量 Uniform[300,400](Mbps) $ {B_{nm}} $ 物理鏈路$ {l_{nm}} $帶寬容量 Uniform[80,100](Mbps) $ F $ SFC集合數(shù)量 30個 $ N_i^{\text{V}} $ VNF集合長度 Uniform[3,5](個) $ {T_{{\text{tot}}}} $ SFC端到端時延限制 30 ms
下載: 導(dǎo)出CSV
-
[1] LI Defang, HONG Peilin, XUE Kaiping, et al. Availability aware VNF deployment in datacenter through shared redundancy and multi-tenancy[J]. IEEE Transactions on Network and Service Management, 2019, 16(4): 1651–1664. doi: 10.1109/TNSM.2019.2936505 [2] QU Kaige, ZHUANG Weihua, YE Qiang, et al. Dynamic flow migration for embedded services in SDN/NFV-enabled 5G core networks[J]. IEEE Transactions on Communications, 2020, 68(4): 2394–2408. doi: 10.1109/TCOMM.2020.2968907 [3] LIU Yicen, LU Hao, LI Xi, et al. An approach for service function chain reconfiguration in network function virtualization architectures[J]. IEEE Access, 2019, 7: 147224–147237. doi: 10.1109/ACCESS.2019.2946648 [4] TANG Lun, HE Xiaoyu, ZHAO Peipei, et al. Virtual network function migration based on dynamic resource requirements prediction[J]. IEEE Access, 2019, 7: 112348–112362. doi: 10.1109/ACCESS.2019.2935014 [5] LIU Yicen, LU Yu, LI Xi, et al. On dynamic service function chain reconfiguration in IoT networks[J]. IEEE Internet of Things Journal, 2020, 7(11): 10969–10984. doi: 10.1109/JIOT.2020.2991753 [6] HUANG Yuzhe, XU Huahu, GAO Honghao, et al. SSUR: An approach to optimizing virtual machine allocation strategy based on user requirements for cloud data center[J]. IEEE Transactions on Green Communications and Networking, 2021, 5(2): 670–681. doi: 10.1109/TGCN.2021.3067374 [7] DAYARATHNA M, WEN Yonggang, and FAN Rui. Data center energy consumption modeling: A survey[J]. IEEE Communications Surveys & Tutorials, 2015, 18(1): 732–794. doi: 10.1109/COMST.2015.2481183 [8] ERAMO V, AMMAR M, and LAVACCA F G. Migration energy aware reconfigurations of virtual network function instances in NFV architectures[J]. IEEE Access, 2017, 5: 4927–4938. doi: 10.1109/ACCESS.2017.2685437 [9] HAN Zhenhua, TAN Haisheng, WANG Rui, et al. Energy-efficient dynamic virtual machine management in data centers[J]. IEEE/ACM Transactions on Networking, 2019, 27(1): 344–360. doi: 10.1109/TNET.2019.2891787 [10] ZHANG Zhongbao, CAO Huafeng, SU Sen, et al. Energy aware virtual network migration[J]. IEEE Transactions on Cloud Computing, 2022, 10(2): 1173–1189. doi: 10.1109/TCC.2020.2976966. [11] GUO Zehua, XU Yang, LIU Yafeng, et al. AggreFlow: Achieving power efficiency, load balancing, and quality of service in data center networks[J]. IEEE/ACM Transactions on Networking, 2020, 29(1): 17–33. doi: 10.1109/TNET.2020.3026015 [12] LI Biyi, CHENG Bo, LIU Xuan, et al. Joint resource optimization and delay-aware virtual network function migration in data center networks[J]. IEEE Transactions on Network and Service Management, 2021, 18(3): 2960–2974. doi: 10.1109/TNSM.2021.3067883 [13] ZHANG Kunpeng, WU Lan, ZHU Zhaoju, et al. A multitask learning model for traffic flow and speed forecasting[J]. IEEE Access, 2020, 8: 80707–80715. doi: 10.1109/ACCESS.2020.2990958 [14] LIU Yi, JAMES J J Q, KANG Jiawen, et al. Privacy-preserving traffic flow prediction: A federated learning approach[J]. IEEE Internet of Things Journal, 2020, 7(8): 7751–7763. doi: 10.1109/JIOT.2020.2991401 [15] ZHANG Zhenyu, LUO Xiangfeng, LIU Tong, et al. Proximal policy optimization with mixed distributed training[C]. The 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), Portland, USA, 2019: 1452–1456. [16] SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[J]. arXiv preprint arXiv: 1707.06347, 2017. [17] BEN YAHIA I G, BENDRISS J, SAMBA A, et al. CogNitive 5G networks: Comprehensive operator use cases with machine learning for management operations[C]. 2017 20th Conference on Innovations in Clouds, Internet and Networks (ICIN), Paris, France, 2017: 252–259. [18] BENDRISS J, BEN YAHIA I G, and ZEGHLACHE D. Forecasting and anticipating SLO breaches in programmable networks[C]. 2017 20th Conference on Innovations in Clouds, Internet and Networks (ICIN), Paris, France, 2017: 127–134. [19] BENDRISS J. Cognitive management of SLA in software-based networks[D]. [Ph. D. dissertation], Institut National des Télécommunications, 2018. -
圖(7) / 表(2)
計量
- 文章訪問數(shù): 1073
- HTML全文瀏覽量: 770
- PDF下載量: 178
- 被引次數(shù): 0
下載:
