基于GPR模型的用户量预测优化方法

doi:10.12305/j.issn.1001-506X.2024.08.19

Abstract

Abstract:

Gaussian process regression (GPR) is a non-parametric Bayesian regression method based on Gaussian processes. It is flexible in adapting to different types of data, and it is used to model and predict complex relationships between different types of data. It has strong fitting capabilities and good generalization abilities. A user quantity prediction optimization method based on GPR is proposed to tackle the problem of real-time user quantity prediction in the context of massive user scenario. Building upon the sliding window method for data processing, the method selects a suitable kernel function and uses k-fold cross-validation to determine the optimal hyperparameter combination for training the GPR model, which enables the real-time prediction of online user quantity. Finally, the performance of the model is evaluated. The experimental results demonstrate that compared with the traditional approach that uses half of the variance of the output data in the training set as the signal noise estimator, the proposed method has higher prediction accuracy and improvements in the four following evaluation metrics of root mean square (RMS), mean absolute error (MAE), mean bias error (MBE) and determination coefficient R² on the test set. Specifically, the MBE shows an improvement of at least 43.3%.

Key words: Gaussian process regression (GPR), user quantity prediction, sliding window, cross-validation, hyperparameter optimization

CLC Number:

TP391.7

Xuehao LIU, Wenxue LIU, Chaosan YANG, Wenjing ZHU, Yu SONG, Jinhai LI. Optimization method of user quantity prediction based on GPR model[J]. Systems Engineering and Electronics, 2024, 46(8): 2721-2729.

Figures/Tables 21

Fig.1

Table 1

Table 2

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Table 3

Fig.13

Fig.14

Table 4

Fig.15

Fig.16

Table 5

References 31

11	LIANG Y Z. An automatic detection method for epileptic EEG signals based on empirical mode decomposition and permutation entropy[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2023.
12	杨雨亭. 基于RF特征优选的ISSA-SVM变压器故障诊断方法[D]. 南京: 南京邮电大学, 2023.
	YANG Y T. ISSA-SVM transformer fault diagnosis method based on RF feature selection[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2023.
13	ZHANG S Y, TAN W A, LI Y B. Survey of kernel extreme learning machine kernel function based on the perspective of kernel parameter optimization time[C]//Proc. of the 4th Annual International Conference on Network and Information Systems for Computers, 2018: 430-433.
14	PARAL P , GHOSH S , CHATTERJEE A , et al. Automatic relevance determination kernel-embedded Gaussian process regression for sonar-based human leg localization with a mobile robot[J]. IEEE Sensors Letters, 2022, 7 (1): 6000504.
15	DONG W H , LI X F , BI D J , et al. Matern kernel adaptive filtering with Nystrom approximation for indoor localization[J]. IEEE Trans.on Instrumentation and Measurement, 2023,
16	TRONARP F, KARVONEN T, SARKKA S. Mixture representation of the Matern class with applications in state space approximations and Bayesian quadrature[C]//Proc. of the IEEE 28th International Workshop on Machine Learning for Signal Processing, 2018.
17	LIU K L , LI Y , HU X S , et al. Gaussian process regression with automatic relevance determination kernel for calendar aging prediction of lithiumion batteries[J]. IEEE Trans.on Industrial Informatics, 2019, 16 (6): 3767- 3777.
18	TAGHAVIFAR H , MARDANI A . Gaussian process with automatic relevance determination predictive model for energy management of electric direct-drive wheels: experimental validation[J]. IEEE Trans.on Vehicular Technology, 2023, 73 (2): 1910- 1917.
19	LIU T, CHAI W, WANG C C. Soft-sensors based on Gaussian process regression for wastewater treatment plants[C]//Proc. of the IEEE 11th Data Driven Control and Learning Systems Conference, 2022: 437-442.
20	杨成飞. 基于高斯过程的智能采样策略研究[D]. 合肥: 中国科学技术大学, 2019.
	YANG C F. Adaptive sampling strategy based on Gaussian process[D]. Hefei: University of Science and Technology of China, 2019.
1	王继民, 彭波. 搜索引擎用户访问量模型[J]. 计算机工程与应用, 2004, (25): 9-11, 30. doi: 10.3321/j.issn:1002-8331.2004.25.003
	WANG J M , PENG B . Modeling quantity of users' access for search engine[J]. Computer Engineering and Applications, 2004, (25): 9-11, 30. doi: 10.3321/j.issn:1002-8331.2004.25.003
2	程鹏超, 杜军平, 薛哲. 基于多路交叉的用户金融行为预测[J]. 智能系统学报, 2021, 16 (2): 378- 384.
	CHENG P C , DU J P , XUE Z . Prediction of user financial behavior based on multi-way crossing[J]. CAAI Transactions on Intelligent Systems, 2021, 16 (2): 378- 384.
3	XIAO H B, XIAO J H, DENG X W, et al. Traffic flow prediction based on traffic and meteorological data fusion in non-stationary environments[C]//Proc. of the International Conference on Electronic Information Technology and Smart Agriculture, 2021: 154-158.
4	LEE J, LEE H Y, KIM N W, et al. A study on online arima algorithms applying various gradient descent optimization algorithms for time series prediction[C]//Proc. of the International Conference on Information and Communication Technology Convergence, 2021: 1104-1106.
5	PENG Y N , XIANG W L . Short-term traffic volume prediction using GA-BP based on wavelet denoising and phase space reconstruction[J]. Physica A: Statistical Mechanics and its Applications, 2020, 549, 123913. doi: 10.1016/j.physa.2019.123913
6	LI R M , HU Y C , LIANG Q H . T2F-LSTM method for long-term traffic volume prediction[J]. IEEE Trans.on Fuzzy Systems, 2020, 28 (12): 3256- 3264. doi: 10.1109/TFUZZ.2020.2986995
7	BECKERS T. An introduction to Gaussian process models[EB/OL]. [2024-01-10]. https://arXivpreprintarXiv:2102.05497, 2021.
8	KRISHNAN V , CHANDRA K . Probability and random proce-sses[M]. Hoboken: John Wiley & Sons, 2016.
9	JAKKALA K. Deep Gaussian processes: a survey[EB/OL]. [2024-01-10]. https://arXivpreprintarXiv:2106.12135, 2021.
10	WILLIAMS C K I , RASMUSSEN C E . Gaussian processes for machine learning[M]. Cambridge: Massachusetts Institute of Technology press, 2006.
11	梁袁泽. 基于经验模态分解与排列熵的癫痫脑电信号自动检测方法[D]. 南京: 南京邮电大学, 2023.
21	ALBRECHT T , RAUSCH T M , DERRA N D . Call me maybe: methods and practical implementation of artificial intelligence in call center arrivals' forecasting[J]. Journal of Business Research, 2021, 123, 267- 278. doi: 10.1016/j.jbusres.2020.09.033
22	BERGMEIR C , COSTANTINI M , BENITEZ J M . On the usefulness of cross-validation for directional forecast evaluation[J]. Computational Statistics & Data Analysis, 2014, 76, 132- 143.
23	CERQUEIRA V , TORGO L , MOZETIC I . Evaluating time series forecasting models: an empirical study on performance estimation methods[J]. Machine Learning, 2020, 109 (11): 1997- 2028. doi: 10.1007/s10994-020-05910-7
24	ARLOT S , CELISSE A . A survey of cross-validation procedures for model selection[J]. Statistics Surveys, 2010, 4 (2010): 40- 79.
25	BERGMEIR C , HYNDMAN R J , KOO B . A note on the validity of cross-validation for evaluating autoregressive time series prediction[J]. Computational Statistics & Data Analysis, 2018, 120, 70- 83.
26	GUI C, SUN B L, SONG Y, et al. Variable length sliding window-based network coding algorithm in MANETs[C]//Proc. of the International Conference on Mobile and Ubiquitous Systems: Networking and Services, 2017.
27	MENG J F , GONG L , XU J . Sliding-window QPS (SW-QPS) a perfect parallel iterative switching algorithm for input-queued switches[J]. ACM Sigmetrics Performance Evaluation Review, 2021, 48 (3): 71- 76. doi: 10.1145/3453953.3453969
28	LI L G, ZHAO J S, QIANG B H, et al. Prediction method of fan main shaft fault state based on sliding window characteristics[C]//Proc. of the 10th International Conference on Internet Computing for Science and Engineering, 2021: 67-73.
29	VARDHAN B V S, KHEDKAR M, SURESH V. Hyper-parameter tuned short term load forecasting using stochastic classifier-regression mapping for power system operator[C]//Proc. of the IEEE PES 14th Asia-Pacific Power and Energy Engineering Conference, 2022.
30	LI Z H, SHOEMAKER C A. Hyper-parameter optimization for deep learning by surrogate-based model with weighted distance exploration[C]//Proc. of the IEEE Congress on Evolutionary Computation, 2021: 917-925.
31	SODA上海开放数据创新应用大赛. 摩拜上海城区用户使用数据[EB/OL]. [2024-01-10]. https://shanghai.sodachallenges.com/data.html.

核函数名	k(z, z′)=
常数核	φ₁²
线性核	z^Tz′
Matérn核$\check{p}=p+\frac{1}{2}, p \in \mathbf{N}$	$\begin{gathered}\varphi_1^2 \exp \left(-\frac{\sqrt{2 \check{p}}\left\\|z-z^{\prime}\right\\|}{\varphi_2}\right) \frac{p!}{(2 p)!} \\\sum\limits_{i=0}^p \frac{(p+i)!}{i!(p-i)!}\left(\frac{\sqrt{8 \check{p}}\left\\|z-z^{\prime}\right\\|}{\varphi_2}\right)^{p-i}\end{gathered}$
平方指数核	$\varphi_1^2 \exp \left(-\frac{\left\\|z-z^{\prime}\right\\|^2}{2 \varphi_2^2}\right)$
由平方指数核得到的周期核	$\varphi_1^2 \exp \left(-\frac{\sin \left(\frac{{\rm{ \mathsf{ π} }}}{p}\left\\|z-z^{\prime}\right\\|\right)^2}{2 \varphi_2^2}\right)$
平方指数自动相关性确定(automatic relevance determination, ARD)核	$\begin{gathered}\varphi_1^2 \exp \left(-\left(z-z^{\prime}\right)^{\mathrm{T}} \boldsymbol{P}^{-1}\left(z-z^{\prime}\right)\right), \\\boldsymbol{P}=\operatorname{diag}\left(\varphi_2^2, \varphi_3^2, \cdots, \varphi_{1+n_z}^2\right)\end{gathered}$

性能评估指标	表达式
RMS	$\sqrt{\frac{\sum\limits_{i=1}^n\left(\hat{x}_i-x_i\right)^2}{n}}$
MAE	$\frac{\sum\limits_{i=1}^n\left\|\hat{x}_i-x_i\right\|}{n}$
MBE	$\frac{\sum\limits_{i=1}^n \hat{x}_i-x_i}{n}$
R²	$1-\frac{\sum\limits_{i=1}^n\left(\hat{x}_i-x_i\right)^2}{\sum\limits_{i=1}^n\left(x_i-\bar{x}_i\right)^2}$

性能指标	不优化Sigma测试集评价	CV优化Sigma测试集评价	优化方法性能提升/%
RMS	20.942 7	15.584 9	25.6
MAE	12.803 2	10.007 6	21.8
MBE	-4.602 8	-2.611 9	43.3
R²	0.917 34	0.954 22	4.0

性能指标	不优化Sigma测试集评价	CV优化Sigma测试集评价	优化方法性能提升/%
RMS	22.412 4	14.593 5	34.9
MAE	13.718 9	9.375 9	31.7
MBE	-5.536	-2.040 9	63.1
R²	0.905 33	0.959 86	6.0

性能指标	不优化Sigma测试集评价	CV优化Sigma测试集评价	优化方法性能提升/%
RMS	20.089 1	15.517 3	22.8
MAE	12.687	10.066 2	20.7
MBE	-4.529 9	-2.391 8	47.2
R²	0.923 94	0.954 62	3.3

Optimization method of user quantity prediction based on GPR model

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 21

References 31

Related Articles 8

Recommended Articles

Metrics

Comments

[1]	Qiuping XU, Kai ZHAO, Detao QU, Gangdun LIU. LCSS-based fast matching method of target trajectory rule [J]. Systems Engineering and Electronics, 2022, 44(4): 1263-1269.
[2]	Gaoyue WANG, Huijun ZHANG, Xian CHEN, Hao LI. Prediction method of spacecraft flight capability in atmospheric entry phase based on Gaussian process regression [J]. Systems Engineering and Electronics, 2020, 42(10): 2334-2339.
[3]	GUAN Xin, GUAN Xin, TAO Li, YI Xiao. Unbiased algorithm of multiple platform passive location based on sliding window [J]. Systems Engineering and Electronics, 2016, 38(9): 1981-1986.
[4]	ZHANG Guo-dong，LIU Zhong. Measurement data pre-processing method based on the improvement of the SVM and the confidence level detection in sliding window type [J]. Journal of Systems Engineering and Electronics, 2013, 35(5): 930-934.
[5]	ZHANG Yi, ZHANG Qiang, WANG Yu, OUYANG Zhi-xin. Adaptive red-black window algorithm for SAR image speckle reduction [J]. Journal of Systems Engineering and Electronics, 2012, 34(8): 1576-1580.
[6]	SHI Bin-bin,QIAN Linjie， CHENG Zhu， HUANGFU Kan. Sliding window RLS method based on CORDIC [J]. Journal of Systems Engineering and Electronics, 2010, 32(11): 2304-2309.
[7]	AO Fu-jiang, DU Jing, YAN Yue-jin, HUANG Ke-di. Online mining closed frequent itemsets over a stream sliding window [J]. Journal of Systems Engineering and Electronics, 2009, 31(5): 1235-1240.
[8]	HOU Wei, YANG Bing-ru, WU Chen-sheng, ZHOU Zhun. Multi-relational pattern frequency update algorithm based on sliding window [J]. Journal of Systems Engineering and Electronics, 2009, 31(3): 671-676.