基于Policy Gradient的机械臂运动跟踪

doi:10.12305/j.issn.1001-506X.2021.09.29

Abstract

Abstract:

Aiming at the parameter self-tuning problem of the manipulator motion tracking controller, a parameter tuning device based on reinforcement learning Policy Gradient method is designed. Firstly, a hybrid dynamic model of manipulator is introduced. According to the system model, the design of proportional-derivative (PD) controller and the stability of Lyapunov are carried out, and the range of parameter matrix is given. Secondly, the parameter tuning device based on Policy Gradient is designed. The integrator is introduced to improve the parameter tuning of gradient, so as to further improve the control effect of PD controller. Finally, a second-order manipulator system is taken as an example to verify the effectiveness and feasibility of the parameter tuning device, which effectively improves the dynamic performance of the proposed system.

Key words: manipulator, motion tracking, Policy Gradient, parameter tuning, proportional-derivative (PD) control

CLC Number:

O231.2

Linxiao HAN, Jianbo HU, Shiyuan SONG, Yingyang WANG, Zihou HE, Peng ZHANG. Parameter tuning of manipulator motion tracking controller based on Policy Gradient[J]. Systems Engineering and Electronics, 2021, 43(9): 2605-2611.

Figures/Tables 7

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

References 31

1	李小华, 杨瑞芳, 刘辉. 一类机械臂系统自适应有限时间有界H∞跟踪控制[J]. 控制理论与应用, 2021, 38 (1): 147- 156.
	LI X H , YANG R F , LIU H . Adaptive finite-time bounded-H∞ tracking control for a class of manipulator system[J]. Control Theory & Applications, 2021, 38 (1): 147- 156.
2	WAI R J , YANG Z W . Adaptive fuzzy neural network control design via a T-S fuzzy model for a robot manipulator including actuator dynamics[J]. IEEE Trans.on Systems, Man, and Cybernetics Part B, 2008, 38 (5): 1326- 1346. doi: 10.1109/TSMCB.2008.925749
3	PILTAN F , SULAIMAN N , ALLAHDADI S , et al. Position control of robot manipulator: design a novel SISO adaptive sliding mode fuzzy PD fuzzy sliding mode control[J]. International Journal of Artificial Intelligence & Expert Systems, 2011, 2 (5): 208- 228.
4	NIKDEL N , BADAMCHIZADEH M A , AZIMIRAD V , et al. Adaptive backstepping control for an n-degree of freedom robotic manipulator based on combined state augmentation[J]. Robotics and Computer Integrated Manufacturing, 2017, 44, 129- 143. doi: 10.1016/j.rcim.2016.08.007
5	PERRUSQUIA A , FLORES J A , TORRES C R . A novel tuning method of PD with gravity compensation controller for robot manipulators[J]. IEEE Access, 2020, 8, 114773- 114783. doi: 10.1109/ACCESS.2020.3003842
6	SU Y X , SUN D , REN L , et al. Integration of saturated PI synchronous control and PD feedback for control of parallel manipulator[J]. IEEE Trans.on Robotics, 2006, 22 (1): 202- 207. doi: 10.1109/TRO.2005.858852
7	NPRPUZI A, KOCH C R. robotic manipulator control using PD-type fuzzy iterative learning control[C]//Proc. of the IEEE Canadian Conference of Electrical and Computer Engineering, 2019.
8	ZHANG H M, GAO X Y, XU G Y. Research on improved PD control of flexible manipulator[C]//Proc. of the Chinese Control and Decision Conference, 2019: 5053-5057.
9	DONG J , HE B , ZHANG C H , et al. Open-closed-loop PD itera-tive learning control corrected with the angular relationship of output vectors for a flexible manipulator[J]. IEEE Access, 2019, 7, 167815- 167822. doi: 10.1109/ACCESS.2019.2930559
10	LIU Y H , SUN D . Stabilizing a flexible beam handled by two manipulators via PD feedback[J]. IEEE Trans.on Automatic Control, 2000, 45 (11): 2159- 2164. doi: 10.1109/9.887656
11	ZHANG C Y . PD plus dynamic pressure feedback control for a direct drive stewart manipulator[J]. Energies, 2020, 13 (5): 1125- 1131. doi: 10.3390/en13051125
12	BENCHAREF S, BOUBERTAKH H. Optimal tuning of a PD control by bat algorithm to stabilize a quadrotor[C]//Proc. of the IEEE 8th International Conference on Modelling, Identification and Control, 2017: 938-942.
13	KHATOON S , NASIRUDDIN I , SHAHID M . Design and simulation of a hybrid PD-ANFIS controller for attitude tracking control of a quadrotor UAV[J]. Arabian Journal for Science and Engineering, 2017, 42, 5211- 5229. doi: 10.1007/s13369-017-2586-z
14	ARROSIDA H, EFFENDI R, AGUSTINAH T, et al. Design of decoupling and nonlinear PD controller for cruise control of a quadrotor[C]//Proc. of the International Seminar on Intelligent Technology and its Applications, 2015: 57-62.
15	LU S T , ZHAO J D . Research on tracking control of circular tra-jectory of robot based on the variable integral sliding mode PD control algorithm[J]. IEEE Acess, 2020, 8, 204194- 204202. doi: 10.1109/ACCESS.2020.3027772
16	KOLATHAYA S . Local stability of PD controlled bipedal walking robots[J]. Automatica, 2020, 114, 108841. doi: 10.1016/j.automatica.2020.108841
17	李洋. PID参数自整定器设计与实现[D]. 黑龙江: 黑龙江大学, 2014.
	LI Y. Design and realization of PID parameter self-tuning device[D]. Heilongjiang: Heilongjiang University, 2014.
18	VERMA B , PADHY P K . Robust fine tuning of optimal PID controller with guaranteed robustness[J]. IEEE Trans.on Industrial Electronics, 2020, 67 (6): 4911- 4920. doi: 10.1109/TIE.2019.2924603
19	QI Z , SHI Q , ZHANG H . Tuning of digital PID controllers using particle swarm optimization algorithm for a CAN-based DC motor subject to stochastic delays[J]. IEEE Trans.on Industrial Electronics, 2020, 67 (7): 5637- 5646. doi: 10.1109/TIE.2019.2934030
20	COATA E B M , SERRA G L O . Self-tuning robust fuzzy controller design based on multi-objective particle swarm optimization adaptation mechanism[J]. Journal of Dynamic Systems Measurement and Control-Transactions of the ASME, 2017, 139 (7): 071009. doi: 10.1115/1.4035758
21	HEKIMOGLU B . Optimal tuning of fractional order PID controller for DC motor speed control via chaotic atom search optimization algorithm[J]. IEEE Access, 2019, 7, 38100- 38114. doi: 10.1109/ACCESS.2019.2905961
22	SU Y P , YU Q M , ZENG L . Parameter self-tuning PID control for greenhouse climate control problem[J]. IEEE Access, 2020, 8, 186157- 186171. doi: 10.1109/ACCESS.2020.3030416
23	HEERTJES M , VERSTAPPEN R . Self-tuning in integral sliding mode control with a Levenberg-Marquardt algorithm[J]. Mechatronics, 2014, 24 (4): 385- 393. doi: 10.1016/j.mechatronics.2013.05.005
24	LI J S , LI W G . On-line PID parameters optimization control for wind power generation system based on genetic algorithm[J]. IEEE Access, 2020, 8, 137094- 137100. doi: 10.1109/ACCESS.2020.3009240
25	YANG B , LIANG G A , PENG J H , et al. Self-adaptive PID controller of microwave drying rotary device tuning on-line by genetic algorithms[J]. Journal of Central South University, 2013, 20 (10): 2685- 2692. doi: 10.1007/s11771-013-1784-4
26	LU L , LIANG S , DAI Y W , et al. Improved quantum bacterial foraging algorithm for tuning parameters of fractional-order PID controller[J]. Journal of Systems Engineering and Electronics, 2018, 29 (1): 166- 175. doi: 10.21629/JSEE.2018.01.17
27	李昭莹. 基于Q学习—变论域模糊PID算法在柴油机调速系统中的设计研究[D]. 哈尔滨: 哈尔滨工程大学, 2016.
	LI Z Y. Research on design of diesel engine speed control system based on Q-learning-variable domain fuzzy PID algorithm[D]. Harbin: Harbin Engineering University, 2016.
28	邓凯. 基于强化学习方法的飞行姿态模拟器控制系统设计[D]. 哈尔滨: 哈尔滨工业大学, 2018.
	DENG K. Design of flight attitude simulator control system based on reinforcement learning method[D]. Harbin: Harbin Institute of Technology, 2018.
29	LIU Y , ZHANG C . Application of dueling DQN and DECGA for parameter estimation in variogram models[J]. IEEE Access, 2020, 8, 38112- 38122. doi: 10.1109/ACCESS.2020.2974926
30	宋仕元, 胡剑波, 王应洋, 等. 滑模控制器参数整定的Actor-Critic学习算法[J]. 电光与控制, 2020, 27 (9): 24- 27, 49.
	SONG S Y , HU J B , WANG Y Y , et al. Actor-critic learning algorithm for parameter tuning of sliding mode controller[J]. Electronics Optics & Control, 2020, 27 (9): 24- 27, 49.
31	刘金琨. 先进PID控制MATLAB仿真[M]. 4版北京: 电子工业出版社, 2004.
	LIU J K . Advanced PID control MATLAB simulation[M]. 4th ed Beijing: Electronic Industry Press, 2004.

[1]	Zijie MA, Yongjun XIE. Dynamic stealth of cruise missile in system combat [J]. Systems Engineering and Electronics, 2022, 44(9): 2826-2831.
[2]	Lingyu MENG, Bingli GUO, Wen YANG, Xinwei ZHANG, Zuoqing ZHAO, Shanguo HUANG. Network routing optimization approach based on deep reinforcement learning [J]. Systems Engineering and Electronics, 2022, 44(7): 2311-2318.
[3]	Dongzi GUO, Rong HUANG, Hechuan XU, Liwei SUN, Naigang CUI. Research on deep deterministic policy gradient guidance method for reentry vehicle [J]. Systems Engineering and Electronics, 2022, 44(6): 1942-1949.
[4]	Qitian WAN, Baogang LU, Yaxin ZHAO, Qiuqiu WEN. Autopilot parameter rapid tuning method based on deep reinforcement learning [J]. Systems Engineering and Electronics, 2022, 44(10): 3190-3199.
[5]	Ang GAO, Zhiming DONG, Liang LI, Jinghua SONG, Li DUAN. Parallel priority experience replay mechanism of MADDPG algorithm [J]. Systems Engineering and Electronics, 2021, 43(2): 420-433.
[6]	Kun ZHANG, Ke LI, Haotian SHI, Zhenchong ZHANG, Zekun LIU. Autonomous guidance maneuver control and decision-making algorithm [J]. Systems Engineering and Electronics, 2020, 42(7): 1567-1574.
[7]	TAN Shili, LEI Humin, WANG Pengfei. Design of tracking differentiator based on tangent Sigmoid function [J]. Systems Engineering and Electronics, 2019, 41(7): 1590-1596.
[8]	WANG Xiao-chen, YAO Yu, FU Shao-wen. Control design of disturbance rejection for a Stewart platform based on the internal model principle [J]. Journal of Systems Engineering and Electronics, 2009, 31(7): 1689-1693.

Parameter tuning of manipulator motion tracking controller based on Policy Gradient

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 31

Related Articles 8

Recommended Articles

Metrics

Comments