系统工程与电子技术 ›› 2020, Vol. 42 ›› Issue (2): 414-419.doi: 10.3969/j.issn.1001-506X.2020.02.21

• 制导、导航与控制 • 上一篇    下一篇

Q-learning强化学习制导律

张秦浩1(), 敖百强1(), 张秦雪2()   

  1. 1. 北京电子工程总体研究所, 北京 100854
    2. 北华航天工业学院计算机学院, 河北 廊坊 065000
  • 收稿日期:2019-07-26 出版日期:2020-02-01 发布日期:2020-01-23
  • 作者简介:张秦浩(1995-),男,硕士,主要研究方向为导弹制导与控制、强化学习与人工智能。E-mail:373480684@qq.com|敖百强(1973-),男,研究员,博士,主要研究方向为导弹制导控制。E-mail:aobqzyj@163.com|张秦雪(1995-),男,硕士,主要研究方向为计算机视觉、人工智能与深度学习。E-mail:1411225811@qq.com
  • 基金资助:
    中国博士后科学基金资助课题(2017M620863)

Reinforcement learning guidance law of Q-learning

Qinhao ZHANG1(), Baiqiang AO1(), Qinxue ZHANG2()   

  1. 1. Beijing Institute of Electronic Engineering, Beijing 100854, China
    2. College of Computer Science, North China Institute of Aerospace Engineering, Langfang 065000, China
  • Received:2019-07-26 Online:2020-02-01 Published:2020-01-23
  • Supported by:
    中国博士后科学基金资助课题(2017M620863)

摘要:

在未来的战场中,智能导弹将成为精确有效的打击武器,导弹智能化已成为一种主要的发展趋势。本文以传统的比例制导律为基础,提出基于强化学习的变比例系数制导算法。该算法以视线转率作为状态,依据脱靶量设计奖励函数,并设计离散化的行为空间,为导弹选择正确的制导指令。实验仿真验证了所提算法比传统的比例制导律拥有更好的制导精度,并使导弹拥有了自主决策能力。

关键词: 比例制导, 制导律, 脱靶量, 机动目标, 强化学习, Q学习, 时序差分算法

Abstract:

As the intelligent missile being a major development trend, it is foreseeable that it will become a precise and effective strike weapon in the future battlefields. On the basis of the traditional proportional guidance law, this paper proposes a guidance algorithm based on reinforcement learning with variable proportional coefficient. Taking the line-of-sight rate as the state, this algorithm designs a discretized action space, as well as a reward function based on the miss distance, to determine the correct guidance command for the missile. The simulation results prove the algorithm possesses better guidance accuracy than the traditional proportional guidance law and endows the missile with the ability of autonomous decision-making.

Key words: proportional guidance, guidance law, miss distance, maneuvering target, reinforcement learning, Q-learning, timing difference algorithm

中图分类号: