基于部分可观察马尔可夫决策过程与前景理论的即时威慑决策模型

doi:10.12305/j.issn.1001-506X.2026.09.24

摘要/Abstract

摘要：

即时威慑是危机爆发后相关敌对实体之间的互动行为，具有内在不确定性、有限理性和不完全信息特点，使用基于工具理性和共识假设的传统博弈论方法难以准确刻画。立足即时威慑中综合运用理性、非理性、不确定性因素，塑造对手益损对比与失控风险的预期，迫使其首先妥协的序贯决策特点，构建了基于部分可观察马尔可夫决策过程的即时威慑一般模型。为进一步刻画决策过程中的非理性因素，引入前景理论对威慑方的目标函数进行建模，设计了融合前景价值的决策算法，并通过算例仿真验证了模型的有效性，可为危机状态下即时威慑决策及风险管控提供新的方法参考。

关键词: 威慑决策, 部分可观察马尔可夫决策过程, 前景理论, 即时威慑

Abstract:

Immediate deterrence refers to the interactive behavior among relevant hostile entities after outbreak of a crisis, characterized by inherent uncertainty, limited rationality, and incomplete information, which cannot be adequately captured by traditional game-theoretic approaches based on instrumental rationality and common knowledge. Based on immediate deterrence, it shapes the adversary’s expectations of gains losses ratio and escalation risks by comprehensively utilizing rational, irrational and uncertain factors compelling them to concession first, a general immediate deterrence decision model is proposed using the partially observable Markov decision process. To better characterize irrational aspects in the immediate deterrence decision-making process, prospect theory is introduced to model the deterrer’s objective function. A decision-making algorithm is designed incorporating prospect value, and the model is verified through a simulation case, which offers a novel methodological reference for immediate deterrence decision-making and risk management in crisis situations.

Key words: deterrence decision, partially observable Markov decision process, prospect theory, immediate deterrence

中图分类号:

E917

邓鹏华, 杨雨鑫, 李新其. 基于部分可观察马尔可夫决策过程与前景理论的即时威慑决策模型[J]. 系统工程与电子技术, 2026, 48(9): 3119-3125.

Penghua Deng, Yuxin Yang, Xinqi Li. Immediate deterrence decision model based on partially observable Markov decision process and prospect theory[J]. Systems Engineering and Electronics, 2026, 48(9): 3119-3125.

图/表 4

图1

表1

表2

表3

参考文献 23

1	劳伦斯·弗里德曼. 威慑[M]. 莫盛凯, 译. 上海: 上海人民出版社, 2022.
2	Morgan P M. Deterrence now[M]. Cambridge: Cambridge University Press, 2003: 9.
3	Schelling T C. The strategy of conflict[M]. Cambridge: Harvard University Press, 1980.
4	Schelling T C. Arms and influence[M]. London: Yale University Press, 1966.
5	Wohlstetter A. Delicate balance of terror[J]. Foreign Affairs, 1959, 37 (2): 211. doi: 10.1163/2468-1733_shafr_sim140110031
6	Harsanyi J C. Games with incomplete information played by “Bayesian” players, I-III, part I. the basic model[J]. Management Science, 1967, 14 (3): 159. doi: 10.1007/978-94-017-2527-9_6
7	Harsanyi J C. Games with incomplete information played by “Bayesian” players, Part II. Bayesian equilibrium[J]. Management Science, 1968, 14 (5): 320. doi: 10.1287/mnsc.14.5.320
8	Harsanyi J C. Games with incomplete information played by “Bayesian” players, Part III. The basic probability distribution of the Game[J]. Management Science, 1968, 14 (7): 486. doi: 10.1287/mnsc.14.7.486
9	Smith J M. Evolution and the theory of games[M]. Cambridge: Cambridge University Press, 1982.
10	王永县, 向钢华. 基于博弈分析的军事威慑理论研究[J]. 清华大学学报（哲学社会科学版）, 2005, 20 (5): 62.
11	向钢华, 王永县. 基于累积前景理论的有限理性威慑模型[J]. 系统工程, 2006, 24 (12): 107. doi: 10.3969/j.issn.1001-4098.2006.12.021
12	Allison G T, Zelikow P. Essence of decision: explaining the cuban missile crisis[M]. 2nd ed. London: Pearson Education, Incorporated, 1999.
13	Karl S. Prospects of deterrence: deterrence theory, representation and evidence[J]. Defence and Peace Economics, 2024, 35 (2): 145. doi: 10.1080/10242694.2022.2152956
14	Ma S, Tauman Y, Zeckhauser R. Deterrence games and the disruption of information[J]. International Journal of Game Theory, 2024, 53, 261. doi: 10.1007/s00182-023-00870-3
15	Kaelbling L P, Littman M L, Cassandra A R. Planning and acting in partially observable stochastic domains[J]. Artificial Intelligence, 1998, 101 (1/2): 99. doi: 10.1016/s0004-3702(98)00023-x
16	Clempner J B, Poznyak A. Optimization and games for controllable Markov chains: numerical methods with application to finance and engineering[M]. Gewerbestrasse: Springer Nature Switzerland AG, 2024: 47.
17	He K, Doshi P, Banerjee B. Modeling and reinforcement learning in partially observable many-agent systems[J]. Autonomous Agents and Multi-Agent Systems, 2024, 38(3): 12.
18	Schwartz J, Newbury R, Kulic D, et al. POSGGym: a library for decision-theoretic planning and learning in partially observable, multi-agent environments[J]. Autonomous Agents and Multi-Agent Systems, 2025, 39(7): 35.
19	Li J C, Cai M Y, Kan Z, et al. Model-free reinforcement learning for motion planning of autonomous agents with complex tasks in partially observable environments[J]. Autonomous Agents and Multi-Agent Systems, 2024, 38(4): 14.
20	Wang Z J, Wang B, Dou H B, et al. Windows deep transformer Q-networks: an extended variance reduction architecture for partially observable reinforcement learning[J]. Applied Intelligence, 2025, 55 (1): 35. doi: 10.1007/s10489-024-05867-3
21	Helmeczi R K, Kavaklioglu C, Cevik M, et al. A multi-objective constrained partially observable Markov decision process model for breast cancer screening[J]. Operational Research, 2023, 23 (4): 30. doi: 10.1007/s12351-023-00774-w
22	Kahneman D, Amos T. Prospect theory: an analysis of decision under risk[J]. Econometrica, 1979, 47 (2): 263. doi: 10.21236/ada045771
23	王彪, 薛源, 陈萍萍, 等. 基于前景理论的多阶段多情景多部门应急决策的矩阵方法[J]. 控制与决策, 2025, 49 (2): 655. doi: 10.13195/j.kzyjc.2023.1707

参数	值	参数	值	参数	值
$ {T}\left({{S}}_{{1}}\|{{S}}_{{1}}{,}{{a}}_{{1}}\right) $	0.9	$ {T}\left({{S}}_{{2}}\|{{S}}_{{1}}{,}{{a}}_{{1}}\right) $	0.1	$ {T}\left({{S}}_{{8}}\|{{S}}_{{1}}{,}{{a}}_{{8}}\right) $	0.9
$ {T}\left({{S}}_{{1}}\|{{S}}_{{2}}{,}{{a}}_{{1}}\right) $	0.3	$ {T}\left({{S}}_{{2}}\|{{S}}_{{2}}{,}{{a}}_{{2}}\right) $	0.5	$ {T}\left({{S}}_{{3}}\|{{S}}_{{2}}{,}{{a}}_{{1}}\right) $	0.2
$ {T}\left({{S}}_{{2}}\|{{S}}_{{3}}{,}{{a}}_{{1}}\right) $	0.2	$ {T}\left({{S}}_{{3}}\|{{S}}_{{1}}{,}{{a}}_{{3}}\right) $	0.6	$ {T}\left({{S}}_{{4}}\|{{S}}_{{1}}{,}{{a}}_{{1}}\right) $	0.2
$ {T}\left({{S}}_{{3}}\|{{S}}_{{4}}{,}{{a}}_{{1}}\right)$	0.1	$ {T}\left({{S}}_{{4}}\|{{S}}_{{1}}{,}{{a}}_{{4}}\right) $	0.7	$ {T}\left({{S}}_{{5}}\|{{S}}_{{1}}{,}{{a}}_{{1}}\right) $	0.2
$ {T}\left({{S}}_{{4}}\|{{S}}_{{5}}{,}{{a}}_{{1}}\right)$	0.1	$ {T}\left({{S}}_{{5}}\|{{S}}_{{1}}{,}{{a}}_{{5}}\right)$	0.7	$ {T}\left({{S}}_{{6}}\|{{S}}_{{1}}{,}{{a}}_{{1}}\right) $	0.2
$ {T}\left({{S}}_{{5}}\|{{S}}_{{6}}{,}{{a}}_{{1}}\right) $	0.1	$ {T}\left({{S}}_{{6}}\|{{S}}_{{1}}{,}{{a}}_{{6}}\right) $	0.7	$ {T}\left({{S}}_{{7}}\|{{S}}_{{1}}{,}{{a}}_{{1}}\right) $	0.2
$ {T}\left({{S}}_{{6}}\|{{S}}_{{7}}{,}{{a}}_{{1}}\right) $	0.1	$ {T}\left({{S}}_{{7}}\|{{S}}_{{1}}{,}{{a}}_{{7}}\right)$	0.7	$ {T}\left({{S}}_{{8}}\|{{S}}_{{1}}{,}{{a}}_{{1}}\right) $	0.2
$ {T}\left({{S}}_{{7}}\|{{S}}_{{8}}{,}{{a}}_{{1}}\right)$	0.1

参数	值	参数	值	参数	值
$ {\varOmega}\left({{o}}_{{1}}\|{{S}}_{{1}}{,a}\right) $	0.7	$ {\varOmega}\left({{o}}_{{2}}\|{{S}}_{{1}}{,a}\right) $	0.2	$ {\varOmega}\left({{o}}_{{3}}\|{{S}}_{{1}}{,a}\right)$	0.1
$ {\varOmega}\left({{o}}_{{1}}\|{{S}}_{{2}}{,a}\right) $	0.5	$ {\varOmega}\left({{o}}_{{2}}\|{{S}}_{{2}}{,a}\right) $	0.3	$ {\varOmega}\left({{o}}_{{3}}\|{{S}}_{{2}}{,a}\right) $	0.2
$ {\varOmega}\left({{o}}_{{1}}\|{{S}}_{{3}}{,a}\right) $	0.3	$ {\varOmega}\left({{o}}_{{2}}\|{{S}}_{{3}}{,a}\right) $	0.4	$ {\varOmega}\left({{o}}_{{3}}\|{{S}}_{{3}}{,a}\right) $	0.3
$ {\varOmega}\left({{o}}_{{1}}\|{{S}}_{{4}}{,a}\right)$	0.2	$ {\varOmega}\left({{o}}_{{2}}\|{{S}}_{{4}}{,a}\right) $	0.5	$ {\varOmega}\left({{o}}_{{3}}\|{{S}}_{{4}}{,a}\right) $	0.3
$ {\varOmega}\left({{o}}_{{1}}\|{{S}}_{{5}}{,a}\right) $	0.1	$ {\varOmega}\left({{o}}_{{2}}\|{{S}}_{{5}}{,a}\right) $	0.4	$ {\varOmega}\left({{o}}_{{3}}\|{{S}}_{{5}}{,a}\right)$	0.5
$ {\varOmega}\left({{o}}_{{1}}\|{{S}}_{{6}}{,a}\right) $	0.1	$ {\varOmega}\left({{o}}_{{2}}\|{{S}}_{{6}}{,a}\right) $	0.3	$ {\varOmega}\left({{o}}_{{3}}\|{{S}}_{{6}}{,a}\right) $	0.6
$ {\varOmega}\left({{o}}_{{1}}\|{{S}}_{{7}}{,a}\right) $	0.05	$ {\varOmega}\left({{o}}_{{2}}\|{{S}}_{{7}}{,a}\right) $	0.25	$ {\varOmega}\left({{o}}_{{3}}\|{{S}}_{{7}}{,a}\right) $	0.7
$ {\varOmega}\left({{o}}_{{1}}\|{{S}}_{{8}}{,a}\right) $	0.05	$ {\varOmega}\left({{o}}_{{2}}\|{{S}}_{{8}}{,a}\right) $	0.15	$ {\varOmega}\left({{o}}_{{3}}\|{{S}}_{{8}}{,a}\right)$	0.8

状态	$ {{a}}_{{1}} $	$ {{a}}_{{2}} $	$ {{a}}_{{3}} $	$ {{a}}_{{4}} $
$ {{S}}_{{1}} $	+3	+3	+3	+3
$ {{S}}_{{2}} $	+2	+2	+2	+2
$ {{S}}_{{3}} $	+1	+1	+1	+1
$ {{S}}_{{4}} $	0	0	0	0
$ {{S}}_{{5}} $	−1	−1	−1	−1
$ {{S}}_{{6}} $	−2	−2	−2	−2
$ {{S}}_{{7}} $	−3	−3	−3	−3
$ {{S}}_{{8}} $	−4	−4	−4	−4

[1]	张煜, 王磊, 卢麟, 叶永平, 万亮. 基于累积前景理论和学识评级的PL-VIKOR群决策[J]. 系统工程与电子技术, 2023, 45(6): 1762-1771.
[2]	尹东亮, 崔国恒, 黄晓颖, 张欢. 基于改进得分函数和前景理论的区间值毕达哥拉斯模糊多属性决策[J]. 系统工程与电子技术, 2022, 44(11): 3463-3469.
[3]	李贺, 江登英. 基于改进符号距离的犹豫模糊前景理论决策方法[J]. 系统工程与电子技术, 2019, 41(12): 2820-2826.
[4]	项华春, 李京峰, 陈云翔, 李正欣. 融合新型偏好度模型和改进累积前景理论的决策方法[J]. 系统工程与电子技术, 2018, 40(12): 2743-2750.
[5]	林松, 汪军, 朱建军, 张世涛. 考虑双重预期的云模型决策[J]. 系统工程与电子技术, 2017, 39(4): 821-828.
[6]	辜磊, 王书宁, 赵峰. 基于期望效能的武器装备体系能力规划模型[J]. 系统工程与电子技术, 2017, 39(2): 329-334.
[7]	陈业华, 王浩, 宋之杰. 基于效用风险熵的突发事件应急方案动态调整[J]. 系统工程与电子技术, 2016, 38(9): 2093-2098.
[8]	常一哲, 李战武, 寇英信, 徐安, 杨海燕. 不确定信息条件下空战接敌队形选择方法[J]. 系统工程与电子技术, 2016, 38(11): 2552-2560.
[9]	陈振颂, 熊升华, 李延来, 钱桂生. 基于参数估计与记分函数联合的直觉梯形模糊随机前景决策方法[J]. 系统工程与电子技术, 2015, 37(4): 851-862.
[10]	王玮, 叶强, 谢春思, 李涛. 基于前景理论的编队作战决心方案优选研究[J]. 系统工程与电子技术, 2015, 37(2): 331-335.
[11]	刘云志1，樊治平1，李铭洋2. 考虑决策者给出参照点的风险型#br# 模糊多属性决策方法[J]. 系统工程与电子技术, 2014, 36(7): 1358-1367.
[12]	刘勇, Jeffrey Forrest, 赵焕焕, 刘思峰, 刘家树. 基于前景理论的多目标灰色局势决策方法[J]. Journal of Systems Engineering and Electronics, 2012, 34(12): 2514-2519.
[13]	马健, 孙秀霞, 郭创. 基于风险-效益比和前景理论的风险性多属性决策方法[J]. Journal of Systems Engineering and Electronics, 2011, 33(11): 2434-2439.