一种基于结构特征的单目视觉惯性里程计

doi:10.12305/j.issn.1001-506X.2023.10.25

系统工程与电子技术 ›› 2023, Vol. 45 ›› Issue (10): 3207-3217.doi: 10.12305/j.issn.1001-506X.2023.10.25

• 制导、导航与控制 • 上一篇

一种基于结构特征的单目视觉惯性里程计

闫莉萍¹^,*, 李程伟¹, 徐柏凯¹, 夏元清¹, 肖波²

1. 北京理工大学自动化学院, 北京 100081
2. 北京邮电大学人工智能学院, 北京 100876

收稿日期:2022-08-17 出版日期:2023-09-25 发布日期:2023-10-11
通讯作者: 闫莉萍
作者简介:闫莉萍(1979—), 女, 教授, 博士, 主要研究方向为数据融合、故障诊断、目标跟踪、组合导航与智能导航
李程伟(1997—), 男, 硕士研究生, 主要研究方向为视觉同时定位与建图
徐柏凯(2000—), 男, 硕士研究生, 主要研究方向为视觉同时定位与建图
夏元清(1971—), 男, 教授, 博士, 主要研究方向为云控制、云数据中心优化调度管理、智能交通、模型预测控制、自抗扰控制、飞行器控制、空天地一体化网络协同控制
肖波(1975—), 男, 副教授, 博士, 主要研究方向为模式识别、机器学习、数据挖掘、网络搜索、网络管理、网络应用研发、信息融合
基金资助:
国家自然科学基金(62073036);国家自然科学基金(62076031);北京市自然科学基金(4202071)

A monocular visual inertial odometry based on structural features

Liping YAN¹^,*, Chengwei LI¹, Bokai XU¹, Yuanqing XIA¹, Bo XIAO²

1. School of Automation, Beijing Institute of Technology, Beijing 100081, China
2. School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing 100876, China

Received:2022-08-17 Online:2023-09-25 Published:2023-10-11
Contact: Liping YAN

摘要/Abstract

摘要：

针对传统的仅依赖点特征的视觉里程计鲁棒性较差的问题，提出了一种结合点特征、线特征以及结构特征的视觉-惯性里程计。结构特征是一个满足曼哈顿世界假设的正交基, 可用于描述结构物体包含的平行与正交信息。结构特征的引入为视觉惯性里程计提供了一个全局的约束, 可有效减少旋转误差的累计。与此同时, 为了保证引入结构特征后系统的实时性, 基于线特征匹配关系以及重力方向, 提出了一种鲁棒且高效的结构特征提取算法。实验结果表明, 相比现有的开源系统, 所提出的系统在保证实时性的同时, 有效减少了旋转累计误差。

关键词: 同时定位与建图, 点特征, 线特征, 结构特征, 惯性测量单元

Abstract:

In response to the problem of poor robustness of traditional visual odometers that only rely on point features, a monocular visual inertial odometry that utilize point features, line features and structural features are deduced. The structural feature is an orthonormal basis that satisfy the Manhattan world hypothesis. This feature contains the parallel and orthogonal information of the structural object. The introduction of structural feature provide an overall constrain for visual inertial odometry and can reduce the accumulative error of rotation effectively. Meanwhile, to ensure the real-time performance of the system after the introduction of structural features, a robust and efficient algorithm is proposed to extract structural feature, which is based on the matching relationship of line features and the direction of gravity vector. The experimental results demonstrate that, compared with the existing open source systems, the proposed system can effectively reduce the accumulated rotation error while maintaining the real-time performance.

Key words: simultaneous localization and mapping, point feature, line feature, structural feature, inertial measurement unit

中图分类号:

TP391

闫莉萍, 李程伟, 徐柏凯, 夏元清, 肖波. 一种基于结构特征的单目视觉惯性里程计[J]. 系统工程与电子技术, 2023, 45(10): 3207-3217.

Liping YAN, Chengwei LI, Bokai XU, Yuanqing XIA, Bo XIAO. A monocular visual inertial odometry based on structural features[J]. Systems Engineering and Electronics, 2023, 45(10): 3207-3217.

图/表 11

图1

图2

图3

图4

图5

图6

图7

图8

表1

表2

表3

参考文献 41

1	陈凤东, 洪炳镕. 基于特征地图的移动机器人全局定位与自主泊位方法[J]. 电子学报, 2010, 38 (6): 1256- 1261.
	CHEN F D , HONG B R . A global localization and self-docking method for mobile robot based on feature map[J]. Acta Electronica Sinica, 2010, 38 (6): 1256- 1261.
2	王晓龙, 刘海颖, 王景琪. 基于分层SLAM的空地多智能体协同导航[J]. 系统工程与电子技术, 2020, 42 (1): 166- 171.
	WANG X L , LIU H Y , WANG J Q . Collaborative navigation of air-ground multi-agent based on hierarchical SLAM[J]. Systems Engineering and Electronics, 2020, 42 (1): 166- 171.
3	陈宝华, 邓磊, 陈志祥, 等. 基于即时稠密三维重构的无人机视觉定位[J]. 电子学报, 2017, 45 (6): 1294- 1300. doi: 10.3969/j.issn.0372-2112.2017.06.003
	CHEN B H , DENG L , CHEN Z X , et al. Instant dense 3D reconstruction-based UAV vision localization[J]. Acta Electronica Sinica, 2017, 45 (6): 1294- 1300. doi: 10.3969/j.issn.0372-2112.2017.06.003
4	王通典, 刘洁瑜, 吴宗收, 等. 基于多尺度光流融合特征点视觉-惯性SLAM方法[J]. 系统工程与电子技术, 2022, 44 (3): 977- 985.
	WANG T D , LIU J Y , WU Z S , et al. Visual-inertial SLAM method based on multi-scale optical flow fusion feature point[J]. Systems Engineering and Electronics, 2022, 44 (3): 977- 985.
5	GUI J J , GU D B , WANG S , et al. A review of visual inertial odometry from filtering and optimisation perspectives[J]. Advanced Robotics, 2015, 29 (20): 1289- 1301. doi: 10.1080/01691864.2015.1057616
6	ZUO X X, XIE X J, LIU Y, et al. Robust visual SLAM with point and line features[C]//Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017: 1775-1782.
7	HE Y J , ZHAO J , GUO Y , et al. PL-VIO: tightly coupled monocular visual-inertial odometry using point and line features[J]. Sensors, 2018, 18 (4): 1-25, 1159. doi: 10.1109/JSEN.2017.2782543
8	赵良玉, 金瑞, 朱叶青, 等. 基于点线特征融合的双目惯性SLAM算法[J]. 航空学报, 2021, 42 (12): 325117.
	ZHAO L Y , JIN R , ZHU Y Q , et al. Stereo visual-inertial SLAM with point and line features[J]. Acta Aeronautica et Astronautica Sinica, 2021, 42 (12): 325117.
9	LEE G H, FRAUNDORFER F, POLLEFEYS M. MAV visual SLAM with plane constraint[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2011: 3139-3144.
10	李海丰, 胡遵河, 陈新伟. PLP-SLAM: 基于点、线、面特征融合的视觉SLAM方法[J]. 机器人, 2017, 39 (2): 214- 220.
	LI H F , HU Z H , CHEN X W . PLP-SLAM: a visual SLAM method based on point-line-plane feature fusion[J]. Robot, 2017, 39 (2): 214- 220.
11	孙沁璇, 苑晶, 张雪波, 等. PLVO: 基于平面和直线融合的RGB-D视觉里程计[J]. 自动化学报, 2023, 49 (10): 1001- 1013.
	SUN Q X , YUAN J , ZHANG X B , et al. PLVO: plane-line-based RGB-D visual odometry[J]. Acta Automatica Sinica, 2023, 49 (10): 1001- 1013.
12	CAMPOSECO F, POLLEFEYS M. Using vanishing points to improve visual-inertial odometry[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2015: 5219-5225.
13	KIM P, COLTIN B. Low-drift visual odometry in structured environments by decoupling rotational and translational motion[C]// Proc. of the IEEE International Conference on Robotics and Automation, 2018: 7247-7253.
14	GUO R , PENG K , ZHOU D , et al. Robust visual compass using hybrid features for indoor environments[J]. Electronics, 2019, 8 (2): 220. doi: 10.3390/electronics8020220
15	KNEIP L, WEISS S, SIEGWART R. Deterministic initialization of metric state estimation filters for loosely-coupled mono-cular vision-inertial systems[C]//Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2011: 2235-2241.
16	WEISS S, SIEGWART R. Real-time metric state estimation for modular vision-inertial systems[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2011: 4531-4537.
17	BLOESCH M , BURRI M , OMARI S , et al. Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback[J]. The International Journal of Robotics Research, 2017, 36 (10): 1053- 1072. doi: 10.1177/0278364917728574
18	LEUTENEGGER S , LYNEN S , BOSSE M , et al. Keyframe-based visual-inertial odometry using nonlinear optimization[J]. The International Journal of Robotics Research, 2015, 34 (3): 314- 334.
19	LI M , MOURIKIS A I . High-precision, consistent EKF-based visual-inertial odometry[J]. The International Journal of Robotics Research, 2013, 32 (6): 690- 711.
20	BLOESCH M , BURRI M , OMARI S , et al. Iterated extended Kalman filter based visual-inertial odometry using direct photometric feedback[J]. The International Journal of Robotics Research, 2017, 36 (10): 1053- 1072.
21	LEUTENEGGER S , LYNEN S , BOSSE M , et al. Keyframe-based visual-inertial odometry using nonlinear optimization[J]. The International Journal of Robotics Research, 2015, 34 (3): 314- 334.
22	CAMPOS C , ELVIRA R , RODRÍGUEZ J J G , et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Trans.on Robotics, 2021, 37 (6): 1874- 1890.
23	FORSTER C , CARLONE L , DELLAERT F , et al. On-Manifold preintegration for real-time visual-inertial odometry[J]. IEEE Trans.on Robotics, 2017, 33 (1): 1- 21.
24	LU X H, J Y Y, LI H, et al. 2-Line exhaustive searching for real-time vanishing point estimation in Manhattan world[C]// Proc. of the IEEE Winter Conference on Applications of Computer Vision, 2017: 345-353.
25	BAZIN J C , DEMONCEAUX C , VASSEUR P , et al. Rotation estimation and vanishing point extraction by omnidirectional vision in urban environment[J]. The International Journal of Robotics Research, 2012, 31 (1): 63- 81.
26	DENIS P, ELDER J H, ESTRADA F J. Efficient edge based methods for estimating Manhattan frames in urban imagery[C]//Proc. of the European Conference on Computer Vision, 2008: 97-210.
27	BAZIN J C, POLLEFEYS M. 3-line RANSAC for orthogonal vanishing point detection[C]//Proc. of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 2012: 4282-4287.
28	MIRZAEI F M, ROUMELIOTIS S I. Optimal estimation of vanishing points in a manhattan world[C]// Proc. of the IEEE International Conference on Computer Vision, 2011: 2454-2461.
29	WILDENAUER H, HANBURY A. Robust camera self-calibration from monocular images of Manhattan worlds[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2012: 2831-2838.
30	TRETYAK E, BARINOVA O, KOHLI P, et al. Geometric image parsing in man-made environments[C]//Proc. of the International Journal of Computer Vision, 2012, 97(3): 305-321.
31	ZHOU H Z , ZOU D P , PEI L , et al. StructSLAM: visual SLAM with building structure lines[J]. IEEE Trans.on Vehi-cular Technology, 2015, 64 (4): 1364- 1375.
32	ZOU D P , WU Y X , PEI L , et al. StructVIO: visual-inertial odometry with structural regularity of man-made environments[J]. IEEE Trans.on Robotics, 2019, 35 (4): 999- 1013.
33	GOMEZ-OJEDA R , MORENO F A , ZUNIGA-NOEL D , et al. PL-SLAM: a stereo SLAM system through the combination of points and line segments[J]. IEEE Trans.on Robotics, 2019, 35 (3): 734- 746.
34	LI H A, YAO J, BAZIN J, et al. A monocular SLAM system leveraging structural regularity in Manhattan world[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2018: 2518-2525.
35	KIM P, COLTIN B. Low-drift visual odometry in structured environments by decoupling rotational and translational motion[C]// Proc. of the IEEE International Conference on Robotics and Automation, 2018: 7247-7253.
36	LUCAS B, KANADE T. An iterative image registration technique with an application to stereo vision[C]//Proc. of the International Joint Conference on Artificial Intelligence, 1981: 674-679.
37	AKINLAR C , TOPAL C . EDLINES: a real-time line segment detector with a false detection control[J]. Pattern Recognition Letters, 2011, 32 (13): 1633- 1642.
38	QIN T , LI P , SHEN S J . VINS-Mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Trans.on Robotics, 2018, 34 (4): 1004- 1020.
39	BARTOLI A , STURM P . Structure-from-motion using lines: representation, triangulation, and bundle adjustment[J]. Computer Vision and Image Understanding, 2005, 100 (3): 416- 441.
40	BURRI M , NIKOLIC J , GOHL P , et al. The EuRoC micro aerial vehicle datasets[J]. International Journal of Robotics Research, 2016, 35 (10): 1157- 1163.
41	GENEVA P, ECKENHOFF K, LEE W, et al. Openvins: a research platform for visual-inertial estimation[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2020: 4666-4672.

序列	OpenVINS		PL-VIO		MF-VIO		FMF-VIO
序列	平移/m	旋转/rad	平移/m	旋转/rad	平移/m	旋转/rad	平移/m	旋转/rad
MH_01_easy	/	/	0.139 788¹	0.026 952	0.172 364²	0.024 544²	0.175 494	0.023 975¹
MH_02_easy	/	/	0.141 971	0.043 020	0.136 833²	0.038 554¹	0.136 259¹	0.038 887²
MH_03_medium	/	/	0.264 665	0.042 052	0.260 564¹	0.037 934¹	0.263 443²	0.038 871²
MH_04_difficult	/	/	0.362 234	0.040 540	0.329 141²	0.039 593¹	0.325 993¹	0.039 655²
MH_05_ difficult	/	/	0.277 990	0.026 338	0.269 991¹	0.025 451²	0.270 585²	0.022 942¹
V1_01_easy	0.064¹	0.020 926¹	0.079 288	0.143 704	0.068 954¹	0.141 590²	0.070 522	0.141 952
V1_03_difficult	0.202	0.103 568²	0.202 742	0.085 312	0.114 108¹	0.112 057	0.181 205²	0.082 737¹
V2_01_easy	0.073	0.027 663¹	0.087 473	0.053 186	0.059 390²	0.029 635	0.058 855¹	0.029 366²
V2_02_medium	0.079¹	0.041 364¹	0.147 834	0.073 479²	0.141 045²	0.077 073	0.141 081	0.077 313
V2_03_ difficult	/	/	0.308 307	0.077 721	0.214 972¹	0.037 335¹	0.257 571²	0.044 254²

序列	MF	GMF	G2MF
MH_01_easy	63.320 76	12.049 29	9.625 10
MH_02_easy	59.049 39	13.145 60	8.789 46
MH_03_ medium	57.212 60	11.961 64	9.586 75
MH_04_difficult	58.012 88	11.695 74	8.921 13
MH_05_ difficult	57.783 94	12.289 53	8.775 61
V1_01_easy	58.892 84	10.977 85	8.411 83
V1_03_difficult	56.624 59	10.368 42	9.129 10
V2_01_easy	56.284 12	9.939 593	9.404 82
V2_02_medium	58.044 27	11.710 93	9.303 44
V2_03_ difficult	58.629 07	10.307 59	9.595 35

模型	操作	PL-VIO	FMF-VIO
前端	点特征提取与匹配	18.729 40	17.150 06
	线段特征提取与匹配	122.447 10	36.727 67
	结构特征提取与匹配	-	8.775 61
后端	非线性优化	30.827 74	33.498 91
后端	边缘化	5.100 24	8.137 24

[1]	鹿珂珂, 王超, 吴俊峰, 钱峰. 基于梯度信息控制曝光的视觉惯性里程计[J]. 系统工程与电子技术, 2023, 45(5): 1496-1502.
[2]	付有斌, 康巧燕, 王建峰, 胡海岩, 赵朔. 标签分割的软件定义飞行自组网控制器智能部署方法[J]. 系统工程与电子技术, 2022, 44(10): 3249-3257.
[3]	杨磊, 李慧娟, 黄博, 刘伟, 李埔丞. 双层稀疏组Lasso高分辨SAR结构特征增强成像[J]. 系统工程与电子技术, 2021, 43(2): 351-362.
[4]	奔粤阳, 魏晓峰, 高倩倩, 赵显鹏, 戴平安, 孙炎. 基于SINS舰船升沉测量误差分析与补偿[J]. 系统工程与电子技术, 2020, 42(7): 1590-1596.
[5]	董春梅, 任顺清, 陈希军. 基于三轴转台误差分析的IMU标定方法[J]. 系统工程与电子技术, 2016, 38(4): 895-901.
[6]	刘丹, 段建民, 于宏啸. 基于自适应渐消EKF的FastSLAM算法[J]. 系统工程与电子技术, 2016, 38(3): 644-651.
[7]	韩萍, 于晓红, 邓豪, 冯青, 石庆研. 稀疏描述与结构特征融合的极化SAR斑点抑制算法[J]. 系统工程与电子技术, 2015, 37(2): 284-290.
[8]	吴晓琳，宋萌，苑晶，孙凤池，陶通. 通讯范围受限条件下的多机器人主动SLAM[J]. Journal of Systems Engineering and Electronics, 2012, 34(10): 2121-2128.
[9]	陈思，杨健，宋小全. SAR图像线特征分析与自动提取[J]. Journal of Systems Engineering and Electronics, 2010, 32(9): 1868-1874.
[10]	孙枫，孙伟. 旋转自动补偿捷联惯导系统技术研究[J]. Journal of Systems Engineering and Electronics, 2010, 32(1): 122-125.

一种基于结构特征的单目视觉惯性里程计

A monocular visual inertial odometry based on structural features

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 41

相关文章 10

编辑推荐

Metrics

本文评价