基于感兴趣区域的物体抓取位姿检测

doi:10.12305/j.issn.1001-506X.2024.06.05

摘要/Abstract

摘要：

在工业生产中, 待抓取物体往往具有种类众多、摆放位置杂乱、形状不规则等特点, 使得难以准确获取物体抓取位姿。针对以上问题, 提出一种基于深度学习的两阶段抓取位姿估计方法。第1阶段, 提出一种基于YOLOv4(you only look once version4)改进的轻量级旋转目标检测算法, 提高目标的检测速度和检测精度。首先, 使用轻量化网络GhostNet和深度可分离卷积对原始网络进行重构, 降低整个模型参数。然后, 在颈部网络中增加自适应空间特征融合结构和无参注意力模块, 提高对感兴趣区域的定位精度; 最后, 使用近似倾斜交并比(skew intersection over union, SkewIoU)损失解决角度的周期性问题。第2阶段, 制作与原始图片尺寸一样的掩膜提取感兴趣区域; 同时, 提出一种改进的DeepLabV3+算法, 用以检测感兴趣区域中物体的抓取位姿。实验结果表明, 改进后的YOLOv4网络检测精度达到92.5%, 改进的DeepLabV3+算法在Cornell抓取数据集上的图像拆分和对象拆分精度分别达到94.6%, 92.4%, 且能准确检测出物体的抓取位姿。

关键词: 深度学习, 掩膜, 感兴趣区域, 轻量化网络, 位姿检测

Abstract:

In industrial production, the objects to be grasped often have the characteristics of varions types, messy placements, irregular shapes, etc., which make it difficult to accurately obtain the grasping pose of the object. In view of the above problems, this paper proposes a two-stage grasp pose estimation method based on deep learning. In the first stage, a lightweight rotating target detection algorithm based on improved you only look once version4 (YOLOv4) is proposed to enhance the detection speed and improve detection accuracy of targets. Firstly, the lightweight network GhostNet and deep separable convolution are used to reconstruct the original network to reduce the parameters of the entire model. Then, the adaptive spatial feature fusion structure and the non-reference attention module are added to the neck network to improve the positioning accuracy of the region of interest. Finally, the approximate skew intersection over union (SkewIoU) loss is used to solve the periodic problem of the angle. In the second stage, a mask extraction region of interest is made with the same size as the original picture. At the same time, an improved DeepLabV3+algorithm is proposed to detect the grasping pose of objects in the area of interest. Experimental results show that the detection accuracy of the improved YOLOv4 network reaches 92.5%, and the improved DeepLabV3+algorithm achieves 94.6% and 92.4% of the image splitting and object splitting accuracy on the Cornell capture dataset, respectively, and can accurately detect the grasping pose of objects.

Key words: deep learning, mask, region of interest, lightweight network, pose detection

中图分类号:

TP242

孙先涛, 江汪洋, 陈文杰, 陈伟海, 智亚丽. 基于感兴趣区域的物体抓取位姿检测[J]. 系统工程与电子技术, 2024, 46(6): 1867-1877.

Xiantao SUN, Wangyang JIANG, Wenjie CHEN, Weihai CHEN, Yali ZHI. Object grasp pose detection based on the region of interest[J]. Systems Engineering and Electronics, 2024, 46(6): 1867-1877.

图/表 21

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

表1

表2

图13

表3

图14

图15

表4

图16

图17

参考文献 30

1	CAI J H , CEN J , WANG H K , et al. Real-time collision-free grasp pose detection with geometry-aware refinement using high-resolution volume[J]. IEEE Robotics and Automation Letters, 2022, 7 (2): 1888- 1895. doi: 10.1109/LRA.2022.3142424
2	CHENG H , WANG Y Y , MENG M Q H . A robot grasping system with single-stage anchor-free deep grasp detector[J]. IEEE Trans.on Instrumentation and Measurement, 2022, 71, 5009712.
3	WEN H T, YAN J H, PENG W L, et al. TransGrasp: grasp pose estimation of a category of objects by transferring grasps from only one labeled instance[C]//Proc. of the 17th European Conference, 2022: 445-461.
4	WEI H , PAN S C , MA G , et al. Vision-guided hand-eye coordination for robotic grasping and its application in tangram puzzles[J]. Artificial Intelligence, 2021, 2 (2): 209- 228.
5	LENZ I , LEE H , SAXENA A . Deep learning for detecting robotic grasps[J]. The International Journal of Robotics Research, 2015, 34 (4/5): 705- 724.
6	MORRISON D , CORKE P , LEITNER J . Learning robust, real-time, reactive robotic grasping[J]. The International Journal of Robotics Research, 2020, 39 (2/3): 183- 201.
7	LIANG H Z, MA X J, LI S, et al. Pointnetgpd: detecting grasp configurations from point sets[C]//Proc. of the International Conference on Robotics and Automation, 2019: 3629-3635.
8	ZHANG L Z , WU D M . A single target grasp detection network based on convolutional neural network[J]. Computational Intelligence and Neuroscience, 2021, (5): 5512728.
9	CHU F J , XU R N , VELA P A . Real-world multi-object, multi-grasp detection[J]. IEEE Robotics and Automation Letters, 2018, 3 (4): 3355- 3362. doi: 10.1109/LRA.2018.2852777
10	AINETTER S, FRAUNDORFER F. End-to-end trainable deep neural network for robotic grasp detection and semantic segmentation from RGB[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2021: 13452-13458.
11	陈丹, 林清泉. 基于级联式Faster RCNN的三维目标最优抓取方法研究[J]. 仪器仪表学报, 2019, 40 (4): 229- 237.
	CHEN D , LIN Q Q . Research on 3D object optimal grasping method based on cascaded Faster RCNN[J]. Chinese Journal of Scientific Instrument, 2019, 40 (4): 229- 237.
12	REN S Q , HE K M , GIRSHICK R , et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems, 2015, 28, 91- 99.
13	孟月波, 黄琪, 韩九强, 等. 基于两阶段的机器人动态多物品定位抓取方法[J]. 激光与光电子学进展, 2023, 60 (6): 288- 297.
	MENG Y B , HUANG Q , HAN J Q , et al. Robot dynamic object positioning and grasping method based on two stages[J]. Laser & Optoelectronics Progress, 2023, 60 (6): 288- 297.
14	安广琳, 李宗刚, 杜亚江, 等. 基于深度学习的多工件抓取点定位方法[J]. 激光与光电子学进展, 2023, 60 (12): 311- 321.
	AN G L , LI Z G , DU Y J , et al. Research on multiple workpiece grasping point localization method based on deep learning[J]. Laser & Optoelectronics Progress, 2023, 60 (12): 311- 321.
15	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
16	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proc. of the European Conference on Computer Vision, 2018: 801-818.
17	HE K M , ZHANG X Y , REN S R , et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1904- 1916. doi: 10.1109/TPAMI.2015.2389824
18	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//Proc. of the IEEE Conference on Computer Cision and Pattern Recognition, 2018: 8759-8768.
19	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
20	HAN K, WANG Y H, TIAN Q, et al. Ghostnet: more features from cheap operations[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 1580-1589.
21	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1251-1258.
22	YANG L X, ZHANG R Y, LI L D, et al. Simam: a simple, parameter-free attention module for convolutional neural networks[C]//Proc. of the International Conference on Machine Learning, 2021: 11863-11874.
23	LIU S T, HUANG D, WANG Y H. Learning spatial fusion for single-shot object detection[C]//Proc. of the International Conference on Computer Vision and Pattern Recognition, 2019.
24	YANG X, YAN J C, FENG Z M, et al. R3det: refined single-stage detector with feature refinement for rotating object[C]//Proc. of the AAAI Conference on Artificial Intelligence, 2021, 35(4): 3163-3171.
25	YANG X, YANG J R, YAN J C, et al. SCRDet: towards more robust detection for small, cluttered and rotated objects[C]//Proc. of the IEEE/CVF International Conference on Computer Vision, 2019: 8232-8241.
26	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-cam: visual explanations from deep networks via gradient-based localization[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 618-626.
27	HAN J M , DING J , LI J , et al. Align deep features for oriented object detection[J]. IEEE Trans.on Geoscience and Remote Sensing, 2021, 60, 1- 11.
28	YANG X, ZHANG G F, LI W T, et al. H2RBox: horizontal box annotation is all you need for oriented object detection[C]//Proc. of the International Conference on Learning Representations, 2023.
29	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
30	李明, 鹿朋, 朱龙, 等. 基于RGB-D融合的密集遮挡抓取检测[J]. 控制与决策, 2023, 38 (10): 2867- 2874.
	LI M , LU P , ZHU L , et al. Densely occluded grasping objects detection based on RGB-D fusion[J]. Control and Decision, 2023, 38 (10): 2867- 2874.

模型	GhostNet	DSC	SimAM	ASFF	近似SkewIoU损失
1	-	-	-	-	-
2	√	√	-	-	-
3	-	-	√	-	-
4	-	-	√	√	-
5	-	-	-	-	√
6	√	√	√	√	√

模型	参数量/M	计算量/G	mAP/%
1	64.363	60.334	80.4
2	11.429	7.088	77.45
3	64.363	60.334	81.01
4	65.231	60.906	82.67
5	66.684	63.476	91.9
6	14.617	10.802	92.5

算法	权重大小/MB	FPS	mAP/%
Faster RCNN	108	17	75.2
YOLOv4	224	36.2	80.4
R3Det	320	14.8	87.8
S2ANet	295	16.9	89.6
H2RBoX	122	22.6	93.8
R-YOLOv4(本文算法)	48	49.8	92.5

方法	检测精度/%		FPS
方法	图像拆分	对象拆分	FPS
文献[5]	73.9	75.6	0.07
文献[8]	88.7	87.2	-
本文(GhostNet)	86.3	83.5	58.2
本文(ResNet-50)	94.6	92.4	47.5

[1]	陈雪梅, 刘志恒, 周绥平, 余航, 刘彦明. 基于HRNet的高分辨率遥感影像道路提取方法[J]. 系统工程与电子技术, 2024, 46(4): 1167-1173.
[2]	张天文, 张晓玲, 邵子康, 曾天娇. 基于掩模注意型交互的SAR舰船实例分割[J]. 系统工程与电子技术, 2024, 46(3): 831-838.
[3]	施端阳, 林强, 胡冰, 杜小帅. 基于YOLO的航管一次雷达目标检测方法[J]. 系统工程与电子技术, 2024, 46(1): 143-151.
[4]	汪萌, 诸兵. 不确定性建模在2D和3D目标检测中的应用[J]. 系统工程与电子技术, 2023, 45(8): 2370-2376.
[5]	邵凯, 杜自群, 王光宇. 基于模型剪枝动态调整压缩率的CSI反馈方法[J]. 系统工程与电子技术, 2023, 45(8): 2615-2622.
[6]	崔天舒, 王栋, 黄振. 面向空间认知通信的轻量化网络自动调制分类方法[J]. 系统工程与电子技术, 2023, 45(7): 2220-2226.
[7]	姜雨, 袁琪, 胡志韬, 吴薇薇, 顾欣. 基于气象因素的机场进离港延误预测[J]. 系统工程与电子技术, 2023, 45(6): 1722-1731.
[8]	陈洋, 廖灿辉, 张锟, 刘建, 王鹏举. 基于自监督对比学习的信号调制识别算法[J]. 系统工程与电子技术, 2023, 45(4): 1200-1206.
[9]	张晔, 侯毅, 欧阳克威, 周石琳. 单变量序列数据分类方法综述[J]. 系统工程与电子技术, 2023, 45(2): 313-335.
[10]	邵正途, 许登荣, 徐文利, 王晗中. 基于LSTM和残差网络的雷达有源干扰识别[J]. 系统工程与电子技术, 2023, 45(2): 416-423.
[11]	陈任飞, 彭勇, 李忠文. 基于持续无监督域适应策略的水面漂浮物目标检测方法[J]. 系统工程与电子技术, 2023, 45(11): 3391-3401.
[12]	陶臣嵩, 陈思伟, 肖顺平. 基于深度学习模型的SAR图像间歇采样转发干扰检测[J]. 系统工程与电子技术, 2023, 45(11): 3465-3473.
[13]	何诗扬, 汪玲, 朱岱寅, 钱君. 基于时空记忆解耦RNN的雷暴预测方法[J]. 系统工程与电子技术, 2023, 45(11): 3474-3480.
[14]	李浩然, 熊伟, 崔亚奇. 基于深度特征融合的SAR图像与AIS信息关联方法[J]. 系统工程与电子技术, 2023, 45(11): 3491-3497.
[15]	刘子超, 王江, 何绍溟. 基于深度学习的时间角度控制制导律[J]. 系统工程与电子技术, 2023, 45(11): 3579-3587.