基于感兴趣区域的物体抓取位姿检测

doi:10.12305/j.issn.1001-506X.2024.06.05

Abstract

Abstract:

In industrial production, the objects to be grasped often have the characteristics of varions types, messy placements, irregular shapes, etc., which make it difficult to accurately obtain the grasping pose of the object. In view of the above problems, this paper proposes a two-stage grasp pose estimation method based on deep learning. In the first stage, a lightweight rotating target detection algorithm based on improved you only look once version4 (YOLOv4) is proposed to enhance the detection speed and improve detection accuracy of targets. Firstly, the lightweight network GhostNet and deep separable convolution are used to reconstruct the original network to reduce the parameters of the entire model. Then, the adaptive spatial feature fusion structure and the non-reference attention module are added to the neck network to improve the positioning accuracy of the region of interest. Finally, the approximate skew intersection over union (SkewIoU) loss is used to solve the periodic problem of the angle. In the second stage, a mask extraction region of interest is made with the same size as the original picture. At the same time, an improved DeepLabV3+algorithm is proposed to detect the grasping pose of objects in the area of interest. Experimental results show that the detection accuracy of the improved YOLOv4 network reaches 92.5%, and the improved DeepLabV3+algorithm achieves 94.6% and 92.4% of the image splitting and object splitting accuracy on the Cornell capture dataset, respectively, and can accurately detect the grasping pose of objects.

Key words: deep learning, mask, region of interest, lightweight network, pose detection

CLC Number:

TP242

Xiantao SUN, Wangyang JIANG, Wenjie CHEN, Weihai CHEN, Yali ZHI. Object grasp pose detection based on the region of interest[J]. Systems Engineering and Electronics, 2024, 46(6): 1867-1877.

Figures/Tables 21

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Table 1

Table 2

Fig.13

Table 3

Fig.14

Fig.15

Table 4

Fig.16

Fig.17

References 30

1	CAI J H , CEN J , WANG H K , et al. Real-time collision-free grasp pose detection with geometry-aware refinement using high-resolution volume[J]. IEEE Robotics and Automation Letters, 2022, 7 (2): 1888- 1895. doi: 10.1109/LRA.2022.3142424
2	CHENG H , WANG Y Y , MENG M Q H . A robot grasping system with single-stage anchor-free deep grasp detector[J]. IEEE Trans.on Instrumentation and Measurement, 2022, 71, 5009712.
3	WEN H T, YAN J H, PENG W L, et al. TransGrasp: grasp pose estimation of a category of objects by transferring grasps from only one labeled instance[C]//Proc. of the 17th European Conference, 2022: 445-461.
4	WEI H , PAN S C , MA G , et al. Vision-guided hand-eye coordination for robotic grasping and its application in tangram puzzles[J]. Artificial Intelligence, 2021, 2 (2): 209- 228.
5	LENZ I , LEE H , SAXENA A . Deep learning for detecting robotic grasps[J]. The International Journal of Robotics Research, 2015, 34 (4/5): 705- 724.
6	MORRISON D , CORKE P , LEITNER J . Learning robust, real-time, reactive robotic grasping[J]. The International Journal of Robotics Research, 2020, 39 (2/3): 183- 201.
7	LIANG H Z, MA X J, LI S, et al. Pointnetgpd: detecting grasp configurations from point sets[C]//Proc. of the International Conference on Robotics and Automation, 2019: 3629-3635.
8	ZHANG L Z , WU D M . A single target grasp detection network based on convolutional neural network[J]. Computational Intelligence and Neuroscience, 2021, (5): 5512728.
9	CHU F J , XU R N , VELA P A . Real-world multi-object, multi-grasp detection[J]. IEEE Robotics and Automation Letters, 2018, 3 (4): 3355- 3362. doi: 10.1109/LRA.2018.2852777
10	AINETTER S, FRAUNDORFER F. End-to-end trainable deep neural network for robotic grasp detection and semantic segmentation from RGB[C]//Proc. of the IEEE International Conference on Robotics and Automation, 2021: 13452-13458.
11	陈丹, 林清泉. 基于级联式Faster RCNN的三维目标最优抓取方法研究[J]. 仪器仪表学报, 2019, 40 (4): 229- 237.
	CHEN D , LIN Q Q . Research on 3D object optimal grasping method based on cascaded Faster RCNN[J]. Chinese Journal of Scientific Instrument, 2019, 40 (4): 229- 237.
12	REN S Q , HE K M , GIRSHICK R , et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. Advances in Neural Information Processing Systems, 2015, 28, 91- 99.
13	孟月波, 黄琪, 韩九强, 等. 基于两阶段的机器人动态多物品定位抓取方法[J]. 激光与光电子学进展, 2023, 60 (6): 288- 297.
	MENG Y B , HUANG Q , HAN J Q , et al. Robot dynamic object positioning and grasping method based on two stages[J]. Laser & Optoelectronics Progress, 2023, 60 (6): 288- 297.
14	安广琳, 李宗刚, 杜亚江, 等. 基于深度学习的多工件抓取点定位方法[J]. 激光与光电子学进展, 2023, 60 (12): 311- 321.
	AN G L , LI Z G , DU Y J , et al. Research on multiple workpiece grasping point localization method based on deep learning[J]. Laser & Optoelectronics Progress, 2023, 60 (12): 311- 321.
15	BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2020.
16	CHEN L C, ZHU Y K, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proc. of the European Conference on Computer Vision, 2018: 801-818.
17	HE K M , ZHANG X Y , REN S R , et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1904- 1916. doi: 10.1109/TPAMI.2015.2389824
18	LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]//Proc. of the IEEE Conference on Computer Cision and Pattern Recognition, 2018: 8759-8768.
19	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
20	HAN K, WANG Y H, TIAN Q, et al. Ghostnet: more features from cheap operations[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 1580-1589.
21	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 1251-1258.
22	YANG L X, ZHANG R Y, LI L D, et al. Simam: a simple, parameter-free attention module for convolutional neural networks[C]//Proc. of the International Conference on Machine Learning, 2021: 11863-11874.
23	LIU S T, HUANG D, WANG Y H. Learning spatial fusion for single-shot object detection[C]//Proc. of the International Conference on Computer Vision and Pattern Recognition, 2019.
24	YANG X, YAN J C, FENG Z M, et al. R3det: refined single-stage detector with feature refinement for rotating object[C]//Proc. of the AAAI Conference on Artificial Intelligence, 2021, 35(4): 3163-3171.
25	YANG X, YANG J R, YAN J C, et al. SCRDet: towards more robust detection for small, cluttered and rotated objects[C]//Proc. of the IEEE/CVF International Conference on Computer Vision, 2019: 8232-8241.
26	SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-cam: visual explanations from deep networks via gradient-based localization[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 618-626.
27	HAN J M , DING J , LI J , et al. Align deep features for oriented object detection[J]. IEEE Trans.on Geoscience and Remote Sensing, 2021, 60, 1- 11.
28	YANG X, ZHANG G F, LI W T, et al. H2RBox: horizontal box annotation is all you need for oriented object detection[C]//Proc. of the International Conference on Learning Representations, 2023.
29	HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2016: 770-778.
30	李明, 鹿朋, 朱龙, 等. 基于RGB-D融合的密集遮挡抓取检测[J]. 控制与决策, 2023, 38 (10): 2867- 2874.
	LI M , LU P , ZHU L , et al. Densely occluded grasping objects detection based on RGB-D fusion[J]. Control and Decision, 2023, 38 (10): 2867- 2874.

模型	GhostNet	DSC	SimAM	ASFF	近似SkewIoU损失
1	-	-	-	-	-
2	√	√	-	-	-
3	-	-	√	-	-
4	-	-	√	√	-
5	-	-	-	-	√
6	√	√	√	√	√

模型	参数量/M	计算量/G	mAP/%
1	64.363	60.334	80.4
2	11.429	7.088	77.45
3	64.363	60.334	81.01
4	65.231	60.906	82.67
5	66.684	63.476	91.9
6	14.617	10.802	92.5

算法	权重大小/MB	FPS	mAP/%
Faster RCNN	108	17	75.2
YOLOv4	224	36.2	80.4
R3Det	320	14.8	87.8
S2ANet	295	16.9	89.6
H2RBoX	122	22.6	93.8
R-YOLOv4(本文算法)	48	49.8	92.5

方法	检测精度/%		FPS
方法	图像拆分	对象拆分	FPS
文献[5]	73.9	75.6	0.07
文献[8]	88.7	87.2	-
本文(GhostNet)	86.3	83.5	58.2
本文(ResNet-50)	94.6	92.4	47.5

[1]	Xuemei CHEN, Zhiheng LIU, Suiping ZHOU, Hang YU, Yanming LIU. Road extraction from high-resolution remote sensing images based on HRNet [J]. Systems Engineering and Electronics, 2024, 46(4): 1167-1173.
[2]	Tianwen ZHANG, Xiaoling ZHANG, Zikang SHAO, Tianjiao ZENG. Mask attention interaction for SAR ship instance segmentation [J]. Systems Engineering and Electronics, 2024, 46(3): 831-838.
[3]	Duanyang SHI, Qiang LIN, Bing HU, Xiaoshuai DU. Target detection method of primary surveillance radar based on YOLO [J]. Systems Engineering and Electronics, 2024, 46(1): 143-151.
[4]	Meng WANG, Bing ZHU. Application of uncertainty modeling in 2D and 3D object detection [J]. Systems Engineering and Electronics, 2023, 45(8): 2370-2376.
[5]	Kai SHAO, Ziqun DU, Guangyu WANG. CSI feedback method for dynamically adjusting compression rate based on model pruning [J]. Systems Engineering and Electronics, 2023, 45(8): 2615-2622.
[6]	Tianshu CUI, Dong WANG, Zhen HUANG. Automatic modulation classification based on lightweight network for space cognitive communication [J]. Systems Engineering and Electronics, 2023, 45(7): 2220-2226.
[7]	Yu JIANG, Qi YUAN, Zhitao HU, Weiwei WU, Xin GU. Airport arrival and departure delay time prediction based on meteorological factors [J]. Systems Engineering and Electronics, 2023, 45(6): 1722-1731.
[8]	Yang CHEN, Canhui LIAO, Kun ZHANG, Jian LIU, Pengju WANG. A signal modulation indentification algorithm based on self-supervised contrast learning [J]. Systems Engineering and Electronics, 2023, 45(4): 1200-1206.
[9]	Ye ZHANG, Yi HOU, Kewei OUYANG, Shilin ZHOU. Survey of univariate sequence data classification methods [J]. Systems Engineering and Electronics, 2023, 45(2): 313-335.
[10]	Zhengtu SHAO, Dengrong XU, Wenli XU, Hanzhong WANG. Radar active jamming recognition based on LSTM and residual network [J]. Systems Engineering and Electronics, 2023, 45(2): 416-423.
[11]	Renfei CHEN, Yong PENG, Zhongwen LI. A novel detector for floating objects based on continual unsupervised domain adaptation strategy [J]. Systems Engineering and Electronics, 2023, 45(11): 3391-3401.
[12]	Chensong TAO, Siwei CHEN, Shunping XIAO. SAR image interrupted sampling repeater jamming detection based on deep learning models [J]. Systems Engineering and Electronics, 2023, 45(11): 3465-3473.
[13]	Shiyang HE, Ling WANG, Daiyin ZHU, Jun QIAN. Thunderstorm prediction method based on spatiotemporal memory decoupling RNN [J]. Systems Engineering and Electronics, 2023, 45(11): 3474-3480.
[14]	Haoran LI, Wei XIONG, Yaqi CUI. An association method between SAR images and AIS information based on depth feature fusion [J]. Systems Engineering and Electronics, 2023, 45(11): 3491-3497.
[15]	Zichao LIU, Jiang WANG, Shaoming HE. Time and angle control guidance law based on deep learning [J]. Systems Engineering and Electronics, 2023, 45(11): 3579-3587.

Object grasp pose detection based on the region of interest

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 21

References 30

Related Articles 15

Recommended Articles

Metrics

Comments