基于RHTC网络的飞机目标检测与精细识别

doi:10.12305/j.issn.1001-506X.2021.12.04

Abstract

Abstract:

Direction detection and fine-gained recognition of aircraft targets is an important task in the field of high-resolution optical remote sensing image interpretation. Aiming at the difficulty of direction detection and recognition of multi-directional densely arranged aircraft in remote sensing images, an aircraft detection and recognition method based on rotating hybrid task cascade (RHTC) network is proposed. Firstly, based on hybrid task cascade (HTC) network, the number of segmented branches is expanded, and the segmented branches and bounding box branches are cascaded at multiple levels to continuously strengthen semantic features. Secondly, a new slant frame regressor is designed and added to the last layer of the mask branch to complete the target direction prediction. Finally, a new directional loss function is added to optimize the training process, so as to complete the construction of RHTC network. In the data preprocessing stage, the fine mask of each type of aircraft target is constructed to enhance the target detail and improve the mask prediction accuracy. Several groups of experiments were carried out on aircraft data sets constructed based on DOTA and public Google images. The results show that compared with other advanced methods, the proposed method has better performance in aircraft detection direction accuracy and category average accuracy. In addition, the designed skew frame regressor and directional loss function also have good performance when embedded into other segmented networks.

Key words: high resolution remote sensing image, aircraft target, rotated hybrid task cascade network, direction detection, fine-grained recognition (RHTC)

CLC Number:

TN957

Xu CAO, Huanxin ZOU, Fei CHENG, Runlin LI, Shitian HE. Aircraft target detection and fine-grained recognition based on RHTC network[J]. Systems Engineering and Electronics, 2021, 43(12): 3439-3451.

Figures/Tables 23

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Fig.9

Fig.10

Fig.11

Fig.12

Table 1

Fig.13

Fig.14

Table 2

Table 3

Fig.15

Table 4

Fig.16

Table 5

Table 6

Table 7

References 35

1	唐小佩, 杨小冈, 刘云峰, 等. 基于深度卷积神经网络的飞机识别研究[J]. 电光与控制, 2018, 25 (5): 72- 76.
	TANG X P , YANG X G , LIU Y F , et al. Research on aircraft recognition based on deep convolutional neural network[J]. Electronics Optics and Control, 2018, 25 (5): 72- 76.
2	MIAO W X, LUO Z. Aircraft detection based on multiple scale faster-RCNN[C]//Proc. of the International Conference on Virtual Reality and Visualization, 2018: 90-93.
3	ZHANG M C , WU X , WANG P . Aircraft recognition via extraction of closed contour and partial feature matching[J]. Computer Simulation, 2006, doi: 10.1016/S1005-8885(07)60042-9
4	刘莉, 吴秀清, 牛宗标, 等. 模糊图像中的飞机识别方法[J]. 计算机仿真, 2006, 23 (3): 164- 166.
	LIU L , WU X Q , NIU Z B , et al. Aircraft recognition method in fuzzy image[J]. Computer simulation, 2006, 23 (3): 164- 166.
5	QIU J B , LI S J , WANG W . A new approach to detect aircrafts in remote sensing images based on corner and edge information fusion[J]. Microelectronics & Computer, 2011, 28 (9): 214- 216.
6	ZHANG Y X , JIAN H U , ZHANG Z , et al. Research on aircraft recognition based on local detection and affine invariant feature[J]. Remote Sensing Information, 2013, 28 (3): 3- 102.
7	ZHANG F , LIU S Q , WANG D B , et al. Aircraft recognition in infrared image using wavelet moment invariants[J]. Image and Vision Computing, 2009, 27 (4): 313- 318. doi: 10.1016/j.imavis.2008.08.007
8	CAI D . Study on aircraft recognition in high spatial resolution remote sensing image based on skeleton characteristics analysis[J]. Advanced Materials Research, 2011, 268-270, 1982- 1985. doi: 10.4028/www.scientific.net/AMR.268-270.1982
9	LIU G , SUN X , FU K , et al. Aircraft recognition in high-resolution satellite images using coarse-to-fine shape prior[J]. IEEE Geoscience and Remote Sensing Letters, 2012, 10 (3): 573- 577.
10	GUO Z H , LI S H . One-dimensional frequency-domain features for aircraft recognition from radar range profiles[J]. IEEE Trans.on Aerospace and Electronic Systems, 2010, 46 (4): 1880- 1892. doi: 10.1109/TAES.2010.5595601
11	REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: towards real-time target detection with region proposal networks[C]//Proc. of the Advances in Neural Information Processing Systems, 2015: 91-99.
12	TANG T Y , ZHOU S L , DENG Z P , et al. Vehicle detection in aerial images based on region convolutional neural networks and hard negative example mining[J]. Sensors, 2017, 17 (2): 336. doi: 10.3390/s17020336
13	DENG Z P , SUN H , ZHOU S , et al. Toward fast and accurate vehicle detection in aerial images using coupled region-based convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10 (8): 3652- 3664. doi: 10.1109/JSTARS.2017.2694890
14	孙嘉赤, 邹焕新, 邓志鹏, 等. 基于级联卷积神经网络的港口多方向舰船检测与分类[J]. 系统工程与电子技术, 2020, 42 (9): 1903- 1910.
	SUN J C , ZOU H X , DENG Z P , et al. Oriented inshore ship detection and classification based on cascade RCNN[J]. Systems Engineering and Electronics, 2020, 42 (9): 1903- 1910.
15	REN S Q , HE K M , GIRSHICK R , et al. Faster R-CNN: towards real-time target detection with region proposal networks[J]. IEEE Trans.on Pattern Analysis & Machine Intelligence, 2017, 39 (6): 1137- 1149.
16	DING J, XUE N, LONG Y, et al. Learning RoI transformer for oriented target detection in aerial images[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 2849-2858.
17	YU D H, ZHAO C, XU J F, et al. A method of aircraft detection using fully convolutional network[C]//Proc. of the International Conference on Computer Systems, Electronics and Control, 2017: 914-918.
18	WANG B, ZHOU Y, ZHANG H, et al. An aircraft target detection method based on regional convolutional neural network for remote sensing images[C]//Proc. of the IEEE 9th International Conference on Electronics Information and Emergency Communication, 2019: 474-478.
19	LI Y , FU K , SUN H , et al. An aircraft detection framework based on reinforcement learning and convolutional neural networks in remote sensing images[J]. Remote Sensing, 2018, 10 (2): 243. doi: 10.3390/rs10020243
20	ZHANG Y H , SUN H , ZUO J , et al. Aircraft type recognition in remote sensing images based on feature learning with conditional generative adversarial networks[J]. Remote Sensing, 2018, 10 (7): 1123. doi: 10.3390/rs10071123
21	YANG J C , ZHU Y , JIANG B , et al. Aircraft detection in remote sensing images based on a deep residual network and super-vector coding[J]. Remote Sensing Letters, 2018, 9 (3): 228- 236. doi: 10.1080/2150704X.2017.1415474
22	LIN J X, LI X D, PAN H. Aircraft recognition in remote sensing images based on deep learning[C]//Proc. of the 33rd Youth Academic Annual Conference of Chinese Association of Automation, 2018: 895-899.
23	ZUO J W , XU G L , FU K , et al. Aircraft type recognition based on segmentation with deep convolutional neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15 (2): 282- 286. doi: 10.1109/LGRS.2017.2786232
24	俞汝劼, 杨贞, 熊惠霖. 基于深度卷积神经网络的航空器检测与识别[J]. 计算机应用, 2017, 37 (6): 1702- 1707.
	YU R J , YANG Z , XIONG H L . Aircraft detection and recognition based on deep convolutional neural network[J]. Journal of Computer Applications, 2017, 37 (6): 1702- 1707.
25	CHEN K, PANG J, WANG J, et al. Hybrid task cascade for instance segmentation[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2019: 4974-4983.
26	XIA G S, BAI X, DING J, et al. DOTA: a large-scale dataset for target detection in aerial images[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 3974-3983.
27	MA J Q , SHAO W , YE H , et al. Arbitrary-oriented scene text detection via rotation proposals[J]. IEEE Trans.on Multimedia, 2018, 20 (11): 3111- 3122. doi: 10.1109/TMM.2018.2818020
28	JIANG Y Y, ZHU X, WANG X, et al. R2CNN: rotational region cnn for orientation robust scene text detection[EB/OL]. [2020-11-30]. https://arxiv.org/abs/1706.09579v2
29	WANG X , SUN H , FU K , et al. Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks[J]. Remote Sensing, 2018, 10 (1): 132- 140. doi: 10.3390/rs10010132
30	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//Proc. of the IEEE International Conference on Computer Vision, 2017: 2961-2969.
31	CAI Z , VASCONCELOS N . Cascade R-CNN: high quality target detection and instance segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2019,
32	CAI Z, VASCONCELOS N. Cascade R-CNN: delving into high quality target detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 6154-6162.
33	LIN T Y, DOLLÁR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition, 2017: 2117-2125.
34	安静. 基于数学形态学的图像增强算法及其应用[D]. 兰州: 西北师范大学, 2016.
	AN J. Image enhancement algorithm based on mathematical morphology and its application[D]. Lanzhou: Northwest Normal University, 2015.
35	OTSU N . A threshold selection method from gray-level histograms[J]. IEEE Trans.on Systems, Man, and Cybernetics, 1979, 9 (1): 62- 66. doi: 10.1109/TSMC.1979.4310076

参数	型号
参数	A-50	An-12	An-24	B-1B	B-2	B-52	C-5	C-17	C-130
S₁₂	398	435	216	879	772	156	1 026	1 005	163
S₃₄	31	15	40	21	14	16	44	63	13

参数	型号
参数	E-3	EP-3	F-15	F-16	Il-76	KC-10	KC-135	MiG-31	P-8A
S₁₂	865	311	256	123	494	1 303	244	287	164
S₃₄	25	4	54	27	22	1	14	1	2

参数	型号
参数	RC-135S	RQ-4	Su-24	Su-27	Tu-22M	Tu-95	Tu-160	Typhoon	其他
S₁₂	460	267	205	353	675	218	863	189	3 272
S₃₄	36	41	3	73	69	6	7	37	34

数量类别	型号
数量类别	A-50	An-12	An-24	B-1B	B-2	B-52	C-5	C-17	C-130
原始数量	75	281	377	43	40	101	116	85	149
扩增数量	1 029	1 335	1 427	1 087	881	1 159	984	1 188	1 621

数量类别	型号
数量类别	E-3	EP-3	F-15	F-16	Il-76	KC-10	KC-135	MiG-31	P-8A
原始数量	36	247	137	85	886	75	202	479	36
扩增数量	798	2 120	1 156	1 090	1 657	1 263	1 370	1 505	1 236

数量类别	型号
数量类别	RC-135S	RQ-4	Su-24	Su-27	Tu-22M	Tu-95	Tu-160	Typhoon	其他
原始数量	34	9	741	1 101	440	189	93	19	2 418
扩增数量	1 017	471	1 317	1 774	2 561	870	687	1 665	5 762

数据集	精准度	方法
数据集	精准度	FRO^[26]	ROI transformer^[16]	RRPN^[28]	R2CNN^[29]	RDFPN^[30]	RHTC
自建数据集	DP	25.50	46.88	47.62	43.50	37.66	79.82
自建数据集	AP	71.08	89.56	84.25	87.20	85.63	93.53
DOTA	DP	40.39	68.33	65.24	66.87	63.50	76.89
DOTA	AP	73.82	86.25	80.92	85.98	81.14	87.13

参数	方法
参数	RHTC	FRO^[26]	ROI transformer^[16]	RRPN^[28]	R2CNN^[29]	RDFPN^[30]
Δθ_max/(°)	135.097	139.088	172.873	157.69	160.256	175.644
Δθ_median/(°)	3.13	61.845	18.294	25.652	17.21	24.251
Δθ_mean/(°)	10.587	71.083	38.904	50.334	40.603	47.247
Δθ_std/(°)	23.15	55.865	46.681	52.79	48.663	51.98

参数	方法
参数	FRO^[26]	ROI transformer^[16]	RRPN^[28]	R2CNN^[29]	RDFPN^[30]	RHTC
mAP	25.5	46.88	47.62	43.5	37.66	79.82

Aircraft target detection and fine-grained recognition based on RHTC network

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 23

References 35

Related Articles 4

Recommended Articles

Metrics

Comments

网络类型	精细掩膜	分割级联	方向损失	DP	mAP
HTC^[25]	—	—	—	52.51	85.42
HTC^[25]+改进	√	—	—	58.66	88.73
	—	√	—	72.36	89.20
	—	—	√	67.23	86.12
	√	—	√	69.35	89.72
	—	√	√	74.66	90.20
	√	√	√	79.82	91.34

评价标准	基线模型1	基线模型1+方向损失+精细掩膜	基线模型2	基线模型2+方向损失+精细掩膜
DP	45.33	62.63	49.28	66.69
mAP	73.40	81.27	75.65	84.11

[1]	Runlin LI, Huanxin ZOU, Xu CAO, Fei CHENG, Shitian HE, Meilin LI. Multi-direction remote sensing ship detection based on center point and semantic information [J]. Systems Engineering and Electronics, 2022, 44(6): 1772-1781.
[2]	Ping HAN, Bo ZHOU, Bin LU, Binbin HAN. Aircraft target detection of PolSAR based on the fusion of environmental feature and scattering characteristics [J]. Systems Engineering and Electronics, 2020, 42(10): 2231-2238.
[3]	YANG Qingqing, FAN Guihua. Battle damage assessment of buildings based on improved fuzzy comprehensive evaluation method [J]. Systems Engineering and Electronics, 2018, 40(9): 2026-2031.
[4]	LUO Chang, WANG Jie, WANG Shiqiang, SHI Tong, REN Weihua. General deep transfer features based high resolution remote scene classification [J]. Systems Engineering and Electronics, 2018, 40(3): 682-691.