基于卷积特征的场景地标检索方法

doi:10.12305/j.issn.1001-506X.2023.05.05

Abstract

Abstract:

Retrieval task of scene landmark matching mainly solves the retrieval and matching problem of geographical targets under the conditions of view angle change, illumination change, and even occlusion. It is also called instance target retrieval. In this paper, we design a three streams' siamese network architecture with high performance convolutional network, the triple loss function, and the regional proposal network is used to accurately define the target area, so generate a accurate and robust image feature representation with fixed length. According to the characteristics of the scene landmark matching, the global features of the image are used for coarse retrieval, and the local features are used for fine retrieval. And combined with the query expansion method, accurate instance target retrieval is achieved. Experiments show that compared with other representative retrieval methods, this method can achieve competitive results in public data set testing.

Key words: convolutional network, landmark matching, siamese network architecture, triplet loss function, instance target

CLC Number:

TP301

Siqiang DONG, Nianmao DENG, Yan LIU, Yubao ZHANG. Scene landmark retrieval method based on convolution feature[J]. Systems Engineering and Electronics, 2023, 45(5): 1297-1304.

Figures/Tables 12

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Table 2

Fig.6

Table 3

Table 4

Table 5

Table 6

References 34

1	YU Q, WANG C F, CETINER B, et al. Building information modeling and classification by visual learning at a city scale[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1910.06391.
2	ZHU P F, WEN L Y, BIAN X, et al. Vision meets drones: a challenge[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1804.07437.
3	RELJA A , PETR G , AKIHIKO T , et al. NetVLAD: CNN architecture for weakly supervised place recognition[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2018, 40 (6): 1437- 1451. doi: 10.1109/TPAMI.2017.2711011
4	LIN T Y, CUI Y, BELONGGIE S, et al. Learning deep representations for ground-to-aerial geolocalization[C]//Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2015: 5007-5015.
5	KNEIP L, SCARAMUZZA D, SIEGWART R. A novel parametrization of the perspective-three-point problem for a direct computation of absolute camera position and orientation[C]//Proc. of the 24th IEEE Conference on Computer Vision and Pattern Recognition, 2011.
6	KRIZHEVSKY A , SUTSKEVER I , HINTON G. E . ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60 (6): 84- 90. doi: 10.1145/3065386
7	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1409.1556.
8	MEI T , RUI Y , LI S P , et al. Multimedia search reranking: a literature survey[J]. ACM Computing Surveys, 2014, 46 (3): 38.
9	REN S Q , HE K , GIRSHICK R , et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
10	EVAN S , JONATHAN L , TREVOR D . Fully convolutional networks for semantic segmentation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2017, 39 (4): 640- 651. doi: 10.1109/TPAMI.2016.2572683
11	TOLIAS G, SICRE R, JEGOU H. Particular object retrieval with integral max pooling of CNN activations[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1511.05879.
12	RAZAVIAN A S, AZIZPOUR H, SULLIVAN J, et al. CNN features off-the-shelf: an astounding baseline for recognition[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1403.6382.
13	RADENOVIC F, TOLIAS G, CHUM O. CNN image retrieval learns from BoW: unsupervised fine-tuning with hard examples[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1604.02426.
14	KALANTIDIS Y, MELLINA C, OSINDERO S. Cross dimensional weighting for aggregated deep convolutional features[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1512.04065.
15	GONG Y C, WANG L W, GUO R Q, et al. Multi-scale order less pooling of deep convolutional activation features[C]//Proc. of the European Conference on Computer Vision, 2014.
16	BOCHKOVSKIY A, WANG C Y, LIAO H. YOLOv4: optimal speed and accuracy of object detection[EB/OL]. [2022-04-06]. https://arxiv.org/abs/2004.10934.
17	CHUM O, PHILBIN J, SIVIC J, et al. Total recall: automatic query expansion with a generative feature model for object retrieval[C]//Proc. of the IEEE 11th International Conference on Computer Vision, 2007.
18	ARANDJELOVIC R. Three things everyone should know to improve object retrieval[C]//Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2012: 2911-2918.
19	JEGOU H , DOUZE M , SCHMID C . Improving bag-of-features for large scale image search[J]. International Journal of Computer Vision, 2010, 87 (3): 316- 336. doi: 10.1007/s11263-009-0285-2
20	ZHANG Y M, JIA Z Y, CHEN T H. Image retrieval with geometry preserving visual phrases[C]//Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2011: 809-816.
21	ZHANG W , NGO C W . Topological spatial verification for instance search[J]. IEEE Trans.on Multimedia, 2015, 17 (8): 1236- 1247. doi: 10.1109/TMM.2015.2440997
22	WEI X S , LUO J H , WU J , et al. Selective convolutional descriptor aggregation for fine-grained image retrieval[J]. IEEE Trans.on Image Processing, 2017, 26 (6): 2868- 2881. doi: 10.1109/TIP.2017.2688133
23	RADENOVIC F , TOLIAS G , CHUM O . Fine-tuning CNN image retrieval with no human annotation[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2019, 41 (7): 1655- 1668. doi: 10.1109/TPAMI.2018.2846566
24	GORDO A , ALMAZ'AN J , REVAUD J , et al. End-to-end learning of deep visual representations for image retrieval[J]. International Journal of Computer Vision, 2017, 124 (2): 237- 254. doi: 10.1007/s11263-017-1016-8
25	HE K , ZHANG X Y , REN S Q , et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2015, 37 (9): 1904- 1916. doi: 10.1109/TPAMI.2015.2389824
26	HE K, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1512.03385.
27	MUJA M , LOWE D G . Scalable nearest neighbor algorithms for high dimensional data[J]. IEEE Trans.on Pattern Analysis and Machine Intelligence, 2014, 36 (11): 2227- 2240. doi: 10.1109/TPAMI.2014.2321376
28	WANG C Y, LIAO H, WU Y H, et al. CSPNet: a new backbone that can enhance learning capability of CNN[C]//Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, 2020.
29	RUSSAKOVSKY O , DENG J , SU H , et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115 (3): 211- 252. doi: 10.1007/s11263-015-0816-y
30	PHILBIN J, CHUM O, ISARD M, et al. Object retrieval with large vocabularies and fast spatial matching[C]//Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2007.
31	PHILBIN J, CHUM O, ISARD M, et al. Lost in quantization: improving particular object retrieval in large scale image databases[C]//Proc. of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2008.
32	GORDO A, ALMAZAN J, REVAUD J, et al. Deep image retrieval: learning global representations for image search[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1604.01325.
33	LIU T Y . Learning to rank for information retrieval[J]. Foundations and Trends in Information Retrieval, 2009, 3 (3): A6- 110.
34	BABENKO A, LEMPITSKY V S. Aggregating local deep features for image retrieval[EB/OL]. [2022-04-06]. https://arxiv.org/abs/1510.07493.

算法/场景	数据集
算法/场景	牛津地标数据集 (简单)	巴黎建筑物数据集 (简单)
局部特征求和池化	78.5	77.1
局部特征最大池化	79.8	84.2

算法/场景	数据集
算法/场景	牛津地标数据集 (简单)	巴黎建筑物数据集 (简单)
局部特征求和池化	78.2	76.5
局部特征最大池化	79.1	83.5

算法(场景)	mAP
算法(场景)	牛津地标数据集	巴黎建筑物数据集
MAC^[11]	41.7	66.2
SPoC^[12]	39.8	69.2
CroW^[19]	42.4	70.4
R-MAC^[11]	49.8	74.0
R-MAC(三元组损失)^[21]	60.9	78.9
GeM(对比损失)^[23]	64.7	77.2
本文方法(全局特征)	67.5	80.1

算法(场景)	mAP
算法(场景)	牛津地标数据集	巴黎建筑物数据集
GeM(对比损失+QE)^[23]	67.2	80.7
本文方法(全局+局部+QE)	71.4	84.0

算法(场景)	mAP
算法(场景)	牛津地标数据集	巴黎建筑物数据集
MAC^[11]	18.0	44.1
SPoC^[12]	12.4	44.7
CroW^[19]	18.5	47.2
R-MAC^[11]	32.4	52.4
R-MAC(三元组损失)^[21]	38.5	59.4
GeM(对比损失)^[23]	13.3	56.3
本文方法(全局特征)	42.9	61.0

Scene landmark retrieval method based on convolution feature

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 34

Related Articles 4

Recommended Articles

Metrics

Comments

[1]	Xiaoling ZHOU, Zhaoxia ZHANG, Ya LU, Qian WANG, Kunkun WANG. SAR image recognition based on improved R-FCN [J]. Systems Engineering and Electronics, 2022, 44(4): 1202-1209.
[2]	Mengchen ZHAO, Xiujuan YAO, Jing WANG, Suhui DONG. Single-channel blind source separation method of spatial aliasing signal based on Stacked-TCN [J]. Systems Engineering and Electronics, 2021, 43(9): 2628-2636.
[3]	Yuyuan ZHANG, Limin ZHANG, Wenjun YAN. SFBC-OFDM recognition method based on cross-correlation feature map and dilated dense convolutional neural networks [J]. Systems Engineering and Electronics, 2021, 43(9): 2657-2664.
[4]	Limeng SHI, Chengzhi YANG, Hongchao WU. Radar signal recognition method based on deep residual network and triplet loss [J]. Systems Engineering and Electronics, 2020, 42(11): 2506-2512.