Abstract: The main task of question answer selection is to sort the candidate answers in the question answering system. The current mainstream methods extract the features for the questions and answers through representation learning under the framework of deep learning, the candidate answers are sorted by the similarity between the feature vector of the given question and the feature vector of the candidate answer. However, the methods ignore the relevance of the question and the answer. To solve this problem, this paper proposes a deep learning method based on multiscale similarity features. Firstly, this method is based on the traditional deep learning model to extract the features of the question and the answer respectively. Secondly, the similarity matrix of the question and answer (Q&A) is obtained by calculating the similarity of the feature at each scale. Finally, three feature learning models are proposed to learn the similarity matrix in order to get joint similarity. This paper evaluates our model on the public data set-WebQA. Experiments show that the results are improved compared with traditional deep learning methods by adding the similarity feature learning method.