The emergence of new infrared decoys poses a severe challenge to the operational effectiveness of conventional infrared imaging air-to-air missiles. In recent years, the research progress of deep learning is rapid, which strongly promotes the development of target tracking and detection. Based on the MDNet framework, the aspect ratio and mean contrast of the artificial features are introduced, and the deep feature and the artificial feature are fused into a tracking framework, which solves the problem that the single feature could not effectively resist the complex interference such as the surface-type decoy in target tracking. In order to evaluate the performance of the algorithm, the simulation sequences and the real shot sequences are tested respectively. Experimental indicates show that the proposed algorithm is better than state-of-the-art trackers in both tracking accuracy and robustness, which is a kind of infrared aerial target tracking method with a strong adaptability.