In consideration of the problem of modulation classification in communication systems, a classification method based on convolutional neural network and sparse filtering is presented. First, the  feasibility of modulation classification based on the two dimensional gray images of the cyclic spectrum is analyzed. Then, down-sampling and clipping technologies are used for preprocessing of cyclic spectrum images, and finally the architecture of the deep convolutional neural network is designed and a pre-training procedure based on sparse filtering is proposed. The simulation results show that,  compared with the state-of-the-art deep learning-based methods, the proposed method has the advantages of simple model, less optimization and best performance in small sample scenarios, and it also has a good value for practice application.