Convolutional neural network (CNN) has been widely used in the field of image classification. Its hyper-parameters have a great impact on the misclassification rate. In order to further optimize CNN hyper-parameters, the change ratio of relative probability based on Softmax regression was introduced. The change ratio of relative probability was used to find the hyper-parameters which have great influence on the image classification problem, and the hyper-parameters were adjusted according to their importance. In order to verify the effectiveness of the change ratio of relative probability, experiments were carried out in Architecture 1 and Architecture 2. The experiment results show that the concept of the change ratio of relative probability introduced in Softmax regression can reflect the effect of CNN hyper-parameters on misclassification rate in both architectures, and it is more helpful to find a better combination of CNN hyper-parameters and reduce the misclassification rate. The comparative experiments on MNIST and CIFAR-10 datasets show that the above conclusions are applicable to different datasets.