Traditional defect detection methods for body stampings face challenges due to the wide range of defect sizes and minimal differences between defect classes, resulting in low accuracy and limited industrial applicability. A deep learning defect detection model SP-DDN with channel self-associative feature pyramid was proposed. The model extended the feature pyramid structure to obtain a new feature fusion CFPN structure, and added a channel correlation analysis attention module to the structure to improve the feature extraction capability of the model for the differences between multiple classes of defects and enhanced the defect detection accuracy of the model. K-means++ was applied to cluster the defect data in the dataset to generate specific pre-checked boxes and solved the problem of mismatch between the default pre-checked box size and the actual defect size. Finally, application validation was carried out using a self-made surface defect dataset of stamped parts and a publicly available surface defect dataset of hot rolled steel. The results show that the method in this paper improves the mAP by 3.1% and 4.0% and the recall by 1.5% and 5.5%, respectively, compared with the benchmark network.