Insulators that connect high-voltage transmission lines may experience various faults due to long-term exposure to the natural environment, leading to safety degradation and reliability issues in power grids. Therefore, detecting defective insulators through daily maintenance and long-term overhaul is crucial. We propose an insulator defect detection method to achieve this objective, using a novel neural network named more information-you only look once (MI-YOLO). MI-YOLO includes several modifications compared to YOLOv5s: a skip connection module with down-sampling in the backbone, a spatial pyramid dilated convolution module in the neck, and a novel serial-parallel spatial-channel attention module in between. These changes enhance the feature abstraction process by providing more information (MI) through the YOLO-like structure, hence the name MI-YOLO. To evaluate the superiority of MI-YOLO, multiple experiments are performed. First, an ablation study demonstrates the effectiveness of novel substructures and their combinations compared to YOLOv5s. Then, MI-YOLO is compared to baseline structures, including faster R-CNN, SSD, and five YOLOs. Furthermore, the performances of MI-YOLO and 10 state-of-the-art object detection methods are evaluated. Lastly, a second dataset is adopted to evaluate the generalization property and heat maps are presented. The experimental results show that the new designs substantially improve the accuracy of insulator fault detection by abstracting more information from the features.