Farmers are often annoyed by crop pests, which can lead to serious losses if not detected in time. Deep learning has gradually penetrated into the agricultural field due to its proven superior performance, promoting the development of smart agriculture. Its real-time performance and effectiveness can help farmers spot pests in time and reduce losses. However, with the increasing application of deep learning technology, its disadvantage of relying heavily on large amounts of data has gradually been exposed. Before training a network, people need to spend a lot of manpower, time, and money to prepare a complete dataset and make the correct labels. To solve this problem, we introduced a less-shot object detection method in crop pest detection. We provide the first few-shot pest detection dataset (FSPD), which contains 20 categories and 2000 samples for this research. We use a few-shot object detection network based on an object pyramid called multiscale positive sample refinement. Numerous experiments are conducted to prove that, with only few (1/2/3/5/10-shot) samples, the accuracy of our method on FSPD is better than that of traditional deep learning methods, such as YOLO and faster R-CNN. After embedding the object pyramid branch, the network performance improves by an average of 5.9% on three splits compared with that without embedding. With input of only 10-shot novel samples, our method can achieve 76%, 79%, and 82% for the mAP on three splits, respectively. As a research study on the detection of crop diseases and pests with less radiation, our work can give some new insight to the future scholars, which is of great significance to the development of smart agriculture.