In this paper, we propose an algorithm model PearlNet and the corresponding detection dataset for freshwater pearls detection, to increase the Degree of Automation and improve the efficiency of existing detection methods based on pearl colors and shapes. PearlNet based on CenterNet. According to the characteristics of the small target of freshwater pearls, the minimum size module of the network is deleted, and the attention mechanism is added at the same time, ignoring the irrelevant background information and focusing on the pearl feature information, which improves the accuracy of recognition. In the transport convolution process, the image quality effect caused by upsampling is reduced by data fusion. The experimental results proved that the PearlNet has a recognition accuracy of 98.4%, which is 15.43%, 9.05% and 5.2% higher than that of CenterNet, Yolo V3 and SSD. PearlNet can accurately identify the color and shape of pearls, which provides a reference for freshwater pearl identification and detection.