Wheat Ear Detection Method Based on Deep Learning

doi:10.13304/j.nykjdb.2021.0612

Abstract

Abstract:

Wheat is one of the most important food crops in the world. In order to solve the problem of low efficiency of artificial field wheat ears counting and yield prediction， this paper proposed a real-time detection method based on deep learning for high resolution small and dense objects. The wheat ear images data set was preprocessed， including image segmentation， marking and enhancement， and the YOLOv4 network model was built based on TensorFlow to improve the transfer learning after adjusted； different models of YOLOv3， YOLOv4-Tiny and Faster R-CNN were trained and used compare to the practicability and limitations with the improved model； the key factors affecting the performance of the wheat ears detection model were discussed. By means of image segmentation， this paper studied the influence of images with different resolutions on model detection. The results showed that the difference between foreground and background could be improved by changing the image resolution， and the optimal pixel ratio of the target could be determined， which had a significant effect on small dense wheat ears and effectively improved the accuracy of the model. By testing the results of the improved model and evaluating the precision recall performance of different models， the model showed high detection accuracy and strong robustness. The mean average precision accuracy of wheat ear images with different resolutions， different varieties and different seasons was 93.7%， and the detection speed of single image was 52 frame·s^-1， which met the real-time detection of wheat ears. The results provided a technical guarantee for wheat spike counting and yield prediction in the field.

Key words: deep learning, object detection, grain, YOLOv4, real-time detection

摘要：

小麦是重要的粮食作物之一，针对人工田间麦穗计数及产量预测效率低的问题，基于深度学习提出了一种高分辨率的小密集麦穗实时检测方法。对麦穗图像数据集进行图像分割、标注、增强预处理，基于Tensorflow搭建YOLOv4网络模型，调整改进后对其进行迁移学习；与YOLOv3、YOLOv4-tiny、Faster R-CNN训练模型进行对比，对改进模型的实用性与局限性进行分析；重点分析影响麦穗检测模型性能的关键因素。通过图像分割的方式，证明了通过改变图像分辨率确定麦穗所占图像最优像素比，可以提高前景与背景差异，对小密集麦穗有显著效果。通过对改进模型的测试，表明该模型检测精度高，鲁棒性强。不同分辨率、不同品种、不同时期的麦穗图像均类平均精度（mAP）为93.7%，单张图片的检测速度为52帧·s^-1，满足了麦穗的高精度实时检测。该研究结果为田间麦穗计数以及产量预测提供技术支持。

关键词: 深度学习, 目标检测, 麦穗, YOLOv4, 实时检测

CLC Number:

S126

Yue ZHAO, Yong WEI, Huiyong SHAN, Zhimin MU, ZHANGJianxin, Haiyun WU, Hui ZHAO, Jianlong HU. Wheat Ear Detection Method Based on Deep Learning[J]. Journal of Agricultural Science and Technology, 2022, 24(9): 96-105.

赵越, 卫勇, 单慧勇, 穆志民, 张健欣, 吴海云, 赵辉, 胡建龙. 基于深度学习的高分辨率麦穗图像检测方法[J]. 中国农业科技导报, 2022, 24(9): 96-105.

Figures/Tables 12

References 24

1	CAO S, XU D, HANIF M, et al.. Genetic architecture underpinning yield component traits in wheat [J]. Theor. Appl. Genet., 2020, 133:1811-1823.
2	XU X, LI H, YIN F, et al.. Wheat ear counting using K-means clustering segmentation and convolutional neural network [J]. Plant Methods, 2020, 106(16): 1-13.
3	岑海燕,朱月明,孙大伟,等.深度学习在植物表型研究中的应用现状与展望[J].农业工程学报,2020,36(9):1-16.
	CEN H Y, ZHU Y M, SUN D W, et al.. Current status and future perspective of the application of deep learning in plant phenotype research [J]. Trans. Chin. Soc. Agric. Eng., 2020, 36(9): 1-16.
4	孙红,李松,李民赞,等.农业信息成像感知与深度学习应用研究进展[J].农业机械学报,2020,51(5):1-17.
	SUN H, LI S, LI Z M, et al.. Research Progress of Image Sensing and Deep Learning in Agriculture [J]. Trans. Chin. Soc. Agric. Mach., 2020 51(5): 1-17.
5	WU W, YANG T, LI R, et al.. Detection and enumeration of wheat grains based on a deep learning method under various scenarios and scales [J]. J. Int. Agr, 2020, 8(19): 1998-2008.
6	MA J, LI Y, DU K, et al.. Segmenting ears of winter wheat at flowering stage using digital images and deep learning [J]. Comput. Electron. Agric., 2020, 2(168): 1-16.
7	HASAN M M, CHOPIN J P, LAGA H, et al.. Detection and analysis of wheat spikes using Convolutional neural networks [J]. Plant Methods, 2018, 100(14): 1-13.
8	MADEC, JINX., LUH., et al.. Ear density estimation from high resolution RGB imagery using deep learning technique [J]. Agric. For. Met., 2019, 3(264): 225-234.
9	DU Z, YIN J, YANG J. Expanding receptive field YOLO for small object detection [J]. IOP Publ., 2019, 3(1314): 1-6.
10	XIAO J. exYOLO: A small object detector based on YOLOv3 object detector [J]. Computer Sci., 2021, 188: 18-25.
11	LabelImg is a graphical image annotation tool and label object bounding boxes in images [EB/OL]. (2017-04-03)[2018-12-03]. .
12	BOCHKOVSKIY A, WANG C, LIAO H M. YOLOv4: Optimal speed and accuracy of object detection [EB/OL]. (2020-04-23) [2021-12-23]. .
13	WANG C, LIAO H M, WU Y, et al.. CSPNet: a new backbone that can enhance learning capability of CNN [C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Seattle, WA, USA: IEEE, 2020:1571-1580.
14	HE K, ZHANG X, REN S, et al.. Spatial pyramid pooling in deep convolutional networks for visual recognition [J]. IEEE Comp. Vis. Patt. Rec., 2015, 9(37): 1904-1906.
15	LIU S, QI L, QIN H, et al.. Path aggregation network for instance segmentation [C]// Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 8759-8768.
16	M-TPHAM, COURTRAI L, FRIGUET C, et al.. YOLO-fine: one-stage detector of small objects under various backgrounds in remote sensing images [J/OL]. RemoteSens., 2020, 12(15): 2501 [2021-12-23]. .
17	XU D, WU Y. Improved YOLO-V3 with DenseNet for multi-scale remote sensing target detection [J/OL]. Sensors, 2020, 20(15):4276 [2021-12-23]. .
18	YANG B, FU X, SIDIROPOULOS ND, et al.. Towards K-means-friendly spaces: simultaneous deep learning and clustering [J]. Proc. Mach. Res., 2017, 70: 3861-3870.
19	LIU M, WANG X, ZHOU X, et al.. UAV-YOLO: small object detection on unmanned aerial vehicle perspective [J/OL]. Sensors, 2020, 20(8): 2238 [2021-12-23]. .
20	XIE Y, CAI J, BHOJWANI R, et al.. A locally-constrained YOLO framework for detecting small and densely-distributed building footprints [J]. Int. J. Geogr. Inf. Sci., 2019, 4(34): 1-25.
21	WU D, LV S, JIANG M, et al.. Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments [J]. Comput. Electron. Agric., 2020, 178: 1-12.
22	XU Z, SHI H, LI N, et al.. Vehicle detection under UAV based on optimal dense YOLO method [C]// Proceedings of 2018 5th International Conference on Systems and Informatics. Nanjing, China: IEEE, 2018: 407-411.
23	NALLDRIN, KRASIN I, PONT-TUSET J, et al.. The open images dataset [EB/OL]. (2016-11-03) [2021-12-23]. .
24	TIAN Y, YANG G, WANG Z, et al.. Apple detection during different growth stages in orchards using the improved YOLO-V3 model [J]. Comput. Electron. Agric., 2019, 157: 417-426.

模型 Model	图片尺寸 Image size	裁剪后分辨率 Cropped resolution	图片数量 Image number
#1	6 000×4 000	500×500	3 000
#2	6 000×4 000	1 000×1 000	3 000
#3	6 000×4 000	1 500×1 500	3 000
#4	6 000×4 000	2 000×2 000	3 000
#5	6 000×4 000	2 500×2 500	3 000
#6	6 000×4 000	3 000×3 000	3 000

模型 Model	图片尺寸 Image size	裁剪后分辨率 Cropped resolution	图片数量 Image number
#1	6 000×4 000	500×500	3 000
#2	6 000×4 000	1 000×1 000	3 000
#3	6 000×4 000	1 500×1 500	3 000
#4	6 000×4 000	2 000×2 000	3 000
#5	6 000×4 000	2 500×2 500	3 000
#6	6 000×4 000	3 000×3 000	3 000

模型Model	方法Method	分辨率Resolution	均类平均精度mAP/%
#1	YOLOv4	500×500	90.6
#2	YOLOv4	1 000×1 000	92.3
#3	YOLOv4	1 500×1 500	93.7
#4	YOLOv4	2 000×2 000	91.6
#5	YOLOv4	2 500×2 500	90.3
#6	YOLOv4	3 000×3 000	89.1

模型Model	方法Method	分辨率Resolution	均类平均精度mAP/%
#1	YOLOv4	500×500	90.6
#2	YOLOv4	1 000×1 000	92.3
#3	YOLOv4	1 500×1 500	93.7
#4	YOLOv4	2 000×2 000	91.6
#5	YOLOv4	2 500×2 500	90.3
#6	YOLOv4	3 000×3 000	89.1

方法Method	损失值Loss	F₁值F₁ score	交并比IoU	类平均精度mAP/%	检测速度FPS
YOLOv3	2.4	0.786	0.842	89.4	57
YOLOv4-tiny	2.8	0.764	0.823	87.3	137
YOLOv4	0.7	0.837	0.886	93.7	52
FasterR-CNN	0.9	0.823	0.874	92.8	7