DOI:
https://doi.org/10.64539/sjer.v2i3.2026.471Keywords:
Agricultural robotics, Vision-based sorting, YouOnlyLookOnce YOLOv8, Object detection, Robotic manipulation, Edge deploymentAbstract
Modern robotic systems address complex engineering challenges using artificial intelligence and machine learning techniques. In agricultural robotics, fruit identification and sorting remain challenging due to variations in size, shape, color, orientation, and lighting conditions. This study presents the design and implementation of a vision-based fruit sorting robotic system integrating YOLOv8-based object detection with robotic manipulation. A custom dataset consisting of images of 2 different fruits (namely banana and strawberry images), including single-fruit and multi-fruit scenarios, was used and manually annotated using bounding boxes in CVAT. The dataset was divided into training, validation, and test subsets to enable robust model development under realistic operational conditions. A lightweight YOLOv8 model was trained using CUDA acceleration and optimized for edge deployment by selecting YOLOv8n to balance inference speed and detection accuracy. The trained model was converted to ONNX format and deployed on a Raspberry Pi 5 for real-time inference using live camera input. Evaluation on an independent test dataset achieved a precision of 0.999, recall of 1.000, mAP@0.5 of 0.995, and mAP@0.5:0.95 of 0.963 under controlled experimental conditions with limited object classes. The modular architecture enables low-cost and scalable deployment and provides a foundation for future enhancements, including closed-loop robotic control, additional object categories, and operation in more dynamic environments.
References
[1] L. E. Montoya-Cavero, R. Díaz de León Torres, A. Gómez-Espinosa, and J. A. Escobedo Cabello, “Vision systems for harvesting robots: Produce detection and localization,” Comput. Electron. Agric., vol. 192, p. 106562, Jan. 2022. https://doi.org/10.1016/J.COMPAG.2021.106562.
[2] Y. Tan, X. Liu, J. Zhang, Y. Wang, and Y. Hu, “A Review of Research on Fruit and Vegetable Picking Robots Based on Deep Learning,” Sensors 2025, Vol. 25, Page 3677, vol. 25, no. 12, p. 3677, Jun. 2025. https://doi.org/10.3390/S25123677.
[3] Y. Yang, Y. Han, S. Li, Y. Yang, M. Zhang, and H. Li, “Vision based fruit recognition and positioning technology for harvesting robots,” Comput. Electron. Agric., vol. 213, p. 108258, Oct. 2023. https://doi.org/10.1016/J.COMPAG.2023.108258.
[4] T. Yoshida, T. Kawahara, and T. Fukao, “Fruit recognition method for a harvesting robot with RGB-D cameras,” ROBOMECH Journal 2022 9:1, vol. 9, no. 1, pp. 15-, May 2022. https://doi.org/10.1186/S40648-022-00230-Y.
[5] G. Ian, C. Aaron, and B. Yoshua, “Deep Learning,” Request exam copy View preview Adaptive Computation and Machine Learning series, pp. 1–23, 2016, Accessed: Feb. 03, 2026. [Online]. Available: http://www.deeplearningbook.org.
[6] S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks”, Accessed: May 10, 2026. [Online]. Available: https://github.com/.
[7] M. H. Ali, K. Aizat, K. Yerkhan, T. Zhandos, and O. Anuar, “Vision-based Robot Manipulator for Industrial Applications,” Procedia Comput. Sci., vol. 133, pp. 205–212, Jan. 2018. https://doi.org/10.1016/J.PROCS.2018.07.025.
[8] M. H. Dairath et al., “Computer vision-based prototype robotic picking cum grading system for fruits,” Smart Agricultural Technology, vol. 4, p. 100210, Aug. 2023. https://doi.org/10.1016/J.ATECH.2023.100210.
[9] C. S. Nandi, B. Tudu, and C. Koley, “An automated machine vision based system for fruit sorting and grading,” Proceedings of the International Conference on Sensing Technology, ICST, pp. 195–200, 2012. https://doi.org/10.1109/ICSENST.2012.6461669.
[10] J. Yu, W. Miao, G. Zhang, K. Li, Y. Shi, and L. Liu, “Target Positioning and Sorting Strategy of Fruit Sorting Robot Based on Image Processing,” International Information and Engineering Technology Association, vol. 38, no. 3, pp. 797–805, Jun. 2021. https://doi.org/10.18280/ts.380326.
[11] B. Purwantana et al., “Performance of an Automatic Sorting System Based on Combination of Robotic Arm and Vision Machine for Agricultural Products,” Social Science Research Network, pp. 1–13, May 2025. https://doi.org/10.2139/SSRN.5253197.
[12] S. B. Nikam, S. B. Lande, G. C. Wakchaure, V. J. Nagalkar, and S. M. Mukane, “ADQS-YOLO: Automatic Dragon Fruit Quality Classification and Sorting Mechanism Using YOLOv8n Model,” Sep. 2025. https://doi.org/10.21203/RS.3.RS-7171300/V1.
[13] H. Zhao et al., “Real-Time Object Detection and Robotic Manipulation for Agriculture Using a YOLO-Based Learning Approach,” Proceedings of the IEEE International Conference on Industrial Technology, 2024. https://doi.org/10.1109/ICIT58233.2024.10540740.
[14] M. Fahmi, A. Yudhana, Sunardi, A.-N. Sharkawy, and Furizal, “Classification for Waste Image in Convolutional Neural Network Using Morph-HSV Color Model,” Scientific Journal of Engineering Research, vol. 1, no. 1, pp. 18–25, Jan. 2025. https://doi.org/10.64539/SJER.V1I1.2025.12.
[15] B. Siciliano, L. Sciavicco, L. Villani, and G. Oriolo, Robotics: Modelling, Planning and Control. Springer, 2008. Accessed: Feb. 01, 2026. [Online]. Available: https://people.disim.univaq.it/~costanzo.manes/EDU_stuff/Robotics_Modelling,%20Planning%20and%20Control_Sciavicco_extract.pdf.
[16] S. E. Mathe, H. K. Kondaveeti, S. Vappangi, S. D. Vanambathina, and N. K. Kumaravelu, “A comprehensive review on applications of Raspberry Pi,” Comput. Sci. Rev., vol. 52, May 2024. https://doi.org/10.1016/J.COSREV.2024.100636.
[17] B. Sebastian, “servo Robot Arm 4DOF by sebastian barrio | Download free STL model | Printables.com,” Printables.com. Accessed: Mar. 25, 2026. [Online]. Available: https://www.printables.com/model/823944-servo-robot-arm-4dof.
[18] A. N. Sharkawy and J. M. Nazzal, “Design and Manufacturing Using 3D Printing Technology of A 5-DOF Manipulator for Industrial Tasks,” International Journal of Robotics and Control Systems, vol. 4, no. 2, pp. 893–909, 2024. https://doi.org/10.31763/IJRCS.V4I2.1456.
[19] Raspberry Pi, “Buy a Raspberry Pi 5 – Raspberry Pi,” Raspberry Pi Ltd. Accessed: May 10, 2026. [Online]. Available: https://www.raspberrypi.com/products/raspberry-pi-5/.
[20] J. S. D. R. G. A. F. Redmon, “(YOLO) You Only Look Once,” Cvpr, vol. 2016-December, pp. 779–788, Dec. 2016. https://doi.org/10.1109/CVPR.2016.91.
[21] G. Jocher, A. Chaurasia and J. Qiu, “YOLO by Ultralytics. - References - Scientific Research Publishing.” 2023, Accessed: May 10, 2026. [Online]. Available: https://www.scirp.org/reference/referencespapers?referenceid=3532980.
[22] OpenMMLab, “Algorithm principles and implementation with YOLOv8 — MMYOLO 0.6.0 documentation,” MMYOLO Documentation. Accessed: May 10, 2026. [Online]. Available: https://mmyolo.readthedocs.io/en/latest/recommended_topics/algorithm_descriptions/yolov8_description.html.
[23] T.-Y. Lin, P. Dollár, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature Pyramid Networks for Object Detection,” Dec. 2016, Accessed: May 10, 2026. [Online]. Available: https://arxiv.org/pdf/1612.03144.
[24] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. C. Chen, “MobileNetV2: Inverted Residuals and Linear Bottlenecks,” Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 4510–4520, Jan. 2018. https://doi.org/10.1109/CVPR.2018.00474.
[25] ONNX Runtime, “Home.” Accessed: May 10, 2026. [Online]. Available: https://onnxruntime.ai/.
[26] D. McHugh, N. Buckley, and E. L. Secco, “A low cost visual sensor for gesture recognition via AI CNNS,” Intelligent Systems Conference (IntelliSys), 2020, Accessed: May 10, 2026. [Online]. Available: https://hira.hope.ac.uk/id/eprint/3016/1/Poster_Daniel.pdf.
[27] N. Buckley, L. Sherrett, and E. L. Secco, “A CNN sign language recognition system with single & double-handed gestures,” Proceedings - 2021 IEEE 45th Annual Computers, Software, and Applications Conference, COMPSAC 2021, pp. 1250–1253, Jul. 2021. https://doi.org/10.1109/COMPSAC51774.2021.00173.
[28] E. L. Secco, D. D. McHugh, and N. Buckley, “A CNN-Based Computer Vision Interface for Prosthetics’ Control,” Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, LNICST, vol. 440 LNICST, pp. 41–59, 2022. https://doi.org/10.1007/978-3-031-06368-8_3.
[29] S. Procter and E. L. Secco, “Design of a Biomimetic BLDC Driven Robotic Arm for Teleoperation & Biomedical Applications,” Journal of Human, Earth, and Future, vol. 2, no. 4, pp. 345–354, Dec. 2021. https://doi.org/10.28991/HEF-2021-02-04-03.
[30] Z. Isherwood and E. L. Secco, “A Raspberry Pi Computer Vision System for Self-driving Cars,” Lecture Notes in Networks and Systems, vol. 507 LNNS, pp. 910–924, 2022. https://doi.org/10.1007/978-3-031-10464-0_63.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Muhammad Afaq, Emanuele Lindo Secco
This work is licensed under a Creative Commons Attribution 4.0 International License.
