Collaboration of 2 Arm Robots in a Pick and Place Task with a Task Allocator Method Based on Computer Vision

Abstract

Robotic arms have become integral in the manufacturing sector for automating repetitive tasks such as pick-and-place operations, which were traditionally performed manually. Manual execution is often limited by human fatigue, reduced accuracy, and the risk of injury, prompting a shift toward robotic automation. While single-arm robotic systems have been widely adopted, recent advancements now enable dual-arm collaboration, which introduces new technical challenges related to synchronization, precision, and object tracking. This study addresses these challenges by implementing an object detection system using YOLOv5 combined with HSV filtering to optimize performance under low computational constraints. The system communicates with robotic pole-arms via Python through Arduino serial communication. A perspective transformation technique is employed to ensure accurate mapping between 2D camera input and the robot's 3D operational space. The trained detection model achieved a mean Average Precision (mAP) of 97.7% at 0.5 and 60% at 0.5:0.95. Object detection testing yielded high evaluation metrics, including a precision of 1.00, a recall of 0.977, and an F1 score of 0.96. In real-world testing, the pick-and-place process demonstrated success rates of 95%, 85%, and 85% across three trials. These results indicate that the proposed system is highly effective for industrial applications and lays a foundation for further research into collaborative robot systems in dynamic environments.