Human-Follower Mobile Robot with High Payload for Cross-Domain Assistance using Shared Autonomy

Abstract

Purpose Relocating heavy payloads during removals in home settings or in the parking of supermarkets might cause physical stress on the human body and affect health of older adults. Whereas fully autonomous social navigations of robots carrying payloads to assist humans remain a challenge (Navarro, et al., 2024), scripted robot motions fail to adapt to dynamic goals and unpredicted events. We therefore address robotized payload carrying while following a human by leveraging on shared autonomy (Guiffo Kaigom, 2023). Our objective is to combine cognitive human skills and machine intelligence for a smart and decent transportation of a wide range of payloads (up to 70 Kg) in multiple domains with negligible efforts, comfort, and self-fulfillment for humans. This assistance benefits from the ability of upskilled mobile robots to spatially track goal-oriented (e.g., collision-free) and dynamic motions of a human to reach unknown locations while carrying a payload. Conversely, humans can contextualize, supervise, and adjust the robot behavior to accommodate events the robot is not aware of and meet highly personalized objectives. The resulting symbiosis thus supports the self-determination of older adults. Method The robot reacts to the human behavior using three functionalities: perception, reasoning, and self-control. Perception is achieved by using a state-of-the-art stereo camera sensor (Stereolabs ZED 2i). The relative distance and orientation of the target person in the 3D space are estimated from image and depth data. Specifically, a pre-trained YOLOv8 model extracts human features from data to reason. The depth map uses the coordinates of a bounding box to determine distance values related to the nearest person by harnessing camera characteristics. Desired linear and angular velocities enforcing the proximity to the person while preserving a safety zone that depends upon the context are autonomously generated by a proportional, integral, and derivative controller and forwarded to the control unit of the robot. The communication occurs via the robot operating system ROS2. Results and Discussion The mobile robot system endowed with a vision-driven human-follower ability can assist people in several tasks. It also leaves room to cope with future challenges. As shown on pictures below, the robot can help move heavy removal boxes from one room to another in home settings. A physical stress on human body is thereby avoided during navigation. Also, the robot can bring shopping baskets to a car in a parking. Older adults can therefore focus on their needs and no longer on the final total weight.  During extensive application validations in practice, we observed that pre-trained neural networks need more robustness to avoid mismeasurements in detection and distance estimation. Enhanced generalization and adaption to changing conditions are investigated. To enhance safety, algorithms for obstacle avoidance based on e.g. LiDAR data (Uguzlar, Cansu, Contarli, & Sezer, 2023) will be implemented. Therefore, redundancy through the fusion of multiple sensor data to improve robustness is pending. All in all, this work forms the basis for follow-up projects with the overarching goal of pervasive and itinerant self-determination fostered by robotics and digitalization (Guiffo Kaigom, 2023).