HARMONY: Assistive Robots for Healthcare

project image

People

Luzia Knoedler - PhD candidate
Dr. Nils Wilde - Postdoctoral researcher
Prof. Javier Alonso-Mora

Funding

This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 101017008.

More Links
Project Website GitHub Youtube Twitter

About the Project

TU Delft partners

Harmony is a Horizon 2020 project which develops assistive robotic mobile manipulation technologies for environments shared with humans, i.e., hospitals. Specifically, Harmony addresses two use cases:

  1. The automation of on-demand delivery tasks around the hospital
  2. The automation of bio-assay sample flow

Current robotic automation solutions only offer “islands of automation” where either mobility or manipulation is dealt with in isolation. Harmony aims to fill this gap in knowledge by combining both robotic mobility and manipulation modalities in complex, human-centred environments. We at AMR focus on providing adaptive task and motion planning with multiple robots in human-centred environments.

Traditionally task scheduling and planning has been decoupled from motion planning. Yet, when robots navigate in critical and dynamic environments, plans may have to be adapted online to take into account congestion and interaction with other robots and human co-workers.

We devise methods for multi-robot motion planning that schedule plans for the robots and adapt them online taking into account the priority of tasks, their associated uncertainty and the preferences and needs of human co-workers. This includes novel methods for Multi-Robot Task Assignment (MRTA) that consider uncertain environments and homogenous robot fleets. Further, we design novel multi-objective planning frameworks that can efficiently explore Pareto-optimal trade-offs. Our results include general solutions for a wide range of motion planning problems as well as specialized methods for multi-objective MRTA.

Multi-Objective Task Assignment
Multi-Objective Task Assignment

Furthermore, we develop local-motion planning approaches which do not only account for collision avoidance but also consider social interactions. We design integrated approaches for navigation as well as mobile manipulation in uncertain and dynamic environments shared with humans, which accounts for social interactions, navigation and coordination tasks, and that provides performance guarantees (in expectation).

Socially interactive local motion planning
Socially interactive local motion planning

Project Demonstrations

Partners

This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 101017008.

TU Delft partners

Current-Based Impedance Control for Interacting with Mobile Manipulators
Jelmer de Wolde, Luzia Knoedler, Gianluca Garofalo, Javier Alonso-Mora. In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024.

arxiv code web
As robots shift from industrial to human-centered spaces, adopting mobile manipulators, which expand workspace capabilities, becomes crucial. In these settings, seamless interaction with humans necessitates compliant control. Two common methods for safe interaction, admittance, and impedance control, require force or torque sensors, often absent in lowercost or lightweight robots. This paper presents an adaption of impedance control that can be used on current-controlled robots without the use of force or torque sensors and its application for compliant control of a mobile manipulator. A calibration method is designed that enables estimation of the actuators’ current/torque ratios and frictions, used by the adapted impedance controller, and that can handle model errors. The calibration method and the performance of the designed controller are experimentally validated using the Kinova GEN3 Lite arm. Results show that the calibration method is consistent and that the designed controller for the arm is compliant while also being able to track targets with fivemillimeter precision when no interaction is present. Additionally, this paper presents two operational modes for interacting with the mobile manipulator: one for guiding the robot around the workspace through interacting with the arm and another for executing a tracking task, both maintaining compliance to external forces. These operational modes were tested in realworld experiments, affirming their practical applicability and effectiveness.
paper image

Learning Social Homologies for Navigation
Diego Martinez-Baselga, Oscar de Groot, Luzia Knoedler, Luis Riazuelo, Javier Alonso-Mora, Luis Montano. In Robotics: Science and Systems (RSS), Workshop on Unsolved Problems in Social Robot Navigation, 2024.

pdf
Navigating mobile robots in social environments remains a challenging task due to the intricacies of human-robot interactions. Most of the motion planners designed for crowded and dynamic environments focus on reaching the goal while avoiding collisions or local social interactions, but do not explicitly consider the high-level navigation behavior (avoiding through the left or right side, letting others pass or passing before others, etc.). In this work, we present a novel motion planner that incorporates topology distinct paths representing diverse navigation strategies around humans. The planner uses a deep neural network model trained on real-world human motion data to estimate how well a topology classes imitate the human behavior. Then, selects the best topology class using it, ensuring socially intelligent and contextually aware navigation. Our system refines the chosen path through an optimization-based local planner in real time, ensuring adherence to desired social behaviors. We evaluate the prediction accuracy of the network with real-world data and the navigation capabilities in both simulation and a realworld platform. Our method demonstrates socially desirable behaviors, smooth and remarkable performance compared to other planners.
paper image

Interaction-Aware Autonomous Navigation among Pedestrians using Social Forces Response Dynamics
Luzia Knoedler, Nils Wilde, Javier Alonso-Mora. In Robotics: Science and Systems (RSS), Workshop on Unsolved Problems in Social Robot Navigation, 2024.

pdf
Ignoring the interactions between agents during motion planning in multi-agent environments can result in overly conservative or opaque navigation behaviors, and in dense crowds, it may lead to the so-called Freezing robot problem. Although coupled planning can mitigate these issues, it typically incurs high computational costs, especially as the number of agents increases. To enhance interaction while limiting the computational complexity, we formulate the interactions as an underactuated system and propose to leverage the Social Forces Model (SFM) as the pedestrians’ response dynamics. The SFM is a well-established model widely used for describing pedestrian behaviors. Unlike deep learning models that require extensive training data, SFM offers interpretability and adaptability across various environments. We use Model Predictive Path Integral Control to solve the optimization problem, demonstrating that by accounting for interactions, the robot can effectively leverage the behavior of other agents. Additionally, we show that when combined with a specific cost function, the robot is able to plan motions that decrease its impact on surrounding pedestrians.
paper image

Simultaneous Synthesis and Verification of Neural Control Barrier Functions through Branch-and-Bound Verification-in-the-Loop Training
Xinyu Wang, Luzia Knoedler, Frederik Baymler Mathiesen, Javier Alonso-Mora. In European Control Conference (ECC), 2024.

pdf code
Control Barrier Functions (CBFs) that provide formal safety guarantees have been widely used for safety-critical systems. However, it is non-trivial to design a CBF. Utilizing neural networks as CBFs has shown great success, but it necessitates their certification as CBFs. In this work, we leverage bound propagation techniques and the Branch-and-Bound scheme to efficiently verify that a neural network satisfies the conditions to be a CBF over the continuous state space. To accelerate training, we further present a framework that embeds the verification scheme into the training loop to synthesize and verify a neural CBF simultaneously. In particular, we employ the verification scheme to identify partitions of the state space that are not guaranteed to satisfy the CBF conditions and expand the training dataset by incorporating additional data from these partitions. The neural network is then optimized using the augmented dataset to meet the CBF conditions. We show that for a non-linear control-affine system, our framework can efficiently certify a neural network as a CBF and render a larger safe set than state-of-the-art neural CBF works. We further employ our learned neural CBF to derive a safe controller to illustrate the practical use of our framework.
paper image

Regret-based Sampling of Pareto Fronts for Multi-Objective Robot Planning Problems
A. Botros, N. Wilde, A. Sadeghi, J. Alonso-Mora, S. L. Smith. In IEEE Transaction on Robotics(T-RO), 2024.

pdf
Many problems in robotics seek to simultaneously optimize several competing objectives. A conventional approach is to create a single cost function comprised of the weighted sum of the individual objectives. Solutions to this scalarized optimization problem are Pareto optimal solutions to the original multiobjective problem. However, finding an accurate representation of a Pareto front remains an important challenge. Uniformly spaced weights are often inefficient and do not provide error bounds. We address the problem of computing a finite set of weights whose optimal solutions closely approximate the solution of any other weight vector. To this end, we prove fundamental properties of the optimal cost as a function of the weight vector. We propose an algorithm that greedily adds the weight vector least-represented by the current set, and provide bounds on the regret. We extend our method to include suboptimal solvers for the scalarized optimization, and handle stochastic inputs to the planning problem. Finally, we illustrate that the proposed approach significantly outperforms baseline approaches for different robot planning problems with varying numbers of objective functions.
paper image

Statistically Distinct Plans for Multi-Objective Task Assignment
Nils Wilde, Javier Alonso-Mora. In IEEE Transaction on Robotics(T-RO), 2024.

pdf
We study the problem of finding statistically distinct plans for stochastic task assignment problems such as online multi-robot pickup and delivery (MRPD) when facing multiple competing objectives. In many real-world settings robot fleets do not only need to fulfil delivery requests, but also have to consider auxiliary objectives such as energy efficiency or avoiding human-centered work spaces. We pose MRPD as a multi-objective optimization problem where the goal is to find MRPD policies that yield different trade-offs between given objectives. There are two main challenges: 1) MRPD is computationally hard, which limits the number of trade-offs that can reasonably be computed, and 2) due to the random task arrivals, one needs to consider statistical variance of the objective values in addition to the average. We present an adaptive sampling algorithm that finds a set of policies which i) are approximately optimal, ii) approximate the set of all optimal solutions, and iii) are statistically distinguishable. We prove completeness and adapt a state-of-the-art MRPD solver to the multi-objective setting for three example objectives. In a series of simulation experiments we demonstrate the advantages of the proposed method compared to baseline approaches and show its robustness in a sensitivity analysis. The approach is general and could be adapted to other multi-objective task assignment and planning problems under uncertainty.
paper image

Scalarizing Multi-Objective Robot Planning Problems Using Weighted Maximization
Nils Wilde, Stephen L. Smith, Javier Alonso-Mora. In IEEE Robotics and Automation Letters (RA-L), 2024.

pdf
When designing a motion planner for autonomous robots there are usually multiple objectives to be considered. However, a cost function that yields the desired trade-off between objectives is not easily obtainable. A common technique across many applications is to use a weighted sum of relevant objective functions and then carefully adapt the weights. However, this approach may not find all relevant trade-offs even in simple planning problems. Thus, we study an alternative method based on a weighted maximum of objectives. Such a cost function is more expressive than the weighted sum, and we show how it can be deployed in both continuous- and discrete-space motion planning problems. We propose a novel path planning algorithm for the proposed cost function and establish its correctness, and present heuristic adaptations that yield a practical runtime. In extensive simulation experiments, we demonstrate that the proposed cost function and algorithm are able to find a wider range of trade-offs between objectives (i.e., Pareto-optimal solutions) for various planning problems, showcasing its advantages in practice.
paper image

Optimizing Task Waiting Times in Dynamic Vehicle Routing
A. Botros, B. Gilhuly, N. Wilde, A. Sadeghi, J. Alonso-Mora, S. L Smith. In IEEE Robotics and Automation Letters (RA-L), 2023.

pdf
We study the problem of deploying a fleet of mobile robots to service tasks that arrive stochastically over time and at random locations in an environment. This is known as the Dynamic Vehicle Routing Problem (DVRP) and requires robots to allocate incoming tasks among themselves and find an optimal sequence for each robot. State-of-the-art approaches only consider average wait times and focus on high-load scenarios where the arrival rate of tasks approaches the limit of what can be handled by the robots while keeping the queue of unserviced tasks bounded, i.e., stable. To ensure stability, these approaches repeatedly compute minimum distance tours over a set of newly arrived tasks. This letter is aimed at addressing the missing policies for moderate-load scenarios, where quality of service can be improved by prioritizing long-waiting tasks. We introduce a novel DVRP policy based on a cost function that takes the p-norm over accumulated wait times and show it guarantees stability even in high-load scenarios. We demonstrate that the proposed policy outperforms the state-of-the-art in both mean and 95th percentile wait times in moderate-load scenarios through simulation experiments in the Euclidean plane as well as using real-world data for city scale service requests.
paper image

Designing Heterogeneous Robot Fleets for Task Allocation and Sequencing
N. Wilde, J. Alonso-Mora. In Proc. IEEE International Symposium on Multi-Robot and Multi-Agent Systems (MRS), 2023.

pdf
We study the problem of selecting a fleet of robots to service spatially distributed tasks with diverse requirements within time-windows. The problem of allocating tasks to a fleet of potentially heterogeneous robots and finding an optimal sequence for each robot is known as multi-robot task assignment (MRTA). Most state-of-the-art methods focus on the problem when the fleet of robots is fixed. In contrast, we consider that we are given a set of available robot types and requested tasks, and need to assemble a fleet that optimally services the tasks while the cost of the fleet remains under a budget limit. We characterize the complexity of the problem and provide a Mixed-Integer Linear Program (MILP) formulation. Due to poor scalability of the MILP, we propose a heuristic solution based on a Large Neighbourhood Search (LNS). In simulations, we demonstrate that the proposed method requires substantially lower budgets than a greedy algorithm to service all tasks.

Multi-Robot Local Motion Planning Using Dynamic Optimization Fabrics
Saray Bakker, Luzia Knoedler, Max Spahn, Wendelin Boehmer, Javier Alonso-Mora. In Proc. IEEE International Symposium on Multi-Robot and Multi-Agent Systems, 2023.

pdf code video
In this paper, we address the problem of real-time motion planning for multiple robotic manipulators that operate in close proximity. We build upon the concept of dynamic fabrics and extend them to multi-robot systems, referred to as Multi-Robot Dynamic Fabrics (MRDF). This geometric method enables a very high planning frequency for high-dimensional systems at the expense of being reactive and prone to deadlocks. To detect and resolve deadlocks, we propose Rollout Fabrics where MRDF are forward simulated in a decentralized manner. We validate the methods in simulated close-proximity pick-and-place scenarios with multiple manipulators, showing high success rates and real-time performance.

Group-based Distributed Auction Algorithms for Multi-Robot Task Assignment
Xiaoshan Bai, Andres Fielbaum, Maximilian Kronmuller, Luzia Knoedler, Javier Alonso-Mora. In IEEE Transactions on Automation Science and Engineering (T-ASE), 2023.

publisher pdf
This paper studies the multi-robot task assignment problem in which a fleet of dispersed robots needs to efficiently transport a set of dynamically appearing packages from their initial locations to corresponding destinations within prescribed time-windows. Each robot can carry multiple packages simultaneously within its capacity. Given a sufficiently large robot fleet, the objective is to minimize the robots' total travel time to transport the packages within their respective time-window constraints. The problem is shown to be NP-hard, and we design two group-based distributed auction algorithms to solve this task assignment problem. Guided by the auction algorithms, robots first distributively calculate feasible package groups that they can serve, and then communicate to find an assignment of package groups. We quantify the potential of the algorithms with respect to the number of employed robots and the capacity of the robots by considering the robots' total travel time to transport all packages. Simulation results show that the designed algorithms are competitive compared with an exact centralized Integer Linear Program representation solved with the commercial solver Gurobi, and superior to popular greedy algorithms and a heuristic distributed task allocation method.

Improving Pedestrian Prediction Models with Self-Supervised Continual Learning
Luzia Knoedler, Chadi Salmi, Hai Zhu, Bruno Brito, Javier Alonso-Mora. In , IEEE Robotics and Automation Letters (RA-L), 2022.

publisher pdf code video
Autonomous mobile robots require accurate human motion predictions to safely and efficiently navigate among pedestrians, whose behavior may adapt to environmental changes. This letter introduces a self-supervised continual learning framework to improve data-driven pedestrian prediction models online across various scenarios continuously. In particular, we exploit online streams of pedestrian data, commonly available from the robot’s detection and tracking pipeline, to refine the prediction model and its performance in unseen scenarios. To avoid the forgetting of previously learned concepts, a problem known as catastrophic forgetting, our framework includes a regularization loss to penalize changes of model parameters that are important for previous scenarios and retrains on a set of previous examples to retain past knowledge. Experimental results on real and simulation data show that our approach can improve prediction performance in unseen scenarios while retaining knowledge from seen scenarios when compared to naively training the prediction model online.
paper image

Do we use the Right Measure? Challenges in Evaluating Reward Learning Algorithms
N. Wilde, J. Alonso-Mora. In Conference on Robot Learning (CoRL), 2022.

pdf
Reward learning is a highly active area of research in human-robot interaction (HRI), allowing a broad range of users to specify complex robot be- haviour. Experiments with simulated user input play a major role in the devel- opment and evaluation of reward learning algorithms due to the availability of a ground truth. In this paper, we review measures for evaluating reward learning algorithms used in HRI, most of which fall into two classes. In a theoretical worst case analysis and several examples, we show that both classes of measures can fail to effectively indicate how good the learned robot behaviour is. Thus, our work contributes to the characterization of sim-to-real gaps of reward learning in HRI.

Online Multi-Robot Task Assignment with Stochastic Blockages
N. Wilde, J. Alonso-Mora. In IEEE Conference on Decision and Control (CDC), 2022.

pdf
In this paper we study the multi-robot task assignment problem with tasks that appear online and need to be serviced within a fixed time window in an uncertain environment. For example, when deployed in dynamic, human centered environments, the team of robots may not have perfect information about the environment. Parts of the environment may temporarily become blocked and blockages may only be observed on location. While numerous variants of the Canadian Traveler Problem describe the path planning aspect of this problem, few work has been done on multi-robot task allocation (MRTA) under this type of uncertainty. In this paper, we introduce and theoretically analyze the problem of MRTA with recoverable online blockages. Based on a stochastic blockage model, we compute offline tours using the expected travel costs for the online routing problem. The cost of the offline tours is used in a greedy task assignment algorithm. In simulation experiments we highlight the performance benefits of the proposed method under various settings.

Error-Bounded Approximation of Pareto Fronts in Robot Planning Problems
A. Botros, A. Sadeghi, N. Wilde, J. Alonso-Mora, S. L. Smith. In 15th Workshop on the Algorithmic Foundations of Robotics (WAFR), 2022.

arxiv pdf
Many problems in robotics seek to simultaneously optimize several competing objectives under constraints. A conventional approach to solving such multi-objective optimization problems is to create a single cost function comprised of the weighted sum of the individual objectives. Solutions to this scalarized optimization problem are Pareto optimal solutions to the original multi-objective problem. However, finding an accurate representation of a Pareto front remains an important challenge. Using uniformly spaced weight vectors is often inefficient and does not provide error bounds. Thus, we address the problem of computing a finite set of weight vectors such that for any other weight vector, there exists an element in the set whose error compared to optimal is minimized. To this end, we prove fundamental properties of the optimal cost as a function of the weight vector, including its continuity and concavity. Using these, we propose an algorithm that greedily adds the weight vector least-represented by the current set, and provide bounds on the error. Finally, we illustrate that the proposed approach significantly outperforms uniformly distributed weights for different robot planning problems with varying numbers of objective functions.
paper image

Learning a Guidance Policy from Humans for Social Navigation
Luzia Knoedler, Bruno Brito, Michael Everett, Jonathan P. How, Javier Alonso-Mora. In Social Robot Navigation: Advances and Evaluation at IEEE International Conference on Robotics and Automation (ICRA), 2022.

publisher pdf
Autonomous mobile robots navigating among humans must not only consider safety and efficiency but also move acceptably in the current social context. A hybrid deep reinforcement learning - model predictive control (DRL-MPC) approach can account for the complex interactions among humans while maintaining the collision avoidance guarantees and feasibility constraints inherent in the MPC formulation. However, encoding socially acceptable behavior through a reward or cost function, along with other objectives such as reaching the goal quickly, is challenging. Therefore, this work proposes a new training strategy that combines supervised and reinforcement learning to exploit human demonstration. Furthermore, it presents first results from real-world experiments.

Integrated Task Assignment and Path Planning for Capacitated Multi-Agent Pickup and Delivery
Z. Chen, J. Alonso-Mora, X. Bai, D. D. Harabor, P. J. Stuckey. In , IEEE Robotics and Automation Letters (RA-L), 2021.

publisher pdf code
Multi-agent Pickup and Delivery (MAPD) is a challenging industrial problem where a team of robots is tasked with transporting a set of tasks, each from an initial location and each to a specified target location. Appearing in the context of automated warehouse logistics and automated mail sortation, MAPD requires first deciding which robot is assigned what task (i.e., Task Assignment or TA) followed by a subsequent coordination problem where each robot must be assigned collision-free paths so as to successfully complete its assignment (i.e., Multi-Agent Path Finding or MAPF). Leading methods in this area solve MAPD sequentially: first assigning tasks, then assigning paths. In this work we propose a new coupled method where task assignment choices are informed by actual delivery costs instead of by lower-bound estimates. The main ingredients of our approach are a marginal-cost assignment heuristic and a meta-heuristic improvement strategy based on Large Neighbourhood Search. As a further contribution, we also consider a variant of the MAPD problem where each robot can carry multiple tasks instead of just one. Numerical simulations show that our approach yields efficient and timely solutions and we report significant improvement compared with other recent methods from the literature.

Where to go next: Learning a Subgoal Recommendation Policy for Navigation in Dynamic Environments
B. Brito, M. Everett, J. P. How, J. Alonso-Mora. In , IEEE Robotics and Automation Letters (RA-L), 2021.

publisher pdf video
Robotic navigation in environments shared with other robots or humans remains challenging because the intentions of the surrounding agents are not directly observable and the environment conditions are continuously changing. Local trajectory optimization methods, such as model predictive control (MPC), can deal with those changes but require global guidance, which is not trivial to obtain in crowded scenarios. This letter proposes to learn, via deep Reinforcement Learning (RL), an interaction-aware policy that provides long-term guidance to the local planner. In particular, in simulations with cooperative and non-cooperative agents, we train a deep network to recommend a subgoal for the MPC planner. The recommended subgoal is expected to help the robot in making progress towards its goal and accounts for the expected interaction with other agents. Based on the recommended subgoal, the MPC planner then optimizes the inputs for the robot satisfying its kinodynamic and collision avoidance constraints. Our approach is shown to substantially improve the navigation performance in terms of number of collisions as compared to prior MPC frameworks, and in terms of both travel time and number of collisions compared to deep RL methods in cooperative, competitive and mixed multiagent scenarios.