Explaining and securing robot planning algorithms using adversarial machine learning

Motion planning algorithms are crucial components of real-world robots: whether in manufacturing, hospital and service robots, inspection, warehouse or other applications. These algorithms compute sequences of movements that make robots meet their goals, such as arriving at their destinations or reaching and grasping objects. Two current problems with motion planning algorithms are the following: 1) They are brittle, and may thus fail or lead to unexpected results upon small changes to the environment (floor unevenness, floor angle, initial posture, object geometry…). 2) They are opaque: meaning that users have trouble anticipating when the algorithms will fail, and anticipating how they will behave. Therefore users will have to fiddle with the algorithms and situations many times until they start getting a good idea about how the robot/planner will behave.

The innovative idea of this thesis is to address both of these problems using adversarial machine learning. The plan is to proceed in the following three stages, though there is flexibility for the student to focus on the specific parts that they are most excited by.

First, the student will develop and use adversarial machine learning algorithms in order to learn to how “break” motion planning algorithms in a variety of ways. This will provide an important contribution at the intersection of adversarial machine learning and robotics, which has yet to be explored. The learned models will show us how ill-intentioned stakeholders can create security “attacks” that lead a planner to fail or take a route that is of interest to the attacker—and they will provide hints for how we can improve current planning methods to be more robust.

Second, the student will develop an algorithm to cluster this model into “examplars” of typical failure modes – which will then be used to build summaries or “datasheets” of planner failure that users will read in order to understand the capabilities and limits of the robot. User studies will be conducted in order to evaluate the effectiveness of such datasheets and to iterate their design. Through this part of the project the thesis will thus also contribute to the area of Explainable AI and Human-Robot Interaction.

Third, the student will use the learned models to develop robust motion planning algorithms that reduce the failure and security issues previously identified. The degree to which the new models improve performance and user trust will then be evaluated.

The thesis will focus on optimal and asymptotically optimal motion planning algorithms, such as A* and RRT*, running on real-world articulated robot collision models (e.g. the HSR office robot and biped humanoid robots) and a dataset of realistic environments (e.g. 3D office scenarios, and disaster response scenarios such as https://github.com/martimbrandao/destruction_scenarios). There is the potential to apply the same methodology to AI Planning problems, in which case the dataset of problems will be collected from the ICAPS International Planning Competitions.

This thesis will be appropriately supervised by two advisors with complementing expertise in robotics, explainability, and adversarial machine learning.It will provide an important contribution to a diverse set of fields – and the student should thus expect a very broad and rewarding experience.

  • Brandao, M., Coles, A., & Magazzeni, D. (2021). Explaining Path Plan Optimality: Fast Explanation Methods for Navigation Meshes Using Full and Incremental Inverse Optimization. In Proceedings of the International Conference on Automated Planning and Scheduling (Vol. 31, pp. 56-64). 
  • Du, Y., Fang, M., Yi, J., Cheng, J., & Tao, D. (2018). Towards query efficient black-box attacks: An input-free perspective. In Proceedings of the 11th ACM Workshop on Artificial Intelligence and Security (pp. 13-24). 

Project ID



Martim Brandaohttps://www.martimbrandao.com/

Yali Du


AI Planning