Collision-free Trajectory Planning in Human-robot Interaction through Hand Movement Prediction from Vision


Robots are increasingly becoming present in factories and households and collaborating with people has become progressively complicated. Modern robots are expected to handle difficult tasks and also need to maximize their chance of success while human users are collaborating with them. With the human in the loop of tasks, the robot controller has to take human motion into consideration when planning the best path to avoid collision, ensure safety, and complete tasks efficiently. As a solution, manufacturers use motion systems that track human hand motions with markers attached to the user, yet markers are extremely unreliable as they can easily be blocked by any object causing a failure in the system. The next generation of manufacturing is composed of human-robot interaction and companies are increasingly looking to make collaboration more efficient and safe.


Researchers at Arizona State University have developed an innovative system that allows collision-free human and robot interactions. To predict collisions a camera captures an image and extracts features recognizing if a human hand is present. Using recurrent neural networks, a system learns spatial-temporal relationships between a user’s hand manipulations and its next several steps of movements. By teaching the system hand movements and manipulations the system is able to predict which steps the human user makes allowing the robot to avoid collision. Furthermore, to assist a smoother and safer trajectory the system predicts five steps into the future in real time. As a safety measure the robot analyzes if abrupt or odd movements are made to quickly react and adapt a safer route.


Potential Applications

  • Automation Robots
  • Medical Robots
  • Household Robots
  • Computer Vision AI


Benefits and Advantages

  • Innovative - System allows for hand motion tracking without the need for markers
  • Fast - Able to predict motions five steps into the future in real-time
  • Safe - Allows for safe human-robot collaboration to increase productivity
  • Intelligent - System is able to quickly react and respond to any anomalies


For more information about the inventor(s) and their research, please see

Yezhou Yang's Research Page

Wenlong Zhang's Directory Page

Download Original PDF

Case ID:
Last Updated:

For More Information, Contact