Wearable Device for Inferring Hand Pose and Gesture in Real Time



Much of human activity requires and revolves around the use of hands; not only are hands used to interact with the physical environment, they are also vital for human communication. Infants communicate with their hands well before using speech; for example, beginning around the age of 10 months they will start to point to objects and can be taught to use signing or symbolic gestures well before talking. At 18 months infants will use gestures at the same time as speech; for example, naming an object while pointing at it. In later years and into adulthood, these gestures most often integrated at a subconscious level but contain a wealth of information. Hand gesture dysfunction is also of interest in medicine, for example, in the detection and monitoring of syndromes such as Parkinson's and essential tremor. Hence, there is benefit in exploring how electronic devices can detect, interpret, and learn from hand motions.


Invention Description

Researchers at Arizona State University have developed a lightweight unobtrusive wearable device which continually monitors the instantaneous hand pose: the position of the wrist relative to one's body and the configuration of the hand. The device infers hand pose in real time and, as such, can be combined with actuators or displays to provide instantaneous feedback to the user. The device is worn on the wrist and all processing can be performed within the device, thus ensuring privacy. Such a device could be used for medical diagnostic purposes, real-time monitoring and guidance in rehabilitation, for training in sports and musical instrument practice, to name but a few applications. Furthermore, the device is designed to operate in real time, allowing perhaps a physiotherapist to monitor and guide patients during in-home exercise rehabilitation.




Potential Applications

•       Personalized activity monitoring (see related technology), including as a memory aid for individuals suffering from long-term neurological effects of COVID-19

•       Medical diagnostics and patient monitoring

•       Rehabilitation and physical therapy (e.g., following hand surgery or injury)

•       Gesture control of connected devices (e.g., household robots, residential lighting)

•       Training for sports and manual skills

•       Musical instrument practice



Research Homepage of Professor Troy McDaniel