Researchers Improve Robot Gesture Recognition With Computer Vision

Researchers from Nanyang Technological University have developed an artificial intelligence system able to recognize gestures more accurately, through the use of wearable strain sensors.

The strain sensor far exceeds previous wearable devices, such as fitness wristwatches or gloves, by being much thinner and flexible. NTU scientists are able to receive data on skin senses and vision, which they then input into a neural network for processing.

SEE ALSO: Researchers Augment Robotic Prosthetics With AI and Computer Vision

“Our data fusion architecture has its own unique bioinspired features which include a man-made system resembling the somatosensory-visual fusion hierarchy in the brain. We believe such features make our architecture unique to existing approaches,” said NTU professor Chen Xiaodong.

“Compared to rigid wearable sensors, our innovation uses stretchable strain sensors that comfortably attaches onto the human skin. This allows for high-quality signal acquisition, which is vital to high-precision recognition tasks.”

Amazon and Tesla both suffered from setbacks in deploying robots to perform human tasks. In Tesla’s case, it found that robots were unable to find the right screw in a box and struggled to lift certain objects.

With better computer vision and precision, those issues may be fixable.

NTU’s AI system was highly accurate even in poor conditions, achieving a 96.7 percent recognition accuracy rating in the dark. The team intends to build a VR and AR system using the AI, which will be used as a show-piece for prospective clients.

“The secret behind the high accuracy in our architecture lies in the fact that the visual and somatosensory information can interact and complement each other at an early stage before carrying out complex interpretation. As a result, the system can rationally collect coherent information with less redundant data and less perceptual ambiguity, resulting in better accuracy,” said Dr. Wang Ming of NTU.