Gesture recognition translates movement into digital inputs, enabling novel —Ìýand sometimes more convenient —Ìýways to interact with machines.
Gesture recognition capabilities have advanced significantly in recent years. From BMW’s ‘in-air’ systems that use precise driver gestures to control navigation and audio systems, to video games to touchless computing terminals used by surgeons, gesture recognition has quickly evolved from novelty, to a convenient and sometimes transformational engagement alternative.
A computer input system that recognizes and interprets physical human gestures.
It can provide engaging, convenient and inclusive experiences for customers and employees alike.
The technology is still immature and may not be sufficiently accurate to deploy in customer-facing systems.
Early applications were in consumer electronics but it’s starting to be used in the automotive industry and in medicine.
What is it?
Gesture recognition refers to technology that uses cameras and haptic devices to capture, recognize, and interpret physical human gestures. Typically, these are simple movements — things like up and down arm motions, the wave of a hand, or the swiping of an arm. However, modern gesture recognition tools can recognize smaller and more nuanced movements, such as pinches, smaller swipes, and individual finger movements.

Gesture recognition can enable your customers to interact with your systems in circumstances where keyboards, voice control or other inputs aren’t convenient or practical. It can enable you to differentiate your offerings by providing compelling customer services.
What’s in for you?
Gesture recognition is enabling novel forms of human-machine interaction, where the users’ context informs this new approach. For instance, the pinch-to-zoom feature that set early iPhones apart from the other smartphones.Ìý
Gesture recognition is a powerful tool for helping customers with unique needs access your services, enabling things like human-to-machine communication through sign language — a truly transformational experience for customers with those needs.Ìý
Following the COVID-19 pandemic, customer appreciation for touch-free digital experiences is likely to be higher than ever. Now, interacting through movement isn’t just a novel alternative, there’s a compelling imperative to offer safer touch-free experiences.
What are the trade-offs?
Ìý
Many of the tools and technologies available today still lack finesse and don’t offer the precision required to enable consistently compelling customer experiences.

As the technology is relatively new, there is a lag between a gesture, recognition and the computer’s reaction to the gesture. As previously mentioned, accuracy and resolution can be a challenge as well, unless multiple gesture technologies are combined together (cameras and haptic devices).
Ìý
Such systems can struggle in poor lightning, busy environments and even different weather conditions. Deploying an overly inaccurate system can set back a product in a market and delay adoption.
How is it being used?
Until now, gesture recognition has remained a novel technology. Most of its applications have been in entertainment — powering things like motion-controlled gaming experiences — or in experimental retail settings, giving customers the chance to swipe through products and interact with terminals without touch.
However, the technology has found many compelling use cases in areas where humans simply cannot physically interact with a machine. Surgeons for example use gesture recognition to interact with screens and systems as they perform demanding procedures—a safe and sanitary alternative to touching a screen.
It’s also making technology a lot more accessible and inclusive for those with physical disabilities. For people that can’t easily physically engage with screens or other input methods, gesture recognition levels the technical playing field, enabling them to perform extremely complex digital tasks.
Would you like to suggest a topic to be decoded?
Just leave your email address and we'll be in touch the moment it's ready.