EgoTouch: On-Body Touch Input Using AR/VR Headset Cameras

Description:

2024-238 EgoTouch: Unlock Bare-Handed Precision Touch for AR/VR Headset Cameras

Benefit

EgoTouch revolutionizes AR/VR interaction by instantly transforming a user’s own hands and arms into intuitive touch surfaces using only the RGB camera built into modern headsets, with no special hardware or calibration needed. This technology offers tactile, comfortable, and highly accurate control that feels natural, unlocking the speed, precision, and haptic feedback familiar from real-world touchscreens, but tailored for immersive reality experiences.

Unlike in-air gesture interfaces, which often lack tactile feedback and are prone to errors, EgoTouch provides reliable, multitouch input even while moving, across all lighting conditions and skin tones. It detects detailed touch events (contact, lift, drag, pressure, and gesture type) with over 95% accuracy, delivering rich input data that supports scrolling, selecting, swiping, and more—directly mapped to users’ skin. This enables seamless, controller-free interaction that’s ergonomic and calibration-free, making everyday AR/VR tasks faster and more comfortable.

By finally making practical, on-skin interfaces achievable and robust in real-world scenarios, EgoTouch paves the way for more natural, intuitive, and accessible XR experiences for everyone.

Market Application

Gaming and Entertainment: Enables immersive, controller-free interactions in VR and AR games, with direct, ergonomic input on hands and arms.
Education and Training: Supports hands-on virtual simulations and instructional interfaces for STEM, medical, industrial, and safety training.
Retail, Shopping, and Customer Experience: Next-gen virtual shopping and product exploration with intuitive touch controls