RF-based micro-motion tracking for gesture tracking and recognition
Inventors
Lien, Jaime • Olson, Erik M. • Amihood, Patrick M. • Poupyrev, Ivan
Assignees
Google LLC • Leland Stanford Junior University
Publication Number
US-12340028-B2
Publication Date
2025-06-24
Expiration Date
2036-04-29
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
This document describes techniques for radio frequency (RF) based micro-motion tracking. These techniques enable even millimeter-scale hand motions to be tracked. To do so, radar signals are used from radar systems that, with conventional techniques, would only permit resolutions of a centimeter or more.
Core Innovation
This invention describes techniques for radio frequency (RF) based micro-motion tracking that enable tracking of millimeter-scale hand motions using radar signals from radar systems that conventionally permit only centimeter or coarser resolution. It overcomes hardware-imposed limits of radar spatial resolution, which depend on antenna beam width and bandwidth, to extract finer displacement information than traditionally achievable. The technique enables finer resolution tracking even using a relatively simple radar system with a single radar-emitting element and single antenna, which is simpler, less costly, and less complex than conventional multi-element radar systems.
The problem addressed is that small-screen devices like smartphones and wearables suffer from difficult and inaccurate virtual keyboard inputs, frustrating users and limiting usability. Existing optical finger- and hand-tracking techniques are often large, costly, or inaccurate, while conventional radar systems struggle to detect small gesture motions without expensive or complex hardware due to resolution limits inherent to radar hardware. Thus, prior radar tracking technologies are unable to detect micro-motions smaller than their hardware's resolution.
The invention addresses these limits by extracting relative dynamics, such as relative velocities and displacements, from radar signals that represent a superposition of reflections from multiple points of a hand within the radar field. By focusing on relative motions between hand points, such as fingertips and knuckles, and using Doppler-based measurements and phase information from the radar signal, the techniques enable computationally light, super-resolution velocity estimation, allowing recognition of fine gestures with millimeter or sub-millimeter scale resolution. This allows use of simpler hardware and enables real-time, precise gesture recognition for controlling applications or devices.
Claims Coverage
The patent claims cover a computer-implemented method and an apparatus for RF-based micro-motion tracking and gesture recognition with multiple inventive features.
Determining relative velocities from Doppler-based velocity profile of multiple points of a non-rigid target
Receiving radar reflections of a radar field off first and second points of a single non-rigid target moving at distinct Doppler velocities, generating Doppler-based measurements representing a velocity profile with higher energy measurements corresponding to those points, and determining relative velocities between the points.
Gesture determination based on changes in relative velocities and displacement
Determining gestures performed by the non-rigid target based on relative velocities at multiple times and integrating these to find changes in displacement, enabling fine resolution gesture recognition finer than the radar wavelength.
Using a moving target indicator filter and range-Doppler-time data cube for signal processing
Identifying target points within radar reflections using a moving target indicator filter and optionally creating a range-Doppler-time data cube to enhance gesture determination robustness and accuracy.
Gesture transmission to applications or devices for control
Passing the recognized gesture to an application or device effective to control or alter displays, functions, or capabilities associated with the receiving application or device.
Low computational complexity and high accuracy with simple radar systems
Determining relative velocities without needing absolute velocities, operating with a broad beam, fully contiguous radar field from a simple radar system, and achieving gesture recognition with high accuracy and low noise compared to other Doppler frequency methods.
The claims collectively cover the novel process of extracting relative dynamic velocity profiles from radar reflections of multiple points on a non-rigid target, such as a hand, and recognizing high-resolution micro-motions and gestures based on these profiles, using computationally efficient methods and simpler radar hardware configurations.
Stated Advantages
Enables tracking of millimeter or sub-millimeter scale hand motions using radar signals from simpler, less costly, and less complex radar systems.
Requires relatively low computational resources due to super-resolution velocity estimation using micro-Doppler centroids and phase unwrapping, suitable for real-time recognition on small or resource-limited devices.
Provides gesture recognition with resolution finer than the radar wavelength or antenna beam width, overcoming traditional hardware-imposed resolution limits.
Offers greater robustness to noise and clutter compared to using Doppler profile peaks.
Allows fine, real-time control of applications or devices through recognition of micro-gestures, improving usability of small-screen and wearable computing devices.
Documented Applications
Gesture recognition for user input on small-screen computing devices including smartphones, wearable computing watches, rings, and bracelets.
Controlling applications or devices such as smart watches to scroll displayed text or control media playback through fine micro-motion gestures.
Tracking fine motions of robotic arms or devices by detecting relative displacements between points on them.
Use in devices including desktop computers, tablets, microwaves, home automation and control systems, entertainment systems, audio systems, security systems, automobiles, and e-readers.
Interested in licensing this patent?