Determining a geographical location based on human gestures
Inventors
Cipoletta, David O. • Ang, Dexter W.
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11157086-B2
Publication Date
2021-10-26
Expiration Date
Abstract
An apparatus includes a sensor constructed to be worn by a person and configured to sense information indicative of a direction intended by the person. The apparatus includes a processor, and storage for instructions executable by the processor. When executing the instructions, the processor determines the direction based on the information sensed by the sensor, and, based on the direction and a location of the person, determines a geographic location. The sensor can include a biopotential signal from which an intended gesture by the person can be determined.
Core Innovation
The invention relates to a wearable human-machine interface using a biopotential sensor worn at a wrist and a wrist location sensor worn at the wrist. The biopotential sensor senses biopotentials at the wrist, and the wrist location sensor provides wrist location data indicative of a location of the wrist. A processor uses these signals together to interpret an intended selection direction and an intention to make a selection.
Based on at least the wrist location data, the system determines a directional vector intended by the person. Based on at least the biopotentials sensed by the biopotential sensor, the system detects a gesture indicating an intention to make a selection. When the gesture indicating the intention to make a selection is detected, the system selects a target indicated by the directional vector as determined at the time of gesture detection.
The disclosure further describes a user interface device and a responsive device with a signal processing pipeline that samples and conditions sensor signals, filters, extracts features, and detects gestures using a machine learning model. The described gesture set includes primitive gestures such as lift, hold, and finger swipe, as well as wake word gestures using wrist motion, and a motion silencing concept that ignores biopotentials during substantial wrist motion while still using wrist location data for determining intent direction.
The disclosure further describes determining a geographic location and/or selecting an object at a location using directional vectors and computed geographic information. In some implementations, location can be derived using triangulation or intersection of pointing vectors from one or multiple users, and confidence or validation is used in connection with the determined location. Context data such as voice/audio, text, image data, video data, map inputs, and optionally depth data may be incorporated to support selection, including selection in 3D surface intersection, vector and location computations, and heads-up display or AR-related presentations.
Claims Coverage
Two independent system claims are provided, both centered on combining a wrist-worn biopotential sensor with wrist location data to determine an intended directional vector, detect an intention-to-select gesture, and select a target based on the directional vector at the time the selection gesture is detected.
Directional vector from wrist location data with intention-based gesture selection
A system comprising a biopotential sensor worn at a wrist and configured to sense biopotentials, a wrist location sensor worn at the wrist and configured to provide wrist location data indicative of a location of the wrist, and a processor; the system determines a directional vector intended by the person based on at least the wrist location data; detects a gesture indicating an intention to make a selection based on at least the biopotentials; and selects a target indicated by the directional vector as determined when the gesture indicating the intention to make a selection is detected.
Processor output based on biopotentials and wrist location for target selection
A system comprising a biopotential sensor worn at a wrist and configured to sense biopotentials and a wrist location sensor worn at the wrist and configured to provide wrist location data indicative of a location of the wrist; the system provides output data based on the biopotentials sensed and the wrist location data sensed to one or more processors that determine a directional vector intended by the person based on at least the wrist location data, detect a gesture indicating an intention to make a selection based on at least the biopotentials, and select a target indicated by the directional vector as determined when the gesture indicating the intention to make a selection is detected.
Both independent claims share the same core coverage: determining a directional vector intended by the person from wrist location data, detecting an intention-to-select gesture from wrist biopotentials, and selecting a target indicated by the directional vector at the time the selection gesture is detected.
Stated Advantages
Documented Applications
Selecting a target indicated by an intended directional vector derived from wrist location data when a gesture indicating intention to make a selection is detected, including geographic location selection and object selection.
Using selected targets for controlling an object by determining one or more commands from wrist location data and biopotentials and transmitting the commands to the object.
Initiating a control mode using a wake word gesture based on wrist location data and biopotentials.
Providing geographic location computation and map-related feedback based on directional vectors and triangulation/intersection from one or multiple users, with confidence/validation of the determined location.
Selecting objects at a computed location for presentation in 3D surface intersection, vector and location computations, and heads-up display or AR-related views.
Controlling another machine, including a drone remote control, based on gestures and derived locations and directions.
Interested in licensing this patent?