Biopotential-based gesture interpretation with machine labeling
Inventors
Ang, Dexter W. • Cipoletta, David O.
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12216822-B2
Publication Date
2025-02-04
Expiration Date
Abstract
System and methods for gesture-based control are described. In some embodiments, a system may include a wearable device having a biopotential sensor and a wrist motion sensor. The biopotential sensor may be configured to output a first data stream indicating actions of a person's hand. The system may further include a second device configured to output a second data stream, which may also indicate the actions of the person's hand. The system may be configured to analyze the first and second data streams to train a machine learning interpreter to classify actions of a person's hand based on at least biopotential data.
Core Innovation
The invention relates to gesture-based control using a wearable device configured to be worn at a wrist of a person. The wearable device includes a biopotential sensor configured to detect biopotentials indicating a state of the hand of the person, and a wrist motion sensor configured to obtain wrist motion data indicating a motion of the wrist. The biopotential sensor outputs a first data stream indicating actions of the hand of the person, and the system uses these biopotentials in conjunction with additional data for machine interpretation.
The system includes a second device that outputs a second data stream also indicating actions of the hand of the person. The system stores first event data comprising a plurality of biopotential data points from the first data stream and biopotential timestamps, and stores second event data comprising one or more data points from the second data stream and one or more second device timestamps.
The invention further provides a training process that determines, based on an analysis of the second event data, that the hand performed a first action, and determines a first action time based on the one or more second device timestamps. Based on the first action time and the biopotential timestamps, the system associates a label with a subset of the first event data indicating that the first action occurred while the first event data was collected. Using at least the label and the first event data, the system trains a machine learning interpreter to generate interpreted outputs based on at least biopotential data.
Claims Coverage
Two independent claims are directed to a gesture-based control system using a wrist-worn biopotential sensor and a wrist motion sensor, with a second device providing a temporally referenced second data stream used to label and train a machine learning interpreter from biopotential data. The claims include four inventive features.
Wrist-worn biopotential gesture detection with wrist motion sensing
A wearable device configured to be worn at a wrist of a person, including a biopotential sensor configured to detect biopotentials indicating a state of the hand of the person and a wrist motion sensor configured to obtain wrist motion data indicating a motion of the wrist, where the biopotential sensor outputs a first data stream configured to indicate actions of the hand.
Dual-data-stream action indication and timestamped event storage
A second device configured to output a second data stream also configured to indicate the actions of the hand of the person, wherein the system stores first event data comprising biopotential data points and biopotential timestamps, and stores second event data comprising one or more data points from the second data stream and one or more second device timestamps.
Action-time determination from the second data stream and labeled biopotential subset selection
The system determines, based on an analysis of the second event data, that the hand performed a first action; determines a first action time at which the first action occurred based on the one or more second device timestamps; and based on the first action time and the biopotential timestamps, associates a label with a subset of the first event data indicating that the first action occurred while the first event data was collected.
Training a machine learning interpreter using labeled biopotential event data
Using at least the label and the first event data, train a machine learning interpreter to generate interpreted outputs based on at least biopotential data; or, in the alternative independent claim, a machine learning interpreter is trained by a training process using at least the label and the first event data.
Across both independent claims, the inventive coverage centers on a wrist-worn wearable that detects hand state via a biopotential sensor with wrist motion sensing, a second device providing a second action-indicating data stream with timestamps, and a training process that determines action time from the second data stream, labels a temporally associated subset of biopotential event data, and trains a machine learning interpreter based on the labeled biopotential data.
Stated Advantages
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?