Monitoring of biometric data to determine mental states and input commands
Inventors
Alcaide, Ramses Eduardo • Stanley, David • Padden, Dereck • HAMET, James • Molnar, Adam • Alders, Jamie • Candassamy, Jegan • Srinivas, Arjun Daniel
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11609633-B2
Publication Date
2023-03-21
Expiration Date
Abstract
Various embodiments of an apparatus, methods, systems and computer program products described herein are directed to an Analytics Engine that receives one more signal files that include neural signal data of a user based on voltages detected by one or more electrodes on a set of headphones worn by a user. The Analytics Engine preprocesses the data, extracts features from the received data, and feeds the extracted features into one or more machine learning models to generate determined output that corresponds to at least one of a current mental state of the user and a type of facial gesture performed by the user. The Analytics Engine sends the determined output to a computing device to perform an action based on the determined output.
Core Innovation
The Analytics Engine passively monitors neural, muscle, and cardiac signals from electrodes integrated in user-worn headphones to infer a user’s current mental state and/or detect facial and head gestures. Extracted features from EEG and EMG are fed into one or more machine learning models to generate a determined output, which is provided to a computing device to perform actions corresponding to the inferred result.
Signal files of data are received based on voltages of human body electrical processes related to facial gestures physically performed by a user. The voltages are detected by one or more electrodes on a set of headphones worn by the user, and the received signal files comprise a channel of electrode data for each electrode. Each channel includes an average amplitude of voltage during a window of time and powerband frequency components of the respective frequency.
Extracted features include at least one feature set based on channel data from a plurality of electrodes. One or more extracted EMG features are fed into a facial gesture machine learning model for determining a type of facial gesture from a plurality of types of facial gestures, where a respective type includes a sequence of facial gestures including at least one jaw clench user fingerprint and represents an attempt to match a passcode. Based on the determined sequence, the system identifies a valid or invalid passcode instance and sends output indicating an instance of verification of an identity associated with the passcode.
Claims Coverage
The provided independent claims are directed to a computer-implemented method, a system, and a computer program product, each covering the same core pipeline with five explicit inventive features for feature extraction from headphone electrode data and facial-gesture machine-learning output for passcode-based identity verification.
Headphone electrodes receive signal files for facial gestures
receiving one or more signal files of data based on voltages of one or more human body electrical processes related to facial gestures physically performed by a user, the voltages detected by one or more electrodes on a set of headphones worn by the user
Channel-based electrode data with window amplitude and powerbands
the one or more received signal files of data comprise a channel of electrode data for each electrode, and each channel comprises an average amplitude of voltage during a window of time and powerband frequency components
Machine-learning feature extraction and feeding for multiple electrodes
extracting features from the received data and feeding the extracted features into one or more machine learning models, with at least one feature set based on channel data from a plurality of electrodes
EMG-driven facial-gesture machine learning using jaw clench user fingerprint passcode sequence
feeding one or more extracted EMG features into a facial gesture machine learning model for determining a type of facial gesture from a plurality of types of facial gestures, where a respective type includes a sequence of facial gestures including at least one jaw clench user fingerprint and representing an attempt to match a passcode
Valid/invalid passcode identification and identity verification output
generating a determined output that identifies a valid or invalid passcode instance and, based on identifying the valid passcode instance, sending output to a computing device indicating an instance of verification of an identity associated with the passcode
The independent claims collectively cover receiving headphone electrode voltages during facial gestures, forming channel-based electrode features with average amplitude and powerband frequency components, extracting features and feeding them into machine learning, and using EMG features in a facial-gesture machine learning model to classify facial-gesture sequences containing a jaw clench user fingerprint for valid versus invalid passcode instances, followed by sending identity verification output to a computing device.
Stated Advantages
Documented Applications
No documented applications found
Interested in licensing this patent?