System and method for training of state-classifiers
Inventors
Stephens, Chad L. • Harrivel, Angela R. • Pope, Alan T. • Prinzel, III, Lawrence J
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-10192173-B2
Publication Date
2019-01-29
Expiration Date
2031-06-22
Abstract
Method and systems are disclosed for training state-classifiers for classification of cognitive state. A set of multimodal signals indicating physiological responses of an operator are sampled over a time period. A depiction of operation by the operator during the time period is displayed. In response to user input selecting a cognitive state for a portion of the time period, the one or more state-classifiers are trained. In training the state-classifiers, the set of multimodal signals sampled in the portion of the time period are used as input to the one or more state-classifiers and the selected one of the set of cognitive states is used as a target result to be indicated by the one or more state-classifiers.
Core Innovation
The invention provides methods and systems for training state-classifiers that classify the cognitive state of an operator based on physiological responses. A set of multimodal signals indicating physiological responses are sampled over a time period during which the operator is subjected to stimuli designed to induce various cognitive states. The corresponding operation by the operator is depicted, and user input selecting a cognitive state for a portion of the time period is used to train the state-classifiers by mapping the sampled physiological signals to the selected cognitive state.
The solution addresses the problem that previous approaches relying on single physiological measures are insufficient to distinguish between different cognitive states with similar cognitive activity levels. Furthermore, physiological responses vary from person to person, requiring individualized training of state-classifiers. The invention employs multiple physiological data sources (multimodal signals) and allows both supervised and unsupervised training, including refinement through specialist review. The system facilitates iterative training by displaying operator operation and cognitive state probabilities, allowing specialists to select portions of time for which cognitive states are verified and used to further train the classifiers.
Claims Coverage
The patent contains three independent claims covering a system and methods for cognitive state classification using multimodal physiological signals and state-classifiers.
System for training and using state-classifiers with user-guided refinement
A system comprising sensors providing multimodal physiological signals of an operator during a time period, a processing circuit configured to train state-classifiers mapping signals to cognitive states, a display depicting operation, data storage for classifiers, and additional processing circuits that determine cognitive states and perform actions based on criteria. The system allows user input selecting a cognitive state and a portion of the time period to retrain classifiers accordingly. Actions include adjusting vehicle operation, providing alerts, or sending alert messages.
Real-time cognitive state assessment and responsive actions
A method sampling multimodal physiological signals, retrieving state-classifiers from storage, determining probabilities of cognitive states using the classifiers, and performing specified actions when probabilities satisfy criteria. Actions include adjusting vehicle operation, providing alerts, or sending messages, for example alerting or engaging autonomous systems if inattentive or unresponsive states are detected.
Simulated environment-based training and cognitive state classification
A method providing simulated stimuli to induce target cognitive states, sampling multimodal physiological signals, retrieving state-classifiers mapping signals to cognitive states including the target state, determining cognitive state probabilities, and performing actions such as adjusting stimuli or alerting trainers based on criteria. The method includes displaying operation, enabling specialist selection of cognitive state portions, and retraining classifiers accordingly. Initial training involves presenting stimuli, sampling signals, and mapping physiological responses to cognitive states.
The claims cover systems and methods that use multimodal physiological signals and user-involved training processes to classify operator cognitive states and perform responsive actions, emphasizing individualized classifier training, real-time operation monitoring, and specialist-guided refinement.
Stated Advantages
Improved accuracy in cognitive state classification through integration of multiple physiological signals to reduce false positives and negatives compared to single-measure approaches.
Individualized training of state-classifiers that accommodates person-to-person variability in physiological responses.
Ability to refine and improve classifiers through specialist review and selection of cognitive states during displayed operation.
Real-time monitoring enables triggering of alerts, adjusting vehicle operation, or autonomous interventions to enhance safety.
Flexibility in applying actions and criteria that can be customized by instructors or operators to suit specific needs.
Documented Applications
Training, assistance, and monitoring of operators of various vehicles including aircraft, trains, trucks, and automobiles.
Enhancing safety in automobile operation by real-time cognitive state evaluation, including providing alerts, disabling controls, or engaging autonomous systems when impaired cognitive states are detected.
Improving human-computer interfaces by adapting video game control and gameplay based on player cognitive state as determined by multimodal physiological signals.
Interested in licensing this patent?