Systems and methods to determine user state
Inventors
Durkee, Kevin • Pappada, Scott • Ortiz, Andres • DePriest, William • Feeney, John • Geyer, Alexandra • Sullivan, Seamus • Wiggins, Sterling
Assignees
Publication Number
US-10265008-B2
Publication Date
2019-04-23
Expiration Date
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
Computer based systems and methods for estimating a user state are disclosed. In some embodiments, the methods comprise inputting a first input at an intermittent interval and a second input at a frequent interval into a user state estimation model to estimate the user state. In some embodiments, the first inputs are enhanced by injecting a noise input to create a plurality of enhanced first inputs whereby the plurality of enhance first inputs correspond to the plurality of second inputs at the frequent interval. In some embodiments, the first input comprises a self-reported input and the second inputs comprise a physiological input, a performance input or a situational input. In some embodiments, a machine learning algorithm creates the state estimation model. In some embodiments, the state estimation model estimates a future user state. In some embodiments, a computer based system for estimating a user state is provided.
Core Innovation
Computer based systems and methods for estimating a user state are disclosed that input a first input at an intermittent interval and a second input at a frequent interval into a user state estimation model to estimate the user state. In some embodiments the first inputs are enhanced by injecting a noise input to create a plurality of enhanced first inputs whereby the plurality of enhanced first inputs correspond to the plurality of second inputs at the frequent interval. In some embodiments the first input comprises a self-reported input and the second inputs comprise a physiological input, a performance input or a situational input, and in some embodiments a machine learning algorithm creates the state estimation model that can estimate a future user state.
The background describes a problem that algorithms estimating human user states in real-time based on physiological inputs have been limited, producing user state estimation algorithms that suffer from low resolution, infrequent updates, and poor generalizability across human operators. The disclosed embodiments address these limitations by providing a real-time, model-based classifier and predictor of user state on a continuous, high resolution scale using multi-modal data source fusion, noise-injected intermittent inputs, on-line adaptation, and predictive techniques to support frequent updates and improved generalization.
Claims Coverage
Independent claims identified: 1, 17, and 20. The following inventive features are extracted from the independent claims.
Receiving a plurality of first inputs at an intermittent interval
receiving a plurality of first inputs at an intermittent interval;
Receiving a plurality of second inputs at a frequent interval
receiving a plurality of second inputs at a frequent interval;
Injecting noise into intermittent inputs to create enhanced inputs
injecting a plurality of third inputs as a noise input into the plurality of first inputs at a non-random interval to create a plurality of enhanced first inputs whereby the plurality of enhanced first inputs correspond to the plurality of second inputs at the frequent interval;
Inputting enhanced and frequent inputs into a state estimation model
inputting the enhanced first inputs and the plurality of second inputs into a state estimation model whereby a user state can be estimated based on the plurality of second inputs;
Estimating the user state based on frequent inputs
estimating the user state based on the plurality of second inputs.
Estimating a future user state
estimating a future state of the user at a future time utilizing the state estimation model.
System comprising processor and non-transitory medium configured to implement the method
a processor; a non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code configured to be executed to implement a method comprising: receiving a plurality of first inputs at an intermittent interval; receiving a plurality of second inputs at a frequent interval; injecting a plurality of third inputs as a noise input into the plurality of first inputs at a non-random interval to create a plurality of enhanced first inputs whereby the plurality of enhanced first inputs correspond to the plurality of second inputs at the frequent interval; inputting the enhanced first inputs and the plurality of second inputs into a state estimation model whereby a user state can be estimated based on the plurality of second inputs; and estimating the user state based on the plurality of second inputs.
The independent claims cover methods and a system that receive intermittent subjective inputs and frequent objective inputs, inject noise into intermittent inputs to create enhanced, temporally aligned inputs, input enhanced and frequent inputs into a state estimation model, estimate user state from frequent inputs, and in some embodiments estimate a future user state; the system claim adds a processor and non-transitory medium configured to implement these method steps.
Stated Advantages
Provides a real-time, model-based classifier and predictor of user state on a continuous, high resolution (0-100) scale.
Enables more frequent estimates of user state, with promising accuracy using updates as frequently as once per second.
Sustains acceptable accuracy at high frequency updates by integrating multi-modal data sources (physiological, situational, task performance, and self-reported factors).
Improves generalization across users and over time via baseline physiological data collection and on-line learning that adapts model weights in real-time.
Supports predictive estimation of user state and its relationship to performance to facilitate proactive adaptive aiding and user interface modification.
Documented Applications
Estimating user states such as workload, task engagement, stress, or trust.
Adaptive augmentation of user interfaces and adaptive aiding, including eliminating unnecessary information sets and displaying necessary information sets based on predicted future state.
Use within computer based training simulators to obtain objective measures of trainee cognitive state (real-time engagement or real-time workload) and to adapt training interfaces based on the objective measure.
Application to remotely piloted aircraft (RPA) task domain and to support resource re-allocation, levels of automation, and adaptive automation decisions.
Interested in licensing this patent?