Brain-computer interface with adaptations for high-speed, accurate, and intuitive user interactions
Inventors
ALCAIDE, Ramses • Padden, Dereck • JANTZ, Jay • HAMET, James • MORRIS, Jr., Jeffrey • Pereira, Arnaldo
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
Embodiments described herein relate to systems, devices, and methods for use in the implementation of a brain-computer interface that integrates real-time eye-movement tracking with brain activity tracking to present and update a user interface that is strategically designed for high speed and accuracy of human—machine interaction. Embodiments described herein also relate to the implementation of a hardware agnostic brain-computer interface with specific user interface adaptations to enable high-speed, intuitive, and accurate user manipulation of applications and/or machines.
Core Innovation
The invention provides a hybrid brain-computer interface that combines eye-tracking and neural recording to determine a user’s point of focus and infer an action intended by the user. An eye-tracking device records eye-movement signals associated with the user while a neural recording device records neural signals associated with the user. A display presents a control interface and a stimulus including a set of control items associated with a set of actions.
The processor determines weights for the eye-movement signals and the neural signals based on a state of the control interface, including the stimulus. Using the determined weights, the processor determines the point of focus of the user based on the eye-movement signals and the neural signals, where the point of focus is associated with at least one control item. Based on the point of focus and the at least one control item, the processor determines an action intended by the user and implements the action via the control interface.
The control interface and stimulus are configured so that determining the point of focus can be supported using different neural signal types and stimulus-driven neural features, including Event Related Potentials and visually evoked neural activity. The disclosed processing can use ensemble signal analysis and includes weighted combination of source signals, with processing constrained in some embodiments to a bagging ensemble classifier algorithm. The system can further adapt the stimulus and the control items, including presenting control items in a three-dimensional virtual space.
Claims Coverage
Independent claim set includes four independent claims (apparatus, non-transitory processor-readable medium, method) that center on a hybrid eye-tracking and neural-recording pipeline with stimulus-driven control items. Across these claims, the core inventive features are: generating/presenting a stimulus with control items, receiving eye and neural information, determining state-dependent weights, determining a point of focus from weighted information, mapping the point of focus to control items, and determining the intended action from the identified control item.
State-dependent weighted fusion of eye and neural signals to determine point of focus
Determine weights for the eye movement signals and the neural signals based on a state of the control interface, including the stimulus; determine, based on the eye-movement signals and the neural signals using the weights, a point of focus of the user, the point of focus being associated with at least one control item from the set of control items.
Stimulus with control items mapped to user actions
Generate and present a stimulus, via the control interface and to the user, the stimulus including a set of control items associated with a set of actions; identify at least one control item from the set of control items associated with the point of focus of the user; determine, based on the at least one control item, an action intended by the user.
Hybrid eye-tracking and neural recording for monitoring user behavior
Receive the eye-movement signals from the eye-tracking device and the neural signals from the neural recording device; receive information from an eye-tracking device and information from a neural recording device, the eye-tracking device and the neural recording device configured to monitor behavior of the user; determine an action intended by the user based on the point of focus derived from the received information.
Implementing intended action through the control interface
Implement the action intended by the user; generate a control interface configured to be manipulated, by a user, to perform a set of actions; determine, based on the at least one control item, an action intended by the user.
The independent claims consistently cover a hybrid BCI apparatus/system where eye-movement and neural-recording signals are fused using state-dependent weights to determine a user’s point of focus, map that focus to control items in a presented stimulus, and then determine and implement the user-intended action through a control interface.
Stated Advantages
Documented Applications
Real-time determination of a user’s point of focus and inference of an intended action for high-speed, accurate human-machine interaction via an updated UI/UX.
Interested in licensing this patent?