Brain-computer interface with adaptations for high-speed, accurate, and intuitive user interactions
Inventors
ALCAIDE, Ramses • Padden, Dereck • JANTZ, Jay • HAMET, James • MORRIS, Jr., Jeffrey • Pereira, Arnaldo
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12001602-B2
Publication Date
2024-06-04
Expiration Date
Abstract
Embodiments described herein relate to systems, devices, and methods for use in the implementation of a brain-computer interface that integrates real-time eye-movement tracking with brain activity tracking to present and update a user interface that is strategically designed for high speed and accuracy of human-machine interaction. Embodiments described herein also relate to the implementation of a hardware agnostic brain-computer interface with specific user interface adaptations to enable high-speed, intuitive, and accurate user manipulation of applications and/or machines.
Core Innovation
The invention provides a hardware-agnostic hybrid brain-computer interface that combines real-time eye-tracking and neural activity recording to support user interaction. Eye-movement signals include 2D/3D gaze and head-mounted/corneal-reflection pupil tracking, and neural signals include electroencephalography (EEG) and other neural modalities. The system ensembles the oculomotor and neural signals to estimate a point of focus of the user and an intent to select and activate actions via a control interface.
A stimulus with a set of control items is generated and presented through a control interface, where each control item is associated with an action from a set of actions. The interface includes a sticky control item, a grabber object, and target control items to realize selection by gaze-driven interaction. In particular, the sticky control item transitions to a picked-up state in response to foveation over the sticky control item, is moved based on eye-movement signals, and transitions to a dropped state on a target control item to activate the target control item.
The system determines the point of focus based on at least one of the eye-movement signals or the neural signals, and associates the sticky control item with the grabber object at a picked-up state based on the determined focus. The system then dissociates the sticky control item from the grabber object when the sticky control item is at the dropped state on the identified control item, associates the sticky control item with the identified control item, and activates the identified control item to implement an action intended by the user. Additional interface behaviors include dynamic stimulus tracking with tagging, 2D/3D UX with depth cues, navigation controls using a velocity-direction overlay, and context-based augmented reality control using image segmentation and device identification/connection.
Claims Coverage
The provided independent claims cover an apparatus and a non-transitory processor-readable medium, plus a method, that together implement a hybrid eye-tracking and neural-signal control interface with action-mapped control items and a gaze-driven sticky control interaction. The independent claims emphasize point-of-focus determination from eye-movement and optionally neural signals, and picked-up to dropped sticky control item selection linked to a determined control item and an intended action.
Hybrid control interface using eye-tracking and neural signals for point of focus
An apparatus comprising an eye-tracking device and a neural recording device, where an interfacing device receives eye-movement signals and neural signals, determines a point of focus of the user based on at least one of the eye-movement signals or the neural signals, and uses the point of focus to identify an associated control item and activate an action intended by the user.
Sticky control item with picked-up, moved, and dropped selection tied to foveation
An apparatus that provides a sticky control item configured to transition to a picked-up state in response to an eye-movement signal indicating foveation over the sticky control item, move based on an eye-movement signal of the user, and transition to a dropped state on a target control item to activate the target control item, including association with a grabber object and dissociation/association to the identified control item.
Stimulus with control items mapped to actions and activation of identified control item
An apparatus that generates and presents a stimulus via a control interface including a set of control items, each control item associated with an action from a set of actions, and activates the identified control item associated with the point of focus to implement an action intended by the user.
Processor-readable medium instructions for sticky control item manipulation and intended action determination
A non-transitory processor-readable medium storing instructions to generate and present a control interface stimulus with control items mapped to actions, receive eye-tracking inputs, determine point of focus, identify a first control item associated with the point of focus, provide a grabber object manipulating a sticky control item, transition the sticky control item picked-up to dropped on a target control item, associate/dissociate the sticky control item with the grabber object and identified control item, and determine an action intended by the user based on the association between the sticky control item and the second control item.
Eye-tracking based point-of-focus identification with time-based control item selection
Instructions that identify a first control item associated with the point of focus at a first time and then determine a point of focus at a second time after the first time to identify a second control item different from the sticky control item, where the resulting association after the dropped state is used to determine the action intended by the user.
Method of gaze-driven sticky-to-target control item selection for intended action
A method presenting a stimulus via a control interface that includes a sticky control item transitioning to a picked-up state in response to foveation, moving based on an eye-movement signal, and transitioning to a dropped state on a target control item, receiving eye-tracking inputs, identifying a foveation over the sticky control item, determining a point of focus at a second time, identifying a second control item different from the sticky control item, and determining an action intended by the user based on the association between the sticky control item and the second control item.
Across the independent claims, the core coverage is a multimodal control interface where eye-tracking and neural signals are used to determine a point of focus, and a sticky control item is picked up via foveation and dropped onto a target control item to activate an action associated with the identified control item, with intended action determined from the association produced by the dropped state.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?