Display system interface using visually-evoked cortical potentials
Inventors
Miller, Michael E. • BORGHETTI, BRETT J. • Kennedy, Kellie D. • Stephens, Chad L. • Pope, Alan T.
Assignees
Government Of Us Represented By Secretary Of Air Force AS • National Aeronautics and Space Administration NASA
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12197648-B2
Publication Date
2025-01-14
Expiration Date
2043-02-02
Abstract
A brain-computer interface system includes a video processor for producing a display signal, a temporal controller for producing a plurality of repetitive visual stimulus (RVS) signals with different respective temporal aspects, a display device that receives the display signal and displays a corresponding image on a plurality of different display regions and receives the RVS signals and displays corresponding RVS in respective ones of the display regions, an electroencephalographic (EEG) sensor for sensing a visually-evoked cortical potential (VECP) signal in a user with eyes fixated on a viewed one of the display regions, and a VECP processor for processing the VECP signal to identify the respective temporal aspect of the respective RVS of the viewed display region to estimate the eye fixation location. The RVS are generated independently of the display update/refresh rate and at sufficiently high frequencies to avoid flicker perceptible to the user.
Core Innovation
The invention relates to a brain-computer interface (BCI) system that includes a video processor for generating a display signal, a temporal controller producing multiple repetitive visual stimulus (RVS) signals each with different temporal aspects, and a display device that simultaneously receives and displays both the image and the different RVS signals in distinct spatial display regions. An electroencephalographic (EEG) sensor senses visually-evoked cortical potential (VECP) signals from a user whose eyes are fixated on one of the display regions, and a VECP processor processes these signals to identify the viewed region based on the temporal aspect of the RVS.
The problem addressed is the limitation in current BCI systems using steady-state visually-evoked potentials (SSVEPs). Traditional displays embed RVS signals in the display image signals and limit RVS to frequencies that are factors of the display's refresh rate, resulting in few selectable stimuli and visible flicker that is perceptible and bothersome to users. These constraints limit practical applications, particularly outside disabled populations. The invention solves this by decoupling the RVS signals from the display update rate to allow high-frequency RVS that avoid visible flicker, increasing the number of potential eye fixation points without introducing perceptible flicker.
The disclosure explains that the temporal controller independently generates RVS signals with temporal aspects such as frequency, phase, amplitude, or duty cycle, which are not embedded in the video processor's image control signal. This allows the display device to present multiple RVS simultaneously in different spatial regions. The RVS frequencies can be sufficiently high (30-120 Hz, preferably around 45-55 Hz) to avoid flicker, and can be independent of or higher than the display refresh rate. Synchronization signals from the temporal controller assist the VECP processor in determining the display region being attended based on the VECP signal's temporal matching to the known RVS. This system overcomes previous limitations by allowing a large number of stimuli with reduced perceptual artifacts in the interface.
Claims Coverage
The patent includes three independent claims, encompassing systems and methods centered on a brain-computer interface utilizing repetitive visual stimuli with distinct temporal aspects separated from and independent of display control signals to identify user eye fixation.
Independent temporal control of repetitive visual stimulus signals separated from display control signals
The system includes a temporal controller configured to produce multiple repetitive visual stimulus (RVS) signals, each with a different temporal aspect, which are separate and independent from and not embedded in the display control signal produced by a video processor.
Mapping RVS signals to spatial regions to identify eye fixation via EEG signals
The system displays an image on a device with multiple display regions, each having light-emitting elements controlled by both a display control signal and independent RVS signals, and uses EEG sensors to detect visually-evoked cortical potentials (VECP) corresponding to these RVS signals to determine which display region the user is viewing.
Use of sufficiently high frequencies for RVS to avoid perceptible flicker
The RVS are generated at sufficiently high frequencies (typically 30 to 120 Hz, preferably 45 to 55 Hz) that do not produce visually perceptible flicker, with frequencies not necessarily being divisors of the display's refresh rate and often greater than half the update rate.
Synchronization signaling between temporal controller and VECP processor
The temporal controller sends synchronization signals indicating pulse onsets of the RVS to the VECP processor, enabling accurate determination of the onset of RVS for each display region during VECP processing.
Distributed temporal controllers managing different portions of the display
The temporal controller can be implemented as multiple temporal controllers in a distributed arrangement, each producing RVS signals with different temporal aspects for controlling spatial display regions.
Incorporation of additional eye movement sensing and reconciliation
An optional electrooculogram (EOG) sensor can sense eye movement or fixation, sending signals to the VECP processor, which performs reconciliation to confirm that the viewed region indicated by EOG matches the region identified via EEG signals.
The independent claims cover a BCI system and method leveraging independent temporal control of visual stimuli at multiple spatial display regions and detecting corresponding EEG signals to accurately estimate eye fixation location, employing high-frequency stimuli to avoid flicker, synchronization between components, distributed control architectures, and optional integration with EOG signals to enhance accuracy.
Stated Advantages
Enables use of a large number of potential eye fixation locations without producing visible flicker.
Overcomes limitations of prior BCI systems that were restricted to low-frequency, flickering stimuli tied to display refresh rates.
Allows independent control of RVS signals separate from image signals, providing flexibility in stimuli frequency, phase, amplitude, and duty cycle.
Improves accuracy and timeliness in estimating eye fixation location, often within 0.1 seconds of fixation onset.
Can combine EEG with eye movement sensors (e.g., EOG) for enhanced confirmation and accuracy of eye fixation detection.
Documented Applications
Use by disabled individuals, such as paraplegics, to control devices like wheelchair movement through brain-computer interface.
Application in avionics displays to provide high-resolution eye fixation detection for user interface control.
Incorporation in head or helmet-worn displays for hands-free control with components arranged to reduce user burden.
Interested in licensing this patent?