Systems and methods decoding intended symbols from neural activity

Inventors

Shenoy, Krishna Vaughn • Henderson, Jaimie M. • Willett, Francis Robert

Assignees

Leland Stanford Junior University

Abstract

Systems and methods for decoding intended symbols from neural activity in accordance with embodiments of the invention are illustrated. One embodiment includes a symbol decoding system for brain-computer interfacing, including a neural signal recorder implanted into a brain of a user, and a symbol decoder, the symbol decoder including a processor, and a memory, where the memory includes a symbol decoding application capable of directing the processor to obtain neural signal data from the neural signal recorder, estimate a symbol from the neural signal data using a symbol model, and perform a command associated with the symbol.

Core Innovation

The invention is a brain computer-interface for symbol decoding that includes a neural signal recorder implanted into a brain of a user and a symbol decoder with a processor and a memory storing a symbol decoding application. The symbol decoding application directs the processor to obtain neural signal data from the neural signal recorder, temporally bin the neural signal data to create a plurality of neural population time series, and convert a portion of the neural population time series covering a predetermined time window into a time probability series using a symbol decoding model.

From the time probability series, the invention identifies a most likely symbol intended by the user as the most likely symbol in the time probability series offset by a delay from a time point in the time probability series at which the probability for a new character exceeds a threshold. After determining the most likely symbol, the invention performs a command associated with the most likely symbol, enabling decoded symbols to drive downstream actions.

The symbol decoding model converts binned neural population time series into a time probability series using trained symbol models, including recurrent neural networks, long short-term memory networks, temporal convolutional networks, and hidden Markov models. The system also supports mapping decoded symbols to commands executed by output devices such as a vocalizer, a display, prosthetics, or a computer system, including mapping alphabet letters to commands that print letters into a text string for vocalization.

The document further describes training and calibration approaches for the decoder RNNs using forced-alignment labeling with HMMs and constructing RNN target probabilities based on inferred symbol start times, along with per-user calibration and training data augmentation from templates. In addition, the document describes an optional “difference maximized” symbol set for alphabet letters.

Claims Coverage

The document provides two independent claims (clm-00001 and clm-00009) that share the same core inventive pipeline: obtaining implanted neural signals, binning into neural population time series, converting a window into a time probability series with a symbol decoding model, selecting a most-likely intended symbol using a delay relative to a new-character threshold, and performing a command associated with the selected symbol. Dependent claims refine this pipeline by specifying neural recorder structure, neural signal characteristics, allowable symbol model types, timing parameters, symbol sets, and symbol-to-command mappings and output targets.

Across both independent claims, the inventive coverage centers on a symbol decoding pipeline that turns temporally binned implanted neural data into a time probability series, then selects a delayed, threshold-triggered most likely intended symbol and performs a command associated with that symbol. Dependent claims further constrain or specify the neural recorder implementation, the recorded neural signal form, the allowed symbol model families, example temporal parameters, symbol sets (including alphabet letters and “difference maximized” symbols), and command/output mappings.

Stated Advantages

Documented Applications