Eye tracking method and system

Inventors

De Villers-Sidani, Etienne • DROUIN-PICARO, Paul Alexandre

Assignees

Innodem Neurosciences

Abstract

A computer-implemented method for determining a gaze position of a user, comprising: receiving an initial image of at least one eye of the user; extracting at least one color component of the initial image to obtain a corresponding at least one component image; for each component image, determining a respective internal representation; determining an estimated gaze position in the initial image by applying a respective primary stream to obtain a respective internal representation for each of the at least one component image; and outputting the estimated gaze position. The processing of the component images is performed using a neural network configured to, at run time and after the neural network has been trained, process the component images using one or more neural network layers to generate the estimated gaze position. A system for determining a gaze position of a user is also provided.

Core Innovation

The invention provides a system and a computer-implemented method for determining a gaze position of a user from an initial image of at least one eye. The system receives the initial image and extracts at least one color component of the initial image to obtain a corresponding at least one component image. The extracted color-component information supports gaze position estimation from the same initial image without relying on dedicated infrared tracking hardware.

For each at least one component image, the system applies a respective primary stream to obtain a respective internal representation for each component image. The internal representations are used to determine an estimated gaze position for the initial image and to output the estimated gaze position. The neural-network approach includes primary streams, an internal representation process, and a gaze position estimation stage.

In additional refinements, the system can identify at least one eye in the initial image using facial landmarks and optionally crop the image to the portion containing the eye before extracting the color components. The described processing also includes mechanisms for head pose invariance using facial landmark features, and for combining contributions of component images based on illuminant values and weight factors. Further refinements include a referential transformation to express the estimated gaze position relative to a device screen and to support user interface interactions, including for paralyzed users.

Claims Coverage

The independent claims are directed to a gaze-position determination pipeline that includes extracting color components from an initial eye image, computing internal representations with respective primary streams, and estimating and outputting an estimated gaze position. The coverage includes neural-network internal representation determination and gaze estimation, with additional architectural refinements reflected in the dependent claim structure referenced for each independent claim.

Overall, the independent-claim coverage is focused on estimating gaze position by extracting color components from an initial image of at least one eye, applying respective primary streams to obtain internal representations for each component image, and determining an estimated gaze position from those internal representations. The provided content also supports a separation between internal-representation generation and gaze-position estimation via stream structure as reflected in the independent-claim family.

Stated Advantages

Documented Applications