Neural network for eye image segmentation and image quality estimation

Inventors

SPIZHEVOY, Alexey • Kaehler, Adrian • Badrinarayanan, Vijay

Assignees

Magic Leap Inc

Abstract

Systems and methods for eye image segmentation and image quality estimation are disclosed. In one aspect, after receiving an eye image, a device such as an augmented reality device can process the eye image using a convolutional neural network with a merged architecture to generate both a segmented eye image and a quality estimation of the eye image. The segmented eye image can include a background region, a sclera region, an iris region, or a pupil region. In another aspect, a convolutional neural network with a merged architecture can be trained for eye image segmentation and image quality estimation. In yet another aspect, the device can use the segmented eye image to determine eye contours such as a pupil contour and an iris contour. The device can use the eye contours to create a polar image of the iris region for computing an iris code or biometric authentication.

Core Innovation

The invention relates to determining eye contours in a semantically segmented eye image that includes a plurality of pixels. The method determines a pupil contour and an iris contour using the semantically segmented eye image. The pupil contour is determined by using a first binary image created based on the semantically segmented eye image, and the iris contour is determined by using a second binary image created based on the semantically segmented eye image.

Based on the pupil contour and the iris contour, the method performs personal biometric identification. The approach uses semantically segmented regions corresponding to the eye image contents such as background, sclera, iris region, and pupil region to support contour determination. The documented workflow includes deriving pupil and iris contours from the segmentation outputs and using those contours for biometric identification.

The document also describes a merged convolutional neural network architecture for producing the semantically segmented eye image and an image quality estimation. The merged architecture includes shared layers whose outputs feed both a segmentation tower and a quality estimation tower. The segmentation output supports contour and downstream iris processing, while the quality estimation provides a good/bad quality classification used alongside the biometric steps.

Claims Coverage

The independent claim is directed to a computer-implemented method that determines pupil and iris contours from a semantically segmented eye image using first and second binary images with specified color values, and then performs personal biometric identification based on the pupil and iris contours. The document also includes an independent claim for a convolutional neural network with segmentation and quality estimation towers using shared layers.

Across the independent claim(s) reflected in the provided text, the main inventive concepts are the determination of pupil and iris contours from a semantically segmented eye image using first and second binary images mapped by specified color values, performing personal biometric identification based on those contours, and generating the semantically segmented output with a merged convolutional neural network that includes segmentation and quality estimation towers sharing layers.

Stated Advantages

Documented Applications