System and method for face recognition with two-dimensional sensing modality
Inventors
Assignees
United States Department of the Army
Publication Number
US-9875398-B1
Publication Date
2018-01-23
Expiration Date
2036-06-30
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
A method and system in which facial image representations stored in a database are defined by facial coordinates in a plane common to other images in the database in order to facilitate comparison or likeness of the facial images by comparing the common plane facial coordinates, the common plane being determined by the locations of the eyes and mouth corners; at least one input operatively connected to the at least one processor and configured to input the corners of the eyes and mouth coordinates; the at least one processor configured to convert inputted coordinates for the corners of the eyes and mouth into estimated common plane coordinates by minimizing the error between the inputted corners of the eyes and mouth coordinates and the estimated coordinates corners of the eyes and mouth obtained from the least square estimation model of the common plane coordinates of the corners of eyes and mouth.
Core Innovation
The invention provides a method and system for face recognition where facial images in a gallery and probe may be from different sensing modalities and have different 3D pose angles because they are acquired by different sensors or at different times. It converts the coordinates of the eyes and mouth corners of a facial image in a randomly oriented photograph into virtual coordinates representing an estimated position as if the subject's head were oriented such that the eye centers and mouth corners lie in a vertical plane with zero roll, pitch, and yaw and unit scale. This estimate is obtained through a model based on a non-linear Gaussian Least Square Differential Correction. The comparison or matching is then performed on these virtual coordinates.
The invention addresses the problem of cross modality recognition between thermal and visible face images, which differ significantly in appearance due to capturing different physical properties (heat vs. light reflectance) and challenges of varying 3D orientations (pose angles) in 2D images. Existing methods struggle with matching thermal to visible images because of differences in edge and texture information and the problem of different 3D poses causing misalignment of facial features in the images. This invention mitigates these issues by using common biometric landmarks visible in both modalities and applying a 3D registration method from a single 2D image frame to normalize for pose differences.
Claims Coverage
The patent includes multiple independent claims covering systems for facial recognition employing estimation of facial feature coordinates in a common plane and using non-linear least square algorithms for pose correction and matching.
Facial coordinate normalization to a virtual vertical plane
A system configured to input coordinates of eyes and mouth corners of a facial image, convert these coordinates into estimated virtual plane coordinates where the facial features lie in a vertical plane with zero roll, pitch, and yaw, and use these virtual coordinates for face recognition matching.
Use of a 9-parameter vector with non-linear estimation
A processor solving for a parameter vector comprising coordinates of eye and mouth corners, and yaw, pitch, and roll angles using a non-linear Gaussian Least Square Differential Correction algorithm, minimizing error between input and estimated feature coordinates.
Common coordinate facial representation for multimodal images
Defining facial image representations stored in a database by coordinates of facial landmarks in a common plane to facilitate likeness comparison of facial images across modalities by comparing these standardized coordinates.
Matching based on multiple hypotheses testing
Performing matching of probe images to gallery images using multiple hypotheses testing theory that minimizes the probability of error, calculating distance measures between the normalized coordinates of facial landmarks.
The claims define systems and methods for landmark-based facial coordinate normalization to a virtual plane using non-linear least square estimation to handle pose variations and modality differences, and perform recognition by comparing normalized landmark coordinates with multiple hypotheses statistical testing to identify matches.
Stated Advantages
Mitigation of degradation in face recognition performance caused by cross modality differences between thermal and visible images by using biometric landmarks common to both.
Capability to perform face recognition using images with different 3D pose angles via a novel 3D registration method using a single 2D image frame.
The unified coordinate system (UCS) is invariant to sensor position and head pose, facilitating matching under uncooperative or uncontrolled conditions.
Reduction of matching error by leveraging multiple frames through temporal information averaging.
Documented Applications
Nighttime personnel target identification where the probe image is a thermal face image and the gallery contains visible face images.
Security and military operations requiring identification despite illumination variations, pose differences, and use of different sensor modalities.
Biometric identification applications including criminal detection, airline screening, access control, surveillance systems, identification of dead bodies, and terrorist watch lists.
Interested in licensing this patent?