Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12374150-B2
Publication Date
2025-07-29
Expiration Date
Abstract
Technologies are described for reconstructing facial models which are preserved images or images captured from security cameras. The reconstructed models can be three-dimensional (3D) point clouds and can be compared to existing facial models and/or other reconstructed models based on physical geometry. The 3D point cloud models can be encoded into one or more latent space feature vector representations which can allow both local and global geometric properties of a face to be described. The one or more feature vector representations of a target face can be used individually or in combination with other descriptors for recognition, retrieval, and classification tasks. Neural networks can be used in the encoding of the one or more feature vector representations.
Core Innovation
The invention addresses facial image reconstruction by extracting a set of facial images for a target individual from an image collection, and sampling one or more contours of each facial image. A sampling rate for each facial image is varied based on at least one of image characteristics of a region of each facial image or external information of a region of each facial image. The approach selects one or more source images from the set of facial images for the target individual and projects a point from the selected source image onto two other facial images to generate first and second projected points on the other facial images.
The method identifies one or more corresponding points between the first and second projected points and projects the one or more corresponding points into a three-dimensional (3D) space. Based at least on the one or more corresponding points, the method generates data indicative of a point cloud of the target individual. In variations described in the document, the process includes determining facial pose and, based at least on facial pose, determining alignment of a selected source image relative to two other facial images.
The document further describes representations derived from the reconstructed 3D facial models, including encoding data indicative of a point cloud into one or more feature vector representations and, in some implementations, classifying or comparing using neural network processing and feature vector representations. It also describes use with a feature vector database for matching via feature vector comparison.
Claims Coverage
The independent claims cover three implementations: a method for facial image reconstruction, a method for facial reconstruction generating data indicative of a point cloud, and a computerized apparatus implementing the point-cloud-indicative reconstruction, totaling three independent claims. Across these, the inventive core is contour sampling with a sampling-rate selection based on image characteristics and/or external information, point projection onto two other facial images to identify corresponding points, and generating 3D data indicative of a point cloud; additional claim coverage includes pose and alignment determination and, in dependent variants, feature-vector encoding and comparison.
Variable contour sampling based on image characteristics or external information
Sampling one or more contours of each facial image, wherein a sampling rate for each facial image is varied based on at least one of image characteristics of a region of each facial image or external information of a region of each facial image.
Point projection between facial images and corresponding-point identification
For each selected source image, projecting a point from the selected source image onto two other facial images to generate a first projected point on a first one of the other facial images and a second projected point on a second one of the other facial images; identifying one or more corresponding points between the first and second projected points.
Generating 3D data indicative of a point cloud from corresponding points
Projecting the one or more corresponding points into a three-dimensional (3D) space to generate data indicative of a point cloud of the target individual.
Facial pose determination and alignment based on corresponding steps
Determining one or more facial pose for each selected source image and for the two other facial images; and based at least on the one or more facial pose, determining an alignment of the selected source image relative to the two other facial images.
Feature-vector encoding and comparison using a feature vector database
Encoding data indicative of the point cloud into one or more feature vector representations; and comparing the one or more feature vector representations to one or more other feature vectors in a feature vector database.
The independent claim set centers on varying contour sampling using image characteristics and/or external information, projecting points from a selected source image onto two other facial images to identify corresponding points, and projecting the corresponding points into a 3D space to generate data indicative of a point cloud. Dependent coverage further adds facial pose determination and alignment, and some implementations encode point-cloud-indicative data into feature vector representations for comparison using a feature vector database.
Stated Advantages
Documented Applications
Facial recognition using 3D facial models reconstructed from multi-view/preserved and security-camera images, including comparing and using reconstructed representations for recognition, retrieval, classification, clustering, and document search.
Surveillance of a target individual, including tracking, extracting and ordering partial facial images, reconstructing a high-resolution dense facial point cloud, and re-rendering high-resolution facial images for recognition.
Document search via vector database queries using feature vectors derived from reconstructed facial models and a facial model reconstruction pipeline over video streams, including facial image extraction, 3D point projection into a point cloud, encoding into feature vectors, and matching via a feature vector database.
Interested in licensing this patent?