Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Assignees
MemberHinge HealthHinge HealthHinge Health delivers a musculoskeletal care platform that combines digital physical therapy, individualized exercise programs, in-person clinical services, and multidisciplinary care for conditions such as pain, mobility loss, pelvic health, gastrointestinal disorders, and menopause-related symptoms. The platform utilizes remote monitoring, motion analysis, and evidence-based care, prioritizing validated clinical outcomes, data security, and integration with benefits systems for employers and health plans.
Hinge Health delivers a musculoskeletal care platform that combines digital physical therapy, individualized exercise programs, in-person clinical services, and multidisciplinary care for conditions such as pain, mobility loss, pelvic health, gastrointestinal disorders, and menopause-related symptoms. The platform utilizes remote monitoring, motion analysis, and evidence-based care, prioritizing validated clinical outcomes, data security, and integration with benefits systems for employers and health plans.
Publication Number
US-12249074-B2
Publication Date
2025-03-11
Expiration Date
Abstract
System and method for extracting human pose information from an image, comprising a feature extractor connected to a database, a convolutional neural network (CNN) with a plurality of CNN layers. Said system/method further comprising at least one of the following modules: a 2D body skeleton detector for determining 2D body skeleton information from the human-related image features; a body silhouette detector for determining body silhouette information from the human-related image features; a hand silhouette detector for determining hand silhouette detector from the human-related image features; a hand skeleton detector for determining hand skeleton from the human-related image features; a 3D body skeleton detector for determining 3D body skeleton from the human-related image features; and a facial keypoints detector for determining facial keypoints from the human-related image features.
Core Innovation
A system extracts human pose information from an image by extracting human-related image features using a feature extractor implemented with a first convolutional neural network (CNN) architecture. The feature extractor is connectable to a database including a dataset of reference images and outputs a plurality of tensors from different CNN layers that collectively preserve the features.
The plurality of tensors produced by the first CNN architecture is used as input to a two-dimensional (2D) body skeleton detector. The 2D body skeleton detector includes a second CNN architecture and outputs, for each of multiple joints, a heat map that visually indicates a location of the joint, enabling pose information determination based on the preserved tensor features.
At least one of the multiple heat maps produced by the 2D body skeleton detector is further provided as input to a facial keypoints detector. The facial keypoints detector includes a CNN architecture and outputs locations of one or more facial keypoints.
Claims Coverage
The partial content includes three independent claims covering a system and two methods for extracting human pose information, using tensor outputs from CNN layers to drive CNN-based joint heat map and facial keypoint detection. The core inventive features are the generation of a plurality of tensors from different CNN layers and the use of those tensors to infer 2D body joint locations and facial keypoint locations.
Tensor-preserving feature extraction from multiple CNN layers for human pose
A feature extractor for extracting features related to human joints and shapes from the image, connectable to a database of reference images and provided with a first CNN architecture including a first plurality of CNN layers, wherein each CNN layer applies a nonlinear activation function to its input data using trained kernel weights, and wherein each CNN layer produces an output as a tensor so that the first CNN architecture outputs a plurality of tensors from different CNN layers that collectively preserve the features.
2D body skeleton detector driven by plurality of tensors to produce joint heat maps
A two-dimensional (2D) body skeleton detector to which the plurality of tensors are provided as input to obtain, for each of multiple joints, a heat map indicating a location of that joint, wherein the 2D body skeleton detector is provided with a second CNN architecture including a second plurality of CNN layers that accepts, as input, the plurality of tensors rather than the image.
Facial keypoints detector using at least one joint heat map
A facial keypoints detector to which at least one of the multiple heat maps produced by the 2D body skeleton detector for the multiple joints is provided as input to obtain locations of one or more facial keypoints.
Method receiving image and extracting human-related features as plurality of tensors for joint heat map inference
Receiving an image; extracting human-related image features from the image using a feature extractor connectable to a database comprising a dataset of reference images and provided with a first CNN architecture including a first plurality of CNN layers, wherein each CNN layer applies a convolutional operation using trained kernel weights to produce an output in the form of a tensor, and wherein the first CNN architecture outputs a plurality of tensors from different CNN layers that collectively preserve the human-related image features; determining human pose information by providing the plurality of tensors to a 2D body skeleton detector as input to obtain, for each of multiple joints, a heat map indicating a location of that joint.
Method providing joint heat maps to facial keypoints detector to obtain facial keypoint locations
Providing at least one of the multiple heat maps produced by the 2D body skeleton detector for the multiple joints to a facial keypoints detector as input to obtain locations of one or more facial keypoints.
Separate tensor-based pipeline determining pose and facial keypoints using 2D body skeleton detector and facial keypoints detector
Applying, to an image, a feature extractor that extracts human-related features in the form of a plurality of tensors, wherein the feature extractor is provided with a first CNN architecture having a first plurality of CNN layers, and wherein each CNN layer applies a convolutional operation to its input using trained kernel weights to produce, as output, one of the plurality of tensors; determining pose information of a human body by providing the plurality of tensors to a 2D body skeleton detector as input to obtain heat maps for multiple joints and providing at least one of the multiple heat maps to a facial keypoints detector as input to obtain locations of one or more facial keypoints.
Across the independent claims, the inventive concept is the use of a feature extractor that outputs a plurality of tensors from different CNN layers to drive a 2D body skeleton detector that produces per-joint heat maps, followed by a facial keypoints detector that obtains facial keypoint locations from at least one of those heat maps.
Stated Advantages
Documented Applications
No documented applications found
Interested in licensing this patent?