Method and system for machine learning classification based on structure or material segmentation in an image
Inventors
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11263497-B2
Publication Date
2022-03-01
Expiration Date
Abstract
A system and method for classifying a structure or material in an image of a subject. The system comprises: a segmenter configured to form one or more segmentations of a structure or material in an image and generate from the segmentations one or more segmentation maps of the image including categorizations of pixels or voxels of the segmentation maps assigned from one or more respective predefined sets of categories; a classifier that implements a classification machine learning model configured to generate, based on the segmentations maps, one or more classifications and to assign to the classifications respective scores indicative of a likelihood that the structure or material, or the subject, falls into the respective classifications; and an output for outputting a result indicative of the classifications and scores.
Core Innovation
A computer-vision and machine-learning system classifies a structure or material in an image of a subject by using segmentation maps that correspond to structures or materials in the image, together with annotated non-image data. A segmenter generates annotated segmentation maps from one or more segmentations formed by segmenting the image into one or more segmentations that correspond to respective structures or materials, and predefined category sets represent pixel or voxel category labels in the segmentation maps.
A classifier consumes the segmentation maps and non-image data to output one or more classifications and scores indicative of a likelihood. The training process implements and trains a classification machine learning model by initializing parameters, updating parameters via a learning algorithm on training data, testing on a testing set, and evaluating whether satisfactory performance is achieved; when performance is not satisfactory, the training steps are repeated.
The training and segmentation approach includes partitioning annotated segmentation maps and annotated non-image data into training and testing sets, and may use additional data partitions for tuning and learning procedure study. The segmentation maps can include structure segmentation maps, material segmentation maps, and optionally abnormality segmentation maps using predefined categories, with optional structure, material, and abnormality segmentation machine learning models.
Claims Coverage
The document includes two independent claims, a computer-implemented method and a system, that share the same core inventive workflow. Across the independent claims, the key inventive features cover training a classification machine learning model using annotated segmentation maps together with annotated non-image data, iteratively updating/testing/evaluating until satisfactory performance, and outputting the trained model for deployment or flagging it as trained.
Training using annotated segmentation maps and annotated non-image data with training and testing sets
Dividing annotated segmentation maps of one or more images and annotated non-image data pertaining to the subject into a training set and a testing set, wherein the annotated segmentation maps are generated from one or more segmentations that correspond to respective structures or materials in the respective image, and wherein each of the training and testing sets includes at least one of the annotated segmentation maps of the image and at least some annotated non-image data.
Iterative classification model parameter updating, testing, and satisfactory-performance evaluation
Implementing a classification machine learning model for classifying a structure or material in an image of a subject, including initializing parameters of the classification machine learning model; updating the parameters of the classification machine learning model by running a learning algorithm on the training data; testing the classification machine learning model on the testing set; evaluating whether the classification machine learning model has satisfactory performance; repeating the implementing, updating, testing, and evaluating steps when the performance is found not to be satisfactory.
Outputting the trained classification model for deployment or flagging as trained
Outputting the classification machine learning model for deployment as a trained classification machine learning model, or flagging the classification machine learning model as a trained classification machine learning model.
Segmentation maps with predefined pixel or voxel category labels
Each of the annotated segmentation maps corresponding to a respective image represents the image, and the segmentation maps are formed by segmenting each of the one or more images into one or more segmentations that correspond to respective structures or materials in the respective image.
System processor configured to carry out training workflow
A system for training a classification machine learning model comprising a processor configured to divide annotated segmentation maps and annotated non-image data into a training set and testing set, implement the classification machine learning model with initialization, update parameters with a learning algorithm on training data, test on the testing set, evaluate satisfactory performance, repeat when not satisfactory, and output the trained model for deployment or flag it as trained.
Both independent claims define training of a classification machine learning model using annotated segmentation maps and annotated non-image data, iterative updating/testing/evaluating until satisfactory performance, and outputting or flagging the trained model. The segmentation maps correspond to structures or materials in the image and represent the image via annotated pixel or voxel category labeling.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Classifying a structure or material in medical images for purposes such as bone fracture risk across timeframes, disease progression, treatment efficacy, and medical conditions, with example outputs described for HRpQCT.
Interested in licensing this patent?