Systems and methods for training a statistical model to predict tissue characteristics for a pathology image
Inventors
Beck, Andrew H. • Khosla, Aditya
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-10650520-B1
Publication Date
2020-05-12
Expiration Date
Abstract
In some aspects, the described systems and methods provide for a method for training a statistical model to predict tissue characteristics for a pathology image. The method includes accessing annotated pathology images. Each of the images includes an annotation describing a tissue characteristic category for a portion of the image. A set of training patches and a corresponding set of annotations are defined using an annotated pathology image. Each of the training patches in the set includes values obtained from a respective subset of pixels in the annotated pathology image and is associated with a corresponding patch annotation determined based on an annotation associated with the respective subset of pixels. The statistical model is trained based on the set of training patches and the corresponding set of patch annotations. The trained statistical model is stored on at least one storage device.
Core Innovation
The invention provides a method for training a statistical model to predict tissue characteristics for a pathology image. The method accesses a plurality of annotated pathology images, each annotated image including at least one annotation describing one of a plurality of tissue characteristic categories for a portion of the image, and addresses learning from annotated tissue regions so that the statistical model generates tissue-category predictions for image portions.
A set of training patches and a corresponding set of annotations are defined using the at least one annotated pathology image. Each training patch includes values obtained from a respective subset of pixels in the annotated pathology image and is associated with a corresponding patch annotation determined based on an annotation associated with the respective subset of pixels. The statistical model is trained based on the training patches and their patch annotations, and the trained statistical model is stored on at least one storage device.
The disclosure further includes embodiments that incorporate sparse polygon/point annotations and patch definitions derived from pixel subsets. It also supports refinements in how training patches are constructed and balanced, including background category patch selection within a pixel-radius of annotations for non-background categories, resampling for data augmentation, interactive validation and retraining with hard negatives, and magnified images with a second statistical model whose outputs can be combined using a logical AND or logical OR operation.
Claims Coverage
The partial content includes two independent claims, a method and a system, that share the same core training workflow. Across the independent claims, multiple inventive features are specified or narrowed, including patch and annotation derivation from pixel subsets, training on patch annotations, optional user-feedback retraining, a convolutional neural network alignment constraint, and multi-model output combination.
Training patches and patch annotations from pixel subsets
Defining a set of training patches and a corresponding set of annotations using the at least one annotated pathology image, wherein each training patch includes values obtained from a respective subset of pixels and is associated with a corresponding patch annotation determined based on an annotation associated with the respective subset of pixels.
Training a statistical model for tissue characteristics from patch data
Training the statistical model based on the set of training patches and the corresponding set of patch annotations.
Storing the trained statistical model on storage device
Storing the trained statistical model on at least one storage device.
Interactive user-confirmed predicted annotation and annotation update loop
Using the trained statistical model to process unannotated portions of pathology images, predict an annotation, present the predicted annotation to a user, receive an indication of whether the predicted annotation is accurate, and annotate the images with the predicted annotations when confirmed accurate.
Feedback-driven retraining after incorrect predicted annotations
If the indication specifies that the predicted annotation is inaccurate, updating the predicted annotation, redefining a set of training patches using the plurality of annotated pathology images, retraining a statistical model using the redefined set of training patches and the respective annotations, and storing the retrained statistical model on a storage device.
Background-category training patch selection within a pixel-radius constraint
Defining a set of training patches such that the set includes at least one background-category training patch that lies within a pixel-radius of annotations for non-background categories.
Convolutional neural network alignment constraint with integer (N−K)/S
The statistical model is a convolutional neural network with multiple layers configured so that a layer with input size N*N, convolution filter size K, and stride S is aligned such that (N−K)/S is an integer.
Logical AND or logical OR combination of outputs from first and second models
Combining the outputs of a first and a second trained statistical model by applying a logical AND or logical OR operation.
The independent claims cover training a statistical model for tissue-category prediction from training patches derived from pixel subsets of annotated pathology images, associating each patch with patch annotations derived from the subset’s annotations, training the model on those patch and annotation pairs, and storing the trained model. Additional claim refinements explicitly include a user-confirmation and correction loop with retraining, a background-category patch inclusion constraint, a convolutional neural network alignment condition expressed by (N−K)/S being an integer, and logical AND/OR output combination for a second trained model.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?