Systems and methods for predicting tissue characteristics for a pathology image using a statistical model
Inventors
Beck, Andrew H. • Khosla, Aditya
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11080855-B1
Publication Date
2021-08-03
Expiration Date
Abstract
In some aspects, the described systems and methods provide for a method for predicting tissue characteristics for a pathology image. A statistical model trained on multiple annotated pathology images is used. Each of the training pathology images includes an annotation describing tissue characteristics for one or more portions of the image. The method includes accessing a pathology image for predicting tissue characteristics. A trained statistical model is retrieved from a storage device. A set of patches is defined from the pathology image. Each of the patches in the set includes a subset of pixels from the corresponding pathology image. The set of patches is processed using the trained statistical model to predict respective annotations for each patch in the set. The predicted annotations are stored on the storage device.
Core Innovation
The invention provides a method and system for predicting tissue characteristics for a pathology image using a trained statistical model. The model is trained on a set of training patches derived from a plurality of annotated pathology images, where each annotated pathology image includes at least one annotation describing tissue characteristics for one or more portions of the image. Each training patch includes one or more values obtained from a subset of pixels in an annotated pathology image and is associated with a corresponding patch annotation determined based on the annotation associated with the subset of pixels.
For an input pathology image different from the annotated pathology images used for training, the method defines a set of patches, where each patch includes a subset of pixels from the input pathology image. The method processes the set of patches using the trained statistical model to predict one or more annotations for each patch, and stores the predicted one or more annotations for each patch on at least one storage device. The same process is implemented by a computer hardware processor using processor-executable instructions stored on at least one non-transitory computer-readable storage medium.
In embodiments, the trained statistical model is a convolutional neural network including a plurality of layers where at least one layer is aligned such that (N−K)/S is an integer, with N as a size of each input dimension, K as a size of a convolution filter, and S as a size of a stride; additionally, no padding is applied to an output of any layer. The training patch dimension is larger than a corresponding dimension of input to the trained statistical model, and additional pixels in the training patch are included when randomly resampling the training patch for data augmentation.
Claims Coverage
The provided set includes four independent claims. Across these independent claims, the inventive coverage centers on patch-based tissue-characteristic prediction using a trained statistical model trained from pixel-subset-derived training patches and patch annotations, with architectural constraints for convolutional neural network layers and storage of predicted patch annotations.
Patch-based tissue-characteristic prediction from pixel-subset training annotations
A method for predicting tissue characteristics for a pathology image using a trained statistical model trained on a plurality of annotated pathology images, wherein each image includes at least one annotation describing tissue characteristics for one or more portions of the image; accessing a pathology image; retrieving a trained statistical model from at least one storage device; defining a set of patches from the pathology image, wherein each patch includes a subset of pixels from the pathology image; processing the set of patches using the trained statistical model to predict one or more annotations for each patch; and storing the predicted one or more annotations for each patch.
Training patches formed from subsets of pixels and corresponding patch annotations
The trained statistical model is trained on a set of training patches and a corresponding set of patch annotations derived from at least one annotated pathology image, wherein each training patch includes one or more values obtained from a subset of pixels in the annotated pathology image and is associated with a corresponding patch annotation determined based on an annotation associated with the subset of pixels.
Random resampling data augmentation using larger training patch dimensions
At least one dimension of each training patch is larger than a corresponding dimension of input to the trained statistical model, and additional pixels in the training patch are included when randomly resampling the training patch for data augmentation.
Convolutional neural network alignment constraint (N−K)/S integer
The trained statistical model comprises a convolutional neural network including a plurality of layers, wherein at least one layer is aligned such that (N−K)/S is an integer, wherein N represents a size of each input dimension of the at least one layer, K represents a size of a convolution filter of the at least one layer, and S represents a size of a stride of the at least one layer.
No padding on outputs of convolutional neural network layers
A convolutional neural network with a plurality of layers is used, wherein no padding is applied to an output of any layer.
System implementation with processor and non-transitory storage medium
A system for predicting tissue characteristics for a pathology image using a trained statistical model trained on a plurality of annotated pathology images, comprising at least one computer hardware processor and at least one non-transitory computer-readable storage medium storing processor-executable instructions that cause the processor to access a pathology image, retrieve the trained statistical model, define a set of patches, process the patches to predict one or more annotations for each patch, and store the predicted one or more annotations.
The independent claims cover predicting tissue characteristics by patching an input pathology image, applying a trained statistical model trained on pixel-subset-derived training patches with corresponding patch annotations, and storing predicted patch annotations. System-level coverage mirrors the method steps using a processor and non-transitory storage medium. Additional dependent coverage highlights CNN-specific constraints such as no padding and (N−K)/S integer alignment.
Stated Advantages
Documented Applications
Predicting tissue-characteristic categories/annotations for unlabeled regions of a pathology image and generating stored predicted annotations per patch.
Using predicted histological/spatial features derived from predicted annotations to train prognostic models for entities including survival time and drug response.
Evaluating performance/specificity using clinical-trial based analyses, including hazard ratio, 95% confidence interval, and Kaplan-Meier curves, with subset survival analysis.
Predicting entities such as patient phenotype/molecular characteristics, mutational burden, tumor molecular characteristics, transcriptomic features, and protein expression features.
Interested in licensing this patent?