Automated whole-slide image classification using deep learning
Inventors
Soans, Rajath Elias • Ianni, Julianna • Sankarapandian, Sivaramakrishnan
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11423678-B2
Publication Date
2022-08-23
Expiration Date
Abstract
Computer-implemented techniques for classifying a tissue specimen are presented. The techniques include obtaining an image of the tissue specimen; segmenting the image into a first plurality of segments; selecting a second plurality of segments that include at least one region of interest; applying an electronic convolutional neural network trained by a training corpus including a set of pluralities of tissue sample image segments, each of the pluralities of tissue sample image segments labeled according to one of a plurality of primary pathology classes, where the plurality of primary pathology classes consist of a plurality of majority primary pathology classes, where the plurality of majority primary pathology classes collectively include a majority of pathologies according to prevalence, and a class for tissue sample image segments not in the plurality of majority primary pathology classes, such that a primary pathology classification is output; and providing the primary pathology classification.
Core Innovation
The invention provides a computer-implemented method for classifying a tissue specimen by processing a computer readable image of the tissue specimen. The method includes segmenting the image into a first plurality of segments and selecting, from among the first plurality of segments, a second plurality of segments that include at least one region of interest. The selected segments are used for primary pathology classification by applying an electronic convolutional neural network trained by a training corpus of labeled tissue sample image segment pluralities.
The electronic convolutional neural network is trained on a set of pluralities of tissue sample image segments, where each plurality comprises image segments from within a same tissue sample image and is labeled according to one of a plurality of primary pathology classes. The plurality of primary pathology classes consist of a plurality of majority primary pathology classes and a class for tissue sample image segments not in the plurality of majority primary pathology classes. The majority primary pathology classes collectively comprise a majority of pathologies of a particular tissue type according to prevalence, and the method outputs and provides the primary pathology classification.
In related dependent implementations, the method supports secondary pathology classification by applying a trained classifier to segments of the tissue sample image to output and provide a secondary pathology classification based on pathology subclasses derived from the training corpus. The invention further refines primary prediction generation by applying the electronic convolutional neural network multiple times using different proper subsets of neurons, producing a plurality of predictions whose sigmoid function outputs are aggregated and selected using a mean-of-sigmoid rule and a maximum-mean selection.
Claims Coverage
The provided independent claims cover a computer-implemented method and a corresponding system that produce and provide a primary pathology classification using a CNN trained on labeled tissue-segment pluralities. Across the dependent claim refinements discussed, the claims include additional inventive features for prediction aggregation/selection, optional secondary pathology subclass classification, and thresholding tied to validation images at a selected confidence level.
Segmenting into segments and selecting regions of interest
Segmenting the image into a first plurality of segments; selecting, from among the first plurality of segments, a second plurality of segments that include at least one region of interest.
CNN trained on labeled tissue segment pluralities with majority and non-majority primary classes
Applying, to the second plurality of segments, an electronic convolutional neural network trained by a training corpus comprising a set of pluralities of tissue sample image segments, each plurality comprising image segments from within a same tissue sample image, each plurality labeled according to one of a plurality of primary pathology classes that consist of a plurality of majority primary pathology classes and a class for tissue sample image segments not in the plurality of majority primary pathology classes, where the plurality of majority primary pathology classes collectively comprise a majority of pathologies of a particular tissue type according to prevalence, whereby a primary pathology classification is output.
Providing the primary pathology classification
Providing the primary pathology classification.
Secondary pathology classification based on pathology subclasses
Applying a trained classifier to at least part of a second plurality of tissue sample image segments to output a secondary pathology classification based on pathology subclasses within the primary classification, and providing the secondary pathology classification.
Multi-run neuron-subset prediction aggregation with combined per-class selection
Applying the electronic convolutional neural network multiple times to the second plurality of segments using different proper subsets of neurons to generate a plurality of predictions; combining, within each primary pathology class, the plurality of predictions to form a combined prediction per class; and selecting a main combined prediction for the primary pathology classification.
Mean of sigmoid outputs and maximum-mean selection
Computing, for each primary pathology class, the mean of sigmoid function outputs for that class across the plurality of predictions and selecting the main combined prediction as the maximum mean.
Thresholding of maximum mean based on validation-image performance at a selected confidence level
Determining that the maximum mean exceeds a threshold value derived from the mean of sigmoid function outputs based on validation images of tissue specimens with known diagnoses across multiple primary pathology classes at a selected confidence level.
The independent claims center on segmenting tissue-specimen images, selecting regions of interest, and applying a trained convolutional neural network that outputs and provides a primary pathology classification using majority primary pathology classes together with a non-majority class. Dependent claim features further add optional secondary pathology subclass classification and refinements to multi-run prediction using proper neuron subsets, aggregation via mean-of-sigmoid outputs, maximum-mean selection, and thresholding informed by validation images at a selected confidence level.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?