Histopathology classification through machine self-learning of “tissue fingerprints”

Inventors

Agus, David • Ruderman, Daniel • Rawat, Rishi • SHA, Fei • Shibata, Darryl

Assignees

University of Southern California USC

Interested in licensing this patent?

MTEC can help explore whether this patent might be available for licensing for your application.

Publication Number

US-12354012-B2

Patent

Publication Date

2025-07-08

Expiration Date


Abstract

Histologic classification of pathology specimens through machine learning is a nascent field which offers tremendous potential to improve cancer medicine. Its utility has been limited, in part because of differences in tissue preparation and the relative paucity of well-annotated images. We introduce tissue recognition, an unsupervised learning problem analogous to human face recognition, in which the goal is to identify individual tumors using a learned set of histologic features. This feature set is the “tissue fingerprint.” Because only specimen identities are matched to fingerprints, constructing an algorithm for producing them is a self-learning task that does not need image metadata annotations. Here, we provide an algorithm for self-learning tissue fingerprints, that, in conjunction with color normalization, can match hematoxylin and eosin stained tissues to one of 104 patients with 93% accuracy. We applied this identification network's internal representation as a tissue fingerprint for use in predicting the molecular status of an individual tumor (breast cancer clinical estrogen receptor (ER) status). We describe a fingerprint-based classifier that predicts ER status from whole-slides with high accuracy (AUROC=0.90), and is an improvement over traditional transfer learning approaches. The use of tissue fingerprinting for digital pathology as a concise but meaningful histopathologic image representation enables a new range of machine learning algorithms leading to increased information for clinical decision making in patient management.

Core Innovation

The invention relates to an unsupervised/self-learning histopathology framework that learns a tissue fingerprint from unannotated stained histologic digital images, including H&E-stained tissue. A self-learning neural network is trained to form a tissue fingerprinting function that maps digital images of unannotated stained tissue samples to a vector of numbers indicative of histologic patterns within the digital images.

Learning occurs by dividing multiple images of characterized or uncharacterized histologic samples into halves and optimizing the network so that the tissue fingerprint correctly pairs the halves. The optimization procedure minimizes an objective loss function that includes a first component promoting learning of the tissue fingerprinting function used to match digital image patches from the same histologic sample while distinguishing those patches from multiple histologic samples.

An optional second component promotes learning that is invariant to sources of pathology artifacts encountered during tissue processing, so that the learned tissue fingerprint remains robust to such artifacts. The trained model is used to determine a status of a test stained tissue sample by obtaining a sample digital image, processing the sample digital image using the trained machine learning model to determine a predicted tissue fingerprint, and matching the predicted tissue fingerprint to tissue fingerprints generated from digital images of annotated histologic samples.

The described framework supports predicting biomarker status from whole-slide images via patch-level fingerprinting and pooled scoring, and includes interpretability outputs such as heatmaps and low-dimensional embedding to show clusters related to ER status.

Claims Coverage

The partial document provides two independent claims. They are directed to determining a status of an unknown histologic sample using a trained self-learning neural network, and to training such a self-learning neural network to produce a tissue fingerprint feature vector. The claims center on tissue fingerprint learning via paired halves, an objective loss with within-sample matching and between-sample discrimination, and an optional pathology-artifact invariance component.

Self-learning tissue fingerprinting from unannotated stained images

A self-learning neural network learns a tissue fingerprinting function by receiving digital images of a histologic sample as inputs and outputting a tissue fingerprint comprising a feature vector indicative of histologic patterns within the digital image, where the function maps digital images of unannotated stained tissue samples to a tissue fingerprint.

Halves pairing learned through objective loss minimization

The untrained machine learning device learns the tissue fingerprinting function by dividing multiple images of characterized or uncharacterized histologic samples into halves and learning a tissue fingerprint that correctly pairs said halves through an optimization procedure that minimizes an objective loss function.

Within-sample patch matching and between-sample discrimination objective

The objective loss function includes a first component promoting learning of the tissue fingerprinting function that can be used to match digital image patches from the same histologic sample but distinguish said digital image patches from multiple histologic samples.

Optional pathology-artifact invariance objective component

The objective loss function optionally includes a second component promoting learning of the tissue fingerprinting function that is invariant to sources of pathology artifacts encountered during tissue processing.

Status determination by predicted fingerprint matching to annotated fingerprints

A test stained tissue sample status is predicted by obtaining a sample digital image, processing the sample digital image using the trained machine learning model to determine a predicted tissue fingerprint, and matching the predicted tissue fingerprint to a set of tissue fingerprints generated from digital images of a set of annotated histologic samples.

Training method using unannotated samples to output tissue fingerprint feature vector

A method for training a machine learning model comprising a self-learning neural network includes obtaining a plurality of digital images of a plurality of unannotated stained tissue samples obtained or derived from a plurality of subjects and training the self-learning neural network to provide the digital images as inputs to a tissue fingerprinting function that outputs a tissue fingerprint feature vector indicative of histologic patterns.

Across the two independent claims, the inventive concept is learning a tissue fingerprint feature vector from unannotated stained images via pairing of halves and optimization of an objective loss that supports same-sample matching and between-sample discrimination, with an optional component to enforce invariance to sources of pathology artifacts. The learned tissue fingerprinting function is then used by predicting a tissue fingerprint for a test sample and matching it to fingerprints generated from annotated histologic samples to determine an unknown histologic sample status.

Stated Advantages

Learned tissue fingerprints enable matching digital image patches from the same histologic sample while distinguishing patches from multiple histologic samples.

An optional objective component promotes invariance to sources of pathology artifacts encountered during tissue processing.

Predicted tissue fingerprints can be matched to tissue fingerprints generated from annotated histologic samples to determine a status of an unknown histologic sample.

Documented Applications

Predicting biomarker status from stained histologic samples, including predicting estrogen receptor (ER) status and PR/HER2 from whole-slide images using patch-level fingerprinting and pooled scoring.

JOIN OUR MAILING LIST

Stay Connected with MTEC

Keep up with active and upcoming solicitations, MTEC news and other valuable information.