Virtual staining for tissue slide images
Inventors
Stumpe, Martin • Nelson, Philip • Peng, Lily
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12367691-B2
Publication Date
2025-07-22
Expiration Date
Abstract
A machine learning predictor model is trained to generate a prediction of the appearance of a tissue sample stained with a special stain such as an IHC stain from an input image that is either unstained or stained with H&E. Training data takes the form of thousands of pairs of precisely aligned images, one of which is an image of a tissue specimen stained with H&E or unstained, and the other of which is an image of the tissue specimen stained with the special stain. The model can be trained to predict special stain images for a multitude of different tissue types and special stain types, in use, an input image, e.g., an H&E image of a given tissue specimen at a particular magnification level is provided to the model and the model generates a prediction of the appearance of the tissue specimen as if it were stained with the special stain. The predicted image is provided to a user and displayed, e.g., on a pathology workstation.
Core Innovation
The described invention performs virtual staining of a tissue specimen by obtaining an input red-green-blue image and supplying the input RGB image to a machine learning predictor model. The model is trained from a multitude of pairs of aligned RGB images of tissue specimens, where one image of each pair comprises the tissue specimen stained with the special stain. After supplying the input RGB image, the predictor model generates a predicted RGB image of the tissue specimen stained with the special stain and outputs the predicted RGB image as a virtual RGB image for viewing as a special-stained appearance.
The disclosed approach is applied to whole-slide imaging contexts by generating virtual special-stain images from input H&E or unstained whole-slide RGB images at various magnifications. The predicted virtual special-stain RGB images can be displayed for pathologists and overlaid onto H&E image views, with image alignment/registration and masking/removing edge artifacts described in the training data generation.
Claims Coverage
The document provides three independent claim sets that cover a method, a system, and a non-transitory computer-readable medium implementing the same virtual special-stain prediction workflow using a machine learning predictor trained on aligned RGB image pairs. Across the independent claim sets, the inventive features center on predicting special-stain RGB images and outputting them as virtual RGB images.
Virtual special-stain RGB image prediction from input RGB
Obtaining an input red-green-blue image of a tissue specimen, supplying the input RGB image to a machine learning predictor model trained from a multitude of pairs of aligned RGB images of tissue specimens, generating a predicted RGB image of the tissue specimen stained with the special stain, and outputting the predicted RGB image as a virtual RGB image.
Training from aligned RGB image pairs with special-stained RGB target
Training the machine learning predictor model from a multitude of pairs of aligned RGB images of tissue specimens in which one image of the pair comprises an RGB image of a tissue specimen stained with the special stain.
Virtual special-stain output as a virtual RGB image
Outputting the predicted RGB image as a virtual RGB image representing the tissue specimen stained with the special stain.
Across the independent claims, the core coverage is the same: use a machine learning predictor trained on aligned RGB image pairs where one image shows a tissue specimen stained with a special stain, predict the special-stain RGB appearance from an input RGB image of a tissue specimen, and output the result as a virtual RGB image.
Stated Advantages
Cost reduction.
Reduced access/latency.
Mitigation of tissue scarcity.
Shortening turnaround time.
Explainability using overlay.
Documented Applications
Previewing multiple immunohistochemical stain outputs, including HER2, PIN4/prostate triple stain, and CD3, for display to pathologists.
Integrating virtual special-stain outputs into pathology workflows and reports.
Interested in licensing this patent?