Multimodal machine learning based clinical predictor
Inventors
VLADIMIROVA, Antoaneta Petkova • PANDIT, Yogesh P. • SHARMA, Vishakha • Klingler, Tod M. • SINGHAL, Hari
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12586685-B2
Publication Date
2026-03-24
Expiration Date
Abstract
Methods and systems for performing a clinical prediction are provided. In one example, the method comprises: receiving first molecular data of a patient, the first molecular data including at least gene expressions of the patient; receiving first biopsy image data of the patient; processing, using a machine learning model, the first molecular data and the first biopsy image data to perform a clinical prediction of the patient's response to a treatment, wherein the machine learning model is generated or updated based on second molecular data including at least gene expressions and second biopsy image data of a plurality of patients; and generating an output of the clinical prediction.
Core Innovation
A method receives first molecular sequencing data and first biopsy image data of a patient, and also receives second molecular sequencing data and second biopsy image data of a plurality of patients. The method executes a multi-stage machine learning model with the received first biopsy image data, including identification of at least one of a shape or a boundary from pixel data of biopsy image data, and a recognition operation using the identified shape or boundary to perform tissue segmentation, cell extraction, or cell component extraction.
The multi-stage machine learning model computes biopsy feature data based on at least one of the tissue and tissue type identified by tissue segmentation, a count of a cell extracted by cell extraction, a density of the cell extracted by cell extraction, or a cell component identified by cell component extraction. The multi-stage machine learning model is trained with the second biopsy image data of a plurality of patients.
A multi-modal machine learning model is then trained using the second molecular sequencing data and the biopsy image feature data computed by applying the multi-stage machine learning model to the second biopsy-image data, to output a clinical prediction of a treatment. The multi-modal machine learning model comprises one or more of a random forest model, a support vector machine model, and/or a neural network model, and is executed with input of the first molecular data and biopsy feature data computed by applying the multi-stage machine learning model to the first biopsy image data.
Claims Coverage
The document provides three independent claims (method, system, and non-transitory computer-readable medium). Across these claims, the inventive coverage centers on a multi-stage biopsy image feature extraction pipeline combined with molecular sequencing data in a trained multi-modal model to output a clinical prediction of a treatment.
Multi-stage biopsy feature extraction from pixel data
Executing a multi-stage machine learning model with received biopsy image data, including identification of at least one of a shape or a boundary from pixel data; recognition using the identified shape or boundary comprising tissue segmentation, cell extraction, or cell component extraction; and computing biopsy feature data based on tissue and tissue type, a count of a cell, a density of the cell, or a cell component.
Training the multi-modal clinical prediction model with molecular sequencing and biopsy feature data
Training a multi-modal machine learning model with molecular sequencing data and biopsy image feature data computed by applying the multi-stage machine learning model to biopsy-image data, to output a clinical prediction of a treatment, where the multi-modal machine learning model comprises one or more of a random forest model, a support vector machine model, and/or a neural network model.
Applying the multi-modal model to output a clinical prediction from patient molecular and computed biopsy features
Executing the multi-modal machine learning model with input of molecular data and input of biopsy feature data computed by applying the multi-stage machine learning model to biopsy image data, to generate an output of a clinical prediction of a treatment.
The independent claims collectively cover multi-stage biopsy image processing to compute biopsy feature data from pixel-level shape or boundary recognition, training a multi-modal model with molecular sequencing data and computed biopsy features, and applying the trained model to produce a clinical prediction of a treatment.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?