Systems and methods to train a cell object detector
Inventors
Zeineh, Jack • Prastawa, Marcel • Fernandez, Gerardo
Assignees
Icahn School of Medicine at Mount Sinai
Publication Number
US-11966842-B2
Publication Date
2024-04-23
Expiration Date
2040-05-22
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
Systems and methods to train a cell object detector are described.
Core Innovation
The invention provides systems and methods to train a cell object detector that can detect rare, hard to consistently identify, or difficult to detect objects within tissue images. The approach includes a computer-implemented method that uses training data consisting of two sets: one set with images and labels identifying the particular cell objects of interest, and another set with images and labels identifying look-alike objects similar in appearance to the particular cell objects. A first neural network is trained to detect candidate objects encompassing both the particular object type and look-alike objects.
A second neural network is then trained to identify the particular type of cell object within the candidate objects detected by the first neural network, enabling the system to distinguish between the actual cell objects and look-alike objects. The training method can include leveraging labeled sets that further identify borderline objects and normal or background objects similar in appearance. The system also employs a sampling technique for the second neural network involving receiving an initial outline of a potential object, moving a sampling window across each pixel within the outline, applying transformations on the sampled image data, and using this for training.
Claims Coverage
The patent contains three independent claims covering a computer-implemented method, a non-transitory computer-readable medium, and a system, each relating to training or using a two-stage neural network machine-learning model for cell object detection.
Training a dual-stage neural network with augmented labeled data
Obtaining training data comprising a first set of cell images with labels identifying particular cell objects and a second set with labels identifying look-alike objects; training a first neural network to detect candidate objects including the particular and look-alike objects by providing both data sets as input, obtaining output labels, comparing them with the training labels, and adjusting network parameters accordingly.
Training a second neural network for fine discrimination
Training a second neural network to identify the particular cell object by inputting candidate objects detected by the first neural network and their associated labels; after training, this network can distinguish between particular cell objects and look-alike objects.
Enhanced sample preparation for second neural network training
Training the second neural network further includes receiving an initial outline of a potential object, moving a sampling window across each pixel within the outline, obtaining corresponding image data, performing transformations (like flips, rotations, translations) on this data to create training images, and providing these for training.
Machine-readable medium with instructions for detection
A non-transitory computer-readable medium containing instructions to perform the training and detection process using the two neural networks as described, including outputting an indication of whether the particular cell object is present in a given cell image.
System configured to perform dual-stage detection
A system with processors executing software that provides a cell image to a trained two-stage neural network model trained using the augmented labeled data, detects candidate objects, discriminates between particular cell objects and look-alikes, and outputs a presence indication of the particular cell object.
The claims collectively cover a two-stage cell object detection approach using augmented labeled training data including look-alikes, where a first neural network detects candidate objects and a second neural network discriminates true objects from look-alikes. The method includes specialized sampling and transformation techniques for training, and is embodied as a computer-implemented method, system, and medium.
Stated Advantages
Provides more accurate and robust model training compared to conventional techniques.
Reduces the time required of clinical experts by focusing expert review on difficult cases such as look-alike and borderline objects.
Enables a two-stage detection process where the first stage achieves high recall with lower precision, allowing efficient coarse detection, and the second stage provides fine discrimination improving precision.
The sampling technique for the second neural network enhances training by generating a richer set of training exemplars through pixel-level sampling within object outlines and applying transformations.
Documented Applications
Training machine learning models to detect mitotic figures in tissue slide images.
Detection of epithelial regions in tissue images.
Biomedical image processing, specifically analyzing tissue samples to detect rare or difficult-to-identify cell objects.
General application in biomedical imaging for detection of cell or non-cell object types that are difficult to distinguish or clinically important, such as neutrophils in colon tissue, tumor cells in lymph nodes, lymphocytes in tumors, or Lewy neurites in brain tissue.
Interested in licensing this patent?