Image processing method for displaying cells of a plurality of overall images
Inventors
MARZAHL, Christian • Gerlach, Stefan • VOIGT, Joern • Kroeger, Christine
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-11854191-B2
Publication Date
2023-12-26
Expiration Date
Abstract
An image processing method is provided for displaying cells from a plurality of pathology images or overall images. A respective overall image represents a respective patient tissue sample or a respective patient cell sample. The method includes the steps of: providing the overall images, detecting individual cell images in the overall images by means of a computer-assisted algorithm, determining classification data by means of the computer-assisted algorithm, wherein the classification data indicate a respective unique mapping of a respective detected cell image to one of a plurality of cell classes, and wherein the classification data further have a respective measure of confidence in respect of the respective unique mapping, generating respective class images for the respective cell classes, wherein a class image of a cell class reproduces the cell images mapped to the cell class in a regular arrangement and with a predetermined order, and wherein further the order of the mapped cell images is chosen on the basis of the measures of confidence of the mapped cell images, and further, displaying a portion of at least one class image.
Core Innovation
The invention relates to an image processing system for displaying cells of a plurality of overall images, each overall image representing a respective patient tissue sample or a respective patient cell sample. The system includes a server and a client in communication via a data network, and the server provides the overall images and detects individual cell images in the overall images by means of a computer-assisted algorithm. The server determines classification data indicating a unique mapping of each detected cell image to one of a plurality of cell classes, together with a measure of confidence for the respective unique mapping.
Based on the classification data, the server generates class images for the respective cell classes. A class image reproduces the cell images mapped to the cell class in a regular arrangement and with a predetermined order, and the predetermined order is chosen on the basis of the measures of confidence. For each class image, the server generates an annotation data record indicating mappings of the respective cell images to the corresponding cell class and respective local positions of the respective cell images within the class images.
To support pathologist verification, the server receives, from the client, a request for a portion of a class image and transmits the requested portion and at least one partial annotation data record corresponding to the portion. The client temporarily stores the partial annotation data record, generates respective optical markings for respective cell images of the portion based on the partial annotation data record, and displays the portion together with the associated optical markings. The system further supports user-driven remapping, synchronization requests, and multiple portions, with client-side markings and server-side updates of annotation and classification data.
Claims Coverage
The provided material includes three independent claims describing a server-client system, a server-executed method, and a client-executed method. Across these independent claims, the inventive features include confidence-based ordering in class images, unique cell-to-class mapping, use of annotation and partial annotation data with local positions, client request and transmission of class-image portions, client-side optical markings, and updating annotation data records based on user selection and modified cell class, including synchronization in server-side variants.
Confidence-based ordering in class images
A class image reproduces the cell images mapped to a cell class in a regular arrangement and with a predetermined order, wherein the order of the mapped cell images is chosen on the basis of the measures of confidence of the mapped cell images.
Unique cell-to-class mapping with confidence
Classification data indicate a respective unique mapping of a respective detected cell image to one of a plurality of cell classes, wherein the classification data further have a respective measure of confidence in respect of the respective unique mapping.
Annotation record with mappings and local positions
The server generates an annotation data record that indicates, for each class image, respective mappings of the respective cell images to a cell class corresponding to the class image and respective local positions of the respective cell images within the class images.
Portion-based class image transmission with partial annotations
The server receives, from the client, a request for a portion of a class image and transmits, to the client, the portion and at least one partial annotation data record corresponding to the portion, the partial annotation data record indicating respective mappings of the respective cell images to a cell class corresponding to the class image and respective local positions of the respective cell images within the class image.
Client-side optical markings from partial annotation data
The client generates respective optical markings for respective cell images of the portion on the basis of at least the partial annotation data record, wherein the respective optical markings indicate respective mappings of the respective cell images to a cell class corresponding to the class image, and displays the portion and the associated optical markings on a display unit.
User selection remapping updating annotation data record
The server receives, from the client, information indicating a selected cell image and a modified cell class, and modifies the annotation data record in correspondence with the selected cell image and the modified cell class; in the client method, the client receives the portion and partial annotation data, temporarily stores the partial annotation data record, generates optical markings, displays the portion, and in a modification workflow receives user input indicating a selected cell image and a modified cell class and generates and displays optical markings reflecting the modified mapping.
Synchronization of classification data with modified annotation data record
The server receives, from the client, a synchronization request and synchronizes classification data with a modified annotation data record.
The independent claims converge on a workflow where individual cell images are uniquely mapped to cell classes with associated confidence measures; class images are generated with a regular arrangement and confidence-based predetermined order; annotation data records and partial annotation data records provide mappings and local positions for portion-based retrieval; the client generates optical markings and displays portions for verification; and user-driven remapping updates corresponding annotation data, with synchronization in server-side variants.
Stated Advantages
Efficient pathologist verification by displaying portions of class images with associated optical markings.
Documented Applications
Displaying cells from a plurality of overall pathology images representing patient tissue samples or patient cell samples.
Class examples discussed include macrophage, lymphocyte, eosinophil, and mitosis/non-mitosis contexts.
Interested in licensing this patent?