Method for managing annotation job, apparatus and system supporting the same

Inventors

Lee, Kyoung Won • Paeng, Kyung Hyun

Assignees

Lunit Inc

Interested in licensing this patent?

MTEC can help explore whether this patent might be available for licensing for your application.

Publication Number

US-12217853-B2

Patent

Publication Date

2025-02-04

Expiration Date


Abstract

A computing device obtains information about a medical slide image, and determines a dataset type of the medical slide image and a panel of the medical slide image. The computing device assigns to an annotator account, an annotation job defined by at least the medical slide image, the determined dataset type, an annotation task, and a patch that is a partial area of the medical slide image. The annotation task includes the determined panel, and the panel is designated as one of a plurality of panels including a cell panel, a tissue panel, and a structure panel. The dataset type indicates a use of the medical slide image and is designated as one of a plurality of uses including a training use of a medical learning model and a validation use of the machine learning model.

Core Innovation

The invention relates to an annotation job management system for medical slide images in which an annotation is performed on a partial area displayed within a medical slide image. A terminal displays an annotation tool including a plurality of selectable tools respectively corresponding to a plurality of annotations and an area in which a partial area of the medical slide image is displayed, and receives an input of an annotator through the annotation tool to perform an annotation on the partial area based on the input of the annotator.

The annotation is used for a machine learning model to perform a target task, and the target task includes classifying at least one of a plurality of classes including a class related to a cell and a class related to a tissue. In a management apparatus workflow, an annotation job for a medical slide image is transmitted to a terminal of an annotator, and a result of the annotation job performed through the annotation tool is received via a communication interface.

To manage annotation quality, the management apparatus performs validation on the result of the annotation job and records the validation result as an evaluation result of the annotator. Validation may include comparing against a second machine learning model output, and if validation indicates that an annotation must be redone, the management apparatus selects another annotator and transmits a new annotation job to that annotator’s terminal. The annotation output includes label information tagged to the partial area, and the partial area is highlighted using a rectangle shape in a medical slice image.

Claims Coverage

The independent claims include 3 claim sets, covering a terminal, a management apparatus, and a terminal method for annotating a partial area of a medical slide image with selectable tools for cell and tissue classification used by a machine learning model.

Terminal with partial area annotation tool for cell and tissue classification

A terminal that displays an annotation tool including a plurality of selectable tools respectively corresponding to a plurality of annotations and an area in which a partial area of the medical slide image is displayed; receives an input of an annotator through the annotation tool; and performs an annotation on the partial area based on the input of the annotator, where the annotation is used for a machine learning model to perform a target task including classifying at least one of a plurality of classes including a class related to a cell and a class related to a tissue.

Management apparatus for annotation job transmission, result reception, validation, and cell and tissue classification

A management apparatus that transmits, via a communication interface, an annotation job for a medical slide image to a terminal of an annotator; receives, via the communication interface, a result of the annotation job performed through an annotation tool from the terminal where the annotation tool includes a plurality of selectable tools respectively corresponding to a plurality of annotations and an area in which a partial area of the medical slide image is displayed; and uses the result of the annotation job for a first machine learning model to perform a target task including classifying at least one of a plurality of classes including a class related to a cell and a class related to a tissue.

Terminal method for partial area annotation tool for cell and tissue classification

A method performed by a terminal comprising displaying an annotation tool including a plurality of selectable tools respectively corresponding to a plurality of annotations and an area in which a partial area of the medical slide image is displayed; receiving an input of an annotator through the annotation tool; and performing an annotation on the partial area based on the input of the annotator, where the annotation is used for a machine learning model to perform a target task including classifying at least one of a plurality of classes including a class related to a cell and a class related to a tissue.

Across the independent claims, the core coverage centers on annotating a partial area of a medical slide image using an annotation tool with selectable tools, using the produced annotation results for machine learning-based target task classification with classes related to cells and tissues, and, in the management apparatus, managing the annotation job workflow including validation and optionally selecting another annotator for redone annotation.

Stated Advantages

Validation of annotation results and recording an evaluation result of the annotator.

Potentially redoing annotation by selecting another annotator based on validation.

Documented Applications

Annotation job management for medical slide image annotation where annotations are used for machine learning model target task classification including classifying cell-related and tissue-related classes.

JOIN OUR MAILING LIST

Stay Connected with MTEC

Keep up with active and upcoming solicitations, MTEC news and other valuable information.