Methods, systems, and media for simultaneously monitoring colonoscopic video quality and detecting polyps in colonoscopy
Inventors
Liang, Jianming • Tajbakhsh, Nima
Assignees
Publication Number
US-10861151-B2
Publication Date
2020-12-08
Expiration Date
2036-08-08
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
Mechanisms for simultaneously monitoring colonoscopic video quality and detecting polyps in colonoscopy are provided. In some embodiments, the mechanisms can include a quality monitoring system that uses a first trained classifier to monitor image frames from a colonoscopic video to determine which image frames are informative frames and which image frames are non-informative frames. The informative image frames can be passed to an automatic polyp detection system that uses a second trained classifier to localize and identify whether a polyp or any other suitable object is present in one or more of the informative image frames.
Core Innovation
The invention provides mechanisms for simultaneously monitoring the quality of colonoscopic video and automatically detecting polyps during a colonoscopy procedure. A quality monitoring system employs a first trained classifier to assess each frame of a colonoscopic video, generating an informativeness score that indicates whether an image frame is informative (e.g., in-focus with high information content) or non-informative (e.g., blurred, dark, or containing artifacts such as bubbles or light reflections). Frames identified as informative are then routed to an automatic polyp detection system.
The automatic polyp detection system applies a second trained classifier to informative image frames in order to localize and identify whether a polyp or similar object is present. The system can further process informative image frames with additional classifiers to determine polyp edges, generate edge maps, and use machine learning techniques, such as random forest classifiers and convolutional neural networks, for the detailed identification and classification of polyps. These steps can include extracting features, clustering them using a bag of visual words model, applying Hough transforms to edge maps, and classifying image patches around polyp candidates.
The problem addressed by this invention is the high rate of polyp miss during colonoscopic procedures, which has been attributed to variable diligence and navigational skills of medical professionals and resulting in poor video quality and reduced visualization. Previous computer-aided detection methods primarily relied on texture or shape features, which have proven insufficient due to variability in polyp appearance and the requirement for well-focused images. The disclosed methods overcome these deficiencies by integrating real-time video quality assessment and polyp detection, aiming to reduce polyp miss rates and improve patient outcomes.
Claims Coverage
The patent includes multiple independent claims covering key inventive features in methods, systems, and computer-readable media for simultaneous video quality assessment and polyp detection in colonoscopic images.
Simultaneous real-time assessment of colonoscopic video quality and polyp detection
A method in which, during a colonoscopy procedure, a hardware processor receives a plurality of image frames in real time and applies a first trained classifier to each image frame to determine an informativeness score. This score indicates if an image frame is informative or non-informative based on factors such as focus, presence of bubbles, light reflection artifacts, and blurring. Informative frames are identified and a second trained classifier is applied to these frames to determine a polyp detection score. An interface presents image frames along with indicators of informativeness and the presence of polyps.
Machine learning-based classification using random forest or convolutional neural network
The first trained classifier can be: - A random forest classifier trained using features extracted from images and clustering them into visual words (bag of visual words model), with histograms representing the number of features per word; or - A convolutional neural network classifier that divides image frames into regions, applies the classifier to each region to obtain informativeness scores, aggregates them, and labels the frame accordingly.
Edge detection and candidate extraction for polyp identification
A third trained classifier may be applied to informative frames to determine pixels likely to contain polyp edges. An edge map is generated based on this classification. Optionally, a Canny edge detector can be used to generate an edge map, including overlap with a ground truth image. A Hough transform is then applied to the edge map to find candidate polyps, from which patches are extracted. The second trained classifier (e.g., convolutional neural network) classifies these patches, aggregates polyp scores, and labels frames accordingly.
User interface indicators for video quality and polyp detection
The invention includes indicators in the user interface representing informativeness scores (e.g., bar indicators, traffic light indicators showing three or more informativeness values), and markers such as bounding boxes to visually indicate detected polyps in image frames.
System and computer-readable medium embodiments
A system comprising hardware processors configured as described above to perform real-time colonoscopic image analysis, and a non-transitory computer-readable storage medium with instructions to execute the same method steps for polyp detection and video quality assessment.
The claims comprehensively cover methods, systems, and computer-readable media for the simultaneous, real-time assessment of video quality and automated polyp detection in colonoscopic images, utilizing a combination of machine learning classifiers, edge detection, candidate extraction, and user interface features.
Stated Advantages
Reduces polyp miss rates by increasing attentiveness and improving examination quality during colonoscopy procedures.
Provides real-time alerts and visual indicators of video quality and polyp presence to assist endoscopists during procedures.
Automatically identifies informative frames and detects polyps, thus assisting in early colorectal cancer detection and potential removal.
Documented Applications
Real-time monitoring of colonoscopic video quality and automated polyp detection during optical colonoscopy procedures.
Interested in licensing this patent?