System and method for analyzing medical images based on spatio-temporal data

Inventors

Galeotti, John • Mathai, Tejas Sudharshan

Abstract

Provided is a system, method, and computer program product for analyzing spatio-temporal medical images using an artificial neural network. The method includes capturing a series of medical images of a patient, the series of medical images comprising visual movement of at least one entity, tracking time-varying spatial data associated with the at least one entity based on the visual movement, generating spatio-temporal data by correlating the time-varying spatial data with the series of medical images, and analyzing the series of medical images based on an artificial neural network comprising a plurality of layers, one or more layers of the plurality of layers each combining features from at least three different scales, at least one layer of the plurality of layers of the artificial neural network configured to learn spatio-temporal relationships based on the spatio-temporal data.

Core Innovation

A method, system, and computer program product are described for analyzing spatio-temporal medical images using an artificial neural network. A series of medical images of a patient is captured with an imaging device, where the series comprises visual movement of at least one entity comprising at least a portion of at least one of the patient and an object. A computing device tracks time-varying spatial data associated with the at least one entity based on the visual movement in one or more images of the series.

The computing device generates spatio-temporal data by correlating the time-varying spatial data with the series of medical images. The spatio-temporal data is input into an artificial neural network, and the series of medical images is analyzed based on the spatio-temporal data using the artificial neural network comprising a plurality of layers.

One or more layers each combine features from at least three different scales, and a layer is configured to learn multi-scale spatio-temporal relationships of features from at least two different scales of the at least three different scales. The one or more layers that combine features from the at least three different scales comprise at least one of dilated convolutions of different scales, dense and/or residual connections between at least a subset of layers comprising features from at least three different scales, or any combination thereof.

Claims Coverage

The independent claims cover a method, a system, and a computer program product for analyzing spatio-temporal medical images using an artificial neural network. Each independent claim includes the same core chain of inventive functions: capturing a series with visual movement, tracking time-varying spatial data, correlating tracking data to generate spatio-temporal data, inputting it into an artificial neural network, and analyzing the image series using multi-scale spatio-temporal feature learning across at least three different scales.

Across the independent claims, coverage centers on analyzing spatio-temporal medical images by correlating tracked time-varying spatial data with a captured image series, then using an artificial neural network whose layers combine features across at least three different scales. The claims specifically require learning multi-scale spatio-temporal relationships using features from the layer and a preceding layer, implemented using dilated convolutions of different scales and/or dense and/or residual connections between layers that carry features from those scales.

Stated Advantages

Documented Applications

No documented applications found