Interpretable deep machine learning for clinical radiology
Inventors
Duncan, James S. • Letzen, Brian • CHAPIRO, JULIUS • Wang, Clinton
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12431244-B2
Publication Date
2025-09-30
Expiration Date
Abstract
One aspect of the invention provides a computer-implemented method of identifying one or more clinical factors associated with an artificial intelligence prediction. The computer implemented method includes: applying a previously trained deep neural network to one or more images for a subject to produce a prediction, the previously trained deep neural network comprising a plurality of nodes; comparing outputs of the nodes to previously identified patterns of node outputs for a plurality of individual clinical factors; and identifying which of the plurality of individual clinical factors are most correlated with the outputs of the nodes.
Core Innovation
The invention relates to an interpretable deep learning system in clinical radiology that applies a previously trained deep neural network to one or more images for a subject to produce an artificial intelligence prediction. The deep neural network comprises a plurality of nodes, and node outputs are used to enable interpretation of the prediction. The system compares outputs of the nodes to previously identified patterns of node outputs for a plurality of individual clinical factors.
The method identifies which of the plurality of individual clinical factors are most correlated with the outputs of the nodes. For selected clinical factors, the method generates a contribution score by measuring changes between predictions or node behaviors produced from altered network inputs or altered network structures. In one approach, contribution score generation includes iteratively removing one or more of the multilayer nodes associated with a previously identified pattern of node outputs for a clinical factor to produce an altered deep neural network, applying the altered deep neural network to the one or more images, and identifying a difference between the altered prediction and the prediction.
In another approach, contribution score generation quantifies the impact of removing the multilayer nodes associated with the previously identified patterns of node outputs for a clinical factor, or quantifies the impact of removing one or more images having a single clinical factor on the prediction. The invention further generates one or more altered images illustrating regions associated with individual clinical factors correlated with the outputs of the multilayer nodes. The method designates pixels or voxels as significant based on correlation and comparability to patterns of node outputs linked to individual clinical factors, and applies an overlay for each clinical factor.
Claims Coverage
The independent claims cover a computer-implemented method for identifying one or more clinical factors associated with an artificial intelligence prediction, with three main inventive blocks: correlating node outputs to previously identified patterns for individual clinical factors, generating contribution scores by altering multilayer nodes or image sets, and producing altered images or overlays highlighting regions associated with correlated clinical factors. Across the five independent claims, the differences lie in how contribution scores are generated and how region-level outputs are produced.
Correlating node outputs to clinical-factor patterns and identifying correlated factors
Comparing outputs of the nodes to previously identified patterns of node outputs for a plurality of individual clinical factors; identifying which of the plurality of individual clinical factors are most correlated with the outputs of the nodes.
Contribution scoring by iteratively removing multilayer nodes and using altered versus original predictions
Generating a contribution score comprising iteratively removing one or more of the multilayer nodes associated with the previously identified patterns of node outputs for one of the plurality of individual clinical factors from the previously trained deep neural network to produce an altered deep neural network; applying the altered deep neural network to the one or more images to produce an altered prediction; and identifying a difference between the altered prediction and the prediction.
Contribution scoring by quantifying impact of removing multilayer nodes
Generating a contribution score comprising quantifying the impact of removing the multilayer nodes associated with the previously identified patterns of node outputs for one of the plurality of individual clinical factors.
Contribution scoring by quantifying impact of removing images having a single clinical factor
Generating a contribution score comprising quantifying the impact of removing one or more images having a single clinical factor on the prediction of the deep neural network.
Region-level altered images using correlation and comparability of multilayer node outputs and applying overlays
Generating one or more altered images illustrating regions of the one or more images associated with the one or more individual clinical factors correlated with the outputs of the multilayer nodes, including identifying which of the multilayer node outputs are most correlated with each clinical factor; comparing the multilayer node outputs for the one or more images used for prediction relative to the multilayer node outputs for the one or more images correlated with individual clinical factors; designating each pixel or voxel in the one or more images used for prediction as significant to the one or more individual clinical factors if the multilayer node outputs associated with the pixel or voxel are correlated with the individual clinical factors and are comparable to the multilayer node outputs found in the one or more images correlated with individual clinical factors; and applying an overlay for each of the one or more individual clinical factors corresponding to the pixels or voxels designated as significant to the individual clinical factor.
Stripped-image contribution scoring with node-output-difference matching to clinical-factor patterns and overlaying correlated pixels or voxels
Generating one or more altered images including iteratively removing one or more pixels or voxels from the one or more images to generate one or more stripped images; reapplying the previously trained deep neural network to the one or more stripped images; identifying one or more differences in outputs of the multilayer nodes for the one or more stripped images relative to the outputs of the multilayer nodes for the one or more images; comparing the one or more differences to the previously identified patterns of node outputs for the plurality of individual clinical factors; designating a pixel or voxel as associated with the one or more individual clinical factors correlated with the outputs of the multilayer nodes if the differences are correlated with the previously identified patterns of node outputs for the plurality of individual clinical factors; and applying an overlay corresponding to the pixels or voxels designated as associated with the one or more individual clinical factors.
The claims collectively cover interpreting an AI prediction by correlating node outputs to stored node-output patterns tied to individual clinical factors, then generating contribution scores by quantifying impacts of removing multilayer nodes or removing images associated with a single clinical factor, and generating altered or overlay images that designate significant pixels or voxels based on correlation and comparability to the learned node-output patterns.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Proof-of-concept experiments for liver lesion classification on multi-phasic MRI using LI-RADS-related lesion entities and features, reporting classification performance and feature-identification, localization, and scoring results.
Interested in licensing this patent?