Saliency prioritization for image processing

Inventors

DOGGETT, ERIKA VARIS • Nguyen, David T. • Tang, Binghao • Zhou, Hailing • Wolak, Anna M. • Qi, Erick Keyu

Assignees

Disney Enterprises Inc • Accenture LLP

Abstract

According to one implementation, a system includes a computing platform having a hardware processor and a system memory storing a software code including a trained neural network (NN). The hardware processor executes the software code to receive an input image including a pixel anomaly, identify, using the trained NN, one or more salient regions of the input image, and determine whether the pixel anomaly is located inside any of the one or more salient regions. The hardware processor further executes the software code to assign a first priority to the pixel anomaly when it is determined that the pixel anomaly is located inside any of the one or more salient regions, and to assign a second priority, lower than the first priority, to the pixel anomaly when it is determined that the pixel anomaly is not located inside any of the one or more salient regions.

Core Innovation

The invention provides a system and method for performing automated saliency prioritization for image processing. The system includes a hardware processor and system memory storing software code with a trained neural network (NN). The processor executes the software code to receive an input image that includes a pixel anomaly, uses the trained NN to identify one or more salient regions within the image, and determines whether the pixel anomaly is located inside any of these salient regions. Based on this determination, the processor assigns a higher priority to pixel anomalies within salient regions and a lower priority to those outside.

The problem addressed by this invention is the high cost, inefficiency, and reliance on human inspectors for correcting pixel errors in images and video frames. Not all pixel anomalies are of equal importance: their necessity for correction can depend on their location in relation to salient regions, such as characters, foreground objects, or areas likely to attract visual attention. Conventional solutions do not automate the prioritization of which pixel anomalies need correction, leading to unnecessary work and expense.

The disclosed solution enables automated prioritization by employing two main approaches: character and foreground object masking, where regions near important image features are identified as salient; and attention mapping, where regions likely to attract human visual attention are mapped using predicted attention scores. Either approach, or a combination, can be employed to prioritize pixel anomalies for correction, reducing unnecessary correction of less important anomalies and streamlining post-production workflows.

Claims Coverage

The patent contains two independent claims: one directed to a system and one to a method, each primarily focused on automated saliency prioritization of pixel anomalies using a trained neural network.

The independent claims cover the automated determination and assignment of priorities to pixel anomalies in images by using a trained neural network to identify saliency, distinguishing between anomalies that should be prioritized for correction and those that can be deprioritized.

Stated Advantages

Documented Applications