Color-perception-guided display power reduction for extended reality

Inventors

Sun, Qi • ZHU, Yuhao • Duinkharjav, Budmonde • Chen, Kenneth • Tyagi, Abhishek • He, Jiayi

Assignees

New York University • University of Rochester

Abstract

Saving power in an extended-reality device can include rendering a video stream on a display, tracking an eye of a user observing the display, ascertaining a gaze direction of the eye based on tracking the eye, calculating a foveal region of the display based on ascertaining the gaze direction in relation to the display, and modulating color hues of the video stream for pixels of the display outside the foveal region, to reduce a power consumption of the display. Modulating color hues is based on human color discrimination and display power consumption correlated to pixel color.

Core Innovation

The disclosure relates to power savings in an extended-reality device by rendering a video stream on a display and tracking an eye of a user observing the display. A gaze direction of the eye is ascertained based on the eye tracking, and a foveal region of the display is calculated based on the gaze direction in relation to the display. Color hues of the video stream are modulated for pixels of the display outside the foveal region to reduce a power consumption of the display.

The disclosed hue modulation is performed based on a computational model of human color discrimination together with a power model that correlates power consumption with pixel color. The computational model uses retinal eccentricity-based discrimination thresholds represented as ellipse regions in DKL/i-DKL color space. The power model correlates OLED/LED power to pixel color to guide hue modulation decisions for pixels outside the foveal region.

The disclosure further states refinements for the color modulation, including real-time implementation via an image-space shader and formulations using constrained convex optimization and/or closed-form chromaticity solutions. Additional refinements include optional luminance intensity modulation and specific hue shift behaviors such as green shift and red shift. The modulation is also described with angular constraints that leave an unmodulated hue within cones around the gaze direction, including 10-degree and 5-degree cones.

Claims Coverage

The partial content includes three independent claims covering a system, a method, and a memory device, each with the same core architecture: eye tracking to determine gaze direction, foveal region calculation, and modulation of color hues outside the foveal region to reduce display power using a computational model of human color discrimination and a power model correlating power with pixel color.

Across the independent claims, the inventive core is gaze-tracked foveal region determination followed by hue modulation for pixels outside the foveal region, where the modulation is based on a computational model of human color discrimination and a power model correlating display power with pixel color.

Stated Advantages

Documented Applications