Diffused light rendering of a virtual light source in a 3D environment

Inventors

Witherspoon, Noah A.

Assignees

Apple Inc

Abstract

Various implementations disclosed herein include devices, systems, and methods that generate a depiction of a portion of a physical environment illuminated based on light provided by identified portions of a virtual light source. For example, an example process may include obtaining sensor data, generating a three-dimensional (3D) reconstruction based on the sensor data, positioning a virtual light source relative to the 3D reconstruction, identifying a portion of the virtual light source based on unobstructed paths between the portion of the physical environment and the virtual light source, associating the portion of the virtual light source with the portion of the physical environment, and generating a view that includes a depiction of a portion of a physical environment illuminated based on light provided by the identified portions of the virtual light source.

Core Innovation

The invention relates to XR illumination using a virtual light source whose visible or unobstructed portions are determined from a 3D reconstruction of a physical environment. Sensor data corresponding to the physical environment is obtained and used to generate the 3D reconstruction, and a virtual light source is positioned relative to the 3D reconstruction so that portions of the virtual light source can be evaluated as unobstructed or obstructed.

Based on the identification of unobstructed versus obstructed portions of the virtual light source, the system determines to illuminate a portion of the physical environment based only on light from the unobstructed portion. A view is then generated comprising a depiction of the portion of the physical environment illuminated based on light from the unobstructed portion.

The document further describes using sensor data that includes depth data and light intensity image data, generating view depictions over multiple successive frames with illumination restricted to unobstructed pixels, and maintaining illumination behavior based on detection of movement of the electronic device relative to a threshold change. Additional rendering behaviors include determining obstructions using objects and generating views that are rendered live while sensor data is streamed.

Claims Coverage

The document provides two independent claims, each centered on determining unobstructed versus obstructed portions of a virtual light source using a 3D reconstruction and using that result to illuminate only corresponding portions of a physical environment and generate a view depicting the illuminated portion. The independent claims define the overall method and the corresponding device implementation, respectively, with multiple dependent claims refining specific sensor data types, device form factors, obstruction logic, rendering across frames, and behavior over time or streaming.

Across both independent claims, the inventive coverage is the combination of generating a 3D reconstruction from sensor data, positioning a virtual light source relative to that reconstruction, identifying unobstructed versus obstructed portions of the virtual light source using unobstructed light paths or direct points through the reconstruction, and illuminating only the physical-environment portion illuminated by the unobstructed portion while generating a view depicting that illuminated portion.

Stated Advantages

Documented Applications