Systems and methods for providing observation scenes corresponding to extended reality (XR) content

Inventors

Wilson, Andrew W. • Hollander, Ari J.

Assignees

Science Applications International Corp SAIC

Abstract

Systems, methods, apparatuses, and computer-readable media for creating an observation video are described. The observation video may comprise a viewport that moves in coordination with motion of an HMD user's head and that shows portions of VR content being output to the HMD user at different times. For each of those times, motion data from the HMD may be used to determine a position, orientation, and/or shape of the viewport. The observation video may also include an element that represents the HMD user, which element may comprise a video element video isolated from video of the user captured by a camera, and/or which may comprise an animated avatar.

Core Innovation

The invention relates to a method of outputting observation scenes showing viewports into an extended reality (XR) environment to an XR user via a moving XR display. For each of a plurality of different XR content segments respectively corresponding to different orientations of the XR display, XR content segment data is received that is renderable to output a visual representation of a portion of the XR environment viewable via the XR display in an orientation that corresponds to the XR content segment, and XR display spatial data corresponding to the XR content segment is received, indicating the orientation that corresponds to the XR content segment.

For each XR content segment, an observation scene viewport sub-region is determined based on the XR display spatial data corresponding to the XR content segment. The determining includes determining a shape of the viewport sub-region based on a vertical distance from a center of the observation scene and a horizontal distance from the center of the observation scene, determining a rotation of the viewport sub-region in a plane of the observation scene based on a rotation of the XR display about a roll rotation axis, and determining a position of the viewport sub-region in the observation scene based on the roll rotation axis rotation and based on rotations of the XR display about pitch and yaw axes. The observation scene is generated by adding a portion of the visual representation to the determined viewport sub-region.

The generated observation scene is output via a second display device as part of an observation video, and the observation scenes corresponding to the XR content segments appear in the observation video as a viewport sub-region that moves in coordination with movements of the XR display. The viewport sub-region moves to show different portions of the XR environment across the observation video, coordinated with changes among the plurality of different orientations of the moving XR display.

Claims Coverage

The partial content includes three independent claims (a method, a computer, and a non-transitory computer-readable medium). Across these independent claims, the core inventive coverage centers on determining a viewport sub-region in an observation scene from XR display spatial data, including roll, pitch, and yaw, and inserting a portion of the rendered XR content into that viewport so the viewport sub-region moves in coordination with XR display motion.

Collectively, the independent claims cover a pipeline that receives renderable XR content segment data and XR display spatial data, determines an observation-scene viewport sub-region using center distances and roll, pitch, and yaw rotations, inserts the rendered portion into that viewport sub-region, and outputs an observation video where the viewport sub-regions move in coordination with XR display movements.

Stated Advantages

Documented Applications

No documented applications found