Interactions with 3D virtual objects using poses and multiple-dof controllers

Inventors

Powderly, James M. • Niles, Savannah • Hamilton, IV, Frank Alexander • Fontaine, Marshal Ainsworth • Hoover, Paul Armistead

Assignees

Magic Leap Inc

Abstract

A wearable system can comprise a display system configured to present virtual content in a three-dimensional space, a user input device configured to receive a user input, and one or more sensors configured to detect a user's pose. The wearable system can support various user interactions with objects in the user's environment based on contextual information. As an example, the wearable system can adjust the size of an aperture of a virtual cone during a cone cast (e.g., with the user's poses) based on the contextual information. As another example, the wearable system can adjust the amount of movement of virtual objects associated with an actuation of the user input device based on the contextual information.

Core Innovation

The invention is an AR/VR wearable interaction system that uses a display system to provide a three-dimensional (3D) view and permit user interaction with objects within a field of view (FOV). A user input device generates an indication of user input, and a hardware processor initiates a cone cast of a virtual cone based on that indication. The virtual cone has an aperture that is dynamic based at least partly on the indication of user input, enabling contextual user-controlled selection or targeting through the cone.

The system determines one or more objects within the virtual cone and performs an action on one or more of the objects within the virtual cone. A visual representation of the cone is rendered, including detection of collisions between the cone and objects with optional occlusion disambiguation, and providing focus indicators. The aperture can be dynamically updated based at least partly on changes in user pose and/or contextual information such as object layout/density, distance to objects, lighting, and user preference.

The invention further includes using contextual information to translate user-input motion into virtual object motion through a distance-based multiplier that is thresholded and can increase with distance to move far objects faster while maintaining one-to-one precision near hand reach. The system supports visualization and interaction with virtual UI objects, including selecting or moving interactable objects based on the determined objects within the cone and performing corresponding actions.

Claims Coverage

The document includes two independent claims, a system claim and a method claim. Both independent claims share a common inventive concept: initiating a cone cast of a virtual cone with a dynamically adjustable aperture based on an indication of user input, determining objects within the cone, and performing an action on those objects.

The independent claims consistently cover a wearable AR workflow where user input indication drives a cone cast of a virtual cone with a dynamically adjustable aperture, enabling identification of objects inside the cone and triggering actions on those objects.

Stated Advantages

Documented Applications