Method for generating a virtual user interface

Inventors

Abovitz, Rony • Schowengerdt, Brian T. • Watson, Mathew D.

Assignees

Magic Leap Inc

Abstract

A waveguide apparatus includes a planar waveguide and at least one optical diffraction element (DOE) that provides a plurality of optical paths between an exterior and interior of the planar waveguide. A phase profile of the DOE may combine a linear diffraction grating with a circular lens, to shape a wave front and produce beams with desired focus. Waveguide apparati may be assembled to create multiple focal planes. The DOE may have a low diffraction efficiency, and planar waveguides may be transparent when viewed normally, allowing passage of light from an ambient environment (e.g., real world) useful in AR systems. Light may be returned for temporally sequentially passes through the planar waveguide. The DOE(s) may be fixed or may have dynamically adjustable characteristics. An optical coupler system may couple images to the waveguide apparatus from a projector, for instance a biaxially scanning cantilevered optical fiber tip.

Core Innovation

The disclosure provides an augmented/virtual reality system that generates a virtual user interface through gesture-based interaction. A first gesture involving a hand of a user is detected, and in real time a first virtual user interface related to the hand is rendered in response. The first virtual interface includes a first level menu item comprising a high level navigation menu item, icon or field that appears on each of a plurality of first portions of the hand.

After detecting the first gesture, the system detects, at each first portion, a second gesture involving the hand of the user. In real time in response to detecting the second gesture, a second virtual user interface related to the hand is rendered. The second virtual user interface includes a plurality of discrete second level menu items, each appearing to be on or attached to an associated first level menu item at an associated first portion, and each second level menu item expands from and is related to an associated first level menu item.

The disclosure further describes additional gesture-driven UI levels, including a third virtual user interface with third level menu items arranged in arcs and moved relative to the hand. Direction-dependent movement based on clockwise versus counter-clockwise circling is also described within the gesture-driven UI framework.

Claims Coverage

The provided material includes one independent claim directed to a gesture-driven method for generating a virtual user interface on multiple portions of a user’s hand. The inventive features are centered on mapping gesture detection to real-time rendering of hierarchical menu items that expand from first level menu items to discrete second level menu items on corresponding hand portions.

Across the independent claim coverage, the core claimed structure uses multi-portion hand gesture detection to render hierarchical virtual menu items: a first level menu item rendered across multiple hand portions, followed by a second gesture detected per hand portion that causes discrete second level menu items to appear on or attached to the associated first level menu items and expand from them. Additional coverage includes a third gesture that drives arc-arranged third level menu items with clockwise or counter-clockwise dependent movement.

Stated Advantages

Documented Applications