Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12469246-B1
Publication Date
2025-11-11
Expiration Date
Abstract
In one implementation, a method of extracting information from a physical environment is performed at a device including an image sensor, one or more processors, and non-transitory memory. The method includes determining a gaze location and a distance to an object in a physical environment at the gaze location. The method includes selecting a field-of-view of the physical environment based on the gaze location and the distance to the object. The method includes obtaining, using the image sensor, an image corresponding to the field-of-view of the physical environment. The method includes extracting information from the image.
Core Innovation
The invention provides depth/gaze-based image processing for an extended reality (XR) device in a physical environment. A gaze location and a distance from the device to an object at the gaze location are determined, and a field-of-view of the physical environment is selected from the device based on the gaze location and the distance. The size of the field-of-view is a first size at a first distance and a second size less than the first size at a second distance greater than the first distance.
The device obtains, using an image sensor, an image corresponding to the selected field-of-view or a selected portion around the gaze. Information is extracted from the obtained image, with the extraction constrained to the selected field-of-view portion. Implementations include selecting among multiple image sensors with different field-of-views and using a dynamic adjustable field-of-view.
The disclosure further describes extracting machine-readable content, including text and QR code, and identifying actionable items from the image. A user interface element includes glints tied to detected actionable items, and input directed to a specific glint is used to perform an associated action. The disclosure compares full-scan, object-scan using semantic segmentation, and area-scan approaches that emphasize reduced computation via the distance-based field-of-view selection.
Claims Coverage
The document contains three independent claims: a method claim, a device claim, and a non-transitory memory claim. Each independent claim is grounded in an identical core sequence of determining gaze location and device-to-object distance, selecting a distance-dependent field-of-view size, obtaining an image corresponding to the field-of-view, and extracting information from the image; the dependent claims further refine field-of-view acquisition/extraction and add actionable-item interaction using glints.
Determining gaze location and distance for object at gaze location
Determine a gaze location and a distance from the device to an object in a physical environment at the gaze location.
Distance-dependent field-of-view size selection
Select a field-of-view of the physical environment from the device based on the gaze location and the distance from the device to the object, wherein a size of the field-of-view is a first size at a first distance and the size of the field-of-view is a second size less than the first size at a second distance greater than the first distance.
Image corresponding to the selected field-of-view
Obtain, using the image sensor, an image corresponding to the field-of-view of the physical environment or the selected field-of-view.
Extracting information from the obtained image
Extract information from the image.
Device components configured for gaze-to-field-of-view imaging and extraction
A device comprising an image sensor, non-transitory memory, and one or more processors configured to determine the gaze location and distance, select the distance-dependent field-of-view, obtain the image using the image sensor, and extract information from the image.
Programs stored in non-transitory memory to perform gaze-to-field-of-view imaging and extraction
A non-transitory memory storing one or more programs that, when executed by one or more processors of a device including an image sensor, determine the gaze location and distance, select the distance-dependent field-of-view, obtain the image using the image sensor, and extract information from the image.
Across the independent claims, the core inventive coverage is a distance-dependent field-of-view selection driven by a determined gaze location and device-to-object distance, followed by image capture corresponding to that field-of-view and information extraction from the captured image. The document’s dependent claims further narrow field-of-view acquisition and extraction to selected portions and include actionable-item detection with glints and input targeted to a particular glint to perform an associated action.
Stated Advantages
Reduced computation by using distance-based area selection rather than full-scan approaches.
Constraining information extraction to the selected field-of-view, including a selected portion around the gaze.
Enabling interaction with actionable items via glints associated with detected actionable items.
Documented Applications
Extended reality (XR) depth/gaze-based image processing in a physical environment to extract information from a gaze-dependent field-of-view.
Machine-readable content extraction including text and QR code from a gaze-dependent field-of-view image.
User interface interaction in XR using glints tied to detected actionable items, where input directed to a specific glint performs an associated action.
Image processing approach selection described as full-scan, object-scan (semantic segmentation), and area-scan emphasizing reduced computation via distance-based field-of-view selection.
Interested in licensing this patent?