Scene understanding using occupancy grids
Inventors
Ramnath, Divya • Dong, Shiyu • Choudhary, Siddharth • Mahendran, Siddharth • Kannan, Arumugam Kalai • Singhal, Prateek • Gupta, Khushi
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12444136-B2
Publication Date
2025-10-14
Expiration Date
Abstract
This document describes scene understanding for cross reality systems using occupancy grids. In one aspect, a method includes recognizing one or more objects in a model of a physical environment generated using images of the physical environment. For each object, a bounding box is fit around the object. An occupancy grid that includes a multiple cells is generated within the bounding box around the object. A value is assigned to each cell of the occupancy grid based on whether the cell includes a portion of the object. An object representation that includes information describing the occupancy grid for the object is generated. The object representations are sent to one or more devices.
Core Innovation
The invention recognizes one or more objects in a model of a physical environment generated using images of the physical environment. For each object, the invention fits a bounding box around the object and generates an occupancy grid within the bounding box that includes a plurality of cells.
The invention assigns a value to each cell of the occupancy grid based on whether the cell includes a portion of each object. For each object, the invention generates an object representation that includes information describing the occupancy grid for the object and sends the object representations to one or more devices.
In further aspects, the invention can assign distinct values to occupancy-grid cells depending on whether a cell includes a portion of an object or does not include a portion of the object. The invention can detect a change to a given object and, in response, generate and send an updated occupancy grid for the given object instead of sending an updated model of the physical environment.
The invention can additionally update a local mesh on one or more devices using the newly generated occupancy grid.
Claims Coverage
The independent claims are directed to a computer-implemented method, a computer-implemented system, and a non-transitory computer-readable medium. Across these, three main inventive features appear: object recognition in an image-generated physical environment model, per-object occupancy-grid generation within fitted bounding boxes with cell values based on object portions, and sending object representations including occupancy-grid information to one or more devices.
Recognizing objects in an image-generated physical environment model
recognizing one or more objects in a model of a physical environment generated using images of the physical environment
Fitting bounding boxes and generating per-object occupancy grids
for each object of the one or more objects: fitting a bounding box around each object; generating an occupancy grid within the bounding box around each object, wherein the occupancy grid includes a plurality of cells
Assigning occupancy-grid cell values based on object portions and sending object representations
assigning a value to each cell of the occupancy grid based on whether the cell includes a portion of each object; generating an object representation that includes information describing the occupancy grid for each object; and sending each object representations to one or more devices
Together, the independent claims cover recognizing objects from an image-generated physical environment model, generating an occupancy grid within each fitted bounding box with cell values based on whether cells include portions of objects, generating object representations that include the occupancy-grid information, and sending those representations to one or more devices.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?