Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12455619-B2
Publication Date
2025-10-28
Expiration Date
Abstract
Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.
Core Innovation
The invention relates to receiving, at a first time, local target data captured via one or more sensors of a first wearable system comprising a first head-wearable device, where the local target data comprises a direction associated with the first wearable system, and image data captured via a camera of the first wearable system, where the image data is associated with a field of view of the first head-wearable device. In response to receiving the local target data and the image data, the invention determines trained target data based on the local target data and further based on the image data, where determining the trained target data comprises applying the local target data and the image data as inputs to a convolutional neural network.
The trained target data is transmitted to a second wearable system. The second wearable system is configured to detect an object in a field of view of the second wearable system and determine, based on the trained target data and based on the convolutional neural network, whether the detected object comprises a target object.
The disclosure further supports updating and deployment architectures in which the trained target data may be updated based on later second local target data and later second image data, and transmitted as updated trained target data to a second wearable system. The disclosure also recites variants where the second wearable system includes or is composed of the first wearable system, or includes a second head-wearable device different from the first head-wearable device, and where the computing device is positioned within a geographic constraint relative to both head-wearable devices.
Claims Coverage
The independent claims are directed to a method, a system, and a non-transitory computer-readable storage medium that define a first wearable capturing local target data and image data, using a convolutional neural network to determine trained target data, and a second wearable detecting an object and determining whether the object comprises a target object based on the trained target data. Across the independent claims, the inventive features center on convolutional neural network-based target determination using paired direction and image data, and the transmission of that trained target data to a second wearable for object and target-object classification.
Direction and image inputs for convolutional neural network target data
Receiving, at a first time, local target data captured via one or more sensors of a first wearable system comprising a first head-wearable device, where the local target data comprises a direction associated with the first wearable system, and receiving image data captured via a camera of the first wearable system, where the image data is associated with a field of view of the first head-wearable device; determining trained target data based on the local target data and further based on the image data, where the determining comprises applying the local target data and the image data as inputs to a convolutional neural network.
Transmit trained target data to a second wearable system
Transmitting, to a second wearable system, the trained target data.
Object detection and target-object determination based on convolutional neural network
Configuring the second wearable system to detect an object in a field of view of the second wearable system and determine, at the second wearable system based on the trained target data, whether the detected object comprises a target object, where the determining is based on the convolutional neural network.
Non-transitory computer-readable storage medium for convolutional neural network trained target data
Storing instructions which, when executed by one or more processors, cause the one or more processors to perform a method comprising receiving local target data and image data at a first time, determining trained target data by applying the local target data and the image data as inputs to a convolutional neural network, and transmitting the trained target data to a second wearable system for object detection and target-object determination based on the convolutional neural network.
Claim coverage centers on CNN-based determination of trained target data using paired local target data and camera image data from a first head-wearable device, transmission of that trained target data to a second wearable system, and convolutional neural network-based determination at the second wearable as to whether a detected object comprises a target object.
Stated Advantages
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?