Visual, depth and micro-vibration data extraction using a unified imaging device

Inventors

Zuta, Yoav • CARMON, Gideon • LEVY, Roi • Raz, Guy

Assignees

Gentex Corp Carbondale

Abstract

A unified imaging device used for detecting and classifying objects in a scene including motion and micro-vibrations by receiving a plurality of images of the scene captured by an imaging sensor of the unified imaging device comprising a light source adapted to project on the scene a predefined structured light pattern constructed of a plurality of diffused light elements, classifying object(s) present in the scene by visually analyzing the image(s), extracting depth data of the object(s) by analyzing position of diffused light element(s) reflected from the object(s), identifying micro-vibration(s) of the object(s) by analyzing a change in a speckle pattern of the reflected diffused light element(s) in at least some consecutive images and outputting the classification, the depth data and data of the one or more micro-vibrations which are derived from the analyses of images captured by the imaging sensor and are hence inherently registered in a common coordinate system.

Core Innovation

A unified imaging device includes a light source adapted to project a predefined structured light pattern constructed of a plurality of diffused light elements onto a scene, and an imaging sensor that captures a plurality of images of the scene. The processor visually analyzes the plurality of images to classify at least one object present in the scene, and in the same captured image data extracts depth data of the at least one object by analyzing a position of at least one diffused light element reflected from the at least one object.

The processor identifies at least one micro-vibration of the at least one object by analyzing a change in a speckle pattern of at least one of the plurality of reflected diffused light elements in at least some consecutive images. The change in the speckle pattern is identified by applying a speckle pattern translation to the speckle pattern with respect to at least one adjacent point in the speckle pattern, and an angular velocity is indicative of the at least one micro-vibration.

The invention emphasizes that the classification, depth data, and micro-vibrations are derived from common captured data obtained from the unified imaging device, enabling inherent spatial registration and temporal synchronization across the visual, depth, and micro-vibration modalities. A vehicle imaging system includes a light source, an imaging sensor spatially aligned with the light source, and a processing unit having an analyzer that analyzes visual content of captured image data and extracted depth data to detect and classify one or more objects in the scene.

Claims Coverage

The document provides three independent claims: clm-00001, clm-00012, and clm-00019. Across these independent claims, there are three core inventive features: unified structured-light imaging for object classification and depth extraction, speckle-pattern translation for micro-vibration detection, and a vehicle imaging system with a neural network that analyzes visual content and extracted depth data.

Overall claim coverage centers on a unified imaging device that uses structured light made of diffused light elements to support object classification and depth data extraction from common image captures, and uses speckle-pattern translation to identify micro-vibrations via angular velocity. A separate independent vehicle imaging system applies a neural network that analyzes both captured image data and extracted depth data to detect and classify objects.

Stated Advantages

Documented Applications