Intelligent video conference cropping based on audio and vision

Inventors

Schubert, Christian • Remijn, Matthew W.

Assignees

Gentex Corp Carbondale

Abstract

A video conferencing system for a vehicle includes a display device disposed in a passenger compartment of the vehicle, the display device comprising a screen, a first imager configured to capture a first image data corresponding to a field of view of a passenger compartment of the vehicle, at least one microphone configured to collect audio data from within the passenger compartment, and a controller. The controller is configured detect a display prompt within the image data corresponding to a portion of the image data received from the first imager, detect an audio prompt within a predetermined zone of the passenger compartment of the vehicle, and selectively crop the first image data according to the portion of the image data corresponding with the display prompt in response to determining a correlation between the portion of the image data and the zone of the passenger department associated with the audio prompt.

Core Innovation

The invention relates to an in-vehicle video conferencing system for a passenger compartment. The system includes a first imager configured to capture first image data corresponding to a field of view of the passenger compartment, and a plurality of microphones in respective predetermined locations corresponding with seating locations within the passenger compartment.

A controller receives information related to a vehicle motion state from a vehicle network and monitors for a display prompt within the image data and an audio prompt at least primarily from one of the plurality of microphones within one of the seating locations. The controller determines a correlation between the portion of the image data and the seating area associated with the audio prompt, including whether the seating area associated with the audio prompt is the driver seating location.

Responsive to vehicle motion state, the controller selectively crops the first image data according to the portion of the image data corresponding with the display prompt. Selective cropping based on the display prompt is performed differently depending on whether the audio prompt is associated with the driver seating location and whether the vehicle is in a motion state or not in a motion state, and the selectively cropped first image data is then used for video conferencing, including selectively transmitting cropped video and corresponding audio for conferencing.

The dependent refinements further define how the display prompt and audio prompt are detected and how the cropped content is prepared. For example, a display prompt can be determined based on detection of movement of one or more features of a vehicle occupant identified in the first image data, and the audio prompt can correspond to detecting a sound identifiable as speech.

Claims Coverage

The independent claim is clm-00001. It defines an in-vehicle video conferencing system with audio-vision prompting, correlation to seating locations, and motion-state-dependent selective cropping conditioned on whether the audio prompt is from the driver seating location. Dependent claims (clm-00002, clm-00003, clm-00004, clm-00006, clm-00008) refine prompt detection, cropping content, and conferencing integration/preprocessing.

The claims cover a vehicle video conferencing system that uses first imager data and seat-associated microphone data to detect a display prompt and an audio prompt, correlates them to seating locations, and performs motion-state-dependent selective cropping. Dependent claims further specify prompt detection based on occupant feature movement and speech-identifiable sounds, constrain cropped content to facial regions, add digital scaling, and define conferencing integration with an external conference server and passenger-compartment display.

Stated Advantages

Documented Applications