Gesture and voice controlled interface device
Inventors
Wagner, Guy • Langer, Leeor • Dahan, Asher
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12340027-B2
Publication Date
2025-06-24
Expiration Date
Abstract
A gesture and voice-controlled interface device comprising one or a plurality of gesture sensors for sensing gestures of a user; one or a plurality of audio sensors for sensing sounds made by the user; and a processor configured to obtain one or a plurality of sensed gestures from said one or a plurality of gesture sensors and to obtain one or a plurality of sensed sounds from said one or a plurality of audio sensors, to analyze the sensed gesture and sensed sounds to identify an input from the user, and to generate an output signal corresponding to the input to a controlled device.
Core Innovation
The invention provides a device that estimates a force applied on an object by a limb of a user using one or a plurality of biopotential sensors and an inertial measurement unit (IMU). The biopotential sensors are configured to be placed on a limb and measure one or more biopotential signals from one or more muscles or one or more nerves. The device obtains the measured biopotential signals and measured inertial properties, and a processor estimates the force applied based on a model constructed from samples of biopotential signals and inertial properties collected from a plurality of users who applied a known force.
The invention further includes gesture-controlled and gesture-and-voice-controlled interface devices that sense user gestures and, in the gesture-and-voice case, sensed sounds. A processor obtains sensed gestures and sensed sounds, analyzes them to identify an input from the user, and generates an output signal corresponding to the input to a controlled device. In the gesture-controlled interface device, the output signal is modulated by a modulating function according to one or more properties of the sensed gestures, where the modulating function simulates a physical parameter relating to a virtual object or a virtual environment selected from weight, texture, friction and resistance.
For a gesture and voice-controlled interface device, the processor identifies an initiating cue in the sensed gestures or in the sensed sounds, extracts first measured gesture values and second measured gesture values, and generates the output signal based on the first measured gesture value and the second measured gesture value. For a further gesture and voice-controlled interface device, the processor identifies a specific person by comparing sensed sounds to a voice signature and verifies the specific person by comparing sensed gestures to a gesture signature, and it generates an output signal only if the gestures are verified as being performed by the specific person.
Claims Coverage
The independent claims cover multiple inventive concepts across force estimation and gesture/voice-controlled interfaces, including biopotential sensors fused with IMU-derived inertial properties, gesture-to-output mapping using a modulating function that simulates virtual physical parameters, initiating-cue gating, and specific person identification and verification before generating an output. The coverage is reflected across independent claims 1, 2, 3, 4, 5, and 9.
Biopotential and IMU force estimation with weight output
A device using one or a plurality of biopotential sensors on a limb to measure biopotential signals from muscles or nerves, and an inertial measurement unit (IMU) to measure inertial properties, with a processor that obtains both and estimates force applied on an object using a model constructed from samples collected from a plurality of users who applied a known force, wherein the processor is configured to estimate weight of the object.
Biopotential and IMU torque estimation
A device using one or a plurality of biopotential sensors on a limb to measure biopotential signals from muscles or nerves, and an inertial measurement unit (IMU) to measure inertial properties, with a processor that obtains both and estimates the force applied on the object by the limb based on a model constructed from samples collected from a plurality of users who applied a known force, wherein the processor is configured to estimate torque applied on a lever.
Biopotential and IMU connector-connection force estimation
A device using one or a plurality of biopotential sensors on a limb to measure biopotential signals from muscles or nerves, and an inertial measurement unit (IMU) to measure inertial properties, with a processor that obtains both and estimates force applied on the object by the limb based on a model constructed from samples collected from a plurality of users who applied a known force, wherein the processor is configured to estimate force applied to successfully connect a connector.
Gesture-controlled output modulation using simulated virtual physical parameters
A gesture-controlled interface device with one or a plurality of gesture sensors and a processor configured to obtain sensed gestures, analyze them to identify an input from the user, and generate an output signal corresponding to the input to a controlled device, wherein the output signal is modulated by a modulating function according to one or more properties of the sensed gestures, the modulating function simulating a physical parameter relating to a virtual object or a virtual environment selected from weight, texture, friction and resistance.
Initiating-cue-based gesture and voice output generation
A gesture and voice-controlled interface device with one or a plurality of gesture sensors and one or a plurality of audio sensors, where the processor obtains sensed gestures and sensed sounds, analyzes them to identify an input, and generates an output signal corresponding to the input to a controlled device, wherein the processor is configured to identify an initiating cue in the sensed gestures or in the sensed sounds, extract a first measured gesture value from the sensed gestures, extract a second measured gesture value from the sensed gestures, and generate the output signal based on the first measured gesture value and the second measured gesture value.
Specific person identification and gesture verification for gated output
A gesture and voice-controlled interface device with one or a plurality of gesture sensors and one or a plurality of audio sensors, where the processor obtains sensed gestures and sensed sounds, analyzes them to identify an input, and generates an output signal corresponding to the input to a controlled device only if verified, wherein the processor is configured to identify a specific person based on comparing sensed sounds to a voice signature of the specific person, verify based on comparing sensed gestures to a gesture signature of the specific person that the sensed gestures were performed by the specific person, and generate the output signal only if it was verified that the sensed gestures were performed by the specific person.
Across the independent claims, the patent covers force estimation based on biopotential sensor measurements combined with IMU-derived inertial properties and a model trained from samples of multiple users who applied known forces, including estimation of weight, torque on a lever, and force for successfully connecting a connector; and gesture and gesture-and-voice interface devices that analyze sensed gestures and sounds to identify user input and generate output signals to a controlled device, with output modulation via simulated virtual physical parameters, initiating-cue-based gesture value usage, and specific-person voice and gesture signature identification/verification gating.
Stated Advantages
Not explicitly described in patent.
Documented Applications
Not explicitly described in patent.
Interested in licensing this patent?