Sensor fusion to validate sound-producing behaviors
Inventors
CHAN, Eduardo • MAULT, James • Wang, David • Leung, Henry • Jones, Sam • Ross, Mark
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12178632-B2
Publication Date
2024-12-31
Expiration Date
Abstract
A method to measure sound-producing behaviors of a subject with a power- and bandwidth-limited electronic device that includes a processor includes measuring, by a microphone communicatively coupled to the processor, sound in a vicinity of the subject to generate an audio data signal that represents the sound. The method also includes measuring, by at least one second sensor communicatively coupled to the processor, at least one parameter other than sound to generate at least a second data signal that represents the at least one parameter other than sound. The method also includes detecting one or more sound-producing behaviors of the subject based on: both the audio data signal and the second data signal; or information derived from both the audio data signal and the second data signal.
Core Innovation
The invention describes a power- and bandwidth-limited electronic device and method for measuring sound-producing behaviors of a subject using sensor fusion between a microphone and at least one sensor that monitors a parameter other than sound. The at least one sensor generates a second data signal, detects wake-up events, and in response to each wake-up event the device provides power to the microphone for a first fixed window of time or a second fixed window of time so the microphone measures sound in the vicinity of the subject to generate corresponding first and second audio data signals.
To manage audio validity under resource constraints, the method evaluates the signal quality of the first audio data signal and discards the first audio data signal when the signal quality is below threshold. The invention detects one or more sound-producing behaviors based on both the second audio data signal and the second data signal, or based on information derived from both, so detection is grounded in both audio and non-audio information.
The described device and method support selection of the parameter other than sound, including acceleration of the chest of the subject, angular velocity, skin temperature or core body temperature, blood oxygenation, blood flow, electrical activity of a heart, and electrodermal activity. The invention also contemplates detecting sound-producing behaviors using extracted audio features from the second audio data signal with subclassification of each audio feature as indicative of a corresponding type of sound-producing behavior.
In addition, the invention describes a feedback mechanism involving subject-confirming annotation input about sound-producing behaviors. The annotation input is reported to a remote server that aggregates inputs from multiple subjects to update an algorithm for detecting sound-producing behaviors, and the updated algorithm is received back by the device.
Claims Coverage
The independent claims are clm-00001 and clm-00011. Together, they define a wake-up-driven, power- and bandwidth-limited microphone capture scheme using a second data signal from a parameter other than sound, discarding low-quality audio, and detecting one or more sound-producing behaviors using both audio and non-audio modalities or derived information.
Power- and bandwidth-limited sound-measurement with parameter-other-than-sound wake-up
A method to measure sound-producing behaviors of a subject with a power- and bandwidth-limited electronic device, measuring at least one parameter other than sound to generate a second data signal, detecting wake-up events within the second data signal, and providing power to a microphone for fixed windows of time to generate first and second audio data signals while the microphone measures sound in the vicinity of the subject.
Signal-quality-based discarding of audio window
Based on a signal quality of the first audio data signal being below threshold, discarding the first audio data signal.
Multi-modal behavior detection using audio and non-audio data
Detecting one or more sound-producing behaviors based on both the second audio data signal and second data signal, or based on information derived from both.
Power- and bandwidth-limited electronic device implementing wake-up and detection operations
A power- and bandwidth-limited electronic device with a processor, a first sensor configured to monitor a parameter other than sound, a microphone, a controllable power source, and instructions that cause the device to receive a second data signal, detect wake-up events, provide microphone power for fixed windows to generate first and second audio data signals, discard the first audio data signal when its signal quality is below threshold, and detect one or more sound-producing behaviors based on both the second audio data signal and second data signal or information derived from both.
Across the independent claims, detection of sound-producing behaviors is defined as wake-up-driven, power- and bandwidth-limited microphone measurement triggered by a second data signal from a parameter other than sound, with discarding of a low-quality audio window and behavior detection using both the second audio data signal and the second data signal or derived information.
Stated Advantages
Enables detection of one or more sound-producing behaviors by combining audio and non-audio modalities using both the second audio data signal and second data signal or information derived from both.
Documented Applications
Validating or measuring sound-producing behaviors of a subject, including detection categories such as sneezing, wheezing, shortness of breath, chewing, swallowing, masturbation, sex, coughing, vomiting, and shouting.
Interested in licensing this patent?