Systems and methods for analyzing time series data based on event transitions
Inventors
Chen, Jixu • Tu, Peter Henry • Chang, Ming-Ching • Kim, Yelin • Lyu, Siwei
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-9984154-B2
Publication Date
2018-05-29
Expiration Date
Abstract
A method for analyzing time series data to identify an event of interest is provided. The method includes receiving, at a computing device, time series data that includes the event of interest, identifying, using the computing device, a start time of the event of interest and an end time of the event of interest by modeling at least one transitional pattern in the time series data, and categorizing, using the computing device, the event of interest based on the at least one transitional pattern.
Core Innovation
The invention analyzes image time series data that includes an event of interest, where the event of interest corresponds to an action performed by a subject. Using a computing device, it identifies a start time of the event of interest and an end time of the event of interest by modeling at least one transitional pattern in the image time series data, and the transitional pattern includes an onset segment and an offset segment for the event.
The onset segment models a transition between a first neutral event and the event of interest as a first function of a joint angle of a joint of the subject. The offset segment models a transition between the event of interest and a second neutral event as a second function of the joint angle. The model therefore uses subject joint angles to represent transitions between neutral states and the event of interest.
After modeling the transitional pattern, the invention categorizes the event of interest based on the transitional pattern. The transitional structure separates neutral event states from the event-of-interest segment types such as neutral, onset, peak, and offset.
In example usages described in the document, the framework is applied to suspicious behavior in smartroom datasets and actions in MAD datasets. The analysis is reported to provide improved performance at frame-level and event-level precision, recall, and F-measure, and to be robust under noisy pose estimation.
Claims Coverage
The document contains three independent claims (method, computing device, and non-transitory computer-readable storage media). The independent claims center on modeling transitional patterns with onset and offset segments defined as functions of joint angles, identifying event start and end times, and categorizing the event of interest based on the transitional pattern.
Onset and offset transitional modeling from joint angles
Modeling an onset segment that represents a transition between a first neutral event and the event of interest as a first function of a joint angle of a joint of the subject, and modeling an offset segment that represents a transition between the event of interest and a second neutral event as a second function of the joint angle.
Joint-angle transition-based event boundary identification
Identifying a start time of the event of interest and an end time of the event of interest by modeling at least one transitional pattern in the image time series data using the onset segment and offset segment.
Categorizing the event based on transitional patterns
Categorizing the event of interest based on the at least one transitional pattern.
Computing device implementation of transitional event analysis
Providing a computing device with a processor configured to receive image time series data including the event of interest, identify the start and end times by modeling transitional patterns including onset and offset segments as functions of joint angles, and categorize the event of interest based on the transitional pattern.
Non-transitory computer-readable instructions for transitional event analysis
Providing at least one non-transitory computer-readable storage media having computer-executable instructions which, when executed by at least one processor, receive image time series data including the event of interest, identify the start and end times by modeling transitional patterns using onset and offset segments as functions of joint angles, and categorize the event of interest based on the at least one transitional pattern.
Across all independent claims, the core claim coverage is the joint-angle-based onset/offset transitional modeling used to identify the event start time and end time, followed by categorizing the event of interest based on the modeled transitional patterns.
Stated Advantages
Improved performance over known methods, reported as quantitative frame-level and event-level precision, recall, and F-measure.
Robustness under noisy pose estimation.
Documented Applications
Smartroom suspicious behavior dataset use for event analysis and categorization.
MAD dataset actions use for event analysis and categorization.
Application to single or group activities.
Application to other time-series domains.
Interested in licensing this patent?