Anomaly recognition in information technology environments
Inventors
Moss, Chris • Kofkin-Hansen, Simon J. • Shamir, Jordan • Conley, Devin • Hoff, James Patrick • Mccown, Iain • Moonen, Scott • Buckland, Bryan M.
Assignees
International Business Machines Corp • Government of the United States of America
Publication Number
US-11321164-B2
Publication Date
2022-05-03
Expiration Date
2040-06-29
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Abstract
A method comprises obtaining a set of log files for a software system. The set of log files applies to an extended window. A periodic pattern in a first set of error-event surges in the set of log files is identified. The error-event surges in the first set is identified as event noise. A second set of log files for the software system is obtained. The second set of log files applies to a shortened window. Timeseries analysis on the second set of log files is performed. A particular error-event surge in a detection period in the second set of log files that is abnormal as compared to the shortened window is detected based on the timeseries analysis. That the particular error-event surge does not fit into the periodic pattern is determined, the particular error-event surge is characterized as an anomaly, based on the determining.
Core Innovation
The invention relates to automatically detecting anomalies in software systems, particularly converged software stacks, by analyzing error-event surges in log files. The method involves obtaining a set of log files over an extended window and identifying periodic patterns in error-event surges, which are characterized as event noise. Then, a second set of log files over a shortened window is obtained and analyzed using timeseries analysis to detect abnormal error-event surges that deviate from the periodic pattern, characterizing such surges as anomalies.
The problem addressed is the challenge of diagnosing system problems through log file analysis in complex software systems, especially converged software stacks that generate numerous and varied logs. Manual log review is often infeasible due to the volume and complexity, and existing automated systems relying on pre-defined event combinations are ineffective in such environments. Additionally, frequent false-positive events (event noise) can cause unnecessary alerts and resource expenditure, complicating health monitoring and problem diagnosis.
Claims Coverage
The patent includes three independent claims covering a method, system, and computer program product for anomaly detection in software systems.
Detection of anomaly based on error-event surge in shortened window
Obtaining a first set of log files for a software system applying to a shortened window; detecting a particular error-event surge in a detection period that is abnormal as compared to the shortened window; determining that the surge does not fit a periodic pattern of error-event surges in an extended window; characterizing the surge as an anomaly.
Identification of periodic patterns in an extended window
Obtaining a second set of log files applying to an extended window; identifying the periodic pattern in a set of error-event surges within the second set, where the identifying involves inserting the second set of log files into a sequence detection algorithm.
Characterizing anomaly window by merging adjacent activity periods
Defining an anomaly window for the characterized anomaly by merging an activity period in which the particular error-event surge occurred with an adjacent period.
The claims cover methods, systems, and computer program products that detect anomaly surges in software system logs by contrasting recent error-event surges against periodic patterns found in extended log sets, with features including aggregation, normalization, and defining merged anomaly windows for precise anomaly characterization.
Stated Advantages
Efficiently identifies actual system anomalies by differentiating them from periodic event noise, improving monitoring accuracy.
Reduces the time and resources IT professionals spend on manual log analysis and false-positive troubleshooting in complex software stacks.
Enables automated classification and resolution recommendation for detected anomalies, assisting IT personnel in rapid problem diagnosis and correction.
Supports modular and configurable algorithms for analysis and detection, allowing adaptation to different software systems and user preferences.
Documented Applications
Monitoring the health and performance of complex software solutions such as converged software stacks, including cloud computing environments.
Automatic detection of anomalies in software systems by analyzing log file error event surges over varied time windows.
Classifying and resolving system anomalies based on detected anomalous log events, including searching knowledge databases for relevant remedies.
Providing alerting systems for IT professionals to efficiently identify when software systems need attention while filtering out false positives.
Interested in licensing this patent?