Viola, L.; Ronchieri, E.; Cavallaro, C. Combining Log Files and Monitoring Data to Detect Anomaly Patterns in a Data Center. Computers2022, 11, 117.
Viola, L.; Ronchieri, E.; Cavallaro, C. Combining Log Files and Monitoring Data to Detect Anomaly Patterns in a Data Center. Computers 2022, 11, 117.
Viola, L.; Ronchieri, E.; Cavallaro, C. Combining Log Files and Monitoring Data to Detect Anomaly Patterns in a Data Center. Computers2022, 11, 117.
Viola, L.; Ronchieri, E.; Cavallaro, C. Combining Log Files and Monitoring Data to Detect Anomaly Patterns in a Data Center. Computers 2022, 11, 117.
Abstract
Context: Anomaly detection in a data center is a challenging task, having to consider different services on various resources. Current literature shows the application of artificial intelligence techniques to either log files or monitoring data: the former created by services at run time, while the latter produced by specific sensors directly on the physical or virtual machine.
Objectives: We propose a model that exploits information both in log files and monitoring data to identify patterns and detect anomalies over time.
Methods: The key idea is to use on one side natural language processing solutions to detect problems at service level, extracting words that represent anomalies. Clustering and topic modeling techniques have been used to identify patterns and group them with respect to topics. On the other side time series anomaly detection technique has been applied to sensors data in order to combine problems found in the log files with problems stored in the monitoring data.
Results: We have tested our approach on a real data center equipped with log files and monitoring data that can characterize the behaviour of physical and virtual resources in production. We have observed a correspondence between anomalies in log files and monitoring data, e.g. an increase in memory usage or machine load. The results are extremely promising.
Conclusion: Our model requires to integrate site administrators' expertise in order to consider all critical scenario in the data center and understand results properly.
Keywords
log analysis; monitoring data; anomaly detection; natural language processing; topic modeling; clustering technique; time series anomaly detection
Subject
Computer Science and Mathematics, Information Systems
Copyright:
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
