Discovering anomalies in big data - areview focused on the application of metaheuristics and machine learning techniques - review

Aug 5, 2026

Upload by Anda

Anomaly detection in big data supports automated identification of outliers for diagnosing faults and cyberattacks when manual labeling is impractical due to massive volumes. The review surveys state-of-the-art methods, emphasizing their advantages, limitations, and the fact that no single approach is universally best across datasets. It explains how data analysis can trigger alarms to prevent failures, reduce maintenance costs, and improve decision-making. It also highlights metaheuristics within machine learning to obtain more robust and efficient tools.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages17

Reading Time43 min

AccessFree to download

Introduction

Key concepts and applications of anomaly detection

Scope and organization of the review

Machine learning methods for real-time anomaly detection

Metaheuristics for optimizing machine learning

Case study and conclusions

TYPE Review PUBLISHED 17 August 2023 DOI 10. 3389/fdata.2023.1179625 OPEN ACCESS EDITED BY A. Fong, Western Michigan University, United States REVIEWED BY Guilherme De Alencar Barreto, Federal University of Ceara, Brazil Erik Cuevas, University of Guadalajara, Mexico *CORRESPONDENCE Claudia Cavallaro [claudia.cavallaro@unict.it](claudia.cavallaro@unict.it) RECEIVED 04 March 2023 ACCEPTED 24 July 2023 PUBLISHED 17 August 2023 CITATION Cavallaro C, Cutello V, Pavone M and Zito F (2023) Discovering anomalies in big data: areview focused on the application of metaheuristics and machine learning techniques. Front. Big Data 6:1179625 . doi: 10.3389/fdata.2023.1179625 COPYRIGHT © 2023 Cavallaro, Cutello, Pavone and Zito. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. Discovering anomalies in big data: a review focused on the application of metaheuristics and machine learning techniques Claudia Cavallaro*, Vincenzo Cutello, Mario Pavone and Francesco Zito Department of Mathematics and Computer Science, University of Catania, Catania, Italy With the increase in available data from computer systems and their security threats, interest in anomaly detection has increased as well in recent years. The need to diagnose faults and cyberattacks has also focused scientiﬁc research on the automated classiﬁcation of outliers in big data, as manual labeling is di􀀈cult in practice due to their huge volumes. The results obtained from data analysis can be used to generate alarms that anticipate anomalies and thus prevent system failures and attacks. Therefore, anomaly detection has the purpose of reducing maintenance costs as well as making decisions based on reports. During the last decade, the approaches proposed in the literature to classify unknown anomalies in log analysis, process analysis, and time series have been mainly based on machine learning and deep learning techniques. In this study, we provide an overview of current state-of-the-art methodologies, highlighting their advantagesand disadvantages and the new challenges. In particular, we will see that there is no absolute best method, i.e., for any given dataset a di􀀀erent method may achieve the best result. Finally, we describe how the use of metaheuristics within machine learning algorithms makes it possible to have more robust and e􀀈cient tools. KEYWORDS anomaly detection, machine learning, metaheuristics, classiﬁcation, deep learning, recurrent neural network, fault detection, security threats 1. Introduction By anomalies, we mean values that deviate signi􀀂cantly from the distribution of a dataset or events that do not conform to an expected pattern. Anomalies can be caused by errors, system tampering, or novelties such as never-observed events. The detection of anomalies can be done by setting a decision threshold that allows an objective criterion to separate the outliers from the normal values, but on large volume data, this or the application of traditional methods becomes quite impractical if not impossible. Anomaly detection (AD) is applied in various 􀀂elds such as log analysis, industrial control systems, diagnostic imaging, cybersecurity, and network monitoring. For instance, identifying medical anomalies allows for the provision of preventive treatments; the analysis of irregular images, videos, and audio has a great social impact because it also allows for the identi􀀂cation of identify fraud, illicit behavior, or fake users on social networks. In log analysis, where the goal is to 􀀂nd text that explains the nature and reas

FAQs

What are anomalies in this review, and what causes them?

Anomalies are values that significantly deviate from a dataset distribution or events that break an expected pattern. They can stem from errors, system tampering, or novel events such as never-observed cases.

Which fields commonly use anomaly detection?

Anomaly detection is applied across log analysis, industrial control systems, diagnostic imaging, cybersecurity, and network monitoring. It is used to support preventive actions such as treatment planning and fraud or illicit behavior identification.

How do metaheuristics relate to machine learning in anomaly detection?

The review explains that metaheuristics can be used within machine learning algorithms to optimize parameters and hyperparameters. This can lead to more robust and efficient tools and improve performance in anomaly detection tasks.

Aug 5, 2026

Upload by Anda