(An) efficient network anomaly detection method based on unsupervised learning

Abstract

With the growing rate of inter-connections among computer systems, reliable network communication is becoming a major challenge. In order to ensure network security, Intrusion Detection Systems (IDS) are being designed to protect the availability, confidentiality and integrity of critical networked information systems. Most current IDSs employ signature based methods and learning algorithms which rely on labeled data to train. Though this approach is highly successful in detecting occurrences of previously known attacks, these methods generally have difficulty in detecting new types of attack and training data is typically expensive. Therefore, the purpose of our anomaly detection scheme is 1) to identify important input features in building a IDS that is computationally efficient and effective and 2) to develop an unsupervised anomaly detection technique in order to learn normal and anomalous patterns from training data and generate classifiers used to detect attacks. To identify important input features, we develop a hybrid feature selection technique in which Principal Components Analysis is combined with optimized K-means clustering technique. Based on this result, we evaluate the performance of intrusion detection based on Self Organizing Map. The Experiment results with KDD Cup 1999 dataset show several advantages in terms of computational complexity and our method achieves significant detection rate which shows possibility of detecting successfully attacks.