Statistical approaches for secure and clean internet

Abstract

In recent years, there has been an increasing need for preventing security issues such as malicious attacks on computers, the leakage of personal information, and the spread of harmful contents through the Internet. In this thesis, statistical approaches are proposed to practically address these issues. The approaches include: (i) an unsupervised data mining technique for online anomaly detection; (ii) a supervised data mining technique for clean Internet; and (iii) a statistical approach for efficient and secure database. First, this study focuses on online anomaly detection for network security. Here, a novel framework is developed for fully unsupervised training and online anomaly detection. The framework is designed so that an initial model is constructed and then it gradually evolves according to the current state of online data without any human intervention. In the framework, a self-organizing map (SOM) that is seamlessly combined with K-means clustering is transformed into an adaptive and dynamic algorithm suitable for real-time processing. The network structure of the SOM is appropriately adapted to incoming data and then dynamically clustered by using the K-means algorithm. In addition, clusters are automatically reconstructed or split up based on runtime accumulative measures. The performance of the proposed approach is evaluated through experiments using the well-known KDD Cup 1999 data set and further experiments using the honeypot data recently collected from Kyoto University. It is shown that the proposed approach can significantly increase the detection rate while the false alarm rate remains low. In particular, it is capable of detecting new types of attacks at the earliest possible time. Second, a hierarchical filtering system is proposed to detect an objectionable video. It consists of three phases using multiple features in different temporal domains. In the first phase, early detection is performed based on hash signatures prior to the ...