Transformation-based community detection from social networks

Abstract

In recent decades social network analysis has become one of the most attractive issues in data mining. In particular, community detection in networks, also known as graph clustering, is a fundamental problem in social network analysis. This is the procedure of identifying the community structure, with many edges joining vertices in the same community and relatively few edges joining vertices of different communities. Many theories, models, and methods have been actively developed for this purpose. However, owing to a wide variety of network structures, there remain challenges to determining community structures from social networks. Among them, we focus on solving four important problems for community detection with different underlying structures. These are identifying the community structure of a graph when it consists of (i) overlapping community structure, (ii) highly mixed community structure, (iii) complex sub-structure, and (iv) highly mixed overlapping community structure.

In this dissertation, we address these interesting problems by developing transformation techniques for the networks. Our key motivation is that the transformation of a given network can provide us an improved structure to identify the community structure of an original network, as a kernel trick does for data mining and machine learning. We propose a transformation-based community detection algorithm for networks that converts an original graph to a transformed graph that reflects the structure of the original network and has a superior structure for each problem. We identify the community structure using the transformed graph and then the membership on the transformed graph is translated back to that of the original graph. Using these techniques, we solve the four problems in this dissertation.

For the first problem, we present a novel notion of the link-space transformation. This notion enables us to combine the advantages of both the original graph and the line graph, thereby conveniently achieving high-quality overlapping communities. Based on this notion, we develop two overlapping community detection algorithms: LinkSCAN and $LinkSCAN^{\ast}$ . The latter is an enhancement of the former for higher efficiency. We conduct extensive experiments using various synthetic and real-world networks. The results on the synthetic networks demonstrate that, in terms of the Normalized Mutual Information (NMI), the proposed algorithms outperform the existing algorithms, especially for networks with many highly overlapping nodes and those with various base structures (e.g., a wide range of average degrees or community densities). Furthermore, for real-world networks, the proposed algorithms demonstrate higher quality and coverage than existing algorithms.

For the second problem, we propose a new community detection algorithm that we call BlackHole. Motivated by the common nature between community detection and graph embedding, we explore the possibility of applying the techniques of graph embedding to community detection. The proposed new graph embedding model attempts to place the vertices of the same community at a single position, similar to how a black hole in space attracts everything nearby. Then, these positions are easily grouped by a conventional clustering algorithm. The lesson from graph embedding is that we should consider not only attractive forces and but also repulsive forces. We theoretically prove the superiority of the proposed algorithm to representative embedding-based algorithms and empirically verify our claims through extensive experiments. The proposed algorithm is proven to achieve extremely high performance regardless of the difficulty of community detection in terms of the mixing parameter, the clustering coefficient, and other factors. If the community structure is not easily detectable, the strength of the proposed algorithm becomes indeed prominent because other algorithms frequently fail to detect communities.

For the third problem, we propose a motif-based embedding method for graph clustering by modeling higher-order relationships among vertices in a graph. The proposed embedding method considers the degree-weighted repulsive forces and motif-based attractive forces to enhance the clustering tendency of points in the output space of graph embedding. We theoretically prove the relationship with graph clustering and verify the performance and applicability through extensive experiments. In particular, we verify the superiority of using triangle- and wedge- based embedding for identifying community structure corresponding to such substructures of interest.

For the fourth problem, we develop an algorithm to address the highly mixed overlapping community detection problem. The proposed algorithm LinkBlackHole is also a transformation-based community detection algorithm

however, its transformation consists of a sequence of two different transformations. By first applying the link-space transformation and then the BlackHole transformation, we can detect highly mixed communities of the link-space graph and highly mixed link communities equivalently. Therefore, a merger of the two transformations uncovers the highly mixed overlapping community structure. We also present a set of experimental results confirming that the algorithm outperforms the existing algorithms for such structures.

The strength of this dissertation is based on a wide variety of community detection problems from social networks. We believe that this work will enhance the quality of community discovery for social network analysis.