LineageBA: A Fast, Exact and Scalable Graph Generation for the Barabasi-Albert Model

Abstract

The Barabasi-Albert (BA) model plays an important role in many domains since it can generate a scale-free graph having the degree exponents that real graphs have. However, due to the dependency among the edges generated at different time steps, the exact generation methods support only a single thread, and the parallel generation methods generate a graph only approximately. There is no method that can generate a large-scale graph following the BA model strictly using multiple threads. We propose a fast, exact, and scalable graph generation method called LineageBA that solves the above issue. We propose the concept of lineage relationship for reducing memory usage significantly and the detection of hash collisions for parallelizing the graph generation. Through extensive experiments, we have shown that LineageBA significantly outperforms the state-of-the-art BA graph generation methods and easily generates 2.5 trillion edges within four hours using a small cluster of PCs.