Tracking and Predicting the Evolution of Research Topics in Scientific Literature

Abstract

The exponential rise in the volume of publications and the prevalence of multidisciplinary practice in scientific domains has made it increasingly difficult to keep track of changes in research trends. In this paper, we propose a framework for determining persistent and emerging research topics in scientific literature. The topics were represented as non-overlapping communities of keywords in a dynamic cooccurrence network derived from 21 million articles in PubMed that were published from 1980 to 2016. We detected a set of communities for each snapshot of the network and traced their instances in consecutive periods using a similarity threshold. Our approach provides a retrospective analysis of changes in research topics: their formation, growth, shrinkage, survival, merging, splitting, and dissolution. We also show that a feature set comprising of 43 temporal and structural attributes from these keyword communities can be used to predict their evolution. In particular, we found that the frequency of cooccurrences and the appearance of new keywords within the community are highly predictive of its persistence or dissolution in the next five years.