Top-K Rank Aggregation From M-Wise Comparisons

Abstract

Suppose one aims to identify only the top-K among a large collection of n items provided M-wise comparison data, where a set of M items in each data sample are ranked in order of individual preference. Natural questions that arise are as follows: 1) how one can reliably achieve the top-K rank aggregation task; and 2) how many M-wise samples one needs to achieve it. In this paper, we answer these two questions. First, we devise an algorithm that effectively converts M-wise samples into pairwise ones and employs a spectral method using the refined data. Second, we consider the Plackett-Luce (PL) model, a well-established statistical model, and characterize the minimal number of M-wise samples (i.e., sample complexity) required for reliable top-K ranking. It turns out to be inversely proportional to M. To characterize it, we derive a lower bound on the sample complexity and prove that our algorithm achieves the bound. Moreover, we conduct extensive numerical experiments to demonstrate that our algorithm not only attains the fundamental limit under the PL model but also provides robust ranking performance for a variety of applications that may not precisely fit the model. We corroborate our theoretical result using synthetic datasets, confirming that the sample complexity decreases at the rate of 1/M. Also, we verify the applicability of our algorithm in practice, showing that it performs well on various real-world datasets.