Learned indexing without parameter tuning using nonparametric density estimation

Abstract

Learned indexes utilize machine learning models to predict the position of a data record for a given lookup key. On read-intensive tasks, it has been shown that learned index structures are more efficient than traditional tree-like structures (e.g., B-Trees) in terms of lookup time and memory consumption. However, existing learned indexes require the tuning of parameters for the index to achieve the best performance. Though parameter tuning can significantly affect lookup time and memory consumption, it often requires too much time and effort to find the optimal parameters for the given environment. In this paper, we propose the Log-Concave Density Estimation (LCDE) index that uses nonparametric density estimation in the process of index construction. LCDE learns the distribution of data under a constrained memory budget, without requiring manual parameter tuning. Through intensive experiments on various workloads, we show that LCDE outperforms other existing methods in most cases in terms of query throughput and space consumption.