Logical distributed counter for scalable file access

Abstract

We address a memory-efficient manycore-scalable distributed reference counter for scalable file access, Logical Distributed Counting(LODIC). The LODIC allocate the local counter on a per-process basis. The per-process based distributed counter design solves the excessive memory pressure and the counter query latency problems of the existing per-core based distributed counting design. The LODIC is designed to dynamically incorporate the three characteristics for reference counter: (i) the population of the object, (ii) the reference brevity, and (iii) the degree of sharing. The key ingredients of the logical distributed counting are Memory mapping, Counter Embedding, and Process-space based reverse mapping. Via mapping a file region to the process address space, LODIC can allocate the local counter at the process address space. With Counter Embedding, the logical distributed counting defines the local counters without the significant changes in the existing kernel code and without introducing significant memory overhead for the local counters. Exploiting the virtual memory segment allocation algorithm of the existing Linux kernel, the process-space based reverse mapping find the local counter of the physical page without the substantial overhead. Logical Distributed Counting increases the throughput by 80× against stock Linux in reading the shared file block. LODIC exhibits as good performance as the ideal scalable reference counter when applied to the NGINX (web server) application.