High-performance networked storage application with cooperative smart NICs

Abstract

The performance of the storage device has been growing rapidly over the last 20 years. As a result, storage software stack, not the latency of storage, become a performance bottleneck recently. For example, software latency of NVM access takes over 90% of total access. Some research works have been tackled the problem by proposing kernel bypass device access. SPDK, a famous user level storage library, reduces software overhead of kernel storage stack such as redundant memory copy and context switching overhead. However, their solutions still have inherent software inefficiencies such as message passing overhead or redundant memory copy which affect to I/O performance. In this paper, we propose a SmartNIC based offloading techniques for high-performance storage server access in storage server. First, we minimize inter-thread communication overhead to increase core scalability of storage server by query distribution offloading on SmartNIC. Query offloading enables that SmartNIC parses query payload and routes to dedicated thread. So, each thread can run without message passing overhead. Second, we minimize CPU utilization by offloads block device access and packet generating which is called context passing. Our experimental result shows that query distribution offloads improves the scalability up to 2.2 times, and context passing shows 1.8x faster throughput and 6x lower tail latency compared to existing methods.