Design and implementation of a TCP stack for offloading file IOs for scalable content delivery

Abstract

The rapid growth of the amount and quality of online content requires high-speed delivery service. Recent advancement of high-bandwidth I/O devices promises scalable content delivery, but the traditional CPU-centric programming model often severely limits the true potential of modern I/O devices. Our experiments reveal that over 70% of CPU cycles are spent on simple tasks such as disk and network I/O operations in online content delivery. In this work, we present IO-TCP, a split TCP stack design that drastically reduces CPU burden on online content delivery. IO-TCP offloads disk I/O and TCP packet transfer to SmartNIC while the rest of the operations are executed on the CPU side. This division of labor realizes the separation of control and data planes of a TCP stack where the CPU side assumes the full control of the stack operation while only the data plane operations are offloaded to SmartNIC for high performance. Our evaluation shows that IO-TCP can saturate four NVMe disks with only one CPU core for a disk-bound workload, as compared to without NIC offloading that cannot saturate the four NVMe disks with all ten CPU cores.