Optimizing performance of Epoll-based usable user-level network stack

Abstract

User level network stack is becoming more and more important as number of connection is expected to rise exponentially with recent IoT trend. mTCP has attracted companies and researchers as it supports developer-friendly socket API, but it has been shown that performance falls behind run-to-completion stacks. We uncover the reasons behind the performance drop of mTCP and show that the overhead does not come from supporting socket API. Overhead comes from synchrozing two kernel level threads, inefficinet cache utilization while processing batch of packets and strict copying application payload. Based on the analysis, we derive three optimization techniques for mTCP: user-level threading, simulated zero-copy ring buffer and packet sorting with header prefetch. Microbenchmark result shows that mTCP's performance is on par with IX when redoing the same experiment conducted in IX paper and macrobenchmark performed with L7 load balancer implementation shows that optimized mTCP performs 20% faster than vanilla mTCP.