Disaggregated memory system providing high availability

Abstract

Resource disaggregation provides a way to improve the efficiency of data center by splitting each hardware resource to separate resource pools such as compute nodes and memory nodes. Resource disaggregation can be an efficient solution to address the resource under-utilization problem in data centers by elastic management of hardware resources. To become a feasible solution, Disaggregated memory systems need to address the costs associated with data movements between compute and memory nodes because the data movements are highly frequent. However, the mechanisms used in most existing disaggregated memory systems still show substantial overhead because they commonly reuse the slow swap system for the ease of implementation. This research proposes a system, TribuOS, for system-level memory disaggregation. TribuOS designs an efficient data path for fetching and evicting remote memory pages, and builds a new subsystem that replaces swap system. The TribuOS’ efficient subsystem reduces the cost of data movements between compute and memory nodes. To reduce fetch latency, TribuOS eliminates the need of using expensive swap cache operation by not evicting shared anonymous pages. TribuOS hides a significant portion of eviction overhead by overlapping network latency and kernel execution. For reliability, TribuOS replicates states of a memory node. The fast eviction performance also contributes to reducing the performance cost of reliability when replicating evicted pages. TribuOS is quantitatively compared with the state-of-the-art disaggregated memory system, Fastswap, showing up to 4x better performance. Our failure and recovery experiment demonstrates that TribuOS can recover replicas and tolerate a cascading failure with negligible downtime due to recovery.