Virtual cache architectures for reducing memory access latencies

Abstract

A cache memory system reduces memory access latencies by storing frequently used data in a fast storage whose size is usually much smaller that that of the main memory. As the speed of a processor increases, it becomes more difficult to design a cache memory that can satisfy the memory requests generated from the fast processor. Addressing cache memory by virtual addresses effectively reduces the cache access time by eliminating the time needed for address translation. The focus of this thesis is to propose virtual cache architectures to reduce the memory access latencies under diverse system architectures, i.e., on fast single processor systems, on shared memory multiprocessors. To achieve this purpose, we investigate the design issues and analyze problems concerned with performance of the virtual caches for those architectures. Then, we have developed three virtual cache schemes as follows in order to overcome the problems caused to be performance constraints. First, due to the fast cache access time, on-chip direct-mapped virtual caches are popular in fast single processor systems. A direct-mapped cache takes less time for accessing data than a set-associative cache because the time needed for selecting a cache line among the set is not necessary. The hit ratio of a direct-mapped cache, however, is lower due to the conflict misses caused by mapping multiple addresses to the same cache line. In order to solve the above problem, we propose a new virtual cache architecture whose access time is almost the same as the direct-mapped cache while the hit ratio is the same as the set-associative caches. The entire cache memory is divided into n banks, and each process is assigned to a bank. Then, each process runs on the assigned bank, and the cache behaves like a direct-mapped cache. A victim for cache replacement is selected from those that belong to a process which is most remote from being scheduled. Trace-driven simulations confirm that the new scheme removes a...