Exploiting bandwidth temporal loads for low latency and high bandwidth memory

Abstract

Increasing clock frequencies and issue rates aggravates the memory latency problem, imposing higher memory bandwidth requirements. While caches can be multi-ported to provide high memory bandwidth, the increase in access latency with the increase in the number of ports limits their potential. The paper proposes a novel technique, called the 'temporal load cache architecture', to reduce load latencies and provide higher memory bandwidths. The key motivation for the technique is that temporal loads - dynamic instances of a static load instruction that access the same address as that accessed by the last dynamic instance of the same static load - constitute 48% of all dynamic loads on average for the SPEC2000 benchmarks. When a load is predicted to be temporal, the data predicted to be accessed by it are read early in the pipeline from a small temporal load cache that stores the temporal data. The proposed temporal load cache architecture has two main advantages. First, since instructions dependent on a temporal load are provided with their data early in the pipeline, they can be issued as soon as they resolve their remaining data dependences and resource conflicts. Second, since a large percentage of loads can be filtered by the temporal load cache, the main data cache can service other (nontemporal) loads better, providing higher memory bandwidth. The experimental results show that the proposed temporal load cache architecture improves performance by 8.3% on average for the SPEC2000 integer benchmarks.