(A) PIM-based SparsePU for transformer model acceleration with bit-slice level sparsity exploitation

Abstract

This paper presents SparsePU, a processing unit capable of leveraging bit-slice level sparsity for accelerating transformer models within a processing-in-memory (PIM) architecture. This processor achieves performance enhancements by utilizing both activation and weight unstructured bit-slice level sparsity, which has been challenging in conventional PIM structures. The proposed accelerator accelerates operations by performing row-wise matrix multiplication for activation sparsity and enables simultaneous acceleration for various ratios of weight sparsity through a row-wise compressed weight data format. It integrates a network within the accelerator for effective and efficient accumulation of compressed weight data. Additionally, it maximizes operational acceleration in high activation sparsity scenarios through a multi-row skipping scheme. The accelerator significantly enhances performance, achieving up to 857.27x faster computation in actual transformer model layers, and reduces the size of sparse weight data to be stored by up to 93.68%.