(A) computing-in-memory-based human pose estimation accelerator with resource-efficient macro for mobile devices

Abstract

Human pose estimation (HPE) is a promising solution for accurately understanding the state and context of human actions in virtual reality (VR). A high frame rate with low-power HPE processing is required for a realistic user interaction experience in battery-limited mobile devices. The proposed HPE accelerator is a computing-in-memory (CIM) based accelerator that computes depth-wise separable convolution (DWSC) of a lightweight HPE network. Three key features contribute to a resource-efficient CIM accelerator: 1) Dual-mode CIM computes DWSC with a reconfigurable homogenous architecture, resulting in $2.68$ times higher throughput than previous analog CIMs. 2) Effective layer-aware unrolling performs bit-parallel computation on dual-mode CIM with fewer ADC operations, achieving 46 times higher throughput than before. 3) Adaptive fused intermacro balancing improves latency balance in layer fusion execution, leading to a $57.0 %$ higher frame rate than before. The proposed HPE accelerator is implemented in $28nm$ CMOS technology. It achieves higher computation resource utilization and operates HPE with a low energy-delay product of $27.6 uJ \cdot s$ in mobile VR devices.