(A) low-power and real-time 3-D object recognition system-on-chip

Abstract

This paper introduces the study of system-on-chip (SoC) designs including hardware-friendly algorithm design, hardware architecture design, ASIC design, and system board design for low-power and low-latency 3-D object recognition SoC in mobile platforms. A 3-D object recognition system consists of 3-D RGB-D data generation and 3-D perception, and the efficient SoC is proposed to accelerate all of them. Specifically, the proposed SoC adopts deep learning-based monocular depth estimation and sensor fusion algorithms with a low-power depth sensor to generate accurate and robust 3-D RGB-D data, which overcomes the disadvantages of previous 3-D RGB-D generation systems that highly rely on high-performance depth sensors. Then, a 3-D point cloud-based neural network is executed on 3-D RGB-D data for accurate 3-D perception. Therefore, this system achieves an accurate and robust 3-D object recognition system with a minimum sensor power consumption. Key features for accelerating this system in mobile platforms are as follows. The window-based search algorithm and its acceleration hardware unit are proposed to efficiently accelerate point processing algorithms. Moreover, band matrix encoding and decoding algorithms are developed to reduce data transactions, and their acceleration hardware units are designed to improve the speed of sensor fusion executions. In addition, zero input and output data acceleration methods through a sparse bit-slice architecture are proposed to maximize the processing speed of matrix multiplication. In addition, the point feature reuse algorithm is proposed to solve redundant operations caused by 3-D point cloud-based neural networks, and its hardware architecture is designed to increase throughput. With these key features, the system-on-chip is fabricated in a silicon chip, and the demonstration system board is designed to evaluate the silicon chip, finally showing the real-time implementation of the 3-D object recognition system. Consequently, the proposed system-on-chip achieves 281.6 mW power consumption and 31.3 ms processing time of the 3-D object recognition system.