(A) stereo matching accelerator using functional CMOS image sensor

Abstract

A low-latency and low-power stereo matching accelerator monolithically integrated together with CMOS image sensor (CIS) for mobile applications. To reduce the overall latency, focal-plane processing is adopted by using the proposed analog census transform circuit (ACTC), and the image readout is pipelined with the following stereo matching process. In addition, a novel focal-plane rectication pixelarray (FRPA) merges the rectification with the image readout without any additional processing latency. For area-efficient pixel design, sparse rectification is proposed, and the image rectification is implemented with only 2 additional switches in each pixel, while 98.57% of depth-map has only 0.96cm or less depth error. A stereo matching digital processor (SMDP) is integrated with the CIS for the cost aggregation. Simulated in 65nm CMOS process, the FRPA, the ACTC, and the SMDP achieve 11.0 ms latency with the complete stereo matching stages, which is suitable for smooth user interface. As a result, 2-chip stereo matching system dissipates $573.9 \mu J/frame$ and achieves $8.76 \times 10^9$ disparity-evaluation/J energy efficiency.