CMOS image sensor-based gaze estimation processor for smart glasses

Abstract

Recently, smart glasses are spotlighted as a next-generation smart device. By using these devices, users can search information from the web or take a picture and upload it to social networks, while doing other jobs with their two hands freely. One thing required in this process is new type of user interface which does not employ hands when manipulating the device.

In the platform of smart glasses, eyes can be used to construct a user interface that satisfies the requirements. Gaze estimation algorithm provides corresponding operations, tracking the pupil center and mapping it to the gaze point. In this system, a CMOS image sensor is integrated with a vision processor to implement a gaze estimation system-on-chip, and full process of the pupil center detection and the gaze es-timation is performed in a single chip. For high accuracy in the pupil center detection, vision processor performs ellipse fitting with floating-point numbers. Logarithmic floating-point datapath has been implemented to reduce the area and power consumption of floating-point ALU. In addition, mixed-signal circuits are proposed to detect the pupil contour and the glint area. These circuits are integrated in a sensor domain, and directly process analog signals provided from pixel array. As a result, the computational load, hence the power consumption, of the vision processor is decreased.

The proposed system has been implemented using 65nm CMOS process. It achieves 34mW power consumption at 30fps real-time operation and 3360x3360 $\mum^2$ area consumption, which includes 320x240 pixel array. In experiments, the system shows less than 1 pixel accuracy in 320x240 resolution for the pupil center detection and 0.5$^\circ$ accuracy for the gaze estimation.