ISeeU: Camera-based user interface for a handheld computer

Abstract

"Tilt scrolling" and "peephole display" are popular user interface ideas for small computers, and inertial sensors were often the choice for the realization of such ideas. Inertial sensors, however, have two fundamental limitations; the frame of reference is not the user but the earth, and drifting errors are difficult to overcome. A possibly better solution, free from such limitations, is machine vision. Machine vision was a luxury for small computers but is becoming a practical solution because a camera is now a common component of small computers and the required vision algorithm is already running in optical mice. The vision algorithm of our prototype, which we called ISeeU, tracks simple features in the user's face and calculates lateral displacements and changes in distance, which in turn are used to control scrolling and zooming. An informal test with scrolling tasks indicates that its performance is comparable with a user interface using arrow keys.