(A) graphics and vision unified processor with pose estimation engine for augmented reality

Abstract

Recently, the interest in augmented reality (AR) has been increased with the enhanced capability of the mobile devices. The goal of AR is to combine a real scene viewed by a user and a computer generated scene. Therefore, the AR allows the user to perceive the virtual object as a real object and acquire information in the real world. This AR application spreads on a wide range from entertainment to education and navigation. Due to its convenience and mobile-friendly applications, the AR system is suitable for mobile environment rather than PC platform. However, the restricted computing capability is a major problem in the mobile AR. Interdisciplinary knowledge is involved in AR, including computer vision for image analysis and computer graphics for image synthesis. Both vision and graphics applications require high computational power, real-time operation, and low power consumption. They also need programmability for wide variety of recognition and rendering targets. To satisfy these requirements, AR processing needs hardware support even in the mobile environment. Hardware support for graphics processing is relatively simple because many handheld devices have been using graphics hardware. Therefore, recent approaches for mobile graphics processor are useful. To support vision processing, two approaches are available. First, since the vision processing requires high computational power, vision processors have been introduced to deal with complex algorithms. However, it is over-head to incorporate both graphics and vision processors within limited area and power budget. Second, the vision computing on the graphics hardware has been researched recently. They utilized the graphics processing units (GPUs) to expand the computational resources that are available to vision and image processing algorithms. However, these researches focus only on the primitive operations using massively parallel processing architecture of GPU which cannot exploit object- and task-lev...