Taking WYSIWYG to the next dimension: Appearance-based spatial user interfaces

Abstract

WYSIWYG, which stands for "what you see is what you get," is an intuitive human-computer interaction paradigm that allows users to directly interact with visible elements. However, application of the WYSIWYG principle, particularly in the design of spatial user interfaces, can be challenging, as interactivity is no longer confined to the pixels on the screen and includes the entire space surrounding the user. The usability of a spatial user interface can quickly degrade without careful consideration of how various hardware capabilities dictate of how visual elements appear to users in interactive spaces, and of how physiological aspects of the human body dictate how users see them. This study presents two novel spatial interaction techniques that explore an appearance-based approach to spatial user interfaces: Binocular Cursor and Projective Windows. Binocular Cursor enables performant, unambiguous selection of distant objects through a transparent display

it deploys a unique strategy to mitigate the problems posed by the binocular parallax. Projective Windows enables easy spatial arrangements of 3D windows by utilizing projective geometry and fluid handovers between windows’ absolute and apparent sizes. With these appearance-based techniques, the user literally gets what is seen. A common English saying is that appearances can be deceiving, but this study argues, based on quantitative evaluations of Binocular Cursor and Projective Windows, that precisely the opposite is true: Appearances are everything in spatial user interfaces.