M.Gesture: An Acceleration-Based Gesture Authoring System on Multiple Handheld and Wearable Devices

Abstract

Gesture-based interaction is still underutilized in the mobile context despite the large amount of attention it has been given. Using accelerometers that are widely available in mobile devices, we developed M.Gesture, a software system that supports accelerometer-based gesture authoring on single or multiple mobile devices. The development was based on a formative study that showed users' preferences for subtle, simple motions and synchronized, multi-device gestures. M.Gesture adopts an acceleration data space and interface components based on mass-spring analogy and combines the strengths of both demonstration-based and declarative approaches. Also, gesture declaration is done by specifying a mass-spring trajectory with planes in the acceleration space. For iterative gesture modification, multi-level feedbacks are provided as well. The results of evaluative studies have shown good usability and higher recognition performance than that of dynamic time warping for simple gesture authoring. Later, we discuss the benefits of applying a physical metaphor and hybrid approach.