Enhancing multimodal interaction of intelligent interactive systems via thermal feedback

Abstract

Although many new technologies have been adopted in intelligent interactive systems, interactions are still experienced primarily through visual or auditory modalities. We investigated the new approach of using thermal feedback to leverage multimodal interactions in intelligent agent systems. We developed thermal output displays and conducted studies to investigate the feasibility and applicability of thermal feedback in two research contexts: 1) portable voice agent and 2) smart car system. In a portable voice agent context, the study revealed that the addition of thermal feedback to voice enhanced the understanding of information, enriched auditory contents and conveyed subsidiary feedback. In a smart car system, the first study on the perception of thermal feedback reported that thermal modality is a viable medium for interactions in a smart car. The second study on the usage of thermal feedback revealed that thermal feedback could increase the intuitiveness of interaction and convey gentle warnings without distracting drivers. Based on these results, We discuss design implications for applying thermal interaction in intelligent interactive systems.