Effect-driven Dynamic Selection of Physical Media for Visual IoT Services using Reinforcement Learning

Abstract

Recent advances in Internet of Things (IoT) technologies have encouraged web services to expand their provision boundary to physical environments by utilizing IoT devices. IoT services that generate and deliver physical effects to users via a space utilize such IoT devices as media to interact within a physical environment. Existing studies on dynamic service selection have only considered network-level quality of service (QoS) attributes, which cannot be used to evaluate the quality of the delivery of physical effects from a user perspective. Furthermore, to provide the services in a continuous manner, a dynamic selection of physical media is essential. Herein we propose a new metric called visual service effectiveness to evaluate how well a visual effect, generated using an IoT device as a medium, can be delivered to a user. Based on this metric, we also propose an effect-driven dynamic medium selection agent (EDMS-Agent) that conducts medium selection to maximize the visual service effectiveness during runtime and can be trained using reinforcement learning algorithms. We evaluated our EDMS-Agent by conducting several experiments in simulated IoT environments. The results show that simple distance-based metric is insufficient to measure the quality of physical effects in the user's perspective, and EDMS-Agent performs better than the baselines in terms of effectiveness, by learning the optimal policy of selecting media.