(A) multi-objective framework for dynamic QoE across users in video streaming

Abstract

Optimizing for user quality of experience (QoE) is a prevailing approach to improving Internet video streaming. The effectiveness of adaptive bitrate (ABR) algorithms is often measured by a weighted combination of conflicting objectives, such as video quality, delay, and smoothness. We highlight the subjectivity of QoE reward formulation, which prevails, but results in various and discrete QoE metrics that assume a correct knowledge of user preferences. We propose MARVEL (A multi-objective approach to reinforcement “video experience” learning), a framework for reformulating QoE as a generalized multi-objective problem with dynamic weights, discarding the assumption of prior knowledge of user preferences. We employ a multi-objective reinforcement learning (MORL) based module within the framework, demonstrating an applied solution that adapts to dynamic user preferences (both across users and within the same session). In an experimental trial with randomized across possible user preferences weights, we show MARVEL learned the frontier solutions of different preferences on the two-dimensional metric. MARVEL outperforms previous RL-based approaches to ABR by 16% ∼ 39% in both adaptations and single objective analysis.