Enhancing live video quality at ingest point utilizing online trained DNNs

Abstract

Live video accounts for a significant volume of today’s internet video traffic and is steadily growing. There has been an effort to enhance user quality of experience (QoE) by looking into both ingest and distribution of the end-to-end live video delivery process. However, there still exists a fundamental limitation in live video delivery: user QoE highly depends on the ingest side bandwidth resource. This bandwidth constraint restricts the upload quality of live video from origin and, as a result, deprives end users of their opportunities to watch high-quality live stream. We present LiveNAS, a new live video ingest framework that enhances the live video quality independent of the ingest side network bandwidth. LiveNAS achieves this by utilizing super-resolution DNNs at the ingest point leveraging its computational power. Due to the unpredictable nature of live video, pre-trained DNNs cannot deliver its training accuracy for unseen/new content. To improve this, we propose a content-adaptive online training approach that consists of two major components. First, origin streamer transmits partial high-quality frames to the ingest point along with encoded real-time video. Second, at the ingest point, server adapts DNNs to live streaming content by learning a mapping from low-quality video to a high-quality version using partial frames received from the streamer. We address several challenges in enabling this key approach such as 1) balancing allocation of upload bandwidth between training data and real-time video, 2) scheduling GPU usage dedicated to online training. Our online training approach improves generic DNN quality by average 1.36 dB in PSNR and achieves an average of 2.57 dB overall video quality improvement in PSNR over state-of-the-art ingest framework.