An objective and subjective quality assessment study of passive gaming video streaming. Issue 3 (22nd November 2018)

• Record Type: Journal Article
• Authors: Barman, Nabajeet; Zadtootaghaj, Saman; Schmidt, Steven; Martini, Maria G.; Möller, Sebastian
• Other Names: Wamser Florian guestEditor.; Alay Özgü guestEditor.; Metzger Florian guestEditor.; Valentin Stefan guestEditor.
• Is Part Of: International journal of network management. Volume 30:Issue 3(2020)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2018-11-22
• Subjects: Computer networks -- Management -- Periodicals; 004.6
• DOI: 10.1002/nem.2054
• Languages: English
• ISSNs: 1055-7148

Abstract

Summary: Passive gaming video‐streaming applications have recently gained much attention as evident with the rising popularity of many Over The Top (OTT) providers such as Twitch.tv and YouTube Gaming. For the continued success of such services, it is imperative that the user Quality of Experience (QoE) remains high, which is usually assessed using subjective and objective video quality assessment methods. Recent years have seen tremendous advancement in the field of objective video quality assessment (VQA) metrics, with the development of models that can predict the quality of the videos streamed over the Internet. A study on the performance of objective VQA on gaming videos, which are artificial and synthetic and have different streaming requirements than traditionally streamed videos, is still missing. Towards this end, we present in this paper an objective and subjective quality assessment study on gaming videos considering passive streaming applications. Subjective ratings are obtained for 90 stimuli generated by encoding six different video games in multiple resolution‐bitrate pairs. Objective quality performance evaluation considering eight widely used VQA metrics is performed using the subjective test results and on a data set of 24 reference videos and 576 compressed sequences obtained by encoding them in 24 resolution‐bitrate pairs. Our results indicate that Video Multimethod Assessment Fusion (VMAF) predicts subjective video quality ratings the best, while Naturalness Image Quality Evaluator (NIQE) turns out to be a promising alternative as a no‐reference metric in some scenarios. Abstract : In this paper, the performance of eight video quality assessment metrics is evaluated for passive gaming video streaming applications. Subjective and objective quality assessment results are provided as an open source dataset. Our results indicate that the full‐reference metric, VMAF, performs the best while pointing towards the need for a custom high performance no‐reference metric for live gaming video streaming applications. It was also found that no particular temporal pooling strategy results in significant performance gain across different quality metrics.