IBC2022 Tech Papers: Encoding optimisations for video game live streaming

Abstract

In recent years, live video-game-centric streaming platforms have experienced dramatic growth, driven by the explosion of the esports market. Besides, live-event rightsholders and broadcasters are successfully ‘gamifying’ their video offerings. Given the rise of video game live streaming, we address in this paper the questions of how to optimise and adapt the encoding strategies for efficient game content coding. We first characterise game content, focusing on how its signal characteristics differ from ‘natural’ content. We then share insights on selected encoding strategies targeting improvements in both compression efficiency and density/encoder run-time for signals with such characteristics. Additionally, we examine the relevance of Screen Content Coding (SCC) tools, as adopted in HEVC SCC and VVC standards, in a video game coding context. Finally, we conclude the paper by highlighting the benefits of using game-content aware technology with better compression efficiency under real-time constraints.

Introduction

The proliferation of IPTV and OTT media delivery technologies have helped establish video game live streaming as a major market segment. In recent years, video-game-centric streaming platforms such as Twitch, YouTube, or Facebook Gaming, have experienced a dramatic growth (101%, 65%, and 238% respectively in 2020). Overall, the related global esports market –the competitive and professional element of the gaming world –generated more than $1 billion in revenues in 2020 and is forecast to hit $1.6 billion by 2023. In addition, we should expect to see live-event rightsholders and broadcasters ‘gamify’ their video offerings with a greater degree of social interaction and mixed-media experiences. For example, the NBA has successfully trialed replacing live action with video game simulations. Likewise, during lockdown SRO Motorsports successfully switched real races to esports, even holding them during their originally scheduled date and time. The unique Fanatec Esports GT Pro Series, initiated in 2021, where real racing drivers compete in a virtual environment for points towards their real-world championship will survive past the lockdown period and return for 2022 streamed live across YouTube, Twitch and Facebook.Given the rise of video game live streaming, questions on how to optimise and adapt the encoding strategies to game content open an important field of research.

In this paper, we discuss some key elements to efficiently encode video game content in real-time and how this can lead to ‘game-content aware’ encoding solutions. We start by characterising game content, focusing on how its signal characteristics differ from ‘natural’ content and may require specific encoder optimisations. We then share insights about some specific encoding strategies (e.g., rate control, adaptive quantisation,in-loop filtering, motion estimation, etc.), targeting improvements in both compression efficiency and density/encoder run-time for signals with such characteristics. Additionally, and since video compression standards, such as HEVC or VVC, include coding tools specifically designed to compress screen content, we examine the relevance of those tools in a video game coding context. Finally, we show the benefits of using game-content aware technology with better compression efficiency under real-time constraints.