On Versatile Video Coding at UHD with Machine-Learning-Based Super-Resolution

Aug 5, 2026

Upload by Patrick

Coding 4K data has become increasingly important as 4K content volume grows worldwide. The work presents a coding chain that applies spatial downscaling before encoding and then performs machine-learning-based single-image super-resolution to restore UHD quality. Experiments using the VVC codec show improved quality over the conventional VVC reference software in low-bitrate scenarios. Average Bjøntegaard delta rate gains up to 12% and 18% are reported for QP ranges above 34 and 42, respectively, while the up-/downscaling approach helps reduce detail loss and compression artifacts.

LanguageEnglish

CategoryTechnology

FormatPDF

Pages6

Reading Time15 min

AccessFree to download

Introduction

Spatial up- and downscaling coding chain

Related work on down-/up-sampling and coding

Proposed approach with VVC and neural super-resolution

©2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. DOI: 10.1109/QoMEX48832.2020.9123140 On Versatile Video Coding at UHD with Machine-Learning-Based Super-Resolution Kristian Fischer, Christian Herglotz, and Andr Kaup Multimedia Communications and Signal Processing Friedrich-Alexander-Universita¨t Erlangen-Nu¨rnberg (FAU) Cauerstr. 7, 91058 Erlangen, Germany {Kristian.Fischer, Christian.Herglotz, [Andre.Kaup](Andre.Kaup}@fau.de)[}](Andre.Kaup}@fau.de)[@fau.de](Andre.Kaup}@fau.de) arXiv :2308 .06570v1 [ ee ss .IV] 12 Aug 2023 Abstract—Coding 4K data has become of vital interest in recent years, since the amount of 4K data is significantly increasing. We propose a coding chain with spatial down- and upscaling that combines the next-generation VVC codec with machine learning based single image super-resolution algorithms for 4K. The investigated coding chain, which spatially downscales the 4K data before coding, shows superior quality than the conventional VVC reference software for low bitrate scenarios. Throughout several tests, we find that up to 12 % and 18 % Bjøntegaard delta rate gains can be achieved on average when coding 4K sequences with VVC and QP values above 34 and 42, respectively. Additionally, the investigated scenario with up-and downscaling helps to reduce the loss of details and compression artifacts, as it is shown in a visual example. Index Terms—VVC, video compression, CNN super-resolution, 4K/UHD video, spatial-resolution scaling, VDSR, RDN I. INTRODUCTION The amount of multimedia data in Ultra-HighDefinition (UHD) resolution which is consumed all over the world is rapidly increasing. According to the Cisco Visual Networking Index [1], the percentage of installed flat panel TVs that are able to show Videos in UHD resolution will be increasing from 23 % in 2017 to 62 % in 2022 . Derived from this statistic, it can be seen that the significance of UHD is growing such that the provision of suitable data is essential. Hence, the question arises how to compress these massive amounts of data in the future, such that the end user can watch UHD content with the best quality at reasonable bitrates. For transmission scenarios with low available bitrates, it has recently been shown [2]–[6] that it is beneficial to downscale the video first before compressing it into the bitstream. In Fig. 1, such a coding chain with spatial up-and downscaling is depicted. First, the original frame Xorig is downscaled by a scaling factor s before encoding. The resulting low resolution frame XLR has s~~12~~ pixels of the original Xorig. Atthe decoder side, the bitstream bLR is decode˜d into the frame with reduced resolution and lower quality XLR . Finally, the compressed low resolution fr˜ ame is upscaled to the original resolution which results in Xscaled. The lower coding branch in Fig. 1 shows the conventional coding process without modifying the spatial resolution. Already in 2003, Bruckstein et al. [2] proposed a coding chain in which they downscaled still images before encoding them with the JPEG standard. This method resulted in superior quality at low bitrates. In a similar approach [3], it was Fig. 1: Top branch: coding chain with spatial up- and downscaling; Bottom branch: conventional coding investigated using a generative adversarial network (GAN) to regain the spatial resolution of the downscaled images that were stored in BPG file format. Nguyen et al. [4] showed for video data that it is beneficial to use a coding chain with an adaptive down-/up-sampling coding scheme at low bitrates for MPEG-2 encoding. There, the rate-distortion performance of the coding chain with

FAQs

What problem does the document address in UHD (4K) video coding?

It tackles the challenge of compressing rapidly growing amounts of UHD/4K data while maintaining high quality at reasonable bitrates, especially under low-bitrate transmission scenarios.

How does the proposed coding chain work?

It spatially downscales 4K frames before VVC encoding, decodes the lower-resolution bitstream, and then uses machine-learning-based single-image super-resolution to upscale back to the original resolution.

What performance gains are reported for the proposed method?

The tests report average Bjøntegaard delta rate gains up to 12% and 18% for coding 4K sequences with VVC at QP values above 34 and 42, respectively, under low-bitrate conditions.

Aug 5, 2026

Upload by Patrick