Versatile Video Coding with Saliency-Enabled Coding Unit Partitioning and Quantization Control

Video is increasingly going towards intelligence after the development of digital, high definition, and ultra high definition, spreading from traditional broadcasting applications to a variety of industries such video communication, medical, and security monitoring. One of the key technologies for video applications is video coding, which focuses on the theory of effective ways for digital video representation and compression, lowering the overhead needed for video transmission and storage. By removing perceptual redundancy during video compression, perceptual video coding (PVC) can significantly increase the efficiency of video coding. PVC is based on a greater understanding of the perceptual properties of the human visual system (HVS) and a higher performance computation model.

Salient target identification, a crucial technology for ROI, can be broadly separated into traditional salient target detection and salient target detection based on deep learning. Traditional saliency target detection generally relies on human intuition or heuristic priors, such as using boundary point priors, chromaticity comparison, and backdrop comparison, etc., to manually extract characteristics and recognize objects. The block partition structure, which serves as the foundation of the coding layer, greatly improves coding performance by taking use of the variable block sizes. While dynamic saliency represents the video's temporal features, static saliency reflects the video's spatial characteristics. The creation of the final saliency map benefits from the efficient fusion of the two.

In this research, a quantitative control method and a perceptual rapid CU division algorithm are combined to form the VVC optimization strategy. In the preparation stage of video encoding, we obtain saliency maps using a fully neural network-based model for detecting video-salient objects. To reduce the complexity of intra encoding, we propose a quick CU partitioning scheme based on computed saliency values at the CU level. This fast CU partitioning scheme includes the recalculation of the CU division depth by computing the Scharr operator and variance as well as the executive decision for intra sub-partitions.

In addition, a quantization control approach based on multi-level classification of saliency values at the CU level is suggested to decrease bitrate. The results of the experiments demonstrate that the suggested perceptual methods in this research significantly reduce the amount of time and bitrate used while sacrificing little in the way of coding effectiveness or perceptual quality.

Source: information