A semantic communication and VVC based hybrid video coding system

Research output: Contribution to journalArticlepeer-review

9 Downloads (Pure)

Abstract

Requirements of next-generation video applications are becoming a challenge for conven-tional video coding systems, although they have evolved over decades to accommodate the most demanding of current video applications. Semantic communications, built on the concept of transmitting just the semantics of a message and allowing the receiver to reconstruct the message based on a shared context, is a non-conventional approach being considered to overcome these challenges and improve performance of video coding systems. In this paper, a first such semantic communication-based video coding system in hybrid mode is proposed, which uses an autoencoder-based semantic encoder for inter coding, augmented by the intra coding capabilities of Versatile Video Coding (VVC) to encode key frames that form the context for the semantic communication and the residuals for improving the fidelity of the output frames. For a range of videos with differing levels of complexity, the proposed system consistently outperforms High Efficiency Video Coding (HEVC) and Advanced Video Coding (AVC) in terms of rate distortion metrics quantified by Bjontegaard Delta Rates. It also outperforms Versatile Video Coding with videos with low or high complexity, but slightly falls behind with videos with medium complexity, which can be improved by addressing the open research areas that stem from this work. The proposed system demonstrates the potential of semantic communication based video coding systems to consistently outperform state-of-the-art conventional video coding systems over a wide range video applications.
Original languageEnglish
Number of pages24
JournalIEEE Access
Early online date9 May 2024
DOIs
Publication statusE-pub ahead of print - 9 May 2024

Keywords

  • autoencoders
  • deep neural network (DNN)
  • semantic communication
  • video coding
  • video communication

Fingerprint

Dive into the research topics of 'A semantic communication and VVC based hybrid video coding system'. Together they form a unique fingerprint.

Cite this