User-action-driven view and rate scalable multiview video coding

Jacob Chakareski, Vladan Velisavljevic, Vladimir Stankovic

Research output: Contribution to journalArticle

23 Citations (Scopus)

Abstract

We derive an optimization framework for joint view and rate scalable coding of multi-view video content represented in the texture plus depth format. The optimization enables the sender to select the subset of coded views and their encoding rates such that the aggregate distortion over a continuum of synthesized views is minimized. We construct the view and rate embedded bitstream such that it delivers optimal performance simultaneously over a discrete set of transmission rates. In conjunction, we develop a user interaction model that characterizes the view selection actions of the client as a Markov chain over a discrete state-space. We exploit the model within the context of our optimization to compute user-action-driven coding strategies that aim at enhancing the client's performance in terms of latency and video quality. Our optimization outperforms the state-of-the-art H.264 SVC codec as well as a multi-view wavelet-based coder equipped with a uniform rate allocation strategy, across all scenarios studied in our experiments. Equally important, we can achieve an arbitrarily fine granularity of encoding bit rates, while providing a novel functionality of view embedded encoding, unlike the other encoding methods that we examined. Finally, we observe that the interactivity-aware coding delivers superior performance over conventional allocation techniques that do not anticipate the client's view selection actions in their operation.

LanguageEnglish
Article number6542763
Pages3473-3484
Number of pages12
JournalIEEE Transactions on Image Processing
Volume22
Issue number9
DOIs
Publication statusPublished - 20 Aug 2013

Fingerprint

Scalable video coding
Set theory
Markov processes
Textures
Experiments

Keywords

  • bit allocation
  • depth-image-based rendering
  • multiview imaging
  • user-action-driven coding
  • view and rate scalable encoding
  • view selection Markov model

Cite this

Chakareski, Jacob ; Velisavljevic, Vladan ; Stankovic, Vladimir. / User-action-driven view and rate scalable multiview video coding. In: IEEE Transactions on Image Processing. 2013 ; Vol. 22, No. 9. pp. 3473-3484.
@article{0b30f449577b4708ac2c9817ea6c0dd6,
title = "User-action-driven view and rate scalable multiview video coding",
abstract = "We derive an optimization framework for joint view and rate scalable coding of multi-view video content represented in the texture plus depth format. The optimization enables the sender to select the subset of coded views and their encoding rates such that the aggregate distortion over a continuum of synthesized views is minimized. We construct the view and rate embedded bitstream such that it delivers optimal performance simultaneously over a discrete set of transmission rates. In conjunction, we develop a user interaction model that characterizes the view selection actions of the client as a Markov chain over a discrete state-space. We exploit the model within the context of our optimization to compute user-action-driven coding strategies that aim at enhancing the client's performance in terms of latency and video quality. Our optimization outperforms the state-of-the-art H.264 SVC codec as well as a multi-view wavelet-based coder equipped with a uniform rate allocation strategy, across all scenarios studied in our experiments. Equally important, we can achieve an arbitrarily fine granularity of encoding bit rates, while providing a novel functionality of view embedded encoding, unlike the other encoding methods that we examined. Finally, we observe that the interactivity-aware coding delivers superior performance over conventional allocation techniques that do not anticipate the client's view selection actions in their operation.",
keywords = "bit allocation, depth-image-based rendering, multiview imaging, user-action-driven coding, view and rate scalable encoding, view selection Markov model",
author = "Jacob Chakareski and Vladan Velisavljevic and Vladimir Stankovic",
year = "2013",
month = "8",
day = "20",
doi = "10.1109/TIP.2013.2269801",
language = "English",
volume = "22",
pages = "3473--3484",
journal = "IEEE Transactions on Image Processing",
issn = "1057-7149",
number = "9",

}

User-action-driven view and rate scalable multiview video coding. / Chakareski, Jacob; Velisavljevic, Vladan; Stankovic, Vladimir.

In: IEEE Transactions on Image Processing, Vol. 22, No. 9, 6542763, 20.08.2013, p. 3473-3484.

Research output: Contribution to journalArticle

TY - JOUR

T1 - User-action-driven view and rate scalable multiview video coding

AU - Chakareski, Jacob

AU - Velisavljevic, Vladan

AU - Stankovic, Vladimir

PY - 2013/8/20

Y1 - 2013/8/20

N2 - We derive an optimization framework for joint view and rate scalable coding of multi-view video content represented in the texture plus depth format. The optimization enables the sender to select the subset of coded views and their encoding rates such that the aggregate distortion over a continuum of synthesized views is minimized. We construct the view and rate embedded bitstream such that it delivers optimal performance simultaneously over a discrete set of transmission rates. In conjunction, we develop a user interaction model that characterizes the view selection actions of the client as a Markov chain over a discrete state-space. We exploit the model within the context of our optimization to compute user-action-driven coding strategies that aim at enhancing the client's performance in terms of latency and video quality. Our optimization outperforms the state-of-the-art H.264 SVC codec as well as a multi-view wavelet-based coder equipped with a uniform rate allocation strategy, across all scenarios studied in our experiments. Equally important, we can achieve an arbitrarily fine granularity of encoding bit rates, while providing a novel functionality of view embedded encoding, unlike the other encoding methods that we examined. Finally, we observe that the interactivity-aware coding delivers superior performance over conventional allocation techniques that do not anticipate the client's view selection actions in their operation.

AB - We derive an optimization framework for joint view and rate scalable coding of multi-view video content represented in the texture plus depth format. The optimization enables the sender to select the subset of coded views and their encoding rates such that the aggregate distortion over a continuum of synthesized views is minimized. We construct the view and rate embedded bitstream such that it delivers optimal performance simultaneously over a discrete set of transmission rates. In conjunction, we develop a user interaction model that characterizes the view selection actions of the client as a Markov chain over a discrete state-space. We exploit the model within the context of our optimization to compute user-action-driven coding strategies that aim at enhancing the client's performance in terms of latency and video quality. Our optimization outperforms the state-of-the-art H.264 SVC codec as well as a multi-view wavelet-based coder equipped with a uniform rate allocation strategy, across all scenarios studied in our experiments. Equally important, we can achieve an arbitrarily fine granularity of encoding bit rates, while providing a novel functionality of view embedded encoding, unlike the other encoding methods that we examined. Finally, we observe that the interactivity-aware coding delivers superior performance over conventional allocation techniques that do not anticipate the client's view selection actions in their operation.

KW - bit allocation

KW - depth-image-based rendering

KW - multiview imaging

KW - user-action-driven coding

KW - view and rate scalable encoding

KW - view selection Markov model

UR - http://www.scopus.com/inward/record.url?scp=84881522084&partnerID=8YFLogxK

U2 - 10.1109/TIP.2013.2269801

DO - 10.1109/TIP.2013.2269801

M3 - Article

VL - 22

SP - 3473

EP - 3484

JO - IEEE Transactions on Image Processing

T2 - IEEE Transactions on Image Processing

JF - IEEE Transactions on Image Processing

SN - 1057-7149

IS - 9

M1 - 6542763

ER -