Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications

Yue Feng, Jinchang Ren, Jianmin Jiang

Research output: Contribution to journalSpecial issue

45 Citations (Scopus)

Abstract

Three-dimensional television (3D-TV) has gained increasing popularity in the broadcasting domain, as it enables enhanced viewing experiences in comparison to conventional two-dimensional (2D) TV. However, its application has been constrained due to the lack of essential contents, i.e., stereoscopic videos. To alleviate such content shortage, an economical and practical solution is to reuse the huge media resources that are available in monoscopic 2D and convert them to stereoscopic 3D. Although stereoscopic video can be generated from monoscopic sequences using depth measurements extracted from cues like focus blur, motion and size, the quality of the resulting video may be poor as such measurements are usually arbitrarily defined and appear inconsistent with the real scenes. To help solve this problem, a novel method for object-based stereoscopic video generation is proposed which features i) optical-flow based occlusion reasoning in determining depth ordinal, ii) object segmentation using improved region-growing from masks of determined depth layers, and iii) a hybrid depth estimation scheme using content-based matching (inside a small library of true stereo image pairs) and depth-ordinal based regularization. Comprehensive experiments have validated the effectiveness of our proposed 2D-to-3D conversion method in generating stereoscopic videos of consistent depth measurements for 3D-TV applications.
LanguageEnglish
Pages500-509
Number of pages10
JournalIEEE Transactions on Broadcasting
Volume57
Issue number2
Early online date21 Apr 2011
DOIs
Publication statusPublished - Jun 2011

Fingerprint

Optical flows
Broadcasting
Television
Masks
Experiments

Keywords

  • 2D-to-3D conversion
  • 3D-TV
  • broadcasting
  • computer vision
  • multimedia systems
  • three dimensional displays
  • cameras
  • cognition
  • estimation
  • optical imaging
  • pixel
  • stereo vision

Cite this

@article{e527fe52bda74b4cbce8e6fb7be5c654,
title = "Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications",
abstract = "Three-dimensional television (3D-TV) has gained increasing popularity in the broadcasting domain, as it enables enhanced viewing experiences in comparison to conventional two-dimensional (2D) TV. However, its application has been constrained due to the lack of essential contents, i.e., stereoscopic videos. To alleviate such content shortage, an economical and practical solution is to reuse the huge media resources that are available in monoscopic 2D and convert them to stereoscopic 3D. Although stereoscopic video can be generated from monoscopic sequences using depth measurements extracted from cues like focus blur, motion and size, the quality of the resulting video may be poor as such measurements are usually arbitrarily defined and appear inconsistent with the real scenes. To help solve this problem, a novel method for object-based stereoscopic video generation is proposed which features i) optical-flow based occlusion reasoning in determining depth ordinal, ii) object segmentation using improved region-growing from masks of determined depth layers, and iii) a hybrid depth estimation scheme using content-based matching (inside a small library of true stereo image pairs) and depth-ordinal based regularization. Comprehensive experiments have validated the effectiveness of our proposed 2D-to-3D conversion method in generating stereoscopic videos of consistent depth measurements for 3D-TV applications.",
keywords = "2D-to-3D conversion, 3D-TV, broadcasting, computer vision, multimedia systems, three dimensional displays, cameras, cognition, estimation, optical imaging, pixel, stereo vision",
author = "Yue Feng and Jinchang Ren and Jianmin Jiang",
year = "2011",
month = "6",
doi = "10.1109/TBC.2011.2131030",
language = "English",
volume = "57",
pages = "500--509",
journal = "IEEE Transactions on Broadcasting",
issn = "0018-9316",
number = "2",

}

Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications. / Feng, Yue; Ren, Jinchang; Jiang, Jianmin.

In: IEEE Transactions on Broadcasting, Vol. 57, No. 2, 06.2011, p. 500-509.

Research output: Contribution to journalSpecial issue

TY - JOUR

T1 - Object-based 2D-to-3D video conversion for effective stereoscopic content generation in 3D-TV applications

AU - Feng, Yue

AU - Ren, Jinchang

AU - Jiang, Jianmin

PY - 2011/6

Y1 - 2011/6

N2 - Three-dimensional television (3D-TV) has gained increasing popularity in the broadcasting domain, as it enables enhanced viewing experiences in comparison to conventional two-dimensional (2D) TV. However, its application has been constrained due to the lack of essential contents, i.e., stereoscopic videos. To alleviate such content shortage, an economical and practical solution is to reuse the huge media resources that are available in monoscopic 2D and convert them to stereoscopic 3D. Although stereoscopic video can be generated from monoscopic sequences using depth measurements extracted from cues like focus blur, motion and size, the quality of the resulting video may be poor as such measurements are usually arbitrarily defined and appear inconsistent with the real scenes. To help solve this problem, a novel method for object-based stereoscopic video generation is proposed which features i) optical-flow based occlusion reasoning in determining depth ordinal, ii) object segmentation using improved region-growing from masks of determined depth layers, and iii) a hybrid depth estimation scheme using content-based matching (inside a small library of true stereo image pairs) and depth-ordinal based regularization. Comprehensive experiments have validated the effectiveness of our proposed 2D-to-3D conversion method in generating stereoscopic videos of consistent depth measurements for 3D-TV applications.

AB - Three-dimensional television (3D-TV) has gained increasing popularity in the broadcasting domain, as it enables enhanced viewing experiences in comparison to conventional two-dimensional (2D) TV. However, its application has been constrained due to the lack of essential contents, i.e., stereoscopic videos. To alleviate such content shortage, an economical and practical solution is to reuse the huge media resources that are available in monoscopic 2D and convert them to stereoscopic 3D. Although stereoscopic video can be generated from monoscopic sequences using depth measurements extracted from cues like focus blur, motion and size, the quality of the resulting video may be poor as such measurements are usually arbitrarily defined and appear inconsistent with the real scenes. To help solve this problem, a novel method for object-based stereoscopic video generation is proposed which features i) optical-flow based occlusion reasoning in determining depth ordinal, ii) object segmentation using improved region-growing from masks of determined depth layers, and iii) a hybrid depth estimation scheme using content-based matching (inside a small library of true stereo image pairs) and depth-ordinal based regularization. Comprehensive experiments have validated the effectiveness of our proposed 2D-to-3D conversion method in generating stereoscopic videos of consistent depth measurements for 3D-TV applications.

KW - 2D-to-3D conversion

KW - 3D-TV

KW - broadcasting

KW - computer vision

KW - multimedia systems

KW - three dimensional displays

KW - cameras

KW - cognition

KW - estimation

KW - optical imaging

KW - pixel

KW - stereo vision

UR - http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=5754617

U2 - 10.1109/TBC.2011.2131030

DO - 10.1109/TBC.2011.2131030

M3 - Special issue

VL - 57

SP - 500

EP - 509

JO - IEEE Transactions on Broadcasting

T2 - IEEE Transactions on Broadcasting

JF - IEEE Transactions on Broadcasting

SN - 0018-9316

IS - 2

ER -