Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos

Yijun Yan, Jinchang Ren, Huimin Zhao, Jiangbin Zheng, Ezrinda Mohd Zaihidee, John Soraghan

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

1 Citation (Scopus)

Abstract

In this paper, we present an efficient approach to detect and track salient objects from videos. In general, colored visible image in red-green-blue (RGB) has better distinguishability in human visual perception, yet it suffers from the effect of illumination noise and shadows. On the contrary, thermal image is less sensitive to these noise effects though its distinguishability varies according to environmental settings. To this end, fusion of these two modalities provides an effective solution to tackle this problem. First, a background model is extracted followed by background-subtraction for foreground detection in visible images. Meanwhile, adaptively thresholding is applied for foreground detection in thermal domain as human objects tend to be of higher temperature thus brighter than the background. To deal with cases of occlusion, prediction based forward tracking and backward tracking are employed to identify separate objects even the foreground detection fails. The proposed method is evaluated on OTCBVS, a publicly available color-thermal benchmark dataset. Promising results have shown that the proposed fusion based approach can successfully detect and track multiple human objects.
LanguageEnglish
Title of host publicationAdvances in Multimedia Information Processing - PCM 2016
EditorsEnqing Chen, Yihong Gong, Yun Tie
Place of PublicationCham, Switzerland
PublisherSpringer
Pages697-704
Number of pages8
ISBN (Print)9783319488950
DOIs
Publication statusE-pub ahead of print - 27 Nov 2016
Event17th Pacific-Rim Conference on Multimedia (PCM 2016) - Xi'an, China
Duration: 15 Sep 201615 Sep 2016

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume9917
ISSN (Print)0302-9743

Conference

Conference17th Pacific-Rim Conference on Multimedia (PCM 2016)
CountryChina
CityXi'an
Period15/09/1615/09/16

Fingerprint

Fusion reactions
Lighting
Color
Hot Temperature
Temperature

Keywords

  • video salient objects
  • pedestrian detection/tracking
  • image fusion
  • visible image
  • thermal image

Cite this

Yan, Y., Ren, J., Zhao, H., Zheng, J., Zaihidee, E. M., & Soraghan, J. (2016). Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos. In E. Chen, Y. Gong, & Y. Tie (Eds.), Advances in Multimedia Information Processing - PCM 2016 (pp. 697-704). (Lecture Notes in Computer Science; Vol. 9917). Cham, Switzerland: Springer. https://doi.org/10.1007/978-3-319-48896-7_69
Yan, Yijun ; Ren, Jinchang ; Zhao, Huimin ; Zheng, Jiangbin ; Zaihidee, Ezrinda Mohd ; Soraghan, John. / Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos. Advances in Multimedia Information Processing - PCM 2016. editor / Enqing Chen ; Yihong Gong ; Yun Tie. Cham, Switzerland : Springer, 2016. pp. 697-704 (Lecture Notes in Computer Science).
@inproceedings{0b998ed28b2d4a9fa1beabdbe3d11506,
title = "Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos",
abstract = "In this paper, we present an efficient approach to detect and track salient objects from videos. In general, colored visible image in red-green-blue (RGB) has better distinguishability in human visual perception, yet it suffers from the effect of illumination noise and shadows. On the contrary, thermal image is less sensitive to these noise effects though its distinguishability varies according to environmental settings. To this end, fusion of these two modalities provides an effective solution to tackle this problem. First, a background model is extracted followed by background-subtraction for foreground detection in visible images. Meanwhile, adaptively thresholding is applied for foreground detection in thermal domain as human objects tend to be of higher temperature thus brighter than the background. To deal with cases of occlusion, prediction based forward tracking and backward tracking are employed to identify separate objects even the foreground detection fails. The proposed method is evaluated on OTCBVS, a publicly available color-thermal benchmark dataset. Promising results have shown that the proposed fusion based approach can successfully detect and track multiple human objects.",
keywords = "video salient objects, pedestrian detection/tracking , image fusion, visible image, thermal image",
author = "Yijun Yan and Jinchang Ren and Huimin Zhao and Jiangbin Zheng and Zaihidee, {Ezrinda Mohd} and John Soraghan",
year = "2016",
month = "11",
day = "27",
doi = "10.1007/978-3-319-48896-7_69",
language = "English",
isbn = "9783319488950",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "697--704",
editor = "Enqing Chen and Yihong Gong and Yun Tie",
booktitle = "Advances in Multimedia Information Processing - PCM 2016",

}

Yan, Y, Ren, J, Zhao, H, Zheng, J, Zaihidee, EM & Soraghan, J 2016, Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos. in E Chen, Y Gong & Y Tie (eds), Advances in Multimedia Information Processing - PCM 2016. Lecture Notes in Computer Science, vol. 9917, Springer, Cham, Switzerland, pp. 697-704, 17th Pacific-Rim Conference on Multimedia (PCM 2016), Xi'an, China, 15/09/16. https://doi.org/10.1007/978-3-319-48896-7_69

Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos. / Yan, Yijun; Ren, Jinchang; Zhao, Huimin; Zheng, Jiangbin; Zaihidee, Ezrinda Mohd; Soraghan, John.

Advances in Multimedia Information Processing - PCM 2016. ed. / Enqing Chen; Yihong Gong; Yun Tie. Cham, Switzerland : Springer, 2016. p. 697-704 (Lecture Notes in Computer Science; Vol. 9917).

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

TY - GEN

T1 - Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos

AU - Yan, Yijun

AU - Ren, Jinchang

AU - Zhao, Huimin

AU - Zheng, Jiangbin

AU - Zaihidee, Ezrinda Mohd

AU - Soraghan, John

PY - 2016/11/27

Y1 - 2016/11/27

N2 - In this paper, we present an efficient approach to detect and track salient objects from videos. In general, colored visible image in red-green-blue (RGB) has better distinguishability in human visual perception, yet it suffers from the effect of illumination noise and shadows. On the contrary, thermal image is less sensitive to these noise effects though its distinguishability varies according to environmental settings. To this end, fusion of these two modalities provides an effective solution to tackle this problem. First, a background model is extracted followed by background-subtraction for foreground detection in visible images. Meanwhile, adaptively thresholding is applied for foreground detection in thermal domain as human objects tend to be of higher temperature thus brighter than the background. To deal with cases of occlusion, prediction based forward tracking and backward tracking are employed to identify separate objects even the foreground detection fails. The proposed method is evaluated on OTCBVS, a publicly available color-thermal benchmark dataset. Promising results have shown that the proposed fusion based approach can successfully detect and track multiple human objects.

AB - In this paper, we present an efficient approach to detect and track salient objects from videos. In general, colored visible image in red-green-blue (RGB) has better distinguishability in human visual perception, yet it suffers from the effect of illumination noise and shadows. On the contrary, thermal image is less sensitive to these noise effects though its distinguishability varies according to environmental settings. To this end, fusion of these two modalities provides an effective solution to tackle this problem. First, a background model is extracted followed by background-subtraction for foreground detection in visible images. Meanwhile, adaptively thresholding is applied for foreground detection in thermal domain as human objects tend to be of higher temperature thus brighter than the background. To deal with cases of occlusion, prediction based forward tracking and backward tracking are employed to identify separate objects even the foreground detection fails. The proposed method is evaluated on OTCBVS, a publicly available color-thermal benchmark dataset. Promising results have shown that the proposed fusion based approach can successfully detect and track multiple human objects.

KW - video salient objects

KW - pedestrian detection/tracking

KW - image fusion

KW - visible image

KW - thermal image

UR - https://link.springer.com/book/10.1007/978-3-319-48890-5

U2 - 10.1007/978-3-319-48896-7_69

DO - 10.1007/978-3-319-48896-7_69

M3 - Conference contribution book

SN - 9783319488950

T3 - Lecture Notes in Computer Science

SP - 697

EP - 704

BT - Advances in Multimedia Information Processing - PCM 2016

A2 - Chen, Enqing

A2 - Gong, Yihong

A2 - Tie, Yun

PB - Springer

CY - Cham, Switzerland

ER -

Yan Y, Ren J, Zhao H, Zheng J, Zaihidee EM, Soraghan J. Fusion of thermal and visible imagery for effective detection and tracking of salient objects in videos. In Chen E, Gong Y, Tie Y, editors, Advances in Multimedia Information Processing - PCM 2016. Cham, Switzerland: Springer. 2016. p. 697-704. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-319-48896-7_69