Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation

Andrew Abel; Amir Hussain; Quoc Dinh Nguyen; Fabien Ringeval; Mohamed Chetouani; Maurice Milgram

doi:10.1007/978-3-642-04391-8_9

Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation

Andrew Abel^*, Amir Hussain, Quoc Dinh Nguyen, Fabien Ringeval, Mohamed Chetouani, Maurice Milgram

^*Corresponding author for this work

Computer And Information Sciences

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

9 Citations (Scopus)

Abstract

In recent years, the established link between the various human communication production domains has become more widely utilised in the field of speech processing. In this work, a state of the art Semi Adaptive Appearance Model (SAAM) approach developed by the authors is used for automatic lip tracking, and an adapted version of our vowel based speech segmentation system is employed to automatically segment speech. Canonical Correlation Analysis (CCA) on segmented and non segmented data in a range of noisy speech environments finds that segmented speech has a significantly better audiovisual correlation, demonstrating the feasibility of our techniques for further development as part of a proposed audiovisual speech enhancement system.

Original language	English
Title of host publication	Biometric ID Management and Multimodal Communication
Subtitle of host publication	Joint COST 2101 and 2102 International Conference
Editors	Julian Fierrez, Javier Ortega-Garcia, Anna Esposito, Andrzej Drygajlo, Marcos Faundez-Zanuy
Place of Publication	Cham, Switzerland
Publisher	Springer
Pages	65-72
Number of pages	8
ISBN (Electronic)	9783642043918
ISBN (Print)	3642043909, 9783642043901
DOIs	https://doi.org/10.1007/978-3-642-04391-8_9
Publication status	Published - 7 Sept 2009
Event	Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID_MultiComm 2009 - Madrid, Spain Duration: 16 Sept 2009 → 18 Sept 2009

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	5707 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID_MultiComm 2009
Country/Territory	Spain
City	Madrid
Period	16/09/09 → 18/09/09

Keywords

canonical correlation
canonical correlation analysis
noisy environment
speech enhancement
visual speech

Access to Document

10.1007/978-3-642-04391-8_9

Cite this

Abel, A., Hussain, A., Nguyen, Q. D., Ringeval, F., Chetouani, M., & Milgram, M. (2009). Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. In J. Fierrez, J. Ortega-Garcia, A. Esposito, A. Drygajlo, & M. Faundez-Zanuy (Eds.), Biometric ID Management and Multimodal Communication: Joint COST 2101 and 2102 International Conference (pp. 65-72). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5707 LNCS). Springer. https://doi.org/10.1007/978-3-642-04391-8_9

Abel, Andrew ; Hussain, Amir ; Nguyen, Quoc Dinh et al. / Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. Biometric ID Management and Multimodal Communication: Joint COST 2101 and 2102 International Conference. editor / Julian Fierrez ; Javier Ortega-Garcia ; Anna Esposito ; Andrzej Drygajlo ; Marcos Faundez-Zanuy. Cham, Switzerland : Springer, 2009. pp. 65-72 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a46b41b11414495bbfa87acd0318a255,

title = "Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation",

abstract = "In recent years, the established link between the various human communication production domains has become more widely utilised in the field of speech processing. In this work, a state of the art Semi Adaptive Appearance Model (SAAM) approach developed by the authors is used for automatic lip tracking, and an adapted version of our vowel based speech segmentation system is employed to automatically segment speech. Canonical Correlation Analysis (CCA) on segmented and non segmented data in a range of noisy speech environments finds that segmented speech has a significantly better audiovisual correlation, demonstrating the feasibility of our techniques for further development as part of a proposed audiovisual speech enhancement system.",

keywords = "canonical correlation, canonical correlation analysis, noisy environment, speech enhancement, visual speech",

author = "Andrew Abel and Amir Hussain and Nguyen, {Quoc Dinh} and Fabien Ringeval and Mohamed Chetouani and Maurice Milgram",

year = "2009",

month = sep,

day = "7",

doi = "10.1007/978-3-642-04391-8_9",

language = "English",

isbn = "3642043909",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "65--72",

editor = "Julian Fierrez and Javier Ortega-Garcia and Anna Esposito and Andrzej Drygajlo and Marcos Faundez-Zanuy",

booktitle = "Biometric ID Management and Multimodal Communication",

note = "Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID_MultiComm 2009 ; Conference date: 16-09-2009 Through 18-09-2009",

}

Abel, A, Hussain, A, Nguyen, QD, Ringeval, F, Chetouani, M & Milgram, M 2009, Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. in J Fierrez, J Ortega-Garcia, A Esposito, A Drygajlo & M Faundez-Zanuy (eds), Biometric ID Management and Multimodal Communication: Joint COST 2101 and 2102 International Conference. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5707 LNCS, Springer, Cham, Switzerland , pp. 65-72, Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID_MultiComm 2009, Madrid, Spain, 16/09/09. https://doi.org/10.1007/978-3-642-04391-8_9

Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. / Abel, Andrew; Hussain, Amir; Nguyen, Quoc Dinh et al.
Biometric ID Management and Multimodal Communication: Joint COST 2101 and 2102 International Conference. ed. / Julian Fierrez; Javier Ortega-Garcia; Anna Esposito; Andrzej Drygajlo; Marcos Faundez-Zanuy. Cham, Switzerland : Springer, 2009. p. 65-72 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5707 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution book

TY - GEN

T1 - Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation

AU - Abel, Andrew

AU - Hussain, Amir

AU - Nguyen, Quoc Dinh

AU - Ringeval, Fabien

AU - Chetouani, Mohamed

AU - Milgram, Maurice

PY - 2009/9/7

Y1 - 2009/9/7

N2 - In recent years, the established link between the various human communication production domains has become more widely utilised in the field of speech processing. In this work, a state of the art Semi Adaptive Appearance Model (SAAM) approach developed by the authors is used for automatic lip tracking, and an adapted version of our vowel based speech segmentation system is employed to automatically segment speech. Canonical Correlation Analysis (CCA) on segmented and non segmented data in a range of noisy speech environments finds that segmented speech has a significantly better audiovisual correlation, demonstrating the feasibility of our techniques for further development as part of a proposed audiovisual speech enhancement system.

AB - In recent years, the established link between the various human communication production domains has become more widely utilised in the field of speech processing. In this work, a state of the art Semi Adaptive Appearance Model (SAAM) approach developed by the authors is used for automatic lip tracking, and an adapted version of our vowel based speech segmentation system is employed to automatically segment speech. Canonical Correlation Analysis (CCA) on segmented and non segmented data in a range of noisy speech environments finds that segmented speech has a significantly better audiovisual correlation, demonstrating the feasibility of our techniques for further development as part of a proposed audiovisual speech enhancement system.

KW - canonical correlation

KW - canonical correlation analysis

KW - noisy environment

KW - speech enhancement

KW - visual speech

U2 - 10.1007/978-3-642-04391-8_9

DO - 10.1007/978-3-642-04391-8_9

M3 - Conference contribution book

AN - SCOPUS:77952056171

SN - 3642043909

SN - 9783642043901

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 65

EP - 72

BT - Biometric ID Management and Multimodal Communication

A2 - Fierrez, Julian

A2 - Ortega-Garcia, Javier

A2 - Esposito, Anna

A2 - Drygajlo, Andrzej

A2 - Faundez-Zanuy, Marcos

PB - Springer

CY - Cham, Switzerland

T2 - Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID_MultiComm 2009

Y2 - 16 September 2009 through 18 September 2009

ER -

Abel A, Hussain A, Nguyen QD, Ringeval F, Chetouani M, Milgram M. Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. In Fierrez J, Ortega-Garcia J, Esposito A, Drygajlo A, Faundez-Zanuy M, editors, Biometric ID Management and Multimodal Communication: Joint COST 2101 and 2102 International Conference. Cham, Switzerland : Springer. 2009. p. 65-72. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-04391-8_9

Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this