Integrating facial expressions into user profiling for the improvement of a multimodal recommender system

Ioannis Arapakis, Yashar Moshfeghi, Hideo Joho, R. Ren, David Hannah, Joemon M. Jose

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

35 Citations (Scopus)

Abstract

Over the years, recommender systems have been systematically applied in both industry and academia to assist users in dealing with information overload. One of the factors that determine the performance of a recommender system is user feedback, which has been traditionally communicated through the application of explicit and implicit feedback techniques. In this paper, we propose a novel video search interface that predicts the topical relevance of a video by analysing affective aspects of user behaviour. We, furthermore, present a method for incorporating such affective features into user profiling, to facilitate the generation of meaningful recommendations, of unseen videos. Our experiment shows that multimodal interaction feature is a promising way to improve the performance of recommendation.
LanguageEnglish
Title of host publication2009 IEEE International Conference on Multimedia and Expo
Subtitle of host publicationICME 2009
Place of PublicationPiscataway, N.J.
PublisherIEEE
Pages1440-1443
Number of pages4
ISBN (Print)978-1-4244-4290-4
DOIs
Publication statusPublished - 1 Jun 2009

Fingerprint

Recommender systems
Feedback
Industry
Experiments

Keywords

  • information filtering
  • search engines
  • user interfaces
  • video signal processing
  • facial expressions integration
  • implicit feedback techniques
  • multimodal interaction feature
  • multimodal recommender system
  • user feedback
  • user profiling
  • video search interface
  • aggregates
  • books
  • computer industry
  • emotion recognition
  • face recognition
  • feedback
  • motion pictures
  • recommender systems
  • support vector machine classification
  • support vector machines
  • affective feedback
  • facial expression analysis
  • muiltimedia retrieval

Cite this

Arapakis, I., Moshfeghi, Y., Joho, H., Ren, R., Hannah, D., & Jose, J. M. (2009). Integrating facial expressions into user profiling for the improvement of a multimodal recommender system. In 2009 IEEE International Conference on Multimedia and Expo: ICME 2009 (pp. 1440-1443). Piscataway, N.J.: IEEE. https://doi.org/10.1109/ICME.2009.5202773
Arapakis, Ioannis ; Moshfeghi, Yashar ; Joho, Hideo ; Ren, R. ; Hannah, David ; Jose, Joemon M. / Integrating facial expressions into user profiling for the improvement of a multimodal recommender system. 2009 IEEE International Conference on Multimedia and Expo: ICME 2009. Piscataway, N.J. : IEEE, 2009. pp. 1440-1443
@inproceedings{a66927b51ae24f3a88d1ecf5b5de9d48,
title = "Integrating facial expressions into user profiling for the improvement of a multimodal recommender system",
abstract = "Over the years, recommender systems have been systematically applied in both industry and academia to assist users in dealing with information overload. One of the factors that determine the performance of a recommender system is user feedback, which has been traditionally communicated through the application of explicit and implicit feedback techniques. In this paper, we propose a novel video search interface that predicts the topical relevance of a video by analysing affective aspects of user behaviour. We, furthermore, present a method for incorporating such affective features into user profiling, to facilitate the generation of meaningful recommendations, of unseen videos. Our experiment shows that multimodal interaction feature is a promising way to improve the performance of recommendation.",
keywords = "information filtering, search engines, user interfaces, video signal processing, facial expressions integration, implicit feedback techniques, multimodal interaction feature, multimodal recommender system, user feedback, user profiling, video search interface, aggregates, books, computer industry, emotion recognition, face recognition, feedback, motion pictures, recommender systems, support vector machine classification, support vector machines, affective feedback, facial expression analysis, muiltimedia retrieval",
author = "Ioannis Arapakis and Yashar Moshfeghi and Hideo Joho and R. Ren and David Hannah and Jose, {Joemon M.}",
year = "2009",
month = "6",
day = "1",
doi = "10.1109/ICME.2009.5202773",
language = "English",
isbn = "978-1-4244-4290-4",
pages = "1440--1443",
booktitle = "2009 IEEE International Conference on Multimedia and Expo",
publisher = "IEEE",

}

Arapakis, I, Moshfeghi, Y, Joho, H, Ren, R, Hannah, D & Jose, JM 2009, Integrating facial expressions into user profiling for the improvement of a multimodal recommender system. in 2009 IEEE International Conference on Multimedia and Expo: ICME 2009. IEEE, Piscataway, N.J., pp. 1440-1443. https://doi.org/10.1109/ICME.2009.5202773

Integrating facial expressions into user profiling for the improvement of a multimodal recommender system. / Arapakis, Ioannis; Moshfeghi, Yashar; Joho, Hideo; Ren, R.; Hannah, David; Jose, Joemon M.

2009 IEEE International Conference on Multimedia and Expo: ICME 2009. Piscataway, N.J. : IEEE, 2009. p. 1440-1443.

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

TY - GEN

T1 - Integrating facial expressions into user profiling for the improvement of a multimodal recommender system

AU - Arapakis, Ioannis

AU - Moshfeghi, Yashar

AU - Joho, Hideo

AU - Ren, R.

AU - Hannah, David

AU - Jose, Joemon M.

PY - 2009/6/1

Y1 - 2009/6/1

N2 - Over the years, recommender systems have been systematically applied in both industry and academia to assist users in dealing with information overload. One of the factors that determine the performance of a recommender system is user feedback, which has been traditionally communicated through the application of explicit and implicit feedback techniques. In this paper, we propose a novel video search interface that predicts the topical relevance of a video by analysing affective aspects of user behaviour. We, furthermore, present a method for incorporating such affective features into user profiling, to facilitate the generation of meaningful recommendations, of unseen videos. Our experiment shows that multimodal interaction feature is a promising way to improve the performance of recommendation.

AB - Over the years, recommender systems have been systematically applied in both industry and academia to assist users in dealing with information overload. One of the factors that determine the performance of a recommender system is user feedback, which has been traditionally communicated through the application of explicit and implicit feedback techniques. In this paper, we propose a novel video search interface that predicts the topical relevance of a video by analysing affective aspects of user behaviour. We, furthermore, present a method for incorporating such affective features into user profiling, to facilitate the generation of meaningful recommendations, of unseen videos. Our experiment shows that multimodal interaction feature is a promising way to improve the performance of recommendation.

KW - information filtering

KW - search engines

KW - user interfaces

KW - video signal processing

KW - facial expressions integration

KW - implicit feedback techniques

KW - multimodal interaction feature

KW - multimodal recommender system

KW - user feedback

KW - user profiling

KW - video search interface

KW - aggregates

KW - books

KW - computer industry

KW - emotion recognition

KW - face recognition

KW - feedback

KW - motion pictures

KW - recommender systems

KW - support vector machine classification

KW - support vector machines

KW - affective feedback

KW - facial expression analysis

KW - muiltimedia retrieval

UR - http://ieeexplore.ieee.org/document/5202773/

U2 - 10.1109/ICME.2009.5202773

DO - 10.1109/ICME.2009.5202773

M3 - Conference contribution book

SN - 978-1-4244-4290-4

SP - 1440

EP - 1443

BT - 2009 IEEE International Conference on Multimedia and Expo

PB - IEEE

CY - Piscataway, N.J.

ER -

Arapakis I, Moshfeghi Y, Joho H, Ren R, Hannah D, Jose JM. Integrating facial expressions into user profiling for the improvement of a multimodal recommender system. In 2009 IEEE International Conference on Multimedia and Expo: ICME 2009. Piscataway, N.J.: IEEE. 2009. p. 1440-1443 https://doi.org/10.1109/ICME.2009.5202773