Integrating facial expressions into user profiling for the improvement of a multimodal recommender system

Ioannis Arapakis, Yashar Moshfeghi, Hideo Joho, R. Ren, David Hannah, Joemon M. Jose

Research output: Chapter in Book/Report/Conference proceedingConference contribution book

38 Citations (Scopus)


Over the years, recommender systems have been systematically applied in both industry and academia to assist users in dealing with information overload. One of the factors that determine the performance of a recommender system is user feedback, which has been traditionally communicated through the application of explicit and implicit feedback techniques. In this paper, we propose a novel video search interface that predicts the topical relevance of a video by analysing affective aspects of user behaviour. We, furthermore, present a method for incorporating such affective features into user profiling, to facilitate the generation of meaningful recommendations, of unseen videos. Our experiment shows that multimodal interaction feature is a promising way to improve the performance of recommendation.
Original languageEnglish
Title of host publication2009 IEEE International Conference on Multimedia and Expo
Subtitle of host publicationICME 2009
Place of PublicationPiscataway, N.J.
Number of pages4
ISBN (Print)978-1-4244-4290-4
Publication statusPublished - 1 Jun 2009


  • information filtering
  • search engines
  • user interfaces
  • video signal processing
  • facial expressions integration
  • implicit feedback techniques
  • multimodal interaction feature
  • multimodal recommender system
  • user feedback
  • user profiling
  • video search interface
  • aggregates
  • books
  • computer industry
  • emotion recognition
  • face recognition
  • feedback
  • motion pictures
  • recommender systems
  • support vector machine classification
  • support vector machines
  • affective feedback
  • facial expression analysis
  • muiltimedia retrieval

Fingerprint Dive into the research topics of 'Integrating facial expressions into user profiling for the improvement of a multimodal recommender system'. Together they form a unique fingerprint.

Cite this