ARtSENSE tackles a very important problem in the modern usage of ICT in cultural heritage domain: bridging the gap between the digital world with the physical in a highly flexible way in order to enable a novel, adaptive cultural experience. ARtSENSE aims to develop an active assistants which looks over the user’s shoulder (physical world) and react on any change in a visitor’s state of interests (user’s world) by adapting the guide (digital world) accordingly. The realization of such an ambitious result will be based on the combination of Augmented Reality for Cultural Heritage, Affective Computing and Complex Event Processing and will be applied and tested in three European museums/art spaces: The Foundation for Art and Creative Technology, Liverpool, UK, the Spanish National Museum of Decorative Arts in Madrid, Spain and the French National Museum of the History of Science and Technology (Musée des Arts et Métiers), in Paris, France.