inHarmony: a Digital Twin for Emotional Well-being

Abstract

A digital twin is an enabling technology that facilitates monitoring, understanding, and providing continuous feedback to improve quality of life and well-being. Thus, a digital twin can consider a solution to enhance one's mood to improve the quality of life and emotional well-being. However, there remains a long road ahead until we reach digital twin systems that are capable of empowering development and the deployment of digital twins. This is because there are so many elements and components that can guide the design of a digital twin. This thesis provides a general discussion for the central element of an emotional digital twin, including emotion detection, emotional biofeedback, and emotion-aware recommender systems. In the first part of this thesis, we propose and study the emotion detection models and algorithms. For emotions, which are known to be highly user dependent, improvements to the emotion learning algorithm can significantly boost its predictive power. We aimed to improve the accuracy of the classifier using peripheral physiological signals. Here, we present a hybrid sensor fusion approach based on a stacking model that allows for data from multiple sensors and emotion models to be jointly embedded within a user-independent model. In the second part of this thesis, we propose a real-time mobile biofeedback system that uses wearable sensors to depict five basic emotions and provides the user with emotional feedback. These systems apply the concept of Live Biofeedback through the introduction of an emotion-aware digital twin. An essential element in these systems guides users through an emotion-regulation routine. The proposed systems are aimed at increasing self-awareness by using visual feedback and provide insight into the future design of digital twins. We focus on workplace environments, and the recommendations are based on human emotions and the regulation of emotion in the construct of emotional intelligence. The objective is to suggest coping techniques to a user during an emotional, stressful episode based on her or his preferences, history of what worked well and appropriateness for the context. The developed solution has been studied based on usability studies and extensively compared to related works. The obtained results show the potentials use as an emotional digital twin. In turn, the proposed solution has been providing significant insights that will guide future developments of digital twins using several scenarios and settings.