19 Novembre – Thesis defense - Nicolas Simonazzi

10 h ENSC Talence

Emotion recognition through interaction with smartphone. Design of methods and tools for the field of customer relations.

The idea of this thesis came from the Orange Labs Service entity in the telecommunication company Orange. The objective of the research project is to improve the quality of the company’s support services through a better understanding of customers’ emotions. Emotions are essential to be considered in a communication situation.
Indeed, they are necessary to adapt to and understand the intentions of a conversational partner. Moreover, smartphones represent a new medium for communicating with customer advisors or chatbot, especially through instant messaging. However, these means of communication deprive the interlocutors of many ways to express their emotional states, such as non-verbal expressions.
The objective of this thesis is therefore to explore the feasibility of recognizing emotions in an automatic and non-invasive way. First, we explored the existing theories in the science of emotions. We based our theoretical framework on the theory of cognitive appraisal and the process-component model in order to identify the components associated with the expression of emotional states. Then, we conducted a literature review of the methods used in the field of affective computing to design automatic emotion recognition systems.
As a result of this research, we have decided to measure the motor expression com ponent of emotional states using inertial sensors (accelerometer and gyroscope) and interactions with the smartphone keyboard. The contribution of this thesis concerns, among other things, the design and development of tools to perform experiments on a smartphone and to record the necessary data. An experimentation application was developed, under an instant messaging format, to induce emotions and to record the necessary data.
Several emotion induction methodologies were tested in the laboratory to collect interaction data from smartphone sensors. The data were labeled according to the participants’ self-report. Following the processing of the temporal signals, we generated a corpus of data suitable for machine learning. This corpus was used to train and evaluate supervised classification models by machine learning. Inter-personal and intra-personal classification models were created to classify self-reported valence levels. The results show a high degree of variability in classification performance across participants.

Event localization