JMIR Publications

ABSTRACT

Background:

Emotional state in everyday life is an essential indicator of health and wellbeing. However, daily assessment of emotional states largely depends on active self-reports, which is often inconvenient and prone to incomplete information. Automated detection of emotional states and transitions based on daily context could be an effective solution to this problem. Yet, the relationship between the emotional transitions and everyday contexts remains unexplored.

Objective:

The objective of this study is two-fold: 1) to explore the relationship between contextual information and emotional transitions/states, 2) to evaluate the feasibility of detecting emotional transitions and states from daily contextual information using Machine Learning (ML) techniques.

Methods:

This study was conducted on 18 persons’ data from a publicly available dataset called ExtraSensory. Contextual data were collected passively using smartphone and smartwatch sensors in a free-living condition, where the number of days for each person varied from three to nine. Sensors include an accelerometer, gyroscope, compass, location services, microphone, phone state indicator, light, temperature, and barometer. The users self-reported approximately 49 discrete emotions at different intervals via a smartphone application throughout the data collection period. We mapped the 49 reported discrete emotions to the three dimensions of the PAD (Pleasure, Arousal, Dominance) model and considered six states of emotions: discordant, pleased, dissuade, aroused, submissive, and dominant. The prevailing emotional states were passively detected every five minutes from the contextual data. We also studied the feasibility of using daily contextual information to detect transitions in emotional states in five minute intervals. The transition detection problem is a binary classification problem that detects whether the person’s emotional state has changed over time or not. In both cases, a wide range of supervised machine learning algorithms were leveraged, in addition to a variety of data preprocessing, feature selection, and data imbalance handling techniques. Lastly, an assessment has been conducted to shed light on the association between everyday context and emotional states.

Results:

This study obtained promising results for emotional state and transition detection. The best accuracy of emotional state detection varied from 88% to 100% across different persons. Despite the highly imbalanced data, the obtained best value of balanced accuracy of emotional transition detection varied from 73% to 85% for different persons. Feature analysis shows that spatio-temporal context, phone state, and activity related information are the most informative for emotional state/transition detections. Our assessment showed that there is an impact of lifestyle on the predictability of emotion.

Conclusions:

Our results demonstrate a strong association of daily context with emotional states and transitions as well as the feasibility of detecting them using data from smartphone and smartwatch sensors. Clinical Trial: N/A