JMIR Publications

ABSTRACT

Background:

The field of mental health technology presently has significant gaps that need addressing, particularly in the domain of daily monitoring and personalized assessments. Current non-invasive devices like wristbands and smartphones are capable of collecting a wide range of data, which has not yet been fully utilized for mental health monitoring.

Objective:

The paper aims to introduce a novel dataset for Personalized Daily Mental Health Monitoring and a new Macro-Micro Framework. This framework is designed to employ multimodal and multitask learning strategies for improved personalization and prediction of emotional states in individuals.

Methods:

Data was collected from 242 individuals using wristbands and smartphones, capturing physiological signals, speech data, and self-annotated emotional states. The proposed framework combines macro-level emotion transformer embeddings with micro-level personalization layers specific to each user. It also introduces a dynamic restrained uncertainty weighting method to effectively integrate various data types for a balanced representation of emotional states. Several fusion techniques, personalization strategies, and multitask learning approaches were explored.

Results:

The proposed framework was evaluated using the Concordance Correlation Coefficient (CCC), resulting in a score of 0.503. This result demonstrates the framework's efficacy in predicting emotional states.

Conclusions:

The paper concludes that the proposed multimodal and multitask learning framework, which leverages transformer-based techniques and dynamic task weighting strategies, is superior for the personalized monitoring of mental health. The study indicates the potential of transforming daily mental health monitoring into a more personalized application, opening up new avenues for technology-based mental health interventions.