JMIR Publications

ABSTRACT

Background:

Calorimetry is both expensive and obtrusive, but provides the only way to accurately measure energy expenditure in daily-living activities of any specific person, as different people can use different amounts of energy despite performing the same actions in the same manner. Deep learning video analysis techniques have traditionally required lots of data to train, but recent advances in few-shot learning, where only a few training examples are necessary, have made developing personalised models without a calorimeter a possibility.

Objective:

The primary objective is to determine which activities are most well suited to calibrate a vision-based personalised deep learning calorie estimation system for daily-living activities.

Methods:

The SPHERE Calorie dataset is used, which features 10 participants performing 11 daily-living activities totalling 4.5 hours of footage. Calorimeter and video data is available for all recordings. A deep learning method is used to regress calorie predictions from video.

Results:

Models are personalised with 32s from all 11 actions in the dataset, and Mean Square Error (MSE) is taken against a calorimeter ground truth. The best single action for calibration is wipe (1.40 MSE). The best pair of actions are sweep and sit (1.09 MSE). This compares favourably to using a whole 30 minute sequence containing 11 actions to calibrate (1.06 MSE).

Conclusions:

A vision-based deep learning energy expenditure estimation system for a wide range of daily-living activities can be calibrated to a specific person with footage and calorimeter data from 32s of sweeping and 32s of sitting.