JMIR Publications

ABSTRACT

Background:

Accurate measurements of children’s digital technology use is essential for understanding its potential implications on health and wellbeing. Wearable cameras can provide such measurements, but the image coding is a high researcher burden. Machine learning based object recognition models have the potential to reduce this burden by identifying images containing technologies.

Objective:

To evaluate the performance of an object recognition model, the Square Eyes model, as a screening tool for identifying technologies in wearable camera images among children for further human review. Additionally, to examine the potential influence of face-blurring methods on the model’s performance.

Methods:

This study used data collected on 48 children (3-14 years of age) during a ~1 hour laboratory session. The children performed various technology related tasks while wearing a camera. A total of 211,226 images were coded by humans and processed through the Square Eyes model. The performance of the Square Eyes model as a screening tool was evaluated by 1) assessing agreement between the model and human coding; 2) evaluating the N-back algorithm, an algorithm embedded in the model aimed to flag images requiring human review, and 3) examining the potential influence of facial-blurring on model performance.

Results:

Humans detected technology in 92,745 (41.9%) images The Square Eyes model detected technologies with an overall accuracy of 78.0%. When considering specific technologies, agreement between the model and human coders was the highest for Television (54.3%) and Laptop (44.5%) and lowest for smaller devices such as Smartphone (31.3%) and Tablet (25.1%). The model’s N-back algorithm effectively flagged images that required further human review, with only 7,144 images (3.2% of all images) not flagged for screening containing a human-coded technology. An explorative analysis indicated that using a square face-blurring with border could have reduced the model’s ability to accurately detect technologies.

Conclusions:

The Square Eyes model demonstrated overall satisfying accuracy in detecting technologies and successfully flagged images that required further review by humans. These findings suggest that the model could be used as an effective screening tool for reducing the burden of human coding. However, the model could be improved for accurately detecting smaller devices and the form of facial blurring in images should be considered.