JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI) applications based on advanced deep learning methods in image recognition tasks can increase efficiency in monitoring of medication adherence through automation. AI has sparsely been evaluated for monitoring of medication adherence in clinical settings. Yet, AI has potential to transform the way health care is delivered even in limited -resource settings such as Africa.

Objective:

We aimed to pilot the development of a deep learning model for simple binary classification and confirmation of proper medication adherence to enhance efficiency in use of video monitoring of patients in tuberculosis treatment.

Methods:

We used a secondary dataset of ~800 video images of medication intake that were collected from consenting adult patients with tuberculosis in an IRB-approved study evaluated Video observed therapy in Uganda. The video images were processed through a series of steps to prepare them for use a training model. First, we annotated videos using a specific protocol to eliminate those with poor quality. After the initial annotation step, 497 videos had sufficient quality for training the models. We used a deep learning framework that leveraged four convolutional neural networks models to extract visual features from the video frames and automatically perform binary classification of adherence or non-adherence. We evaluated the diagnostic properties of the different models using sensitivity, specificity, F1 score and positive precision. The area under the curve (AUC) was used to assess the discriminative performance and the speed per video review as a metric for model efficiency. We conducted a five-fold internal cross-validation to determine the diagnostic and discriminative performance of the models. We did not conduct external validation due to a lack of publicly available datasets with specific medication intake video frames.

Results:

Diagnostic properties and discriminative performance from internal cross-validation were moderate to high in the binary classification tasks with four selected automated DL models. The sensitivity ranged from 92.8 to 95.8%, specificity 43.5 to 55.4%, F1 score 0.91 to 0.92, precision 88.0 to 90.1% and AUC 0.78 to 0.85. The 3D ResNet model had the highest precision, AUC and speed.

Conclusions:

All four deep learning models showed comparable diagnostic properties and discriminative performance. The findings serve as a reasonable proof-of-concept to support the potential application of artificial intelligence in binary classification of video frames to predict medication adherence.