JMIR Publications

ABSTRACT

Background:

Medication adherence remains a significant concern in both clinical practice and public health. Non-adherence to prescribed medication regimens is associated with poorer health outcomes, higher rates of hospitalization, and increased financial burdens on healthcare systems. Despite its critical role in ensuring treatment efficacy, adherence assessment is still largely dependent on patient self-reports, pharmacy refill records, or caregiver observations, methods that are often subjective, inconsistent, and unreliable. Therefore, there is an urgent need for objective, automated solutions to accurately monitor medication, particularly the information on the pills taken and intake behavior. Artificial intelligence (AI) techniques, especially computer vision, are promising solutions; however, their application in assessing medication adherence has several key pitfalls in maintaining robustness under environment variation that limit their reliability and real-world applicability.

Objective:

This study aimed to develop a reliable and accurate computer vision framework for pill detection and counting using mobile phone images captured under diverse real-world conditions to support medication adherence assessment.

Methods:

This study is carried out on a public dataset on pill detection (N = 152) and a curated dataset (N = 60) on medical adherence assessment. Inspired by mixture of experts (MoE), we propose and implement a robust pill detection framework that integrates two expert-level object detection models, You Only Look Once (YOLOv12) and Faster Reginal-based Convolution Neural Network (Faster R-CNN), with a Segment Anything Model (SAM). The pill counting performance was evaluated using mean absolute error (MAE) and accuracy. We further conduct a pilot study on a curated dataset that mimics a subject taking medicine for a period of 20 days. We evaluate capability sensitivity and specificity in medical adherence assessment.

Results:

The proposed multi-model framework substantially reduces both error types, achieving a low MAE of 0.17 on the public test set in pill counting and an accuracy of 0.93 in pill detection. In medical adherence simulation, the proposed framework achieves high accuracy (0.9167), specificity (0.90), and sensitivity (0.93), significantly outperforming existing AI models (p<0.05). Performance remained robust on the self-collected dataset in medicine taken scenarios in the settings of household, restaurant, office, and outdoor environments.

Conclusions:

The AI-enhanced approach achieves high performance in pill counting, detection, and medical adherence detection. This proof-of-concept study has demonstrated the potential to enhance medication adherence among patients and support more reliable treatment monitoring.