Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Currently submitted to: JMIR AI

Date Submitted: Jul 8, 2026

Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.

Developing a Standardized Approach for Reviewing Video-Observed Treatment Tuberculosis Medication Adherence: Towards Human-AI Collaborative Protocol

  • Patrick Kaggwa; 
  • Yanqiao Wang; 
  • Suchendra Bhandarkar; 
  • Jin Sun; 
  • Juliet Nabbuye Sekandi

ABSTRACT

Background:

Video Directly Observed Therapy (VDOT) supports remote monitoring of tuberculosis (TB) treatment adherence, but daily manual review of medication-intake videos creates a major burden for healthcare workers. Artificial intelligence (AI) may reduce this burden, but standardized protocols for classifying TB medication videos are limited.

Objective:

To develop and validate a standardized four-category protocol for reviewing TB medication-intake videos to support consistent human review and human-AI monitoring.

Methods:

We developed a four-category protocol that classifies medication intake videos in either Positive, Negative, Uncertain, or Poor Quality based on lighting, face visibility, ingestion action, and pill evidence. To evaluate the protocol’s utility, we applied it to 782 videos from 69 adults who received TB treatment in Uganda. Validation included human review with adjudication, a supervised action-recognition model, and a zero-shot vision-language model (VLM). Performance was measured using agreement, accuracy, and F1 scores.

Results:

The protocol supported high human alignment, with an initial majority-vote agreement of 72.3%, increasing to 94.1% after consensus adjudication. Guided by the protocol categories, the zero-shot VLM classified the videos with 75.7% accuracy and a F1 score of 0.78, performing strongly on Positive (F1=0.84) and Poor Quality (F1=0.82) classes. The supervised action-recognition model achieved 63.7% accuracy and an F1 score of 0.54, performing well on Positive videos (F1=0.79) but poorly on Negative videos (F1=0.00).

Conclusions:

The standardized four-category protocol improves consistency in human review and provides a practical framework for reviewing VDOT videos in digital adherence monitoring programs. It also supports AI-based video interpretation, creating a foundation for scalable human-AI collaborative workflows.


 Citation

Please cite as:

Kaggwa P, Wang Y, Bhandarkar S, Sun J, Sekandi JN

Developing a Standardized Approach for Reviewing Video-Observed Treatment Tuberculosis Medication Adherence: Towards Human-AI Collaborative Protocol

JMIR Preprints. 08/07/2026:106518

DOI: 10.2196/preprints.106518

URL: https://preprints.jmir.org/preprint/106518

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.