JMIR Publications

ABSTRACT

Background:

Antiretroviral therapy (ART) has transformed HIV from a fatal illness to a chronic disease. HIV programs employ a range of approaches to support individuals in adherence to ART and in reengaging those who interrupt treatment. These interventions can often be time consuming and costly, and thus providing for all may not be sustainable.

Objective:

To describe our early experiences in developing a machine learning (ML) model to predict interruption in treatment (IIT) at 30 days among people living with HIV (PLHIV) newly enrolled on ART in Nigeria and our experiences integrating the model into the routine information system. To ascertain health worker perceptions and use of the model’s outputs for case management.

Methods:

Routine program data collected from January 2005 through February 2021 was used to identify individual characteristics associated with interruption in treatment (IIT) among PLHIV receiving ART and to develop a ML model (boosting tree and extreme gradient boosting) to predict future IIT. Individuals were defined as having IIT if they were provided a 30-day supply of antiretrovirals (ARVs) but did not return for a refill within 28 days of their scheduled follow-up visit date. The model was applied to all new individuals enrolled on ART from July-August 2022. Outputs were shared weekly with health care workers at selected facilities.

Results:

After data cleaning, complete data for 136,747 clients were used for the analysis (74.7% of the clinic dataset). The percentage of IIT cases decreased from 58.6% before 2017 to 14.2% during October 2019 through February 2021. There were significant changes to ART and the ART treatment program during that time. Overall IIT was higher among clients sicker at enrollment. Several models were initially developed, and the selected model was successfully integrated into the national electronic medical records database. Variables significantly associated with IIT rates were missing TB, pregnancy, and breastfeeding status and facility characteristics (location, service level, and service type). The model was had a sensitivity of 81% and specificity of 88%. During field testing, the majority of users had a positive outlook on the model’s ability to identify high risk individuals early, but in this limited testing time some had mixed views regarding the usefulness and application of ML.

Conclusions:

Despite initial challenges, we were able to successfully develop and deploy a ML model into Nigeria’s routine HIV/AIDS information system. There was a high level of acceptance of the ML model among staff yet the need for continued training and guidance during scale up to ensure all health care workers understand the benefits.