JMIR Publications

ABSTRACT

Background:

Frontline health workers (FLWs) in low- and middle-income countries (LMICs) often face barriers that compromise care quality, including limited training, high patient loads, inadequate access to updated clinical guidance, and resource constraints. Working in underserved settings further exacerbates challenges to providing accurate, complete, and empathetic responses. Addressing these gaps requires innovative tools to support FLWs in real time.

Objective:

This study aims to evaluate the effectiveness of large language models (LLMs) in enhancing the quality of FLWs’ responses to health inquiries in Lagos, Nigeria.

Methods:

In this observational study, 36 licensed FLWs (doctors, nurses, community health workers) practicing general or primary care in Lagos, Nigeria, generated responses to 15 patient-generated health questions covering maternal and neonatal health, family planning, and sexually transmitted infections. Each FLW produced responses using their own knowledge and available resources (“human-only”) and with ChatGPT-3·5 assistance (“GPT-aided”). A panel of clinicians evaluated responses using a 5-point Likert scale for accuracy, empathy, contextualization, completeness, safety, and overall preference. In addition, 8 non-clinician health seekers assessed a subset of responses for trustworthiness, clarity, and perceived quality of care.

Results:

Across 1080 responses from 36 FLWs, ChatGPT-aided responses significantly outperformed human-only responses on all clinician-rated metrics (P<.001), with the largest gains in completeness, empathy, and overall preference - particularly among non-physician FLWs. Non-clinician evaluators also preferred ChatGPT-aided responses, especially for trustworthiness. The findings highlight LLMs' potential to improve the quality, empathy, and trustworthiness of FLWs’ responses, particularly in low-resource settings. There is a need to explore the safe integration of LLMs into clinical workflows, alongside capacity-building and further evaluation to ensure effectiveness, equity, and patient safety.

Conclusions:

This study provides empirical evidence that a large language model enhances the accuracy, empathy, and perceived trustworthiness of responses by frontline health workers in Lagos, Nigeria. These findings underscore the potential of LLM integration to strengthen healthcare delivery in resource-limited settings, highlighting the necessity for further investigation into its safe, equitable, and effective implementation.