JMIR Publications

ABSTRACT

Background:

The exponential growth of biomedical literature and clinical guidelines contributes to information overload for healthcare professionals, exacerbating clinician burnout and impeding evidence-based practice. Traditional resources often fail to deliver rapid, accessible information at the point of care. Retrieval-augmented generation (RAG) architectures using large language models (LLMs) offer potential for efficient, provenance-bound clinical decision support, but require evaluation in real-world settings to ensure safety and utility, particularly in public health informatics contexts like UK healthcare.

Objective:

To describe the design of iatroX, a UK-centred RAG-based clinical reference platform, and report early real-world adoption, usability, and perceived clinical utility from a formative implementation evaluation.

Methods:

This mixed-methods formative evaluation combined (i) a retrospective observational analysis of platform usage data across web, iOS, and Android applications over 16 weeks (8 April–31 July 2025) and (ii) a cross-sectional, in-product intercept survey. Usage metrics (unique users, engagement events, clinical queries) were sourced from Google Analytics 4, Apple App Store Connect, and Google Play Console, with bot filtering applied. A client-side script randomized single-item survey prompts to ~10% of web sessions from a predefined battery assessing usefulness, reliability, and adoption intent; responses were anonymous ("Yes/No/Don't Know"). Proportions were reported with Wilson 95% confidence intervals; qualitative comments underwent thematic content analysis. Reporting adhered to STROBE and CHERRIES guidelines. This was classified as a UK service evaluation, requiring no ethics review.

Results:

The platform achieved rapid adoption with 19,269 unique web users, 202,660 engagement events (~10.5 per active user), and ~40,000 clinical queries across platforms. Mobile downloads included 1,960 iOS and steady Android growth (peak >750 daily active users). The survey elicited 1,223 item-level responses: perceived usefulness 86.2% (95% CI 74.8–93.9%; 50/58); time saved 60.9% (95% CI 38.5–80.3%; 14/23); would use again 93.3% (95% CI 68.1–99.8%; 14/15); willingness to recommend 88.4% (95% CI 75.1–95.9%; 38/43); perceived accuracy 75.0% (95% CI 58.8–87.3%; 30/40); perceived reliability 79.4% (95% CI 62.1–91.3%; 27/34). Themes included speed, guideline-grounded responses, and UK specificity.

Conclusions:

iatroX demonstrated strong early uptake and positive user perceptions among UK clinicians, highlighting RAG's potential to mitigate information overload in public health informatics. Limitations include small per-item sample sizes and early-adopter bias; future work should include objective accuracy audits and prospective impact studies on workflows and care quality.