Accepted for/Published in: JMIR Human Factors
Date Submitted: Sep 3, 2023
Date Accepted: Dec 1, 2024
Comparison of a novel machine learning-based clinical query platform with traditional guideline searches for hospital emergencies: A prospective pilot study of user experience and time efficiency.
ABSTRACT
Background:
Identification of clinically relevant guidance is of particular importance in emergency care. The potential use of artificial intelligence (AI) in signposting locally relevant guidelines has not been examined.
Objective:
To develop and test a machine learning algorithm for retrieval of information in hospital care.
Methods:
Clinical information searches by doctors in training caring for acutely unwell patients with complex needs in Acute Medicine were observed during 10 working days. Based on findings and the results of a focus group with 14 clinicians a context sensitive search engine was implemented, and clinical practice observed during a further 10 working days.
Results:
Doctors in the control group had a median of 23 and in the intervention group a median of 54 months clinical experience. Participants in the intervention group undertook less searches. There were no differences in user satisfaction or likelihood of a search result solving the query between the two phases. Searches with the novel proof-of-concept engine took 43 seconds longer. Clinicians rated the application with a favourable Net Promotor Score of 20.
Conclusions:
We report a successful feasibility pilot of an AI driven search engine for clinical guidelines. Further development of the engine including the incorporation of Large Language Models might improve accuracy and speed. More research is required to establish clinical impact in different user groups. Focusing on new staff at beginning of their post might be the most suitable study design. Clinical Trial: n.a.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.