Accepted for/Published in: JMIR Formative Research
Date Submitted: Aug 29, 2022
Date Accepted: Jan 31, 2023
Implementing a Machine Learning Screening Tool for Malnutrition: Insights from Qualitative Research Applicable to Other ML-Based CDSS
ABSTRACT
Background:
Machine learning (ML)-based CDSS are popular in clinical practice settings but are often criticized for being limited in usability, interpretability and effectiveness. Evaluating implementation of ML-based CDSS is critical to ensure CDSS is acceptable and useful to clinicians and helps them deliver high quality health care. Malnutrition is a common and underdiagnosed condition among hospital patients that can have serious adverse impacts. Early identification and treatment of malnutrition is important.
Objective:
To evaluate the implementation of a ML tool, Malnutrition Universal Screening Tool (MUST)-Plus, that predicts hospital patients at high risk for malnutrition and identify implementation best practices applicable to this and other ML-based clinical decision support systems (CDSS).
Methods:
We conducted a qualitative post-implementation evaluation using in-depth interviews with registered dietitians (RDs) who use MUST-Plus output in their everyday work. After coding the data, we mapped emergent themes onto select domains of the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) framework.
Results:
We interviewed 17 of the 24 RDs approached (70.8%), representing 36.9% of those who use MUST-Plus output. Several themes emerged: (1) Enhancements to the tool were made to improve accuracy and usability; (2) MUST-Plus helped identify patients that would not otherwise be seen. Perceived usefulness was highest in the original site; (3) Perceived accuracy varied by respondent and site; (4) RDs valued autonomy in prioritizing patients; (5) Depth of tool understanding varied by hospital and level; (6) MUST-Plus was integrated into workflows and electronic health records; (7) RDs expressed a desire to eventually have one automated screener. Our findings suggest that continuous involvement of stakeholders at new sites given staff turnover is vital to ensure buy-in. Qualitative research can help identify potential bias of ML tools and should be widely used to ensure health equity.
Conclusions:
Ongoing collaboration between CDSS developers, data scientists, and clinicians may help refine CDSS for optimal use and improve acceptability of CDSS in the clinical context. Clinical Trial: N/A
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.