Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Jul 27, 2023
Open Peer Review Period: Jul 27, 2023 - Sep 21, 2023
Date Accepted: Oct 6, 2023
(closed for review but you can still tweet)
AI Dietitian for Type 2 Diabetes Mellitus Management Based on Large Language Model and Image Recognition Model: A Preclinical Concept Validation Study
ABSTRACT
Background:
Nutritional management for diabetes patients in China is limited due to low supply of registered clinical dietitian. To address this, we've created an AI-based nutritionist program that uses advanced language and image recognition models. This program can identify ingredients from images of a patient's meal and offer nutritional guidance and dietary recommendations.
Objective:
The primary objective of this study is to conduct a preclinical evaluation to validate the conceptual framework of this AI-based nutritionist program.
Methods:
In this preclinical study, we evaluated the potential of an AI nutritionist program for T2DM patients through a multi-step process. Firstly, a survey was conducted among T2DM patients and endocrinologists to identify knowledge gaps in dietary practices. The AI program was then tested through the Chinese Registered Dietitian examination to assess its proficiency in providing evidence-based dietary advice. The AI's responses to open-ended questions about MNT were compared with expert responses to evaluate its proficiency. The model's food recommendations were scrutinized for consistency with expert advice. A deep learning-based image recognition model was developed for food identification at the ingredient level, and its performance was compared with existing models. Finally, a user-friendly app was developed, integrating the capabilities of language and image recognition models to potentially improve care for T2DM patients.
Results:
We collected questionnaires from 206 patient and 26 doctor, and revealed that most patients demand more immediate and comprehensive nutritional management and education. Both ChatGPT and GPT 4.0 passed the Chinese Registered Dietitian examination. The food recommendations were mostly in line with best practices, except for certain foods like root vegetables and dry beans. The responses to common questions were largely positive, with 162 out of 168 favorable reviews. The multi-label image recognition model evaluation showed that the Dino V2 model achieved an average F1 score of 0.943, indicating high accuracy in recognizing ingredients.
Conclusions:
Our promising research led us to develop FoodMed Companion, a nutritional management tool. This mini program can identify food ingredients from user-uploaded images and inform the user about the suitability of these ingredients. Patients can also interact with ChatGPT API, which is limited to provide nutritional education and advice. This tool is now ready for a supervised pilot study.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.