JMIR Publications

ABSTRACT

Background:

Traditional Chinese Medicine (TCM) emphasizes the concept of medicine food homology (MFH), integrating dietary therapy into healthcare. However, applying MFH often requires extensive expert knowledge and manual interpretation, creating challenges for automating MFH-based dietary recommendations. While large language models (LLMs) show potential in healthcare decision support, they often generate inaccurate or misleading information when involving knowledge in specific domains, such as TCM. Retrieval-augmented generation (RAG) addresses this by integrating external knowledge, but it struggles to model the inherent heterogeneity and uncertainty within TCM knowledge. Integrating uncertain knowledge graphs (UKGs) with LLMs offers a promising solution, as it not only provides a structured yet flexible representation of TCM's individualized principles but also enhances LLMs' ability to generate more accurate MFH-based dietary recommendations.

Objective:

This study introduces Yaoshi-RAG, a framework that leverages a UKG to enhance LLMs' capabilities in generating accurate and personalized MFH dietary recommendations based on TCM principles.

Methods:

The proposed framework began by constructing a comprehensive MFH knowledge graph (KG) through LLM-driven Open Information Extraction, which extracted structured data from multiple sources. It then employed UKG reasoning to complete missing triples and measured the confidence of the extracted triples. When processing user queries, query entities were identified and linked to the MFH KG, enabling retrieval of appropriate reasoning paths. These reasoning paths were then ranked based on triple confidence scores and entity importance to enhance accuracy. Finally, the retrieved knowledge was integrated through prompt engineering, enabling the LLM to generate personalized dietary recommendations that aligned with both individual user requirements and established TCM principles. The framework underwent validation through both automated metrics and human evaluation.

Results:

The constructed MFH KG comprised 24,984 entities, 22 relations, and 29,292 triples. Integration of this KG significantly enhanced LLM performance across all evaluation metrics, yielding an average increase of 14.5% in Hits@1 and 8.7% in F1-score, respectively. Among evaluated base LLMs, DeepSeek-R1 demonstrated superior performance, achieving 84.2% in Hits@1 and 71.5% in F1-score, respectively. Human evaluation corroborated these quantitative findings, confirming that Yaoshi-RAG consistently outperformed baseline models across all assessed quality dimensions.

Conclusions:

This study presents Yaoshi-RAG, a new framework that enhances LLMs' capabilities in generating MFH dietary recommendations through the knowledge retrieved from a UKG. By constructing a comprehensive MFH knowledge representation, our framework effectively extracts and utilizes TCM principles. Experimental results demonstrate the effectiveness of our framework in synthesizing traditional wisdom with advanced language models, facilitating personalized dietary recommendations that address individual health conditions while providing evidence-based explanations grounded in TCM principles.