JMIR Publications

ABSTRACT

Background:

Endometrial cancer incidence is rising globally, yet early detection remains hampered by the subjectivity and resource intensity of conventional diagnostics. There is a critical need for non-invasive, interpretable tools that integrate structured clinical data with unstructured medical knowledge.

Objective:

To develop and validate an AI-Agent–Based Endometrial Cancer Prediction (AIECP) model designed to enhance risk assessment accuracy and clinical interpretability.

Methods:

A total of 3,959 patient records were collected, and twelve machine learning algorithms were evaluated. The top five performing models were integrated using weighted soft voting to enhance predictive accuracy. Semantic embeddings of medical dialogues were generated using Sentence-BERT and stored in a vector database to enable context-aware retrieval. A locally fine-tuned large language model was then employed to synthesize classification results and retrieved knowledge, providing interpretable diagnostic explanations. The performance of these models was evaluated using several metrics: accuracy, precision, recall, F1-score, ROC-AUC, and PR-AUC, supplemented by a usability and feasibility evaluation involving 100 patients suspected of endometrial cancer.

Results:

The soft voting ensemble achieved a PR-AUC of 0.898 and a ROC-AUC of 0.724, outperforming all individual models. Given the pronounced class imbalance in the dataset, PR-AUC was emphasized as the primary performance metric, as it provides a more clinically meaningful assessment for early cancer risk stratification. Sentence-BERT embeddings demonstrated superior performance compared to conventional embedding methods in document classification tasks, achieving F1-scores of 0.95. In the case study, the predictions generated by the AIECP model exhibited a high level of concordance with clinical diagnoses. Additionally, user feedback indicated a high satisfaction rate, with an average rating of 4.8 out of 5.

Conclusions:

By integrating ensemble learning, knowledge retrieval, and contextual reasoning, the AIECP model effectively bridges data-driven inference and clinical decision support, facilitating real-world clinical translation and future deployment.