JMIR Publications

ABSTRACT

Background:

Talaromycosis and cryptococcosis are prevalent in Southern China and Southeast Asia and are frequently misclassified due to overlapping lesion morphology and limited access to confirmatory testing.

Objective:

To evaluate the zero-shot diagnostic performance of multimodal large language models in identifying and differentiating cutaneous lesions of talaromycosis and cryptococcosis

Methods:

Published clinical photographs of cutaneous lesions of talaromycosis and cryptococcosis were systematically retrieved up to 31 August 2025, and seven representative multimodal large language models were benchmarked under a strictly zero-shot setting using a standardized prompt template and a predefined output schema. Latency, unanswerable/invalid response rates, and diagnostic performance were evaluated using accuracy, precision, sensitivity, specificity, F1-score, and Matthews correlation coefficient. For explanation quality assessment, model-generated texts were independently rated by two clinicians across five dimensions, and hallucination events were quantified.

Results:

In total, 214 articles (95 for talaromycosis and 119 for cryptococcosis), including 244 talaromycosis cutaneous lesion images and 236 cryptococcosis cutaneous lesion images, were collected for zero-shot evaluation. Most models achieved acceptable performance recognition, among them, ChatGPT-5 achieved the best performance. For comprehensive performance comparison, ChatGPT-5 ranked first across six indicators but exhibited relatively lower sensitivity. Evaluation of the output text quality demonstrated that the diagnostic texts generated by GPT-5 were excellent. The EQI was 70.08, with a hallucination rate of 21.76%.

Conclusions:

ChatGPT-5 demonstrates feasibility in the recognition of cutaneous lesions of talaromycosis and cryptococcosis under zero-shot conditions and can serve as a potential tool for assisting in the analysis of infectious skin disease images.