JMIR Publications

ABSTRACT

Background:

Periodontitis is a chronic gum disease affecting approximately 42% of adults aged 30 and older in the United States. Training dental students to accurately diagnose and manage periodontitis is a critical component of dental education and clinical care. Recent advances in large language models (LLMs) offer new opportunities to support both domains, yet their performance in periodontal diagnosis remains largely unexplored—particularly for newer models such as Generative Pre-trained Transformer 5 (GPT-5).

Objective:

This study conducted an exploratory evaluation of GPT-5’s ability to stage and grade periodontitis.

Methods:

Twenty-five publicly available clinical cases explicitly reporting periodontitis stage and grade were identified through Google and PubMed searches. Each case description was entered into GPT-5 using a zero-shot prompting approach to assess guideline-based reasoning without exemplar conditioning. The model’s predictions were compared with the published reference diagnoses. Performance was measured using accuracy, 95% confidence intervals (CI), unweighted, and weighted Cohen’s kappa.

Results:

Across these cases, GPT-5 showed marked class-dependent performance and a tendency to overestimate disease severity. Grading performance was notably imbalanced, with high recall for Grade C but substantially lower discrimination for Grade B. GPT-5 achieved a staging accuracy of 68.0% (95% CI: 0.484-0.828) and a grading accuracy of 77.3% (95% CI: 0.566-0.899), with corresponding Cohen’s kappa values of 0.454 (95% CI: 0.110-0.756) and 0.179 (95% CI: -0.158-0.638), respectively. While staging performance showed fair agreement beyond chance, the low kappa for grading indicates poor agreement and limited reliability in distinguishing periodontal disease severity.

Conclusions:

These findings suggest that although GPT-5 demonstrates potential for guideline-based periodontitis staging and grading, its current diagnostic performance, particularly for periodontitis grading, limits its utility in clinical assessment and educational training. Meaningful application in periodontal diagnosis and training will require substantial improvements in reliability and rigorous validation in larger, more diverse, and prospectively collected datasets.