JMIR Publications

ABSTRACT

Background:

Diseases of the oral cavity, including oral squamous cell carcinoma (OSCC) pose major challenges to healthcare worldwide due to its late diagnosis and complicated differentiation of oral tissues. Endoscopic hyperspectral imaging (eHSI) represents a promising approach to the demand for modern, non-invasive tissue diagnostics. This dataset is designed to enhance the performance of deep learning models by providing comprehensive spectral data essential for distinguishing between healthy and pathological oral tissue conditions.

Objective:

To develop and validate a clinical dataset of endoscopic hyperspectral imaging (eHSI) of the oral cavity and to evaluate the performance of deep learning-based semantic segmentation models for automated tissue classification.

Methods:

This clinical study included 226 participants (166 women, 60 men, aged 24-87). eHSI data were collected using an endoscopic hyperspectral sensor system, capturing spectral data in the range of 500-1000 nm. Each participant underwent five standardized intraoral hyperspectral scans of the cheek, palate, tongue, and teeth. RGB and eHSI images were archived in NPY format for Python analysis. Oral structures were annotated using RectLabel Pro©. DeepLabv3 with a ResNet-50 backbone was adapted for eHSI segmentation by modifying the first convolutional layer. The model was trained for 50 epochs on 70% of the dataset, with 30% for evaluation. Performance metrics (Precision, Recall, F1-score) confirmed its efficacy in distinguishing oral tissue types.

Results:

Preliminary analysis revealed that the Coefficient of Variation exceeded 15% for most spectral bands, indicating high variability in spectral signatures. DeepLabv3 (ResNet-101) achieved strong segmentation performance (F1-score: 0.856, Precision: 0.8506, Recall: 0.8634), excelling in key structures—mucosa (F1: 0.915), retractor (F1: 0.940), tooth (F1: 0.902), and palate (F1: 0.900). Moderate results for gingiva (F1: 0.760) and lip (F1: 0.706) suggest potential for further refinement. To assess dataset robustness, additional models were tested, including DeepLabv3 (ResNet-50), FCN (ResNet-50/101), PSPNet (ResNet-50/VGG16), and U-Net (EfficientNet-B0/ResNet-50). DeepLabv3 (ResNet-101) and U-Net (EfficientNet-B0) emerged as top performers, particularly in retractor, mucosa, and tooth segmentation.

Conclusions:

Conducting an in-depth hyperspectral analysis of oral tissue has facilitated the development of robust deep learning algorithms, thereby enhancing diagnostic accuracy and clinical applicability. This study integrates advanced imaging technologies with deep learning, offering significant potential to enhance non-invasive detection, classification, and therefore individualized treatment of oral diseases. Clinical Trial: Deutsche Forschungsgemeinschaft (DFG) - Projectnumber 516210826 https://gepris.dfg.de/gepris/projekt/516210826