JMIR Publications

ABSTRACT

Background:

Orbital fractures are common facial injuries, yet subtle fractures are frequently missed on CT due to visual fatigue and subjective interpretation. Missed fractures can lead to preventable complications including enophthalmos, diplopia, and permanent visual impairment if surgical intervention is delayed. Deep learning offers a promising solution for automated detection, but evidence on its diagnostic accuracy remains fragmented and has not been systematically synthesized.

Objective:

We sought to evaluate the overall diagnostic performance of deep learning (DL) for detecting orbital fractures on computed tomography (CT), and to explore heterogeneity by comparing analysis units (slice-level vs. non-slice-level). Additionally, we aimed to conduct exploratory evaluations of neural network architectures (CNNs vs. Transformers) and fracture subtype classification (trap-door vs. depressed).

Methods:

A systematic literature search was conducted across PubMed, Embase, Web of Science, Cochrane Library, and Scopus up to July 6, 2026. Methodological quality was assessed using QUADAS-2 and PROBAST+AI. For the primary diagnostic performance and unit-of-analysis comparisons, pooled sensitivity, specificity, and the area under the curve (AUC) of the optimal independent models were calculated using a bivariate random-effects model, strictly ensuring no patient overlap. Conversely, to prevent optimism bias, we exhaustively extracted data from all available models to conduct an exploratory comparison between CNN and Transformer architectures, acknowledging the inherent patient overlap. Furthermore, due to the availability of only a single study for subtype classification (trap-door vs. depressed), diagnostic metrics were solely extracted without meta-analytic pooling. This review was prospectively registered in PROSPERO (CRD420261442640).

Results:

Five studies encompassing 31,546 diagnostic instances were included. Without patient overlap, the optimal pooled DL models demonstrated excellent overall diagnostic performance with moderate certainty of evidence, yielding a sensitivity of 0.96 (95% CI: 0.92-0.98), specificity of 0.96 (95% CI: 0.87-0.99), and an AUC of 0.98 (95% CI: 0.95-0.99). Subgroup analysis indicated that slice-level evaluations artificially inflated specificity compared to non-slice-level evaluations (0.96 vs. 0.83, P < 0.001). In the exhaustive exploratory analyses containing patient overlap, CNNs maintained a robust sensitivity of 0.88, whereas Transformers were compromised (0.41); additionally, algorithms showed high accuracy (0.83-0.92) in classifying trap-door versus depressed subtypes within the single available cohort.

Conclusions:

DL demonstrates excellent diagnostic accuracy for detecting orbital fractures on CT, showing great potential as a triage tool. However, slice-level assessments can artificially inflate specificity. Exploratory comparisons suggest CNNs may outperform Transformers and show potential in identifying critical trap-door subtypes, though these specific findings are limited by inherent patient overlap and lack of multicenter external validation. Future large-scale prospective validations adhering to standardized guidelines are urgently required to verify these findings.