Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Currently submitted to: JMIR AI

Date Submitted: Nov 6, 2026

Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.

Machine Learning Prognostic Models in Spine Trauma: Systematic Review, Methodological Assessment, and Benchmark Meta-Analysis

  • Caitlyn Smith; 
  • Zheqing Zhang; 
  • Jeremiah Oluwatomi Itodo Daniel; 
  • Anushka Acharya; 
  • Kamala Thompson; 
  • Najib E El Tecle

ABSTRACT

Background:

Machine learning (ML) prognostic models for spine trauma are proliferating, typically reporting areas under the receiver operating characteristic curve (AUCs) above 0.85. At least 8 reviews have cataloged them, but none has tested whether reported performance is trustworthy, whether ML improves on established clinical rules, or whether models predict what patients need.

Objective:

This study aimed to determine whether the performance reported by ML prognostic models in spine trauma reflects real, transportable predictive value.

Methods:

We searched PubMed, Embase, Scopus, Web of Science, and the Cochrane Library (January 2015 to May 2025), supplemented by cross-checking 8 previous reviews, reference lists, and targeted web searching (October 2026). Eligible studies developed or validated ML models predicting clinical outcomes after traumatic spinal cord injury (SCI) or spinal column injury. Besides PROBAST+AI (Prediction Model Risk of Bias Assessment Tool for artificial intelligence), we assessed each full text against 11 prespecified criteria, including data leakage, minimum sample size, calibration, comparison with regression or clinical rules, performance above the no-information rate, and numerical consistency. For independent walking, the only outcome with a shared benchmark (the van Middendorp rule), we pooled AUCs and paired AUC differences using Hartung-Knapp-Sidik-Jonkman random-effects meta-analysis.

Results:

Of 28 included studies, 27 were retrospective and 4 (14%) were externally validated. Outcome-informed leakage was found in 13 studies (46%; 95% CI 30%-64%), most often from feature selection outside resampling (n=8). Only 4 of 19 assessable studies had adequate sample sizes (median events per candidate parameter 3.2, IQR 1.7-6.3). Calibration was assessed in 8 studies (29%); none reported calibration slope or intercept. Three of 16 studies reporting accuracy did not exceed the no-information rate, and 3 reported metrics irreconcilable with each other or with prevalence. AUCs fell by a median of 0.051 (up to 0.177) from development to held-out data. Within studies, flexible ML exceeded regression by a median of 0.009 AUC. For independent walking, ML (pooled AUC 0.908, 95% CI 0.876-0.933) performed no better than the van Middendorp rule (0.918, 95% CI 0.885-0.942; pooled paired difference −0.004, 95% CI −0.032 to 0.024). In the only clinician benchmark, pooled predictions of 6 neurologists matched ML (accuracy 0.89 vs 0.88-0.90). No study predicted sexual function, neuropathic pain, or quality of life, which people with SCI prioritize highly.

Conclusions:

Reported AUCs for ML prognostic models in spine trauma substantially overstate their readiness: nearly half of studies are affected by leakage, most are underpowered and uncalibrated, and where a fair benchmark exists, ML adds no discrimination over a 5-variable clinical rule or pooled clinician judgment. Future models should target patient-prioritized outcomes, embed every data-dependent step within resampling, meet sample-size requirements, report calibration and baselines, and be benchmarked against established rules and clinicians. Clinical Trial: PROSPERO CRD420251128049; https://www.crd.york.ac.uk/PROSPERO/view/CRD420251128049


 Citation

Please cite as:

Smith C, Zhang Z, Oluwatomi Itodo Daniel J, Acharya A, Thompson K, El Tecle NE

Machine Learning Prognostic Models in Spine Trauma: Systematic Review, Methodological Assessment, and Benchmark Meta-Analysis

JMIR Preprints. 06/11/2026:116363

DOI: 10.2196/preprints.116363

URL: https://preprints.jmir.org/preprint/116363

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.