JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI) is rapidly transforming medical practice, with emerging applications in perioperative care and anesthesiology. However, the clinical implementation of AI-assisted decision-making systems in anesthetic management remains challenging and requires comprehensive evaluation.

Objective:

This study aimed to assess the performance and clinical applicability of an AI-assisted decision support system (ZW-AA-001) for general anesthesia management by comparing its decisions with those of experienced anesthesiologists across six medical centers.

Methods:

A multicenter retrospective study was conducted using perioperative data from 1008 patients who underwent elective non-cardiac surgeries under total intravenous anesthesia. The AI system's recommendations for anesthetic and hemodynamic medication adjustments were compared with anesthesiologists' decisions. Key outcomes included decision concordance rates, temporal performance, and consistency across centers. Advanced statistical methods, including prevalence-adjusted and bias-adjusted kappa (PABAK) and Gwet's AC1, were used to evaluate agreement metrics.

Results:

The study included 1008 patients, with a median age of 50 years (IQR: 37-59) and female predominance (61.4%). During anesthesia maintenance, the AI system demonstrated general overall decision agreement with anesthesiologists (73.3%, 95% CI: 72.4%-74.3%; P<.001). Analysis of center-specific data revealed consistent performance across all six centers. For propofol management, high concordance was observed in dosage adjustment decisions (91.1%, 95% CI: 90.4%-91.8%; P<.001), though moderate agreement was found in adjustment direction (68.6%, 95% CI: 67.3%-69.8%; P<.001). The AI system showed significantly faster decision-making time compared to anesthesiologists for the adjustment of propofol (median difference: -71.0 seconds, 95% CI: -75.0 to -68.0; P<.001). Although esmolol-related decisions showed a numerically higher concordance of 71.4%, this was not statistically significant (P=.45; PABAK=0.429, P=.21; AC1=0.622, P=.09). Decisions for atropine, ephedrine, and urapidil demonstrated substantially lower agreement (17.4%–29.8%). The AI system recommended hemodynamic interventions more frequently than anesthesiologists across all medications.

Conclusions:

The AI-assisted decision support system demonstrated varying levels of concordance with anesthesiologists in managing surgery, with the highest agreement observed in propofol administration. However, the system's lower agreement in hemodynamic medication management highlights the need for further optimization and validation. These findings underscore the potential of AI systems to enhance anesthetic decision-making, improve efficiency, and address workforce challenges in anesthesiology. Future research should focus on expanding the system's capabilities, validating its performance in prospective clinical trials, and ensuring its integration into diverse clinical settings. Clinical Trial: no.MR-31-23-023706