JMIR Publications

ABSTRACT

Background:

Chatbots are increasingly valuable in clinical settings, offering rapid access to medical information, aiding documentation, and improving perioperative patient education. Their adaptability makes them promising tools for personalized risk stratification and anesthesia planning throughout surgical phases. However, their role in preoperative setting is not yet defined.

Objective:

This systematic review explores the current applications of chatbots in preoperative patient risk assessment and states their use recommendation based on evidence quality.

Methods:

This review followed PRISMA guidelines. The review protocol was defined by all Authors before the search and registered on PROSPERO (ID: CRD42025642357). A comprehensive search of electronic databases (PubMed, MeSH, MEDLINE, Scopus and Embase) was conducted up to August 2025. Primary outcome measured chatbot performance in perioperative risk prediction and planning compared to clinician judgment. Studies quality was assessed using Methodological Index for Non-Randomized Studies (MINORS) and Grading of Recommendations Assessment, Development, and Evaluation (GRADE) system. No funding was received for this paper.

Results:

Seven studies published between 2023 and 2025 were included with a total of 88.654 patients. Two papers reached a high-quality score of 16 or above according to MINORS Score. Main evaluation metrics were heterogeneous across papers (AUROC, AUPRC, sensitivity, specificity, Cohen’s kappa, percent agreement) limiting a structured comparison. Evidence suggests that: 1) chatbots may serve as a clinical support system in risk stratification but they are not autonomous in general anesthetic planning; 2) they have a good performance in classification tasks, but they struggle in unstructured one; 3) ChatGPT is efficient in predicting American Society of Anesthesiologists Physical Status score (ASA), but performance in the application of other clinical scores is not studied yet; 4) in only one study a chatbot (Gemini) seems to have a high concordance with clinicians’ anesthesia choice; 5) domain-specific models show comparable accuracy to the NSQIP, but slightly lower sensitivity.

Conclusions:

This study is subjected to several limitations. Selection bias because of the heterogeneous geographical distribution and surgical settings of the studies; detection bias because of the highly variable definitions of “performance” across studies and reporting bias due to the absence of raw data for independent re-analysis, reducing transparency and reproducibility. According to GRADE System the quality of the evidence is low to very low with a weak for using recommendation if the chatbot does not substitute the clinician’s judgment but acts as a clinical support system. Clinical Trial: PROSPERO ID: CRD42025642357