JMIR Publications

ABSTRACT

Background:

Early detection and monitoring of wound-related complications after total knee arthroplasty is critical to optimize outcomes and preserve implants. Conventional postoperative monitoring relies on in-person assessment and patient-reported symptoms, which may delay recognition. This study evaluated the technical feasibility of a multimodal artificial intelligence system integrating electro-optical (EO) imaging and infrared (IR) data for postoperative surveillance and characterized comorbidity patterns and baseline laboratory profiles associated with early complications.

Objective:

This aim of this study was to evaluate the technical feasibility and performance of a multimodal artificial intelligence system integrating electro-optical and infrared imaging for postoperative complication surveillance following total knee arthroplasty, and to explore clinical factors associated with early complications.

Methods:

We conducted a single-center prospective cohort study of 749 patients undergoing primary total knee arthroplasty. Patients underwent standardized multimodal data collection during the first 30 postoperative days. Models trained on EO data localized surgical landmarks and segmented complication‑oriented features; a fine‑tuned vision–language model provided protocol checks, complex scene understanding, and identification of patient-specific cues. Electro-optical landmarks were mapped to thermal space using a deterministic EO-to-IR alignment algorithm, where IR ROI (Region of Interest) segmentation and a thermal trained classification model quantified abnormal thermal pattern. Agreement with clinician-adjudicated infection-related complications served as the primary outcome. Clinical data, including comorbidities and baseline laboratory values, were analyzed to contextualize complication risk.

Results:

Among 749 patients, 26 (3.5%) experienced postoperative complications. Landmark detection achieved mAP@0.5 = 0.818 (primary landmarks > 0.92). Electro-optical segmentation reached AUROC 0.990, PRAUC 0.786. In the evaluated dataset, the finetuned visual language model reached 94.8% accuracy for infection-related complications, 96.9% for hematologic events, and >98% for surgical site detection. Infrared ROI extraction achieved IoU 0.9850 and ROCAUC 0.9997. The radiometric classifier yielded ROCAUC 0.937; at Youden’s J: sensitivity 0.826, specificity 0.884. Exploratory analyses showed nominal associations between chronic kidney disease, diabetes mellitus, and postoperative complications, although these did not remain significant after multiplicity correction.

Conclusions:

In this single-center formative study, a EO + IR multimodal sensor data demonstrated technical feasibility for postoperative wound image analysis following TKA in both a controlled clinical and an in-home setting. Exploratory comorbidity analyses identified clinical risk profiles associated with complications that may help generate hypotheses for future prospective validation. This formative work provides preliminary proof-of-concept for multimodal imaging analysis and highlights key technical requirements for future prospective clinical validation studies. Clinical Trial: N/A - This study was an observational cohort study and did not meet the criteria for mandatory registration on ClinicalTrials.gov.