JMIR Publications

ABSTRACT

Background:

Artificial Intelligence (AI), particularly large language models (LLMs) like GPT-4, is increasingly being integrated into healthcare to enhance data mining capabilities, especially for unstructured medical records. The application of AI in neurosurgery has the potential to streamline data extraction, reduce operational inefficiencies, and improve clinical outcomes, but its effectiveness in extracting structured data from unstructured medical reports remains unclear.

Objective:

The aim of this proof-of-concept study was to evaluate the feasibility and accuracy of using GPT-4o for extracting structured data from unstructured neurosurgical reports, specifically focusing on patient data related to vestibular schwannomas (VS).

Methods:

This retrospective study included ten consecutive patients diagnosed with VS who underwent surgical treatment between August and December 2023. Raw medical reports, including discharge reports, surgery reports, histopathology reports, and 3-month follow-up reports, were provided to GPT-4o for data extraction. The AI’s performance in extracting structured data was evaluated by comparing its outputs with human expert evaluations. Data accuracy was assessed for both non-interpreted structured information (e.g., patient ID, tumor grade) and interpretive data (e.g., symptoms, complications, postoperative deficits).

Results:

GPT-4o demonstrated high accuracy in extracting straightforward data categories such as patient ID, date of surgery, histopathological diagnosis, and WHO tumor grade, with a 100% accuracy rate. However, extraction of variables like intraoperative complications and postoperative deficits initially showed lower accuracy (50%), which improved significantly (90-100%) after model adjustment using refined prompts.

Conclusions:

GPT-4o successfully extracted structured data from unstructured neurosurgical reports, achieving high accuracy for most data categories. This study highlights the potential of AI in enhancing clinical workflows, particularly in data mining and automated medical coding, which can reduce manual workloads and improve research and clinical operations. Further studies are required to refine these techniques and address regulatory and ethical concerns before widespread implementation.