JMIR Publications

ABSTRACT

Background:

Medical record quality is a key focus of healthcare quality management, directly impacting clinical decision-making and the accuracy of research data. Traditional manual medical record quality control suffers from low efficiency, inability to achieve full-sample coverage, and poor consistency. Existing intelligent quality control systems have significant deficiencies in real-time intervention and quantitative assessment of application effects.

Objective:

This study aims to construct a medical record real-time quality control system (MRQCS) deeply integrated with clinical workflows, and evaluate its long-term impact on improving medical record defect detection efficiency, quality control consistency, and overall medical record quality.

Methods:

MRQCS implements a two-level quality control process on the physician side: non-blocking real-time prompts during the "writing stage" and blocking mandatory verification during the "completion stage". For QC reviewer, it provides AI-assisted defect detection and one-click write-back functionality.Quality control records from a tertiary hospital during 2019-2020 (non-AI-assisted) and 2024-2025 (AI-assisted) were compared and analyzed to evaluate system application effects, using statistical methods including Welch's ANOVA, one-way ANOVA, and Mann-Whitney U test.

Results:

By 2025, the MRQCS quality control rule library expanded from 40 rules initially to 597 rules. The AI-assisted defect write-back rate increased significantly from 12.35% in 2021 to 98.23% in 2024, with AI assistance becoming the core method of terminal quality control. The defect rate of sampled cases showed a trend of initial increase followed by decrease, reaching a peak of 61.87% in 2024 and declining to 60.65% in 2025. The high-risk sampled case defect rate (defect score ≥10 points) peaked at 9.7% in 2024 and decreased to 4.33% in 2025, a reduction of over 50%. Regarding quality control consistency, there was no significant difference in defect detection rates among different QC reviewer in the AI-assisted group (F=1.340, p=0.266), significantly better than the within-group differences in the non-AI-assisted group (F=22.874, p<0.001). Significant differences were found in defect detection rates for various document types before and after AI assistance, with substantial enhancement in content defect detection capabilities for key documents such as progress notes and admission records.

Conclusions:

Through the collaborative model of two-level real-time quality control and AI-assisted terminal quality control, MRQCS effectively reshaped the medical record quality management system, significantly improving medical record defect detection efficiency, homogenization of QC reviewer work, and overall medical record quality, providing a replicable practical model for the construction of medical record quality management systems.