JMIR Publications

ABSTRACT

Background:

Although new technologies have increased the efficiency and convenience of medical care, patients still struggle to identify specialized outpatient departments in Chinese tertiary hospitals due to a lack of medical knowledge.

Objective:

The objective of our study was to develop a precise and subdividable outpatient triage system to improve the experiences and convenience of patient care.

Methods:

We collected 395,790 EMRs and 500 medical dialogue groups. 387,876 (98%) of each dataset was used to design and train the triage model, 3,957 (1%) for testing and validation. The triage system was evaluated by recommendation accuracies in Xinhua Hospital, using the cancellation rate in 2021 and 2022, from 29 October to 5 December. Finally, a prospective observational study containing 306 samples was conducted to compare the system’s performance with triage nurses’, which was evaluated by calculating precision, accuracy, recall@3 (recall of the Top 3 recommended departments), and time consumption.

Results:

With 3,957 records each, the testing set and validation set achieved accuracies of 0.8945 and 0.8941, respectively. Implemented in Xinhua Hospital, our triage system could accurately recommend 79 subspecialty departments and reduce the number of registration cancellations from 16037 of the total 418714 (3.82%) to 15338 of the total 434200 (3.53%), P <.001. In comparison to the triage system, the performance of the triage nurses is more accurate (0.9803 vs 0.9153) and precise (0.9213 vs 0.9049) since the system could identify subspecialty departments, whereas triage nurses or even general physicians can only recommend main departments. Besides, our triage system significantly outperformed in recall@3 (0.6230 vs 0.5266, P <.001) and time consumption (10.11s vs 14.33s, P <.001).

Conclusions:

The triage system demonstrates high accuracy in outpatient triage of all departments and excels in subspecialty department recommendations, which could decrease the cancellation rate and time consumption. It improves the efficiency and convenience of clinical care, to fulfill better utilization of medical resources, to expand hospital effectiveness, and to improve patient satisfaction in Chinese tertiary hospitals.