JMIR Publications

ABSTRACT

Background:

The rapid deployment of DeepSeek, an open-source large language model has sparked concerns of its impact on patient outcomes and safety. However, little is known about how DeepSeek is used and regulated in these facilities.

Objective:

This study aimed to 1) systematically review the characteristics of deployed DeepSeek in the top 100 hospitals in China; and 2) compare performances and risks from hospital disclosure with research evidence.

Methods:

We performed a scoping review of gray and white literature, collecting data from the top 100 Chinese hospitals. We extracted basic characteristics of DeepSeek, its aim, evaluation approach, performance, risk and hospital regulation. A coding framework was developedcovering LLMs application scenario, evaluation dimension and source of risk.

Results:

We identified a total of 58 DeepSeek models in 48 out of the top 100 Chinese hospitals as well as 27 studies. We observed deployed DeepSeek mainly intended to assist clinical decision making, such as patient diagnosis and treatment recommendation. However, only 36.2% hospital-deployed models clearly indicated a pre-deployment assessment, 22.4% presented assessment results, and 8.6% identified potential risks and countermeasures. We found poor transparency in hospital reporting, with none presenting evaluation details. Hospitals were likely to report DeepSeek’s higher performance and fewer risks.

Conclusions:

The irresponsible deployment of DeepSeek in Chinese leading hospitals poses potential risks to patient outcomes and safety. We highlight the urgent need that existing regulations should be expanded to the downstream developers and users and hospitals need to perform a more rigorous validation and transparent reporting.