JMIR Publications

ABSTRACT

Background:

Infant incubator adverse events refer to various harmful incidents occurring during normal use of marketed infant incubators that result in or may result in human injury. However, in recent years, the number of reported adverse events has been steadily increasing, with a large volume of unstructured data also flooding into monitoring systems. This has made the monitoring of infant incubator adverse events—a task characterized by strong interdisciplinary knowledge requirements and a high volume of unstructured data—time-consuming and labor-intensive when relying solely on manual processing. While general-purpose large language models (LLMs) boast logical rigor, strong relevance, and high generalizability, they still suffer from domain knowledge gaps and hallucination issues in specialized fields. Fine-tuning technology enables LLMs to adapt to specific application scenarios, while Retrieval-Augmented Generation (RAG) enhances models' capabilities for knowledge-intensive tasks. Therefore, LLMs combining these two technologies hold significant potential for addressing monitoring challenges.

Objective:

This study aims to develop an adverse event analysis model for infant incubators by integrating RAG with dual-adapter joint fine-tuning, thereby enhancing the intelligent monitoring of adverse events in medical devices.

Methods:

The study leveraged adverse event data from Chinese infant incubators to construct a high-quality dataset through prompt engineering. Technologically, it innovatively combined two parameter-efficient fine-tuning methods—LoRA and (IA)³—to achieve efficient adaptation on the Qwen2-7B base model. Simultaneously, it introduced the FINBGE embedding model with supervised contrastive semantic optimization to build a knowledge retrieval system that mitigates hallucination issues.

Results:

This study comprises 1,565 general pediatric disease question-answering corpora, 2,530 specific corpora on adverse events in infant incubators, and 1,488 regulatory corpora.Extensive experiments have demonstrated the superiority of this analytical model across various metrics. Compared to baseline models, it achieves improvements of 56.9% and 132.7% on text generation metrics such as BLEU-4 and ROUGE-L, respectively. The element recall rate reaches 0.815, while the accuracy rate in regulatory clause question-answering tasks attains 0.938.

Conclusions:

The analytical model proposed in this study demonstrates significant advantages in analyzing adverse events related to infant incubators, while also achieving substantial improvements in text generation metrics. When combined with Retrieval-Augmented Generation (RAG), it not only effectively mitigates hallucination issues but also enhances knowledge timeliness. Experiments confirm that this research successfully achieves intelligent analysis of high-risk medical device adverse event monitoring data through fine-tuning large language models and RAG technology, providing a new technical paradigm for medical device regulation.