JMIR Publications

ABSTRACT

Background:

Medical information extraction requires automatically identifying disease names and their associated terms in text. This task, known as Named Entity Recognition (NER), relies on expert-annotated data that are costly to produce and only available in limited quantities. The method of data augmentation (DA) aims to increase available training data, however standard techniques, including synonym substitution and back-translation, sometimes use inappropriate substitutions and do not maintain proper entity-label alignments, which are essential for sequence labeling tasks. Although current Large language models (LLMs) can generate text effectively, they often struggle to maintain factual accuracy.

Objective:

The research investigated how persona-driven, document-level DA using LLM can enhance disease NER performance under low-resources scenarios by rephrasing medical documents into diverse synthetic versions that introduce linguistic bias without altering meaning or entity factuality.

Methods:

We designed a DA framework using multiple personas that vary in medical expertise, personality, tone, and narrative style. Each persona rephrased documents from a medical disease corpus while preserving annotated entity spans through XML-constrained prompting. We measured semantic fidelity and lexical diversity to categorize personas into high-, balanced-, and low-fidelity groups. Models based on bert-base-cased were fine-tuned under four conditions: using (1) gold-standard (GS) data only, (2) single-persona augmentation, (3) persona subsets, and (4) all-persona augmentation. Performance were evaluated by micro-averaged F1-scores, with additional breakdowns by entity type (Disease, RareDisease, Sign, and Symptom).

Results:

Combining GS data with persona-generated texts substantially improved performance. The curated Top-3 mix (P2, P7, P8) raised F1-scores from 68.13 to 71.18, while augmentation with all ten personas achieved 72.29. Entity-level scores improved across categories, particularly Symptoms (from 59.64 to 67.26). Under low-resource settings (60% of GS), persona-augmented models surpassed the baseline trained on full GS data. Error analysis showed reduced confusion among closely related entities.

Conclusions:

Persona-driven augmentation enhances biomedical NER by improving accuracy, reducing entity-type confusion, and requiring less GS data. By preserving span integrity while introducing lexical diversity, the method offers a scalable, robust approach for low-resource biomedical text settings.