JMIR Publications

ABSTRACT

Synthetic data generation (SDG) holds promise for healthcare research but may produce limiting hallucinations. While commonly observed in text generation, this study investigated whether hallucinations also occur in tabular synthetic data, whether their frequency increases with training data complexity and whether they impact the utility of synthetic data for downstream prognostic AI/ML modeling. Using 6,354 training datasets of varying complexity, created by including different subsets of variables from 12 real-world healthcare datasets, synthetic data was generated with 7 SDG models. The hallucination rate (HR) was the proportion of hallucinations in a synthetic dataset and ranged from 0.3% to 100% (median of 99.1%, IQR [98.5, 100.0]). The HR increased with training data complexity but did not or not meaningfully affect AI/ML prognostic model performance. These findings suggest that hallucinations can be very common in synthetic tabular health data but do not necessarily impair its utility for prognostic modeling.