JMIR Publications

ABSTRACT

Background:

Insufficient accrual is a major challenge in clinical trials and can result in underpowered studies, as well as exposing study participants to toxicity and additional costs, with limited scientific benefit.

Objective:

Evaluate whether generative models can be used to simulate additional virtual patients to compensate for insufficient accrual in clinical trials.

Methods:

We performed a retrospective analysis using ten datasets from nine fully accrued, completed and published breast cancer trials. For each trial we removed the latest recruited patients, trained a generative model on the remaining patients, and simulated virtual patients to replace the removed ones using the generative model to augment the available data. We then replicated the published analysis on this augmented dataset to determine if the findings are the same. Four different generative models were evaluated: sequential synthesis with decision trees, Bayesian network, generative adversarial network, and a variational autoencoder. These generative models were compared to sampling with replacement (bootstrap) as a simple alternative. Replication of the published analysis utilized four metrics: decision agreement, estimate agreement, standardized difference, and confidence interval overlap.

Results:

All approaches struggle when the trial result is marginal, indicating a general sensitivity to that particular scenario. Otherwise, sequential synthesis performed well on the replication metrics for the removal of up to 40% of the last recruited patients (decision agreement: 88% to 100% across datasets, estimate agreement 100%, cannot reject standardized difference null hypothesis: 89% to 100%, and CI overlap: 0.8 to 0.92), and the Bayesian network performed relatively well on the smallest datasets. There was no evidence of a monotonic relationship in the estimated effect size with recruitment order across these studies. This suggests that patients recruited earlier in a trial are not systematically different than those recruited later, at least partially explaining why generative models trained on early data can effectively simulate patients recruited later in a trial.

Conclusions:

For a study with poor accrual, sequential synthesis is relatively effective and can enable the simulation of the full dataset had the study continued accruing patients. For the smaller datasets, a Bayesian network should be used. These results demonstrate the potential for generative models to rescue poorly accruing clinical trials. Clinical Trial: NCT02861859; NCT02721433; NCT00066573; NCT00009945; NCT02428114; NCT02816164; NCT02632435; NCT00295646; NCT03664687; NCT00127205