JMIR Publications

ABSTRACT

Background:

Despite the rapid growth of research in artificial intelligence/machine learning (AI/ML), little is known about the extent of disclosed results years after study completion.

Objective:

To estimate the proportion of result disclosures of AI/ML research through ClinicalTrials.gov or peer-reviewed publications indexed in PubMed or Scopus.

Methods:

Using data from the Clinical Trials Transformation Initiative Aggregate Analysis of ClinicalTrials.gov, we identified studies initiated and completed between January 2010 to December 2023 and contained AI/ML-specific terms in study descriptions, official title, brief summary, interventions, conditions, detailed descriptions, or keywords. For 842 completed studies, we searched PubMed and Scopus for publications containing study identifiers and AI/ML-specific terms in the title, abstract, and keywords. We calculated disclosure rates within 3 years of study completion and median times to disclosure—from the ‘primary completion date’ to the ‘results first posted date’ on ClinicalTrial.gov or the earliest date of journal publication.

Results:

Of 842 completed studies (357 interventional; 485 observational), 5.5% (46/842) disclosed results on ClinicalTrials.gov, 13.9% (117/842) in journal publications, and 17.7% (149/842) through either route within 3 years of completion. Higher disclosure rates were observed for trials: 10.4% (37/357) on ClinicalTrials.gov, 19.3% (69/357) in journal publications, and 26.1% (93/357) through either route. Randomized controlled trials had even higher disclosure rates: 11.3% (23/203) on ClinicalTrials.gov, 24.6% (50/203) in journal publications, and 32.0% (65/203) through either route. Nevertheless, most study findings (82.3%; 693/842) remained undisclosed after 3 years of completion. Trials employing randomization (vs. nonrandomized) or masking (vs. open-label) had higher disclosure rates and shorter times to disclosure. Most trials (85%; 305/357) had sample sizes of ≤1000, yet larger trials (n>1000) had higher publication rates (30.8%; 16/52) than smaller trials (n≤1000) (17.4%; 53/305). Academia (15.1%; 39/259) and hospitals (12.4%; 42/340) published the most, followed by industry (13.7%; 20/146). High-income countries accounted for 82.4% (89/108) of published studies. Of studies with disclosed results, the median times to report through ClinicalTrials.gov and in journal publications were 505 days (IQR: 399-676) and 407 days (IQR: 257-674), respectively. Open-label trials accounted for 60% (214/357). Single-center studies were prevalent among trials (83.3%; 290/348) and observational studies (82.3%; 377/458).

Conclusions:

For over 80% of AI/ML studies completed during 2010–2023, study findings remained undisclosed even 3 years after study completion, raising concerns about the representativeness of data accessible on completed research. While methodological rigor was generally associated with higher publication rates, the predominance of single-center designs and high-income countries may limit the generalizability of the results currently accessible.