JMIR Publications

ABSTRACT

Background:

Traditional healthcare models have evolved to increasingly recognize patients perspectives as key to improving the quality of care, especially in oncology. In such, Patient-Centered Measures (PCMs), including Patient-Reported Outcome Measures (PROMs) and Patient-Reported Experience Measures (PREMs), can enhance Patient-Healthcare Provider (HCP) communication while facilitating individualized care. This tailored approach not only improves patient outcomes but also underscores the importance of research methodologies that actively account for variability in patient experiences, especially across different sociodemographic and clinical backgrounds. However, identifying PCMs in clinical research has traditionally been time-consuming.

Objective:

We hypothesized that Artificial intelligence (AI) could automate and accelerate this identification. We conducted a study that aimed to estimate the proportion of clinical studies that included PROMs/PREMs, using either a traditional expert-based identification method or an AI-enriched approach.

Methods:

In a retrospective cross-sectional study using the ClinicalTrial.gov database, we focused on oncology studies between 2012 and 2022. Two methods were assessed for identifying PROMs/PREMs: i) a traditional expert-based method, where an algorithm identified PROMs/PREMs from a list of 346 oncology-specific PROMs/PREMs (extracted from the PROQOLID™ database, Mapi Research Trust) and/or 11 PROMs/PREMs-specific terms; and ii) an AI-enriched method using a Bidirectional Encoder Representations from Transformers (BERT) model, trained on 2,399 outcomes labeled by experts. To evaluate the performance of the algorithms, we compared the results returned by each method to the gold standard (expert decision). Each study was classified as reporting using (or not) at least one PROM/PREM and was described accordingly (logistic regression). To better identify which PROMs/PREMs were the most frequently used in clinical research, a Named Entity Recognition (NER) model was then used.

Results:

A total of 24,491 studies were included. According to the traditional expert-based algorithm, 7,549 studies (31%) used at least one PROM/PREM, as compared to 8,029 studies (33%) identified by the AI-enriched algorithm, increasing from 2012 to 2022 (Chi-squared test, p<0.001). With 90% accuracy, the AI-enriched algorithm outperformed (Chi-squared test, p<0.001) the traditional algorithm (83%) in identifying PROMs/PREMs. Breast and digestive cancers accounted for nearly 50% of all oncology studies using PROMs/PREMs, with the EORTC QLQ-C30 being the most frequently used. As expected, phases 2 to 4 trials more frequently included PROMs/PREMs than preclinical or early-phase 1 studies (OR[95% CI]=1.8[1.1-2.8] for phase 2; 3.6[2.3-5.8] for phase 3; 2.6,[1.6-4.4] for phase 4). In observational studies, cross-sectional and prospective studies incorporated PROMs/PREMs three times more frequently than retrospective studies (OR[95% CI]=4.6[3.3–6.4] and 3.2[2.5–4.1], respectively).

Conclusions:

Both approaches relied on human expertise; however, the one that involved training AI has proven to be more effective than the traditional method in identifying PROMs/PREMs in clinical studies. Future studies should explore the applicability of this approach to a broader field of pathologies and assess its applicability to other additional open-access databases. Clinical Trial: NA