Accepted for/Published in: JMIR Medical Informatics
Date Submitted: Feb 8, 2021
Date Accepted: Mar 7, 2021
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Accuracy of an Artificial Intelligence System for Cancer Clinical Trial Eligibility Screening
ABSTRACT
Background:
Screening patients for eligibility for clinical trials is labor intensive. It requires abstraction of data elements from multiple components of the longitudinal health record and matching them to inclusion and exclusion criteria for each trial. Artificial intelligence (AI) systems have been developed to improve the efficiency and accuracy of this process.
Objective:
This study aims to evaluate the ability of an AI clinical decision-support system (CDSS) to identify eligible patients for a set of clinical trials.
Methods:
This study included the de-identified data from a cohort of breast cancer patients seen at the medical oncology clinic of academic medical center between May and July 2017 and assessed patient eligibility for four breast cancer clinical trials. CDSS eligibility screening performance was validated against manual screening. Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for eligibility determinations were calculated. Disagreements between manual screeners and the CDSS were examined to identify sources of discrepancies. Interrater reliability between manual reviewers was analyzed using Fleiss’ kappa, and significance of differences was determined by Wilcoxon signed-rank test.
Results:
Three hundred eighteen breast cancer patients were included. Interrater reliability for manual screening was 0.64, indicating substantial agreement. The overall accuracy of breast cancer trial-eligibility determinations by the CDSS was 87.6%. CDSS sensitivity was 81.1% and specificity was 89%.
Conclusions:
The AI CDSS in this study demonstrated accuracy, sensitivity, and specificity of greater than 80% in determining eligibility of patients for breast cancer clinical trials. CDSSs can accurately exclude ineligible patients for clinical trials and offers the potential to increase screening efficiency and accuracy. Additional research is needed to explore whether increased efficiency in screening and trial matching translates to improvements in trial enrollment, accruals, feasibility assessments, and cost.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.