Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Sep 12, 2023
Open Peer Review Period: Sep 12, 2023 - Sep 28, 2023
Date Accepted: May 6, 2024
(closed for review but you can still tweet)
Evaluation of the Clinical Efficacy and Trust in Artificial Intelligence-Assisted Embryo Ranking: A Survey-Based Prospective Study
ABSTRACT
Background:
Current embryo assessment methods for in vitro fertilization (IVF) depend on subjective morphological assessments. Recently, artificial intelligence (AI) has emerged as a promising tool for embryo assessment; however, its clinical efficacy and trustworthiness remains unproven. Simulation studies may provide additional evidence, provided that they are meticulously designed to mitigate bias and variance.
Objective:
The primary objective of this study was to evaluate the benefits of an AI model for predicting clinical pregnancy through well-designed simulations. The secondary objective was to identify the characteristics of and potential bias in the subgroups of embryologists with varying degrees of experience.
Methods:
This simulation study involved a questionnaire based survey conducted on 61 embryologists with varying levels of experience from twelve IVF clinics. Inter- and intra-observer assessments and the accuracy of embryo selection from 360 day 5 embryos before and after AI guidance were analyzed for all embryologists and subgroups of senior and junior embryologists.
Results:
With AI guidance, the inter-observer agreement increased from 0.355 to 0.527 and from 0.440 to 0.524 for junior and senior embryologists, respectively, thus reaching similar levels of agreement. The overall accuracies of the embryologists only, embryologists with AI guidance, and AI only were 37.7%, 50%, and 65.5%, respectively. Without AI, the average accuracy of the junior group was 33.516 (37.2%), while that of the senior group was 35.967 (40.0%). With AI’s guidance, the junior group’s accuracy improved to 46.581 (51.8%), reaching a level similar to that of the senior embryologists, 44.833 (49.8%). The junior embryologists had a higher level of trust in the AI score.
Conclusions:
This study demonstrates the potential benefits of AI in selecting embryos with high chances of pregnancy, particularly for embryologists with less than or equal to 5 years of experience, possibly due to their trust in AI. Thus, using AI as an auxiliary tool in clinical practice has the potential to improve embryo assessment and increase the probability of a successful pregnancy.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.