JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI) based diagnostic prediction models could aid primary care (PC) in decision making for faster and more accurate diagnoses. AI has the potential to transform electronic health records (EHR) data into valuable diagnostic prediction models. Different prediction models based on EHR have been developed. However, there are currently no systematic reviews that evaluate AI-based diagnostic prediction models for PC using EHR data.

Objective:

To provide an overview of diagnostic prediction models based on AI and EHR in primary care and to evaluate the content of each model, including risk of bias and applicability.

Methods:

This systematic review was performed according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. MEDLINE, EMBASE, Web of Science and Cochrane were searched. We included observational and intervention studies using AI and primary care EHRs and developing or testing a diagnostic prediction model for health conditions. Two independent reviewers used a standardised data extraction form. Risk of bias and applicability were assessed using PROBAST (Prediction model Risk Of Bias ASsessment Tool).

Results:

From 10,657 retrieved records, a total of 15 papers were selected. Most EHR papers focused on one chronic healthcare condition (n=11). From the 15 papers, 13 described a study that developed a diagnostic prediction model and 2 described a study that externally validated and tested the model in a primary care setting. Studies used a variety of AI techniques. The predictors used to develop the model were all registered in the EHR. We found no papers with a low risk of bias, high risk of bias was found in 9 papers. Biases covered an unjustified small sample size (n=5), not excluding predictors from the outcome definition (n=2) and the inappropriate evaluation of the performance measures (n=2). Unclear risk of bias was found in 6 papers, as no information was provided on the handling of missing data (n=10) and no results were reported from the multivariate analysis (n=9). Applicability was unclear in 10 papers, mainly due to lack of clarity in reporting the time interval between outcomes and predictors.

Conclusions:

Most AI-based diagnostic prediction models based on EHR data in primary care focused on one chronic condition. Only two papers tested the model in a primary care setting. The lack of sufficiently described methods led to a high risk of bias. Our findings highlight that the currently available diagnostic prediction models are not yet ready for clinical implementation in primary care.