JMIR Publications

ABSTRACT

Background:

Electronic health record (EHR) data are increasingly used in predictive models of PTSD, but it is unknown how multivariable prediction of an EHR-based diagnosis might differ from prediction of a more rigorous diagnostic criterion.

Objective:

To compare predictive models using the same predictors to predict an EHR-based versus semi-structured interview-based PTSD diagnostic criterion.

Methods:

To examine this question, we compared performance of several machine learning models predicting EHR-based PTSD diagnosis to models predicting semi-structured interview-based diagnosis (SCID-5) in a nationwide sample of 1,343 U.S. veterans who completed SCID-5 interviews and had clinic visit data extracted from the Veterans Affairs (VA) EHR. All models used a nearly identical set of predictors.

Results:

Models predicting EHR-based PTSD performed very well (AUCs .85-.9, MCCs .58-.69), whereas those predicting interview-based PTSD performed only moderately well overall (AUCs .71-.76, MCCs .24-.28). Sensitivity analyses showed that participants’ frequency of VA visits played a role in these differences.

Conclusions:

Results suggest that predictive models of PTSD built using only EHR data are biased relative to models predicting diagnosis from more a rigorous structured clinical interview. Researchers building clinical prediction models should not assume that diagnosis in the EHR is a sufficient proxy for the true criterion of interest.