JMIR Publications

ABSTRACT

Background:

Prior suicide attempts are a strong risk factor for future attempts. There is growing interest in using longitudinal electronic health record (EHR) data to derive risk prediction models for suicidal behavior. However, model performance may be inflated by an unrecognized form of “data leakage” during model training: codes for suicide attempt outcomes may refer to prior attempts also included as predictors.

Objective:

We aimed to develop an automated rule for determining when documented suicide attempt codes identify distinct events.

Methods:

From a large healthcare system’s EHR, patients with at least two suicide attempt codes were randomly sampled (N = 369). Chart reviewers assigned clinical setting, attempt method, and inter-code interval to each pair of suicide attempt codes. The probability that the second code in a pair referred to a distinct attempt from its preceding code was calculated by setting, method, and inter-code interval.

Results:

Of the 1,253 code pairs reviewed, 78% were non-independent (i.e., two codes referred to the same event). When the second code in a pair was assigned in a setting other than the emergency department (ED), it represented a distinct event less than 15% of the time. Code pairs in which the second code was assigned in the ED at least 5 days after its preceding code had a positive predictive value (PPV) of 0.90.

Conclusions:

Suicide attempt codes documented in an ED setting at least 5 days after a preceding code can be confidently treated as new events in EHR-based suicide risk prediction models. Clinical Trial: NA