JMIR Publications

ABSTRACT

Background:

There is urgent demand for efficient use of clinical datasets to detect self-injurious thoughts and behaviors (SITB) during childhood and adolescence. Scarce information is available on the accuracy of prevalent methods of detecting children who have received acute mental health treatment for suicide-related emergencies.

Objective:

To evaluate the performance of ICD-10-CM codes and chief complaints in identifying child emergency department visits for SITB, explore the variation in detection performance of ICD-10-CM codes and chief complaints by child sociodemographics and type of SITB (non-suicidal self-injury, suicidal ideation, preparatory acts, suicide attempts, and completed suicide), and compare detection performance with supervised machine learning classifiers.

Methods:

The study utilized electronic health record data from a large urban health system to establish a gold-standard classification of SITB through manual review of clinical notes from 600 mental health-related emergency department visits by children aged 10-17 years occurring between 2015-2019. The accuracy, sensitivity, and specificity of detection with ICD-10-CM encounter diagnoses and suicide-related chief complaint were compared with four machine learning classifiers incorporating a broad range of structured data elements and trained on the gold-standard classification.

Results:

Self-injurious thoughts and behaviors accounted for 284 (47.3%) visits. Compared with manual review, the sensitivity of each indicator was as follows: diagnostic codes alone 0.70, chief complaint alone 0.45, diagnostic codes and chief complaint 0.38 (both affirmed), diagnostic codes or chief complaint 0.77 (either affirmed). Sensitivity was significantly lower for boys compared with girls (0.69 [0.61-0.77] vs. 0.84 [0.78-0.90]) and preteens compared with adolescents (0.66 [0.54-0.78] vs. 0.86 [0.80-0.92]). Specificity was significantly lower for detecting preparatory acts (0.68 [0.64-0.72]) and attempts (0.67 [0.63-0.71]) compared with ideation (0.79 [0.75-0.82]). Four machine learning classifiers considering additional structured data elements significantly improved detection with area-under-the-curve 0.95-0.96 (vs. 0.88).

Conclusions:

Diagnostic codes and chief complaint under-detected children with suicide-related emergencies, particularly among males and preteens. Use of additional structured data elements strengthened detection sensitivity. Going forward, future developmentally informed research is needed to strengthen bridges between advances in informatics approaches to phenotyping and clinical child mental health research.