JMIR Publications

ABSTRACT

Background:

Although female firearm ownership and firearm suicide deaths have increased significantly in recent years, the circumstances preceding firearm suicide and prevention opportunities are under studied. The National Violent Death Reporting System (NVDRS) is a comprehensive source of data on violent deaths and includes unstructured incident narrative reports from coroners/medical examiners (CME) and law enforcement (LE). Conventional natural language processing (NLP) approaches have been used to identify common circumstances preceding female firearm suicide deaths but failed to identify rarer circumstances due to insufficient training data.

Objective:

To leverage a large language model approach to identify infrequent circumstances preceding female firearm suicide in the unstructured CME and LE narrative reports available in the National Violent Death Reporting System.

Methods:

We used the narrative reports of 1,462 female firearm suicide decedents in NVDRS from 2014-2018. We coded 9 infrequent circumstances preceding female firearm suicides. We experimented with predicting those circumstances by leveraging a large language model approach in a yes/no question-answer way.

Results:

Our large language model outperformed a conventional support vector machine (SVM) supervised machine learning approach by a wide margin. Compared to the SVM model, which had F1 scores below 0.2 for most infrequent circumstances, our large language model approach achieved an F1 score of over 0.6 for four circumstances and 0.8 for two circumstances.

Conclusions:

The use of a large language model approach shows promise. Researchers interested in using NLP to identify infrequent circumstances in narrative report data may benefit from large language models.