JMIR Publications

ABSTRACT

Background:

Online crowdfunding has become a popular method to raise money for medical expenses. However, crowdfunding data are largely composed on unstructured text, and categorizing crowdfunding campaigns into clinically meaningful groups is challenging due to vague and misspelled medical terminology. Previous studies have employed methods that have either failed to address these challenges or that are poorly scalable to large sample sizes. To enable further research on this emerging funding mechanism in healthcare, better methods are needed.

Objective:

We sought to validate an algorithm to identify 11 disease categories in online medical crowdfunding campaigns to facilitate further work in this field.

Methods:

Web scraping was used to collect medical crowdfunding campaigns from GoFundMe. Using pre-trained named entity recognition (NER) and entity resolution (ER) models from Spark NLP for Healthcare in combination with targeted keyword searches, we constructed an algorithm to identify conditions in the campaign description, translate conditions to ICD-10-CM codes, and predict the presence or absence of 11 disease categories in the campaign. The classification performance of the algorithm was evaluated against 200 manually annotated campaigns.

Results:

We collected 87,449 crowdfunding campaigns through web scraping. Inter-rater reliability for detecting the presence of broad disease categories in the campaign description was high (Cohen’s kappa, range 0.69-0.98). The NER and ER models identified 6,555 unique (272,722 total) ICD-10-CM codes among all crowdfunding campaigns in our sample. Word search identified 1,255 additional campaigns for which a medical condition was not otherwise detected with the NER model. When averaged across all disease categories and weighted by the number of true positives for each category in the reference set, the algorithm demonstrated a precision of 0.83 (range 0.50-0.95), recall of 0.78 (range 0.43-0.97), F1 of 0.79 (range 0.46-0.96), and accuracy of 95% (range 90%-98%).

Conclusions:

A disease identification algorithm combining pre-trained natural language processing models and an ICD-10-CM code hierarchy was able to detect 11 disease categories in medical crowdfunding campaigns with high precision and accuracy.