JMIR Publications

ABSTRACT

Background:

Numerous medical articles regarding adverse drug events (ADEs) have been reported by pharmaceutical companies for drug safety information purposes. Although policies governing reporting to regulatory bodies vary among countries and regions, reporting may be precision- or recall-based. Recall-based reporting implemented in Japan requires to report any possible ADE and can introduce numerous false negatives and/or substantial amounts of noise, which is difficult to accomplish using limited human labor.

Objective:

We aim to develop an automated system that identifies adverse drug event-related medical articles, which can support recall-based reporting and alleviate human labor in Japanese pharmaceutical companies.

Methods:

Given medical articles, the proposed system based on natural language processing applies document-level classification which extracts articles containing ADEs for replacing human labor in first screening and sentence-level classification which extracts sentences within those articles that imply ADEs for supporting experts in second screening. We used 509 Japanese medical articles annotated by a medical engineer to evaluate the performance of the proposed system.

Results:

Document-level classification yielded F1 = 0.903 while the F1 score for sentence-level classification was 0.413. These were averages of five-fold cross-validations.

Conclusions:

A simple automated system may alleviate the human labor involved in screening drug safety-related medical articles in pharmaceutical companies. After improving the accuracy of the sentence-level classification by considering a wider context, we intend to apply this system towards real-world post-marketing surveillance.