JMIR Publications

ABSTRACT

Background:

After a patient consultation, a clinician determines the steps in the management of the patient. A clinician may for example request to see the patient again or refer them to a specialist. Whilst many clinicians will typically record their intent as ‘next steps’ in the patient’s clinical notes, in some cases the clinician may forget to indicate their intent as an order or request, e.g. failure to place the follow-up order. This consequently results in patients becoming ‘lost-to-follow up’ and may in some cases lead to adverse consequences.

Objective:

Train a machine learning model to detect a clinician’s intent to follow up with a patient from the patient’s clinical notes.

Methods:

Trained a machine learning model to detect clinical intent using a dataset of annotated clinical notes taken from the bariatric clinic from the University College London Hospitals (UCLH) NHS trust. A total of 3000 notes were annotated by three blinded annotators. This dataset was then used to train a natural language processing (NLP) multilabel classification model.

Results:

Annotators systematically identified 22 possible types of clinical intent and annotated 3000 Bariatric clinical notes. The annotation process revealed a class imbalance in the labeled data and we found that there was only sufficient labeled data to train 11 out of the 22 intents. We used the data to train a BERT based multilabel classification model and reported the following average accuracy metrics for all intents: macro-precision: 0.91, macro-recall: 0.90, macro-f1: 0.90.

Conclusions:

NLP models can be used to successfully detect clinical intent in clinical free text notes.