JMIR Publications

ABSTRACT

Background:

Alarm fatigue, a multifactorial desensitization of staff to alarms, can harm both patients and healthcare staff in intensive care units (ICU), especially due to false and non-actionable alarms. Increasing amounts of routinely collected alarm and ICU patient data are paving the way for training machine learning (ML) models that may be able to reduce the number of non-actionable alarms and thus alarm fatigue. Today, however, there exists no publicly available dataset or process that routinely collects information on alarm actionability, whether an alarm triggered a medical intervention or not—a key feature for developing meaningful ML models for alarm management. Furthermore, case-based manual annotation is too slow and resource intensive for large amounts of data.

Objective:

Our objective was to propose a method to annotate patient monitoring alarms regarding their actionability; while the method is aimed to be used primarily in our institution, other clinicians, scientists and industry stakeholders should be able to reuse it to build their own datasets.

Methods:

The interdisciplinary research team followed a mixed methods approach to develop the annotation method, using data-driven, qualitative and empirical strategies. The iterative process consisted of six steps:1) defining alarm terms; 2) reaching consensus on an annotation concept and documentation structure; 3) defining physiological alarm conditions, related medical interventions and time windows to assess; 4) developing mapping tables; 5) creating the annotation rule set; and 6) evaluating the generated content. All decisions were made based on feasibility criteria, clinical relevance, frequency of occurrence, data availability and quantity, structure, and storage mode. The annotation guideline development process was preceded by analyses of the institution’s data and systems, of device manuals and a rapid literature review.

Results:

In a multidisciplinary, consensus-based approach, we defined pre-processing steps and a rule-based annotation method to classify alarms as either actionable or non-actionable based on data from the patient data management system. We present our experience in developing the annotation method and provide resources that we generated: The method focuses on respiratory and medication management interventions and include eight general rules in a tabular format that are accompanied by graphical examples. Mapping tables enable to handle unstructured information and are referenced in the annotation rule set.

Conclusions:

Our annotation method will enable large amounts of alarms to be labelled semi-automatically, retrospectively, and time-savingly, and provide information on their actionability based on further patient data. This will make it possible to generate annotated datasets for ML models in alarm fatigue research. We believe that our annotation method and the resources provided are universal enough and could be used by others to prepare data for future ML projects, even beyond the topic of alarms. Clinical Trial: Not applicable