JMIR Publications

ABSTRACT

Background:

The reporting of machine learning modeling prognostic and diagnostic studies is often inadequate, making it unnecessarily difficult to understand and replicate such studies. To address this, multiple consensus and expert reporting guidelines for such ML studies have been published. However, these guidelines cover different parts of the analytics life cycle and individually none of them provides a comprehensive set of reporting requirements.

Objective:

Consolidate the ML reporting guidelines in the literature for prognostic and diagnostic studies.

Methods:

A review and assessment of the literature on ML reporting guidelines identified 25 articles. These were consolidated by the authors into a single set of reporting items.

Results:

In total, 36 reporting items were identified. None of the source guidelines covered all 36 reporting items. Data and model evaluation guidelines were the most common in the source literature, with methodological guidance (i.e., data preparation descriptions of model training) had the least coverage in the literature.

Conclusions:

The items were converted into a checklist that can be used by authors, journal editors and reviewers to ensure consistency in reporting of ML modeling studies, and can also be used to score and monitor reporting quality. This is expected to improve understandability and reproducibility of work in this area.