Accepted for/Published in: JMIR Research Protocols
Date Submitted: Jun 13, 2021
Date Accepted: Dec 31, 2021
Reporting of model performance and statistical methods in studies using machine learning to develop clinical prediction models: a systematic review protocol
ABSTRACT
Background:
With growing excitement of the potential benefits of using machine learning and artificial intelligence in medicine, there has been a recent increase in the number of published clinical prediction models using these approaches. Unfortunately, there are many examples that suggest the reporting of machine learning-specific aspects within these studies is poor. Further, there are no reviews assessing reporting quality or broadly accepted reporting guidelines for these aspects.
Objective:
To assess the reporting quality of machine learning-specific aspects in studies using machine learning to develop clinical prediction models.
Methods:
We will include studies that use a supervised machine learning algorithm to develop a prediction model for use in clinical practice (i.e., diagnosis, prognosis, or identification of candidates for health care interventions). We will search the MEDLINE database for studies published in 2019, pseudo-randomly sort the records, and screen until we obtain 100 studies meeting our inclusion criteria. We will assess reporting quality with a novel checklist being developed in parallel with this review using existing reporting guidelines, textbooks, and consultations with experts. The checklist will cover four key areas where the reporting of machine learning studies is unique: modelling steps (order and data used for each step), model performance (e.g., reporting the performance of each model compared), statistical methods (e.g., describing the tuning approach), and presentation of models (e.g., specifying which predictors contributed to the final model).
Results:
We recently completed data analysis and manuscript writing will commence shortly with an expected submission of results to a peer-reviewed journal in early 2022.
Conclusions:
This review will contribute to more standardized and complete reporting in this field through the identification of areas where reporting is poor and can be improved. Clinical Trial: PROSPERO registration number: CRD42020206167
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.