Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Accepted for/Published in: Journal of Medical Internet Research

Date Submitted: Feb 10, 2023
Date Accepted: Mar 2, 2024

The final, peer-reviewed published version of this preprint can be found here:

Evaluating Artificial Intelligence in Clinical Settings—Let Us Not Reinvent the Wheel

Cresswell K, De Keizer NF, Magrabi F, Williams R, Rigby M, Prgomet M, Kukhareva P, Wong ZSY, Scott P, Craven CK, Georgiou A, Medlock S, Brender J, Ammenwerth E

Evaluating Artificial Intelligence in Clinical Settings—Let Us Not Reinvent the Wheel

J Med Internet Res 2024;26:e46407

DOI: 10.2196/46407

PMID: 39110494

PMCID: 11339570

Evaluation frameworks for Artificial Intelligence in clinical settings: let's not reinvent the wheel

  • Kathrin Cresswell; 
  • Nicolette F De Keizer; 
  • Farah Magrabi; 
  • Robin Williams; 
  • Michael Rigby; 
  • Mirela Prgomet; 
  • Polina Kukhareva; 
  • Zoie Shui-Yee Wong; 
  • Philip Scott; 
  • Catherine K Craven; 
  • Andrew Georgiou; 
  • Stephanie Medlock; 
  • Jytte Brender; 
  • Elske Ammenwerth

ABSTRACT

Background:

There is an urgent need to incorporate theory-informed health information technology evaluation frameworks into existing and emerging guidelines for the evaluation of Artificial Intelligence (AI). Such frameworks can help developers, implementers, and strategic decision makers to build on existing experience and the existing empirical evidence base.

Objective:

To provide an overview of existing frameworks.

Methods:

We here provide concrete examples on how existing theory-informed health information technology evaluation frameworks may be used to inform the safe implementation of AI in healthcare settings.

Results:

AI-based evaluation frameworks need to consider technology, design, use by health and care professionals and organisations, and implementation strategy within and across organisations over time.

Conclusions:

There is an urgent need to refine and tailor existing theoretical approaches to AI-based health information technologies.


 Citation

Please cite as:

Cresswell K, De Keizer NF, Magrabi F, Williams R, Rigby M, Prgomet M, Kukhareva P, Wong ZSY, Scott P, Craven CK, Georgiou A, Medlock S, Brender J, Ammenwerth E

Evaluating Artificial Intelligence in Clinical Settings—Let Us Not Reinvent the Wheel

J Med Internet Res 2024;26:e46407

DOI: 10.2196/46407

PMID: 39110494

PMCID: 11339570

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.