Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Accepted for/Published in: JMIR Formative Research

Date Submitted: Jan 14, 2022
Date Accepted: Jun 24, 2022

The final, peer-reviewed published version of this preprint can be found here:

Qualitative Evaluation of an Artificial Intelligence–Based Clinical Decision Support System to Guide Rhythm Management of Atrial Fibrillation: Survey Study

Stacy J, Kim R, Barrett CD, Sekar B, Simon S, Banaei-Kashani F, Rosenberg MA

Qualitative Evaluation of an Artificial Intelligence–Based Clinical Decision Support System to Guide Rhythm Management of Atrial Fibrillation: Survey Study

JMIR Form Res 2022;6(8):e36443

DOI: 10.2196/36443

PMID: 35969422

PMCID: 9412903

Qualitative Evaluation of an Artificial Intelligence-based Clinical Decision Support System to Guide Rhythm Management of Atrial Fibrillation: A Survey Study

  • John Stacy; 
  • Rachel Kim; 
  • Christopher D Barrett; 
  • Balaviknesh Sekar; 
  • Steve Simon; 
  • Farnoush Banaei-Kashani; 
  • Michael A. Rosenberg

ABSTRACT

Background:

Despite the numerous studies evaluating various rhythm-control strategies for patients with atrial fibrillation, determination of the optimal strategy in a single patient is often based on trial-and-error, with no one-size-fits-all approach based on international guideline recommendations. The decision, therefore, remains an individualized one that lends itself well to the aid of a clinical decision support system, specifically one guided by artificial intelligence (AI).

Objective:

In this study, we performed qualitative evaluation of a novel, AI-based, clinical decision support system (CDSS) for rhythm management of AF called QRhythm, which uses a combination of supervised and reinforcement learning to recommend either a rate-control or one of three types of rhythm-control strategies—external cardioversion, antiarrhythmic medication, or ablation—based on individual patient characteristics.

Methods:

QRhythm utilizes a two-stage machine learning model to identify the optimal rhythm-management strategy in a given patient based on a set of clinical factors, in which the model first uses supervised learning to mimic an expert clinician, and then applies reinforcement learning to identify the best strategy to obtain the best clinical outcome—a composite of symptomatic recurrence, hospitalization, and stroke. 33 clinical providers, including cardiologists and internal medicine physicians, performed an assessment of QRhythm, followed by a survey to assess relative comfort with automated CDSS in rhythm-management, and examine areas for future development.

Results:

33 providers were surveyed with training levels ranging from resident to fellow to attending. Of the characteristics of the app surveyed, safety was most important to providers with an average importance rating of 4.7 out of 5 (SD 0.72). This priority was followed by a desire for the advice provided to make clinical sense with an importance of 4.5 out of 5 (SD 0.9), transparency in the population used to create the algorithm, 4.3 out of 5 (SD 0.65), and reasoning behind the decisions made, 4.3 out of 5 (SD0.88), and the ability to challenge the decisions made by the model, 3.85 out of 5 (SD 0.83). Providers that used the app ranked their highest concern with ongoing clinical use of the model as accuracy of recommendations followed by ineffectiveness of the application and patient data security. Trust in the app was varied. 16.7% of providers responded that they somewhat disagreed with the statement “I trust the recommendations provided by the QRhythm app,” 33.3% responded with neutrality to the statement, and 50% somewhat agreed with the statement.

Conclusions:

Safety of machine learning applications was the number one priority of the providers surveyed, and trust of such models remains varied. Widespread clinical acceptance of machine learning in health care is dependent on how much providers trust the algorithms. Building this trust involves ensuring transparency and interpretability of the model.


 Citation

Please cite as:

Stacy J, Kim R, Barrett CD, Sekar B, Simon S, Banaei-Kashani F, Rosenberg MA

Qualitative Evaluation of an Artificial Intelligence–Based Clinical Decision Support System to Guide Rhythm Management of Atrial Fibrillation: Survey Study

JMIR Form Res 2022;6(8):e36443

DOI: 10.2196/36443

PMID: 35969422

PMCID: 9412903

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.