Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.
Who will be affected?
Readers: No access to all 28 journals. We recommend accessing our articles via PubMed Central
Authors: No access to the submission form or your user account.
Reviewers: No access to your user account. Please download manuscripts you are reviewing for offline reading before Wednesday, July 01, 2020 at 7:00 PM.
Editors: No access to your user account to assign reviewers or make decisions.
Copyeditors: No access to user account. Please download manuscripts you are copyediting before Wednesday, July 01, 2020 at 7:00 PM.
Diagnostic Accuracy of a Mobile AI-based Symptom Checker and a Web-based Self-Referral Tool in Rheumatology: Results from a Multicenter Randomized Controlled Trial
Johannes Knitza;
Koray Tascilar;
Franziska Fuchs;
Jacob Mohn;
Sebastian Kuhn;
Daniela Bohr;
Felix Muehlensiepen;
Christina Bergmann;
Hannah Labinsky;
Harriet Morf;
Elizabeth Araujo;
Matthias Englbrecht;
Wolfgang Vorbrüggen;
Cay Benedikt von der Decken;
Stefan Kleinert;
Andreas Ramming;
Jörg HW Distler;
Peter Bartz-Bazzanella;
Nicolas Vuillerme;
Georg Schett;
Martin Welcker;
Axel Hueber
ABSTRACT
Background:
Unspecific symptoms and the lack of rheumatologists delay the diagnosis of inflammatory rheumatic diseases (IRD). Digital diagnostic decision support systems (DDSS) promise to accelerate diagnosis and guide patients more effectively through the healthcare system.
Objective:
The aim of this study was to assess the diagnostic accuracy of a mobile artificial intelligence (AI)-based symptom checker (Ada) and a web-based self-referral tool (Rheport) regarding IRD.
Methods:
In a prospective, multicenter open-label controlled randomized crossover trial patients newly presenting to three rheumatology centers were randomly assigned to complete a symptom assessment with Ada or Rheport. The primary outcome was correct identification of a patient with IRD by the DDSS, defined as the presence of any IRD in the list of suggested diagnoses with Ada or a pre-specified threshold score with Rheport. Rheumatologists’ diagnosis was the gold standard.
Results:
600 patients were included, among whom 214 (36%) patients were diagnosed with an IRD. Rheport showed a sensitivity of 62% and specificity of 47% for IRD. Ada’s top 1 (D1) and top 5 disease suggestions (D5) showed a sensitivity of 52% and 66% and a specificity of 68% and 54% concerning IRD, respectively.
Conclusions:
To our knowledge, this is the largest comparative DDSS trial with actual use of DDSS by patients. The diagnostic accuracy of both DDSS for IRD was not promising in this high prevalence patient population. DDSS may lead to a misuse of scarce healthcare resources and our results indicate that strict regulation and drastic improvement is necessary to ensure DDSS safety and effectiveness. Clinical Trial: German Register of Clinical Trials (DRKS): DRKS00017642
Citation
Please cite as:
Knitza J, Tascilar K, Fuchs F, Mohn J, Kuhn S, Bohr D, Muehlensiepen F, Bergmann C, Labinsky H, Morf H, Araujo E, Englbrecht M, Vorbrüggen W, von der Decken CB, Kleinert S, Ramming A, Distler JH, Bartz-Bazzanella P, Vuillerme N, Schett G, Welcker M, Hueber A
Diagnostic Accuracy of a Mobile AI-Based Symptom Checker and a Web-Based Self-Referral Tool in Rheumatology: Multicenter Randomized Controlled Trial