Accepted for/Published in: JMIR Mental Health
Date Submitted: Aug 24, 2021
Date Accepted: Nov 9, 2021
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Can an App-Based Symptom Checker Accurately Detect Mental Disorders? A Comparative Study in Psychotherapy Outpatients
ABSTRACT
Background:
Digital technologies have become a common starting point for health-related information. Web- or app-based symptom checkers (SCs) aim to provide rapid and accurate self-diagnoses and triage advice but have not yet been investigated for mental disorders in routine health care settings.
Objective:
This study aimed to test the diagnostic performance of a widely available SC against the gold-standard of expert diagnoses with structured clinical interviews in the diagnosis of mental disorders.
Methods:
Adult patients from an outpatient psychotherapy clinic used the app-based symptom checker “Ada – check your health” (ADA) at intake. The percentage of agreement of the first (M1) and one of the first of five (M5) condition suggestions of ADA with at least one of the interview-based diagnoses were checked. Additionally, self-reported usability (assessed by the System Usability Scale [SUS]) and acceptance of ADA (assessed by an adapted feedback questionnaire) were evaluated.
Results:
A total of 49 patients (61.2% female; 33.41yrs, SD 12.79) were included in this study. The interview-based diagnoses matched with ADA’s first condition-suggestion (M1) in 46.9% [95% CI 32.5-61.7] and with one of the first of five condition-suggestions (M5) in 71.4% [56.7-83.4] of cases. For depressive disorders, M1 agreement was 61.3%, for somatoform disorders 22.2%, and for anxiety disorders 20.8%. The usability of ADA was rated high in the SUS (M = 81.51, SD 11.59, score range 0-100). Most patients (71.4%) would have preferred a face-to-face over the app-based diagnostic.
Conclusions:
Our results can be interpreted as a low to moderate overall diagnostic accuracy of a widely available SC when compared to interview-based expert diagnoses of mental disorders. Accuracies varied considerably between common diagnostic categories. While SCs have some potential to complement the diagnostic process as a screening tool, the diagnostic performance should be tested in larger samples and against further diagnostic instruments.
Citation
