Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Jul 7, 2021
Date Accepted: Jan 30, 2022
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Triage Accuracy of Symptom Checker Apps: Five-Year Follow-Up Evaluation
ABSTRACT
Background:
Symptom checkers (SCs) are digital tools assisting laypersons in self-assessing the urgency and potential causes of their medical complaints. They are widely used but face concerns from both patients and healthcare professionals, especially regarding their accuracy. A 2015 landmark study substantiated these concerns using case vignettes to demonstrate that SCs commonly err in their triage assessment.
Objective:
Our study revisits the landmark index study to investigate whether and how SCs’ capabilities have evolved since 2015, and how they currently compare to laypersons stand-alone triage appraisal.
Methods:
In early 2020, we searched for smartphone and web-based applications providing triage advice. We evaluated these apps on the same 45 case vignettes as the index study. Using descriptive statistics, we compared our findings to the index study and to publicly available data on laypersons’ triage capability.
Results:
We retrieved 22 SCs providing triage advice. The median triage accuracy in 2020 (55.8%; IQR=15.1%) is close to that in 2015 (59.1%; IQR=15.5%). The apps in 2020 are less risk-averse (Odds=1.11:1, over-triage errors to under-triage errors) than in 2015 (Odds=2.82:1), missing more than 40% of emergencies. Few apps outperformed laypersons either in deciding whether emergency care was required or whether self-care was sufficient. No app outperformed the laypersons on both decisions.
Conclusions:
SCs’ triage performance has on average not improved over the course of five years. It decreased in two use cases (advice on when emergency care is required and when no healthcare is needed for the moment). However, triage capability varies widely within the sample of SCs. Whether it is beneficial to seek advice from SCs depends on the app chosen and on the specific question to be answered. Future research should develop resources (eg, case vignette repository) to audit the capabilities of SCs continuously and independently and provide guidance on when and to whom it should be recommended.
Citation
