JMIR Publications

ABSTRACT

Background:

Symptom checkers are tools developed to provide clinical decision support to laypersons. Apart from suggesting probable diagnoses they commonlyP< advise on what level of care the user should seek (“triage advice”). Symptom checkers gain increasing popularity despite prior studies rating their performance as mediocre. To date it is unclear, whether symptom checkers triage better than their potential users.

Objective:

Comparison of triage accuracy between symptom checkers and their potential users (ie laypersons).

Methods:

We recruited 91 adults living in the United States without a background in health care online on Amazon MTurk. In an online survey the participants evaluated 45 fictitious clinical case vignettes. Data for symptom checkers was obtained from a previous study. As main outcome measures we assessed (1) the accuracy of triage assessments of participants and symptom checkers, for each of three triage levels (ie emergency care, non-emergency care, self-care) and overall, (2) for every symptom checker the proportion of participants outperforming it in terms of accuracy, and (3) the risk-aversion of participants and symptom checkers by comparing the proportions of cases under- and over-triaged.

Results:

Mean overall triage accuracy was similar for participants (60.9%; 95% CI 59.5%-62.3%) and symptom checkers (58%). Participants outperformed all but five symptom checkers. On average, symptom checkers (80.6%) more reliably detected emergencies than laypersons (67.5%; 95% CI 64.1%-70.8%). Although both symptom checkers and participants struggled with cases requiring self-care (the least-urgent triage category), symptom checkers wrongly classified these cases as emergencies more often (25%) than laypersons (4%).

Conclusions:

Most symptom checkers have no greater triage capability than an average layperson. However, the best symptom checkers have a triage accuracy superior to the participants. Symptom checkers might improve early detection of emergencies but may also needlessly increase resource utilization in health care. Laypersons require support in deciding when to rely on self-care, but here symptom checkers perform the worst. Further research should investigate for whom and for under which circumstances symptom checkers are most and least helpful.