JMIR Publications

ABSTRACT

Background:

Lyme disease (LD) is the most common vector-borne disease in the United States, but is difficult to diagnose, especially since it mimics numerous other conditions and testing protocols may not be sufficient. While the Centers for Disease Control and Prevention (CDC) recommend a two-tiered serologic testing approach for LD diagnosis, many patients are instead diagnosed clinically based on criteria such as the Erythema Migrans (EM) rash and/or, particularly when the rash is not present, various symptom patterns, exposure history, and other information and clinician observations. Against this backdrop, online symptom checkers, employing algorithmic and natural language processing techniques, are increasingly used for diagnostic information and resources. With LD as a use case, this study employs a modified capabilities approach to examine the effectiveness and accuracy of artificial intelligence (AI) based tools in application and comparison to serologically (CDC+) and clinically based diagnoses.

Objective:

The overarching goal of this research was to explore whether AI-assisted diagnostic tools can complement more traditional medical assessment approaches in detecting complex infectious diseases. To assess the diagnostic accuracy of online symptom checker platforms with LD as a use case, this study (1) evaluated platform performance in identifying LD across different diagnostic cohorts; (2) compared symptom patterns and severity distributions among online LD diagnoses; and (3) identified the most frequently co-occurring conditions misclassified as LD or vice versa.

Methods:

Data was drawn from a structured survey of patients with confirmed or probable LD, including diagnostic pathway (CDC + and/or clinical), symptom profiles and severity, treatment history, and time to diagnosis. These patient cases were then entered into three leading AI-based symptom checker platforms — MediFind, Isabel, and WebMD — to examine diagnostic performance. Descriptive analytics, logistic regressions, and post-estimation analyses were used to identify patterns of diagnostic accuracy and interaction effects among cohort type, symptom severity, and AI-based platform.

Results:

Diagnostic accuracy (DA) varied significantly across platforms and symptom severity thresholds. DA improved at higher symptom-severity thresholds (≥ 3; p = 0.0001) and was slightly higher among clinically diagnosed cohorts compared to CDC+ patients (p = 0.208). Marginal analyses revealed that clinically diagnosed respondents were more sensitive to changes in severity, but with different levels of platform consistency across conditions.

Conclusions:

Findings suggest that AI-based symptom checkers may supplement early diagnostic reasoning in complex conditions like LD, particularly when symptom severity is high. However, inconsistency across platforms and diagnostic categories highlights the need for algorithmic refinement and standardized validation frameworks to ensure diagnostic reliability in AI-based tools.