JMIR Publications

ABSTRACT

Background Machine learning or deep learning algorithms for clinical diagnosis are inherently dependant on the availability of large-scale clinical datasets. Lack of such datasets and inherent problems such as over-fitting often necessitate the development of innovative solutions. Probabilistic modelling closely mimics the rationale behind clinical diagnosis and represents a unique solution. Objectives & Methods Numerical values of symptom-disease associations were utilised to mathematically represent medical domain knowledge. These values served as the core engine for the probabilistic model. For the given set of symptoms, the model was utilised to produce a ranked list of differential diagnoses. This list was compared to the differential diagnosis constructed by a physician in a consult. Practising medical specialists were integral in the development and validation of this model. Clinical vignettes were utilised to compare the accuracy of doctors and the model against the assumed gold standard. The accuracy analysis was carried out over the following metrics: Top 3 Accuracy, Precision and Recall. Results The model demonstrated a statistically significant improvement (P=.002) in diagnostic accuracy (85%) as compared to the doctor’s performance (66.5%). This advantage was retained across all the three categories of clinical vignettes, 100% vs 81.5% (P<.001) for highly specific disease presentation, 83% vs 64.5% for moderately specific disease presentation (P=.005) and 72% vs 48.8% (P<.001) for non-specific disease presentation. The model performed better than the doctors average in precision (62% vs 59.6%, P=.43) but trailed in recall (53% vs 55.6%, P=.27). Discussion & Conclusion The present research demonstrates a drastic improvement over previously reported results that can be attributed to the development of a stable probabilistic framework utilising symptom-disease associations to mathematically represent medical domain knowledge. The current iteration relies on static, manually curated values for calculating the degree of association. Shifting to real-world data-derived values represents the next evolutionary step.