JMIR Publications

ABSTRACT

Background:

Low diagnostic accuracy is a major concern in automated medical history-taking systems with differential diagnosis generators. Extending the concept of collective intelligence to the field of differential diagnosis generators such that the accuracy of judgment becomes higher when accepting an integrated diagnosis list from multiple persons than when accepting a diagnosis list from a single person may be a possible solution.

Objective:

To assess whether the combined use of several differential diagnosis (DDx) generators improves the diagnostic accuracy of DDx lists.

Methods:

We used medical history data and the top 10 DDx lists (index DDx lists) generated by an artificial intelligence (AI)-driven automated medical history-taking system from 103 patients with confirmed diagnoses. Two research physicians independently created other top 10 DDx lists (second and third DDx lists) per case by imputing key information into the other two DDx generators based on the medical history generated by the automated medical history-taking system, without reading the index lists generated by the automated medical history-taking system. We used the McNemar test to assess the improvement in diagnostic accuracy from the index DDx lists to the three types of combined DDx lists: (a) simply combining DDx from the index, second, and third lists; (b) creating a new top 10 DDx list using a 1/n weighting rule; and (c) creating new lists with only shared diagnoses among DDx lists from the index, second, and third lists. We treated the data generated by two research physicians from the same patient as independent cases. Therefore, the number of cases included in analyses in the case using two additional lists was 206 (103 cases × two physicians input).

Results:

The diagnostic accuracy of the index lists was 47/103 (45.6%). Diagnostic accuracy was improved by simply combining the other two DDx lists (133/206, 64.6%, P<.001), whereas the other two combined DDx lists did not improve the diagnostic accuracy of the DDx lists (106/206, 51.5%, P=.052 in the collective list with 1/n weighting rule; and 29/206, 14.1%, P<.001 in the only shared diagnoses among the three DDx lists).

Conclusions:

Simply adding each of the top 10 DDx from additional DDx generators increased the diagnostic accuracy of the DDx list by approximately 20%, suggesting that the combinational use of DDx generators early in the diagnostic process is beneficial.