Accepted for/Published in: JMIR Medical Informatics
Date Submitted: Apr 17, 2020
Date Accepted: Nov 3, 2020
Development of a Human-Algorithm Integration System for Hip Fracture Detection on Plain Radiography – Synergy Between Human and Algorithm-based Detection.
ABSTRACT
Background:
Hip fractures are the leading fracture of elderly individuals. The application of human-based deep learning algorithm in plain pelvic radiographs (PXR) potentially improves the accuracy of hip fracture diagnosis.
Objective:
To develop and validate the feasibility and efficacy of human-algorithm integration (HAI) system to improve the diagnostic accuracy of hip fracture in real clinical environment.
Methods:
The HAI system was developed using a deep learning algorithm trained on trauma registry data and PXR. Thirty-four physicians were recruited to compare the diagnostic performance before HAI and after HAI system assistance using an independent testing dataset. The participant’s accuracy, sensitivity, specificity and the agreement with the algorithm were analysed. The subgroup analyses regarding specialty and experience were also performed. Furthermore, we applied the HAI system in the emergency departments of different hospitals to validate its accuracy and efficacy in the real world.
Results:
With the support of the algorithm with 91% accuracy, the diagnostic performance including sensitivity (pre-HAI, median 95% ; HAI, median 99%; p<0.05), specificity (pre-HAI, median 90%; HAI, median 95%; p<0.05), accuracy (pre-HAI, median 90%; HAI, median 96%; p<0.05) and human-algorithm agreement (pre-HAI κ, median 0.69 [IQR 0.63-0.74] ; HAI κ, median 0.80 [IQR 0.76-0.82]; p<0.05) were significantly improved in the independent testing dataset. The primary physician significantly improved the diagnostic performance comparable to consulting physicians using HAI, and both the experienced and less-experienced participants benefit from HAI. After the HAI system was applied in three departments for five months, a total of 587 images were used. The sensitivity of the HAI system was 97%, the specificity was 95.7%, and the accuracy was 96.08% for detecting hip fractures.
Conclusions:
HAI currently impacts healthcare, and integrating this technology into emergency departments is feasible. The human-algorithm collaboration system can enhance the diagnostic performance of hip fracture of physicians.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.