JMIR Publications

ABSTRACT

Background:

Osteoporosis is a bone disease characterized by reduced bone mineral density and mass, which increases the risk of fragility fractures in patients. Artificial intelligence can mine imaging features specific to different bone densities, shapes, and structures, and fuse other multimodal features for synergistic diagnosis to improve prediction accuracy.

Objective:

The aim of this study is to develop a multimodal model that fuses chest X-rays and clinical parameters for opportunistic screening of osteoporosis, and to compare and analyze the experimental results with existing methods.

Methods:

We used multimodal data from a total of 1780 patients at Chongqing Daping Hospital from January 2019 to August 2024, including chest X-ray images and clinical data. We adopted a probability fusion strategy to construct a multimodal model. In our model, we used a convolutional neural network as the backbone network for image processing and fine-tuned it using a transfer learning technique to suit the specific task of this study. In addition, we introduced a gradient-based wavelet feature extraction method and combined it with an attention mechanism to assist in feature fusion, which enhanced the model's focus on key regions of the image and further improved its ability to extract image features.

Results:

The multimodal model proposed in this paper outperforms the existing methods in the four evaluation metrics of AUC value, accuracy, sensitivity, and specificity. Compared with using only the X-ray image model, the multimodal model improved the AUC value from 0.951 to 0.975, the accuracy from 89.32% to 92.36%, the sensitivity from 89.82% to 91.23%, and the specificity from 88.64% to 93.92% significantly.

Conclusions:

Multimodal model fusing chest X-ray images and clinical data outperforms unimodal models and existing methods.