JMIR Publications

Background:

Data linkage in pharmacoepidemiological research is commonly employed to ascertain exposure and outcomes, or to obtain more information about confounding variables. However, to protect patient confidentiality usually unique patient identifiers are not provided; thus, makes data linkage between various sources challenging. The Saudi Real-Evidence Researches Network (RERN) aggregates Electronic Health Records from various hospitals, which may require a robust linkage technique.

Objective:

To evaluate and compare the performance of deterministic, probabilistic, and machine learning approaches for linking de-identified multiple sclerosis (MS) patient data from the RERN and Ministry of National Guard Health Affairs (MNGHA) EHR systems.

Methods:

We applied a simulation-based validation framework before linking real-world data sources. Deterministic linkage was based on predefined rules, while probabilistic linkage was based on a similarity-score matching. We applied both similarity-score and classification approach in machine learning¬¬¬¬— models including neural networks (NN), logistic regression (LR), and random forest (RF). Performance of each approach was assessed using confusion matrix focusing on sensitivity, positive predictive value (PPV), F1-score, and computational efficiency.

Results:

The study included linked data of 2,247 MS patients (spanning from 2016 to 2023). The deterministic approach resulted in an average F1-score of 97.2% in the simulation and demonstrated varying match rates in real-work linkage: 1,046 out of 2,247 (46.6%) to 1,946 out of 2,247 (86.6%). This linkage was computationally efficient with a run time of <1 second per rule. Using a probabilistic approach, provided an average F1-score of 93.9% in the simulation, with real-world match rates ranging from 1,472 out of 2,247 (65.5%) to 2,144 out of 2,247 (95.4%), and processing times ranged from ~0.1 second to ~5 second per rule. Although that machine learning approaches achieved high performance (F1-score reached 99.8%), they were computationally expensive. Processing time ranged from approximately 13 to 16,936 seconds for the classification approach and from approximately 13 to 7,467 seconds for the similarity-score approach. Real-world match rates from the machine learning models were highly variable depending on the method used; the similarity-score approach identified 789 out of 2,247 (35.1%) matched pairs, whereas the classification approach identified 2,014 out of 2,247 (89.6%).

Conclusions:

Probabilistic linkage offers high linkage capacity by recovering matches missed by deterministic methods, proving to be both flexible and efficient method, especially in real-world scenarios where unique identifiers are lacking. Probabilistic linkage achieved a great balance between recall and precision, enabling better integration of various data sources that could be useful in MS research.