Accepted for/Published in: JMIR Medical Informatics
Date Submitted: Jul 21, 2022
Date Accepted: Nov 13, 2022
(closed for review but you can still tweet)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Exploring Electronic Medical Record Diagnostic Text and Unstructured Text Data for Validation of a Pan-Canadian Definition for PTSD: Retrospective Observational Study.
ABSTRACT
Background:
The availability of electronic medical record (EMR) free-text data for research varies. However, access to short diagnostic text fields is more widely available.
Objective:
This study assessed agreement between free-text and short diagnostic text data from primary care EMR for identification of posttraumatic stress disorder (PTSD).
Methods:
This retrospective cross-sectional study used EMR data from a pan-Canadian repository representing 1,574 primary care providers, at 265 clinics using 11 EMR vendors. Medical record review using free-text and short diagnostic text fields of the EMR produced reference standards for PTSD. Agreement was assessed with sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy.
Results:
Our reference set contained 327 patients with free-text and short diagnostic text. Among these patients (n=327) agreement between free text and short-diagnostic text had an accuracy of 93.6% (CI 90.4-96.0%)). In a single Canadian province, case definition 1 and 4 had a sensitivity (82.6%, CI 74.4-89.0%) and specificity (99.5%, CI 97.4-100%). However, when the reference set was expanded to a pan-Canada reference (n=12,104 patients), Case definition 4 had the strongest agreement (sensitivity (91.1%, CI90.1-91.9), specificity (99.1%, CI 98.9-99.3)).
Conclusions:
Inclusion of free-text encounter notes during medical record review did not lead to improved capture of PTSD cases, nor did it lead to significant changes in case definition agreement. Within this pan-Canadian database, jurisdictional differences in diagnostic codes and EMR structure suggested the need to supplement diagnostic codes with natural language processing (NLP) to capture PTSD. When unavailable, short diagnostic text can supplement free-text data for reference set creation and case validation. Application of the PTSD case definition can inform PTSD prevalence and characteristics.
Citation
