Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Jan 11, 2023
Open Peer Review Period: Jan 11, 2023 - Mar 8, 2023
Date Accepted: Mar 30, 2023
(closed for review but you can still tweet)
Eliciting Insights from Chat Logs of the 25x5 Symposium to Reduce Documentation Burden: A Novel Application of Topic Modeling
ABSTRACT
Background:
Addressing clinician documentation burden through “targeted solutions” is a growing priority for many organizations including government, academia, and industry. Between January and February 2021, the 25 by 5: Symposium to Reduce Documentation Burden on U.S. Clinicians by 75% (25x5 Symposium) convened across six weekly, two-hour sessions among experts and stakeholders to generate actionable goals for reducing clinician documentation over the next five years. Throughout the virtual symposium, we passively collected—with their knowledge that the content would be deidentified and made publicly available—attendee contributions to a chat functionality. This presented a novel opportunity to synthesize and understand participants’ perceptions and interests from chat messages. We performed a content analysis of 25x5 Symposium chat logs to identify themes on reducing clinician documentation burden.
Objective:
The objective of this study was to explore unstructured chat log content from the virtual 25x5 Symposium to elicit latent insights on clinician documentation burden among clinicians, healthcare leaders, and other stakeholders using topic modeling.
Methods:
Across the six sessions, we captured 1,787 messages among 167 unique chat participants cumulatively; fourteen were private messages not included in the analysis. We implemented a latent Dirichlet allocation (LDA) topic model on the aggregated dataset to identify clinician documentation burden topics mentioned in the chat logs. Coherence scores and manual examination informed optimal model selection. Five domain experts independently and qualitatively assigned descriptive labels to model-identified topics and classified them into higher-level categories, which were finalized through a panel consensus.
Results:
We uncovered ten topics using the LDA model: (1) determining data and documentation needs (23.8%), (2) collectively reassessing documentation requirements in electronic health records (EHR) (14.2%), (3) focusing documentation on patient narrative (9.1%), (4) documentation that adds value (8.3%), (5) regulatory impact on clinician burden (8.0%), (6) improved EHR user interface and design (7.2%), (7) addressing poor usability (6.9%), (8) sharing 25x5 Symposium resources (6.9%), (9) capturing data related to clinician practice (6.4%), and (10) role of quality measures and technology on burnout (6.2%). Among these 10 topics, five high-level categories emerged: consensus building (46.3%), burden sources (20.6%), EHR design (14.1%), patient-centered care (9.1%), and symposium comments (6.9%).
Conclusions:
We conducted a topic modeling analysis on 25x5 Symposium multi-participant chat logs to explore the feasibility of this novel application and to elicit additional insights on clinician documentation burden among attendees. Based on the results of our LDA analysis, consensus building, burden sources, EHR design and patient-centered care may be important themes to consider when addressing clinician documentation burden. Our findings demonstrate the value of topic modeling in discovering topics associated with clinician documentation burden using unstructured textual content. Topic modeling may be a suitable approach to examine latent themes presented in virtual symposium chat logs.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.