Accepted for/Published in: JMIR Medical Informatics
Date Submitted: Dec 7, 2018
Open Peer Review Period: Dec 11, 2018 - Feb 5, 2019
Date Accepted: Sep 26, 2019
(closed for review but you can still tweet)
Measuring regional quality of care using unsolicited online data: creating more detailed insight using text analyses
ABSTRACT
Background:
Regional population health management (PM) initiatives require insight into experienced quality of care at the regional level. Unsolicited online provider ratings have shown potential for this use. This study explored the addition of comments accompanying unsolicited online ratings to regional analyses.
Objective:
The goal was to create additional insight for each PM initiatives as well as overall comparisons between these initiatives by attempting to determine the reasoning and rationale behind a rating.
Methods:
The Dutch Zorgkaart database provided the unsolicited ratings (period: 2008 – 2017) for analyses. All ratings included both quantitative ratings as well as qualitative text comments. Nine PM regions were used to aggregate ratings geographically. Sentiment analyses were performed by categorizing ratings into negative, neutral and positive ratings. Per category, as well as per PM initiative, word frequencies (unigrams and bigrams) were explored. Machine learning (naive Bayes) was applied to identify the most important predictors for rating overall sentiment and for identifying PM initiatives.
Results:
449,263 unsolicited ratings were available in the Zorgkaart database, 303,930 positive ratings, 97,739 neutral and 47,592 negative ratings. Bigrams illustrated that feeling like not being “taken serious” was the dominant bigram in negative ratings, while bigrams in positive ratings were mostly related to listening, explaining and perceived knowledge. Comparing bigrams between PM initiatives showed a lot of overlap, but several differences were identified. The naive Bayes machine learning was able to predict sentiments of comments, but unable to distinguish between specific PM initiatives.
Conclusions:
Adding information from text comments that accompany online ratings to regional evaluations provides insight for PM initiatives into the underlying reasoning. They provide useful overarching information for healthcare policy makers, but due to a lot of overlap it adds little region specific information. Specific outliers for some initiatives are insightful. Clinical Trial: The Medical Research Involving Human Subjects Act (WMO) does not apply to this study, and official approval was not required [1]. Participants agreed to the terms of service of Zorgkaart Nederland, which states that their submission can be used anonymously for research purposes [2].
Citation
Per the author's request the PDF is not available.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.