Accepted for/Published in: JMIR Mental Health
Date Submitted: Jul 19, 2020
Date Accepted: Jul 27, 2021
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Automated thematic annotations: Systematic review of existing machine learning algorithms for small databases and implementation in Avatar Therapy.
ABSTRACT
Background:
Avatar therapy (AT) is a modern therapeutic alternative for schizophrenic patients with persistent auditory hallucinations. The effectiveness of psychotherapy is generally measured by semi-structured interviews or self-reported questionnaires. To complement, an increasing number of research teams have started to use qualitative analyses, but these approaches can be criticized because of inherent biases. Text mining is one of the few techniques used to derive data from the massive number of interactions obtained from therapy. However, in-person therapies can yield databases that are small. A machine learning algorithm applicable for small databases is therefore needed for such cases.
Objective:
The objective of this study is threefold: the first is to conduct a systematic review of the use of machine learning for automated text classification for small datasets in the fields of psychiatry, psychology and social sciences to determine the best algorithm, the second is to identify and implement this algorithm in our dataset and the third objective is to assess if this classification is comparable to the classification done by human evaluators.
Methods:
A systematic search was performed in the electronic databases of MedLine (PubMed), Web of Science, PsycNet (PsycINFO) and Google Scholar from their inception dates until 2020. Algorithms were implemented using the Python programming language and the Sk-learn libraries.
Results:
Seven articles were included in the analysis. From the identified articles and our performance pretest, the Linear Support Vector Classifier is suited to perform automated theme classifications on Avatar therapy transcripts with the use of limited datasets with the accuracy of 67.74% and substantial classification Scott’s Pi agreement of 0.647.
Conclusions:
Thus, our results are showing that it is possible to annotate automatically an un-annotated transcript basing ourselves solely on a small database. These results open the door to additional researches such as predicting the outcome of therapy and specifically AT.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.