JMIR Publications

ABSTRACT

Background:

The study of doctor-patient-computer interactions is a key research area for examining doctor-patient relationships. However, studying these interactions is costly and obtrusive as researchers usually set up complex mechanisms or intrude into consultations, collect data, and manually analyze it.

Objective:

We aim to facilitate human-computer and human-human interaction research in clinics by providing a computational ethnography tool: an unobtrusive automatic classifier of basic doctor-patient-computer interactions.

Methods:

The classifier’s input are videos taken by doctors using their computers' internal camera and microphone. By estimating the key points of the doctor's face and the presence of voice activity, we estimate the type of interaction that is taking place. The output is a classification of video segments into four interaction classes: doctor-patient-computer, doctor-patient, doctor-computer, and Other interactions.

Results:

We evaluated the classifier using 30 minutes of video provided by five doctors simulating consultations in their clinics both in semi and fully inclusive layouts. The classifier achieved an overall accuracy of 0.83, a performance similar to a human coder. In addition, similarly to the human coder, the classifier had a better accuracy in a fully-inclusive layout in comparison to a semi-inclusive layout.

Conclusions:

The proposed classifier can be used by researchers, care providers, designers, medical educators, and others who are interested in exploring and answering questions related to doctor-patient-computer interactions during consultations.