JMIR Publications

ABSTRACT

Background:

User-Centered Design (UCD) is seen as a vital determinant of a healthcare informatics’ success. Yet, an increasing number of software companies work according to the Agile software development method, which is difficult to integrate with UCD practices. Log file analysis may provide opportunities for integrating UCD practices in the Agile process. However, research within healthcare information technology mostly has a theoretical approach and is often focused on the researcher’s interpretation of log file analyses. No studies have been reported on the Agile professionals’ interpretation of log file analyses, and therefore an opportunity exists for coupling these interpretations to concrete steps in the Agile development process.

Objective:

We propose a systematic approach to log file analysis (including pre-processing, analysis and various visualizations) in this study, and present these to developers to explore how they react and interpret them in the context of a real world healthcare information system, in an attempt to answer the following question: How may log file analyses contribute to increasing the match between the healthcare system and its users within the Agile development method according to Agile team members?

Methods:

This study consisted of two phases to answer the research question. In the first phase, log files were collected from a healthcare information system, and subsequently analyzed (summarizing sequential patterns, heatmapping, and clustering). In the second phase, the results of these analyses are presented to Agile professionals during a focus group interview. The interpretations of the Agile professionals are analyzed by open axial coding.

Results:

In the first phase, log file data of 17924 user sessions, and in total 176678 activities were collected. We found that the Patient Timeline is mainly visited, with 23707 (13.42%) visits in total. The page Change Conversation topic was least visited (n = 3; 0.0%). The main unique user session occurred in 5.99% of all user sessions, and consisted of ‘Insert Measurement Values for Patient’, ‘Patient Timeline’ followed by the page ‘Patient Settings’ and lastly ‘Patient Treatment Plan’. In the heatmap, we found that users often navigate to the pages ‘Insert Measurement Values’ and ‘Load Messages Collaborate’. Moreover, we found that there is a high probability that users repeatedly navigate from page ‘Address Book’ towards ‘Address Book’ again. Lastly, in the cluster analysis we found five clusters, namely the Information-seeking cluster (SS = 96.16), the Collaborative cluster (SS = 99.27), the Mixed cluster (SS = 193.40), the Administrative cluster (177.57), and the Patient-Oriented cluster (SS = 378.02). The total sum of squares within groups was 944.42 and the between sum of squares was 561.49. In the second phase, we found that the interpretations of these results by Agile professionals are related to stating hypotheses (n = 34), comparing paths (n = 31), benchmarking (n = 22), and prioritizing (n = 17).

Conclusions:

We found that analyzing log files provides Agile professionals valuable insights into users’ behavior. Therefore, we argue that log files analyses should be used within Agile development to inform professionals about users’ behavior. In this way, further UCD research can by informed by these results, making the methods less labor-intensive. Moreover, we argue that these translations to an approach for further UCD research will be carried out by UCD specialists, since they are able to infer which goals the user had when going through these paths when looking at the log data.