JMIR Publications

Background:

Several models suggest how the qualities of a product or service influence user satisfaction. Models such as the Customer Satisfaction Index (CSI), Technology Acceptance Model (TAM), and Delone and McLean Information Systems Success demonstrate those relations and have been used in the context of health information systems.

Objective:

This study aimed to investigate which qualities foster greater satisfaction among patient and professional users. In addition, we are interested in knowing to what extent improvement in those qualities can explain user satisfaction and whether this makes user satisfaction a proxy indicator of those qualities.

Methods:

The Unified eValuation using ONtology (UVON) method was used to construct an ontology of the required qualities for 7 electronic health (eHealth) apps being developed in the Future Internet Social and Technological Alignment Research (FI-STAR) project, a European Union (EU) project in electronic health (eHealth). The eHealth apps were deployed across 7 EU countries. The ontology included and unified the required qualities of those systems together with the aspects suggested by the Model for ASsessment of Telemedicine apps (MAST) evaluation framework. Moreover, 2 similar questionnaires for 87 patient users and 31 health professional users were elicited from the ontology. In the questionnaires, the user was asked if the system has improved the specified qualities and if the user was satisfied with the system. The results were analyzed using Kendall correlation coefficients matrices, incorporating the quality and satisfaction aspects. For the next step, 2 partial least squares structural equation modeling (PLS-SEM) path models were developed using the quality and satisfaction measure variables and the latent construct variables that were suggested by the UVON method.

Results:

Most of the quality aspects grouped by the UVON method are highly correlated. Strong correlations in each group suggest that the grouped qualities can be measures that reflect a latent quality construct. The PLS-SEM path analysis for the patients reveals that the effectiveness, safety, and efficiency of treatment provided by the system are the most influential qualities in achieving and predicting user satisfaction. For the professional users, effectiveness and affordability are the most influential. The parameters of the PLS-SEM that are calculated allow for the measurement of a user satisfaction index similar to CSI for similar health information systems.

Conclusions:

For both patients and professionals, the effectiveness of systems highly contributes to their satisfaction. Patients care about improvements in safety and efficiency, whereas professionals care about improvements in the affordability of treatments with health information systems. User satisfaction is reflected more in the users’ evaluation of system output and fulfillment of expectations but slightly less in how far the system is from ideal. Investigating satisfaction scores can be a simple and fast way to infer if the system has improved the abovementioned qualities in treatment and care.