This paper has been accepted and is currently in production.
It will appear shortly on 10.2196/103570
The final accepted version (not copyedited yet) is in this tab.
Usability Evaluation of a Redesigned Feasibility Query Tool in a Federated FHIR-Based Data Portal: A Mixed Methods Study
ABSTRACT
Background:
Access to distributed routine clinical data for research is challenged by fragmented systems, heterogeneous standards, and complex governance. The German Research Data Portal for Health (FDPG) hosts a feasibility tool enabling federated cohort queries against the Medical Informatics Initiative (MII) Core Data Set. In 2024, the tool’s interface was redesigned to support a structured, multi-step workflow and improved search capabilities over an expanding ontology.
Objective:
To evaluate whether the redesigned interface supports cohort authoring in a federated FHIR environment, reduces cognitive load, and scales with the increasing semantic complexity of the MII Core Data Set
Methods:
We conducted a remote mixed methods usability study with 15 participants from Data Integration Centers and related technical backgrounds. Participants completed think-aloud scenario tasks, a task-specific questionnaire, and the System Usability Scale (SUS). Think-aloud recordings were qualitatively analyzed, while questionnaire responses and SUS scores provided quantitative measures of perceived usability.
Results:
The mean SUS score was 72.5 (SD = 15.1), indicating above-average usability. Participants positively evaluated the visual design, search functionality, and support for defining inclusion and exclusion criteria. Think-aloud protocols identified seven usability issues, including two critical problems related to the discoverability of core actions and one major issue concerning visibility and navigation within the workflow. Participants reported improved efficiency after an initial familiarization phase. Suggested improvements focused on enhancing interaction feedback, workflow transparency, and support for understanding complex data structures.
Conclusions:
The redesigned workflow-oriented interface demonstrates good usability for technical users and effectively mitigates some complexity introduced by a large ontology. However, high-severity interaction issues and the study's limited participant diversity highlight the need for targeted design fixes and broader evaluations that include clinicians and real query execution with real world cohort scenarios.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.