JMIR Publications

ABSTRACT

Digital health tools, platforms and AI/ML-based clinical decision support systems are increasingly part of health delivery approaches, with an ever greater degree of system interaction. Critical to the successful deployment of these tools is their functional integration into existing clinical routines and workflows. This depends on system interoperability and on intuitive and safe user-interface design. The importance of minimizing emergent workflow stress through human factors research, and of purposeful design for integration cannot be overstated. Usability of tools in practice is as important as algorithm quality. Regulatory and health technology assessment frameworks recognised the importance of these factors to a certain extent, but their focus remains mainly on the individual product rather than on emergent system and workflow effects. Measurement of performance and user experience is so far performed in ad hoc non-standardised ways by individual actors using their own evaluation approaches. We propose that a standard framework for system level and holistic evaluation could be built into interacting digital systems to enable systematic and standardized system-wide, multi-product, post-market surveillance and technology assessment. Such a system could be made available to developers through regulatory/assessment bodies as an application programming interface and could be a requirement for digital tool certification, just as interoperability is. This would enable health systems and tool developers to collect system level data directly from real device use cases enabling the controlled and safe delivery of systematic quality assessment/improvement studies suitable for the complexity and interconnectedness of clinical workflows using developing digital health technologies.