Accepted for/Published in: Journal of Medical Internet Research
Date Submitted: Apr 25, 2023
Date Accepted: Jul 26, 2023
A comprehensive, valid, and reliable tool to assess the degree of responsibility of digital health solutions that operate with or without artificial intelligence: Three-phase mixed method study
ABSTRACT
Background:
Clinicians’ scope of responsibilities is steadily being transformed by digital health solutions that operate with or without artificial intelligence (hereafter called “D/AI” solutions). Most tools developed to foster ethical practices lack rigour and do not concurrently capture the health, social, economic, and environmental issues such solutions raise.
Objective:
To support clinical leadership in this field, we aimed to develop a comprehensive, valid, and reliable tool that measures the responsibility of D/AI solutions by adapting the multidimensional and already validated Responsible Innovation in Health Tool.
Methods:
We conducted a three-phase mixed method study. Relying on a scoping review of available tools, Phase 1 concept mapping led to a preliminary version of the ‘Responsible D/AI Solutions Assessment Tool.’ In Phase 2, an international two-round e-Delphi expert panel rated on a five-level scale the importance, clarity, and appropriateness of the Tool’s components. In Phase 3, two raters independently applied the revised Tool to a sample of D/AI solutions (n=25), interrater reliability was measured, and final minor changes were brought to the Tool.
Results:
The mapping process identified a comprehensive set of responsibility premises, screening criteria, and assessment attributes specific to D/AI solutions. e-Delphi experts critically assessed these new components, provided comments to increase content validity (n=293), and after Round 2 consensus was reached on 22 of the 26 items surveyed. Interrater agreement was “substantial” for a sub-criterion and “almost perfect” for all other criteria and assessment attributes.
Conclusions:
The Responsible D/AI Solutions Assessment Tool offers a comprehensive, valid, and reliable means to assess the degree of responsibility of D/AI health solutions. Because regulation remains limited, this forward-looking Tool has the potential to change practice towards more equitable as well as economically and environmentally sustainable digital health care. Clinical Trial: NA
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.