JMIR Publications

ABSTRACT

With the rapid development of artificial intelligence (AI), particularly large language models, there is growing interest in adopting AI approaches within academic medical centers (AMCs). However, the vast amounts of data required for AI and the sensitive nature of medical information pose significant challenges to developing high-performing models at individual institutions. Furthermore, recent changes in government funding priorities may result in the decentralization of biomedical data repositories that risk creating significant barriers to effective data sharing and robust model development. This has generated significant interest in federated learning (FL), which enables collaboratively model training without transferring data between institutions, thereby enhancing the protection of proprietary and sensitive information. Nonetheless, the complexity of FL introduces additional risks, as the distributed nature of training can expose new vulnerabilities related to both the data and the models themselves. These risks often present uncharted territory for AI governance, security, and privacy leadership. Using the common federated learning framework NVIDIA FLARE, we examine the inherent risks associated with platform-defined roles, privacy and security configurations, and identify key artifacts essential for a comprehensive risk-management approach. We introduce a risk framework for FL designed to assist AMC leaders in security, privacy, IT, and AI governance in effectively managing these emerging challenges. Finally, we propose platform-agnostic guidance for AI governance committees focused on model certification and evaluation.