JMIR Publications

ABSTRACT

Background:

The proliferation of artificial intelligence (AI) models in health has been accompanied by a fundamental but largely unexamined assumption: that the availability of health data and computational capacity is, in itself, sufficient justification for training new models. This assumption has driven an exponential increase in publications reporting AI applications in clinical and public health domains, yet the proportion of these models that demonstrably improve patient outcomes, clinical decision-making, or health system efficiency remains remarkably low. The field lacks a systematic pre-training evaluation framework that distinguishes between what is technically trainable and what is scientifically and socially necessary.

Objective:

This paper proposes a methodological framework for determining when training an AI model in health is scientifically justified and when it is not. We argue that the decision to train should be treated as a consequential resource allocation decision, subject to the same standards of justification applied to any health intervention, rather than as a neutral technical exercise.

Methods:

We conducted a conceptual and integrative review of the current AI-in-health literature, examining prevailing training practices, outcome reporting standards, and the gap between predictive performance and clinical impact. Drawing on frameworks from health technology assessment, implementation science, and philosophy of science, we synthesized a set of pre-training criteria organized around four dimensions: scientific necessity, clinical relevance, economic justification, and social accountability.

Results:

We present the TRAIN-H (Training Rationale Assessment for AI in Health) framework, a structured decision tool comprising seven core questions that must be affirmatively answered before model development is justified. The framework formalizes the principle that prediction without consequence is not impact, and that accuracy without altered conduct, workflow, or cost does not constitute a valid reason to build a model. We identify six explicit conditions under which training should not proceed and discuss implications for research ethics, peer review, funding allocation, and graduate education.

Conclusions:

Training AI in health is an investment of computational, human, institutional, and public resources. Like any health intervention, it requires a clear hypothesis, a defined population benefit, and a measurable endpoint. The absence of these elements does not merely weaken a study—it eliminates the justification for the model’s existence. We call for the adoption of pre-training justification standards across research institutions, funding bodies, and editorial boards.