JMIR Publications

ABSTRACT

Background:

Artificial intelligence (AI) is increasingly used in literature retrieval, evidence synthesis, data analysis, interpretation, and manuscript development. Current guidance from journals and professional organizations emphasizes disclosure of AI use, transparency, and human accountability. However, disclosure alone may not reveal how AI-mediated processes influence evidence selection, reasoning pathways, or the scientific claims ultimately reported. Given growing concerns regarding fabricated citations, unsupported assertions, inconsistent terminology, and nontransparent evidence transformations, additional mechanisms are needed to ensure scientific trustworthiness.

Objective:

To propose TRAIL-AI (Transparent Reporting and Audit of AI-Mediated Literature, Evidence, and Inference), a reporting framework designed to improve transparency, traceability, and accountability for AI use that materially influences scientific evidence, analysis, interpretation, or conclusions.

Methods:

We reviewed emerging recommendations and reporting guidance addressing generative AI in scientific publishing and biomedical research and identified gaps related to scientific provenance and evidence traceability. Building on these observations, we developed TRAIL-AI as a conceptual framework focused on consequence-sensitive reporting of AI-mediated scientific activities. The framework consists of 13 reporting items organized into five domains: (1) task and threshold, (2) resources and retrieval, (3) AI-mediated actions, (4) inspection and verification, and (5) lineage and responsibility. TRAIL-AI introduces a proportional reporting model with three levels: Level 1 for expressive or mechanical assistance, Level 2 for evidence-influencing activities, and Level 3 for inference-influencing activities requiring claim-level provenance documentation. The framework additionally incorporates semantic provenance through the use of recognized clinical terminologies and emphasizes human verification and investigator accountability.

Results:

TRAIL-AI extends existing AI reporting approaches by linking AI-mediated operations to the evidence and claims they influence. The framework introduces a structured provenance statement for evidence-influencing use and a claim-level ledger for selected high-consequence scientific assertions. Key reporting elements include documentation of AI systems and accessible resources, retrieval and screening processes, affected evidence and claims, verification of source existence and claim support, management of evidentiary uncertainty, terminology normalization, preservation of audit records, and designation of accountable investigators. The framework prioritizes reconstructability, allowing reviewers and readers to inspect consequential AI-assisted scientific processes without requiring exact reproduction of nondeterministic model outputs.

Conclusions:

TRAIL-AI proposes a practical approach for strengthening transparency and accountability in AI-assisted scientific research. By moving beyond simple disclosure toward provenance-based reporting, the framework seeks to ensure that material scientific claims remain traceable to identifiable evidence, documented AI-mediated actions, standardized clinical concepts, and accountable human judgment. Future work should evaluate the framework through consensus development, pilot implementation, and prospective studies assessing validity, reliability, feasibility, and its impact on the identification of unsupported or inadequately documented scientific claims.