JMIR Publications

ABSTRACT

Background:

Developing Software as a Medical Device (SaMD) and clinical artificial intelligence (AI) systems requires rigorous lifecycle traceability under IEC 62304 and national regulations such as Brazilian Health Regulatory Agency (ANVISA) Resolution RDC 657/2022. However, academic groups and early-stage engineering teams frequently maintain source code repositories lacking structural evidence of dependency control, automated verification, fault handling, or model provenance. Conversely, submitting raw repositories directly to large language models (LLMs) risks regulatory hallucination, loss of line-level traceability, and exposure of proprietary code.

Objective:

This study aims to (1) present the architecture and implementation of TraceAST-62304 (v1.0.0), an open, local-first, evidence-traceable pre-audit screening framework for Python medical software repositories; (2) report preliminary technical evaluation results across controlled fixtures and 20 public medical software/AI repositories; and (3) publish a structured protocol inviting independent domain experts for external validation.

Methods:

TraceAST-62304 decouples four layers: (1) static inspection and Abstract Syntax Tree (AST) extraction without executing target code; (2) a versioned deterministic rules engine evaluating eight technical indicators (R01-R08) with physical file and line-range verification; (3) local bilingual Retrieval-Augmented Generation (RAG) indexing 26 normative and template documents (ANVISA RDC 657/2022 and OpenRegulatory IEC 62304 templates; 257 chunks); and (4) an optional, privacy-preserving LLM explanation layer that receives only sanitized metadata and cannot alter deterministic statuses. We evaluated the system using a 17-test automated suite, synthetic fixtures, an internal RAG retrieval benchmark (top_k=3), and a technical benchmark of 20 public Python medical software repositories (10 well-structured engineering repositories and 10 experimental/deficient research repositories; 160 rule evaluations).

Results:

All 17 automated unit, integration, and negative tests passed (100% pass rate in 6.23 seconds). Across the 20 external benchmark repositories, TraceAST-62304 inspected 4,395 Python files and successfully parsed 4,390 via AST (99.89% parse coverage; 5 SyntaxError failures isolated to 1 legacy repository) in 204.92 seconds, extracting 5,204 classes, 24,524 functions/methods, 976 try-except blocks, and 2,686 candidate I/O operations. Automated testing evidence (R03) and continuous integration (R04) were observed in 10/10 (100%) and 9/10 (90%) well-structured repositories versus 2/10 (20%) and 0/10 (0%) experimental repositories, respectively. In preliminary internal RAG testing across 8 rule queries, document-level hit-rate Recall@3 was 1.00 (8/8) and mean fractional Recall@3 was 0.5417. Independent human expert validation has not yet been conducted; all normative mappings remain tagged as pending_expert_review.

Conclusions:

TraceAST-62304 demonstrates technical feasibility, computational scalability, and deterministic file-and-line traceability for preliminary pre-audit screening of Python medical software repositories. Through this preprint, we invite independent specialists in IEC 62304, ISO 13485, SaMD regulatory affairs, and medical AI to critically review the rule catalog and participate in a blinded multi-rater validation study.