JMIR Publications

ABSTRACT

Background:

Academic institutions face increasing challenges in grant writing due to evolving federal and state policies that restrict the use of specific language. Manual review processes are labor-intensive and may delay submissions, highlighting the need for scalable, secure solutions that ensure compliance without compromising scientific integrity.

Objective:

To develop a secure, AI-powered tool that assists researchers in writing grants consistent with evolving state and federal policy requirements.

Methods:

GrantCheck was built on a private AWS Virtual Private Cloud, integrating a rule-based natural language processing engine with large language models (LLMs) accessed via Amazon Bedrock. A hybrid pipeline detects flagged terms and generates alternative phrasing, with validation steps to prevent hallucinations. A secure web-based front end enables document upload and report retrieval. Usability was assessed using the System Usability Scale.

Results:

GrantCheck achieved high performance in detecting and recommending alternatives for sensitive terms, with a precision of 1.000, recall of 0.73, and an F1 score of 0.84—outperforming general-purpose models including GPT-4o (F1 = 0.43), Deepseek R1 (F1 = 0.40), Llama 3.1 (F1 = 0.27), Gemini 2.5 Flash (F1 = 0.58), and even Gemini 2.5 Pro (F1 = 0.72). Usability testing among 16 faculty and staff participants yielded a mean System Usability Scale (SUS) score of 82.2, indicating a positive user satisfaction with the tool’s interface, functionality, and workflow integration.

Conclusions:

GrantCheck demonstrates the feasibility of deploying institutionally hosted, AI-driven systems to support compliant and researcher-friendly grant writing. Its hybrid architecture ensures high performance and privacy while reducing administrative burden in navigating shifting language policies.