JMIR Publications

ABSTRACT

Background:

Large language models (LLMs) can draft manuscript sections on demand. General-purpose conversational tools produce text whenever asked, which can yield manuscripts the nominal author did not reason through, raising educational and academic-integrity concerns and doing little to train the author.

Objective:

We describe and demonstrate a Socratic AI writing tutor whose defining feature is an automatic engagement gate: the tool refuses to produce any draft text until a secondary model judges that the author has substantively engaged with at least three guiding questions.

Methods:

The web application guides the users through eight sequential manuscript modules. Before each turn, a secondary LLM classifies the author's latest reply as substantive or not; draft generation is unlocked only after three substantive replies, and even then, a draft is merely offered for the author to accept and edit, never inserted automatically. The guidance is aligned with the relevant reporting standards for the chosen study design. We illustrate the tool with a fictional retrospective case series (atezolizumab plus bevacizumab in hepatocellular carcinoma) by constructing two manuscript sections. Because this is a demonstration of the intended behavior, both the trainee and tutor turn in the illustrative sessions were generated by a large language model (Claude Opus 4.8); no real users were involved

Results:

In the illustrative sessions, with the gate active, the tutor declined write-for-me requests, converted vague claims into exact figures, corrected two technical errors (censoring and misplaced interpretation), and assembled drafts only from the author's own statements once the threshold was met.

Conclusions:

An automatic engagement gate is a feasible mechanism for keeping an LLM in a tutoring role rather than a ghostwriting role, distinguishing this tool from general-purpose chatbots. Clinical Trial: Not Applicable