Accepted for/Published in: JMIR Formative Research
Date Submitted: Aug 23, 2024
Date Accepted: Mar 18, 2025
Using a hybrid of artificial intelligence and template-based method in automatic item generation to create multiple-choice questions in medical education: Hybrid AIG
ABSTRACT
Background:
Template-based automatic item generation (AIG) is more efficient than traditional item writing but it still heavily relies on expert effort in model development. While non-template-based AIG, leveraging artificial intelligence (AI), offers efficiency, it faces accuracy challenges.
Objective:
We aimed to propose a Hybrid AIG to demonstrate whether it is possible to generate item templates using AI.
Methods:
This is a mixed-methods methodological study with proof-of-concept elements. We proposed the Hybrid AIG method that utilizes AI to generate item models (templates) and cognitive models to combine the advantages of the two AIG approaches. The Hybrid AIG consists of seven steps. The first five steps are carried out by an expert in a customized AI environment. Following a final expert review (Step 6), the content in the template can be used for item generation through a traditional (non-AI) software (Step 7). We used two multiple-choice questions for demonstration.
Results:
We demonstrated that AI is capable of generating item templates for AIG under the control of a human expert in a mere 10 minutes. Leveraging AI in template development made it less challenging.
Conclusions:
The Hybrid AIG method transcends the traditional template-based approach by marrying the “art” that comes from AI as a “black box” with the “science” of algorithmic generation under the oversight of expert as a “marriage registrar”. It does not only capitalize on the strengths of both approaches but also mitigates their weaknesses, offering a human-AI collaboration to increase efficiency in medical education.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.