JMIR Publications

ABSTRACT

Background:

The Japanese National Medical Licensing Examination (NMLE) is mandatory for all medical graduates to become licensed physicians in Japan. Given the cultural emphasis on summative assessment, the NMLE has had a significant impact on Japanese medical education. Although the NMLE Content Guidelines have been revised approximately every five years over the last two decades, there is an absence of objective literature analyzing how the actual exam itself has evolved.

Objective:

To provide a holistic view of the trends of the actual exam over time, this study used a combined rule-based and data-driven approach. We primarily focused on classifying the questions according to the perspectives outlined in the NMLE Content Guidelines, while complementing this approach with a natural language processing technique called topic modeling to identify latent topics.

Methods:

Publicly available NMLE data from 2001 to 2024 were collected. Six exam iterations (2,880 questions) were manually classified from three perspectives (Level, Content, and Taxonomy) based on pre-established rules derived from the guidelines. Temporal trends within each classification were evaluated using the Cochran-Armitage test. Additionally, topic modeling was conducted for all 24 exam iterations (11,540 questions) using the BERTopic framework. The temporal trends of each topic were traced using linear regression models of topic frequencies to identify topics growing in prominence.

Results:

In Level classification, the proportion of questions addressing common or emergent diseases increased from 60% to 76% (p < 0.001). In Content classification, the proportion of questions assessing knowledge of pathophysiology decreased from 52% to 33% (p < 0.001), whereas the proportion assessing practical knowledge of primary emergency care increased from 20% to 29% (p < 0.001). In Taxonomy classification, the proportion of questions that could be answered solely through simple recall of knowledge decreased from 51% to 30% (p < 0.001), while the proportion assessing advanced analytical skills, such as interpreting and evaluating the meaning of each answer choice according to the given context, increased from 4% to 19% (p < 0.001). Topic modeling identified 25 distinct topics, and 10 topics exhibited an increasing trend. Non-organ-specific topics with notable increases included “Comprehensive Clinical Questions,” “Accountability in Medical Practice and Patients’ Rights,” “Care, Daily Living Support, and Community Healthcare,” and “Infection Control and Safety Management in Basic Clinical Procedures.”

Conclusions:

This study identified significant shifts in the Japanese NMLE over the past two decades, suggesting that Japanese undergraduate medical education is evolving to place greater importance on practical problem-solving abilities than on rote memorization. This study also identified latent topics that showed an increase, possibly reflecting underlying social conditions. Clinical Trial: NA