JMIR Publications

ABSTRACT

Background:

It is vital for residents to have a longitudinal view of their educational progression and it is crucial for the medical education team to have a clear way to track resident progress over time. Current tools for aggregating resident data are difficult to use and don’t provide a comprehensive way to evaluate and display resident educational advancement.

Objective:

This work describes the creation and assessment of a system created to improve the longitudinal presentation, quality, and synthesis of educational progress for trainees. We designed and programmed a new system for residency progress management with three goals in mind: 1) long-term and centralized location for residency education data, 2) a clear and intuitive interface that is easy to access for both the residents and faculty involved in medical education, 3) automated data input, transformation, and analysis. The system consists of 22 components including items such as administrative checklists, scholarly activity, and multi-source evaluations. This work aims to introduce and describe the system, as well as provide evaluations regarding whether residents use and find it useful; and whether the faculty like the system and perceive that it helps them save time with administrative duties.

Methods:

The system was created using a suite of Google Workspace tools including Forms, Sheets, Gmail, and a suite of Apps Scripts triggered on various times and events. To assess whether the system had an effect on the residents, we surveyed residents, asking them to self-report on how often they accessed the system and interviewed them as to whether they found it useful. To understand what the faculty thought of the system we conducted a 14-person focus group and asked the faculty to self-report their time spent preparing for residency progress meetings before and after the system debut.

Results:

The system went live in February 2022 as a quality improvement project, growing and evolving through multiple iterations of feedback and assessment. The authors found that the system was accessed differently by different postgraduate years (PGY), with the most usage reported in the PGY1 class (weekly), and the least amount of usage in the PGY3 class (once or twice). However, all of the residents reported finding the system useful, specifically for aggregating all of their evaluations in the same place. Faculty members felt that the system enabled a more high-quality bi-annual Clinical Competency Committee (CCC) meeting. The faculty members also reported a combined time savings of 8 hours in preparation for each CCC as a result of reviewing resident data through the system.

Conclusions:

Our work reports on the creation of an automated, instantaneous, and comprehensive resident progress management system. The system has been shown to be well-liked by both residents and faculty. Younger PGY classes reported more frequent system usage than older PGY classes. Faculty reported that it helped facilitate more meaningful discussion of training progression and reduced the administrative burden by 8 hours per bi-annual session.