JMIR Publications

ABSTRACT

Background:

The 13 Core Entrustable Professional Activities (EPAs) are key competency-based learning outcomes in the transition from undergraduate to graduate medical education. Five of these EPAs (EPA2: prioritizing differential, EPA3: recommending and interpreting tests, EPA4: entering orders and prescriptions, EPA5: documenting clinical encounters, and EPA10: recognizing urgent and emergent conditions) are uniquely suited for online assessment.

Objective:

For this pilot study, we created a web-based simulation platform for diagnostic assessment of these EPAs and examined its feasibility and acceptability.

Methods:

Four simulation cases underwent three rounds of consensus panels and pilot testing. Incoming emergency medicine interns (n=15) completed all cases, and up to 4 “look for” statements, which encompassed specific EPAs, were generated for each participant: 1) performing harmful or missing actions, 2) narrow differential or wrong final diagnosis, 3) having errors in documentation, and 4) lack of recognition and stabilization of urgent diagnoses. Finally, we interviewed a sample of interns (n=5) and residency leadership (n=5) and analyzed the responses using thematic analysis.

Results:

All participants had at least 1 missing critical action and 40% participants performed at least one harmful action across all 4 cases. The final diagnosis was not included in the differential diagnosis in more than half of assessments (53%). Other errors included choosing the incorrect documentation (40%) and indiscriminately applying oxygen (60%). The themes to the interviews included: psychological safety of the interface, ability to assess learning, and fidelity of cases. The most valuable feature cited was the ability to place orders in a realistic electronic medical record interface.

Conclusions:

This study demonstrates the feasibility and acceptability of this platform for diagnostic assessment of specific EPAs. This approach rapidly identifies potential areas of concern for incoming interns using an asynchronous format, provides this feedback in a manner appreciated by residency leadership, and informs individualized learning plans.