Previously submitted to: JMIR Medical Education (no longer under consideration since Apr 08, 2026)
Date Submitted: Oct 9, 2025
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
Validity of the Family Medicine Certification Exam through the Online OSCE: A Cross-Sectional Study.
ABSTRACT
Background:
Medical certification is a crucial mechanism for ensuring quality and safety in healthcare. In Mexico, the Mexican Board of Certification in Family Medicine (CMCMF) conducts certification through a two-phase process that includes an online Objective Structured Clinical Examination (OSCE). The shift to virtual environments after the COVID-19 pandemic has raised concerns about the validity and reliability of these assessments.
Objective:
To analyze the psychometric properties of the CMCMF online OSCE applied in the 65th (December 2023) and 66th (August 2024) certification exams.
Methods:
An observational, cross-sectional, descriptive study was conducted with 3,083 family physicians (1,797 from Cohort 65 and 1,286 from Cohort 66) who completed the practical phase of certification. Sociodemographic variables were described and compared across cohorts. Psychometric analyses included item discrimination using t-tests for extreme groups, reliability through Cronbach’s alpha, exploratory factor analysis with varimax rotation, and hierarchical multiple regression to assess predictive capacity.
Results:
Women represented 64.4% of the sample, with a higher proportion in Cohort 66 (p = 0.026). Item discrimination confirmed that the 26 items differentiated between high- and low-performing candidates (p < 0.001). Cronbach’s alpha was 0.93 in both cohorts, indicating excellent internal consistency. Factor analysis identified four stable dimensions—preventive measures, clinical interview skills, doctor–patient relationship, and clinical interpretation skills—explaining 59% of the variance. Overall performance was comparable between cohorts (78.3 vs. 79.0). Hierarchical regression analysis showed that Factor 1 alone explained 70.2% of the variance; the inclusion of all factors increased the explained variance to R² = 1.000.
Conclusions:
The CMCMF online OSCE demonstrated high reliability, a stable factor structure, and strong predictive capacity, supporting its validity as a tool for assessing clinical competencies in family medicine. Further research should evaluate its long-term applicability and generalization across specialties.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.