Assessing Performance on Professional, Health Advocate and Scholar During Crisis Simulation

Abstract

Abstract The objective was to assess Professional, Health Advocate and Scholar CanMEDS competencies in anesthesia residents using a Generic Integrated Objective Structured Assessment Tool (GIOSAT) during simulated scenarios. Methods Twenty one anesthesia residents participated in managing two scenarios: “do not resuscitate” and “morphine overdose”. Four trained blinded raters analyzed video recordings using the GIOSAT. Internal structure was examined using generalizability analysis. Results Results of the g-study focused on PHAS and ME components alone, participants accounted for 20% of the scores variance (G-coefficient 0.66). A D-study indicated two raters and eleven or more scenarios would be required for a G-coefficient > 0.80. Conclusion This study demonstrates construct validity for assessing PHAS competencies for low stakes assessment. Results address the gap of assessment performance for PHAS competencies, describe methodology and produce recommendations for summative assessments using simulation.