Implementation of entrustable professional activities for interns within the new Australian National Framework: a pilot study using action-research to explore utility, and clarify challenges and enablers

Abstract

Abstract Background Programs worldwide are introducing programmatic assessment and Entrustable Professional Activities (EPAs) with strong theoretical foundations and growing research evidence regarding potential benefits. In Australia, the new National Framework for Prevocational Medical Training includes four EPAs for doctors in PGY1-2, who rotate through different specialties to gain broad experience: EPA1:Clinical assessment, EPA2:Recognition and Care of the Acutely Unwell Patient, EPA3:Prescribing, EPA4:Team communication. Little is known about how this new education model will work in practice. Hence a pilot study was conducted to (1) Investigate how feasibility, acceptability, educational value and overall utility of EPAs can be maximised, and (2) Clarify challenges and enablers of EPA implementation, operationalised via EPA assessments. Methods The study was conducted during intern terms 2–4, 2024 (10 weeks each) in a metropolitan health service. EPA assessments (EPAA) were implemented for interns working in eleven medical units across four specialties (medicine, surgery, mental health, emergency). Change strategies were designed using Kotter’s 8-step and COM-B models. Action-research methodology was used and each 5-week cycle involved: enacting plans, gathering user data, analysing data, and co-designing improvements with expert and stakeholder panels. Quantitative and qualitative user data were collected from all EPAA e-forms, plus surveys and focus groups conducted each cycle. Results In total, 738 EPAAs were submitted by 119 interns and 283 assessors. The frequency order was: EPA1(highest)-EPA4-EPA3-EPA2. The majority took 10 min or less to complete, and most were assessed by registrars, except in Emergency where consultants predominated. EPAA activity varied substantially across individual units, even within one specialty. User perspectives indicated that EPAAs' educational value depended primarily on the quality of the feedback discussion. Key challenges identified were establishing a trained assessor cohort, fitting EPAAs into workflow, refining the form design and rating scale (National Framework scale used in Phase 1 and modified Ottawa-scale in Phases 2–3). Multiple revisions to implementation were co-designed and executed across the pilot. Practical strategies to support implementation were clarified. Conclusions This large pilot provides valuable insights into EPA implementation according to the National Framework. Clarifying challenges and enablers may assist others. Optimising EPAA quantity, rating scale, form and feedback quality is important due to impacts on realising educational value and providing robust performance data for progress decisions.