Document resource
Background and Importance Since July 2022, the Taiwan Society of Hospital Pharmacists has promoted Competency-Based Pharmacy Education (CBPE). In November 2023, the Education Development Committee established a national framework integrating pharmacists’ six core competencies with a 2 year training curriculum through the implementation of Entrustable Professional Activities (EPAs).Aim and Objectives Following the initial implementation, the first CCC was convened to review learners’ performance and calibration across assessment tools. This study aimed to examine the agreement between traditional workplace-based assessments and CCC entrustment decisions, and to identify the key sources of discrepancy related to assessment design, rater variability, and contextual limitations.Material and Methods This single-centre deliberative study analysed data from the first CCC convened after the implementation of CBPE. The participants were pharmacy trainees enrolled in the 2 year postgraduate year training program, evaluated during the final phase of their training (approximately 20–24 months after program entry). The CCC reviewed each trainee’s performance using multiple workplace-based assessments, including the Mini-CEX, DOPS, ad-hoc EPAs, OSCE, and longitudinal learning portfolios. Quantitative analyses examined the agreement between these assessment tools and CCC entrustment decisions using weighted kappa coefficients, McNemar tests, and descriptive statistics. Qualitative thematic analysis of CCC meeting transcripts was conducted to identify sources of discrepancy, focusing on assessment design, rater calibration, and contextual factors influencing entrustment judgements.Results Twelve pharmacy trainees were evaluated during the final phase of the 2 year PGY program. All trainees passed the Mini-CEX, DOPS, and portfolios, while 10 of 11 passed the OSCE. In the ad-hoc EPA for medication dispensing, all reached Level 4 and one achieved Level 5. However, only 10 trainees (83.3%) were judged as entrustable by the CCC. Agreement rates between traditional assessments and CCC decisions were moderate (83.3%; κ = 0.00–0.40), suggesting ceiling effects and limited discrimination. The portfolio trend, reflecting longitudinal progress, showed the highest agreement (91.7%; κ = 0.63), indicating that continuous monitoring of learning trajectories provided stronger evidence for entrustment than single-point ‘pass/fail’ assessments.Conclusion and Relevance Data-informed CCC reviews highlight the need for longitudinal, evidence-based assessment to guide training improvement. Future training should integrate continuous, data-informed EPA assessments to enhance entrustment decision quality.Conflict of Interest No conflict of interest