By: Holly Caretta-Weyer (@holly_cw)
The transition from undergraduate (UME) to postgraduate medical education (PGME) training ought to be a continuum characterized by the transmission of honest assessment data and a developmentally focused learner handover. Instead, the transition is fraught with discontinuity, competing priorities, and opaque communication between medical schools and postgraduate training programs. The main culprit, particularly in North America, is arguably the postgraduate selection process.
The Misalignment of Selection and CBME
Ironically, as stakeholders in both UME and PGME have sought to implement competency-based medical education (CBME) across the continuum of medical training, the chasm between UME and PGME has grown wider. While CBME demands growth-focused assessment, the current postgraduate selection process retains an intense reliance on norm-referenced data to stratify students for interview selection and ranking. This encourages students to focus on clerkship grades and test scores instead of professional growth and development. Given this intense focus on metrics instead of beginning to develop specialty-specific skills and defining areas for growth as medical students transition to postgraduate training, we should not be surprised that program directors find students ill-prepared to meet what is expected of them on day one of postgraduate training. This dichotomy between achievement-focused metrics and the desire for a transparent, growth-focused system has led many to question the postgraduate selection process.
As such, the postgraduate selection process is the single greatest threat to operationalizing a continuum of medical education under CBME. How then are we to achieve the promises of CBME? In order to do so, the postgraduate selection process must evolve to align with CBME.
In the 2018 Ottawa conference consensus statement, Patterson and colleagues present three main approaches to postgraduate selection: individually focused processes (centered around academic success and subsequent readiness), competency-based processes (requisite knowledge, skills, and behaviors that drive readiness to perform a job), and societal expectations (student diversity, social accountability, workforce planning, and inclusion). Each of these approaches focuses on a different priority outcome within the selection process. Much of postgraduate selection to date has centered around individually focused processes and metrics at the expense of competency-based processes and societal expectations.
An Outcomes-Oriented Approach to Selection
In order to move forward, the adoption of CBME pushes us to consider postgraduate selection from an outcomes-oriented lens focused on a combination of competency-based processes and societal expectations. This does not neglect individually focused processes as programs and students must still select one another on an individual basis. It instead refocuses that selection process itself around relevant programmatic, specialty, and societal outcomes within which students and programs are better able to align with one another and relevant higher order outcomes.
When considering an outcomes-oriented approach to the postgraduate selection process that centers competency and societal expectations, we can learn a great deal from the field of organizational psychology. This field focuses on how an organization is successful in achieving its defined outcomes by optimizing employee selection and focusing on the transparent alignment of organizational culture with employee motivations and values.
Integrating the principles of organizational psychology is central to optimizing the postgraduate selection process. Foremost, a job analysis is essential to defining the desired outcomes of postgraduate training. A job analysis utilizes interviews, task analyses, and direct observation of workers, in this case residents and practicing physicians in a given specialty, to elucidate the necessary knowledge, skills, abilities, and overall competencies required to successfully perform their work and provide optimal care for patients. In emergency medicine, this may range from the competencies necessary to provide resuscitative care based on the distribution of acute presentations within the local patient population to the skills necessary to achieve the defined scholarly publication outcomes of the postgraduate training program within its specific areas of focus. In much the same way that competency-based medical education (CBME) requires an outcomes framework around which to scaffold curricula and assessment, organizational psychology begins with the outcomes in mind and works backward to define measures for selection.
Call to Action: A Stakeholder Coproduction Model
Aligning selection with an outcomes-oriented approach requires us to consider the coproduction of macro (society), meso (specialty), and micro (program) level outcomes using the principles of organizational psychology to guide the development of a rigorous and transparent selection process. Once a novel process is defined and tools created to assess applicant and program alignment, various logistical interventions at the level of application submission and review would be better positioned to reduce application numbers and mitigate the burden of file review on programs. However, this will require national organizations, specialty societies, medical schools, postgraduate training programs, and applicants to work together to devise a mutually acceptable way forward.
Each stakeholder group has a role in the coproduction model to achieve the necessary outcomes within an outcomes-driven approach to postgraduate selection. Examples include the following:
- National Organizations: Organizations spanning the UME-PGME continuum must evaluate their responsibility of training physicians to meet the needs of society. This would lead to societal-level outcomes, such as consideration of social determinants of health, to inform the postgraduate selection process. Additionally, national organizations should contribute policy around developing and implementing an outcomes-oriented selection process uniformly across schools and programs so that it is not reliant on a small group of pilots or early adopters.
- Specialty Societies: Each specialty society must examine the competencies required to perform the job of a given profession as part of a specialty-level job analysis. This would provide requisite specialty-level outcomes to focus selection priorities based on the necessary knowledge, skills, attitudes, and competencies required to care for a broad range of patients in a variety of contexts within a given specialty.
- Postgraduate Training Programs: Each program leadership team must reflect on the mission and values of their individual training program, what it does well, its desired training priorities, and previous applicants who have ultimately thrived in their program. This may be aided significantly by a programmatic job analysis to define program-specific outcomes. Program level outcomes are subsequently developed and refined and become the primary driver of the postgraduate selection process from application screening to final ranking.
- Medical Schools: Refocusing on outcomes-oriented data allows for better alignment between competency-based assessment data and the postgraduate selection process. Focusing on school-level outcomes and student priorities creates opportunities for individualized learning, driven by students. This allows for optimal alignment between a program and an applicant secondary to transparency around program-level outcomes and applicant priorities while allowing for continued growth in the transition from UME to PGME.
- Applicants: Focusing on societal, specialty, and programmatic outcomes instead of current metrics such as grades allows applicants to concentrate on their growth within a competency framework in medical school while simultaneously developing their personal values and priorities. By reflecting on their priorities and areas for growth, applicants can focus on their alignment with a given specialty and program and apply accordingly in order to “match” at a place where they will thrive.
It is time for postgraduate selection processes to pivot to focus on competency-based priorities and societal expectations using an outcomes-based approach to align with continuous growth and development. This model provides the first steps on a path forward.
ABOUT THE AUTHOR: HOLLY CARETTA-WEYER, MD, MHPE, IS THE ASSOCIATE RESIDENCY PROGRAM DIRECTOR AND DIRECTOR OF EVALUATION AND ASSESSMENT FOR THE STANFORD UNIVERSITY EMERGENCY MEDICINE RESIDENCY PROGRAM AS WELL AS EPA/CBME IMPLEMENTATION LEAD AT THE STANFORD UNIVERSITY SCHOOL OF MEDICINE.
References
1.Wolf SJ, TM Lockspeiser, J Gong, G Guiton. Students’ perspectives on the fourth year of medical education: A mixed-methods analysis. Acad Med. 2014;89:602-607.
2. O’Brien BC. What to do about the transition to residency? Exploring problems and solutions from three perspectives. Acad Med. 2018;93:681-684.
3. Pereira AG, M Woods, AP Olson, S van den Hoogenhof, BL Duffy, R Englander. Criterion-based assessment in a norm-based world: how can we move past grades? Acad Med. 2018;93(4):560-564.
4. Patterson F, E Ferguson, S Thomas. Using job analysis to identify core and specific competencies: Implications for selection and recruitment. Med Educ. 2008:42(12);1195-1204.
5. Gardner AK, BJ Dunkin. Pursuing excellence: The power of selection science to provide meaningful data and enhance efficiency in selecting surgical trainees. Ann Surg. 2019;270(1):188-192.
6. Patterson F, C Roberts, MD Hanson, et al. Ottawa consensus statement: Selection and recruitment to the healthcare professions. Med Teach. 2018;40(11):1091-1101.
The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page