By: Cory Rohlfsen (@CoryRohlfsen)
As a junior attending physician, millennial, and clinical educator track director who is fascinated by self-determination theory, I’ve become increasingly sensitive to how competency-based medical education (CBME) intersects with trainee motivation, goal-setting, and self-regulated growth. The start of graduate medical education is marked by a brisk transition in formative assessment in which trainees experience rapid progression across a range of competencies. Unlike pre-clinical year(s) where medical students have objective benchmarks (e.g., test scores), residents must adapt to a model of subjective, workplace-based judgments. Otherwise, low stakes feedback can be misinterpreted as high stakes, summative assessment.1 To make matters more confusing, the roadmap to improve is more complex, the finish line is less clear, and a trainee’s locus of control may feel more external than internal.2 Most importantly, residents are learning more than medicine. Through transformative experiences, they are forging their professional identities as they negotiate a tension between self-image (ego) and milestone-based feedback. CBME ensures there is no shortage of “disorienting dilemmas” in which trainees are stretched to improve.3 For most, it feels less like a linear path and more like a “tug-of-war” in which change is resisted, embraced, resisted, and then embraced again. This tension is normal particularly as newly minted doctors navigate the enormous responsibility of how to take care of a patient. We should not be surprised when an intern who is placing insulin orders for the first time and lacks confidence in DKA management becomes defensive when their attending points out a missed opportunity for advocacy. Managing psychological stress is how we grow without being broken and it’s best summarized by the inverted U-shaped stress and learning curve4 (Figure 1).
While much can be said about the responsibility incumbent on clinical educators to steward this delicate balance of optimal arousal, I’d like to focus this post on the role CBME may play in the tension (x-axis) from a system perspective. As currently implemented, does CBME exacerbate or ameliorate the tension? How could it better facilitate an optimal balance? What blind spots is CBME missing with respect to its implementation science?
Lessons learned the hard way: “milestone” is a dirty word
The day I became aware of the gap between how CBME is intended and how it is experienced by trainees, I honestly wasn’t looking for it. The University of Nebraska Medical Center (UNMC) launched a competency-based, clinical educator track for residents and fellows in 2021. We drafted milestones to outline the progressive sequencing of competencies required to be an expert teacher in the clinical learning environment. Participants with high proclivity for growth mindset were selected from an application pool to join the inaugural cohort. Accepted trainees were then asked to assess themselves throughout the curriculum across fourteen domains of competence using the milestone-based rubric. The rubric was not used for performance evaluation(s) nor grading purposes but rather to facilitate individualized goal setting. Participants were supported by a community of practice including mentors, coaches, and peers to help foster reflective teaching experiences. At the end of the first year, we conducted a mixed methods program evaluation using a grounded theory approach. Much to my chagrin, the novel competency-based self-assessment rubric ranked dead last amongst a long list of curricular components in an otherwise well received, extracurricular training program. What went wrong?
A theme emerged during the program evaluation that “milestone” is a dirty word. For most residents and fellows, it was difficult to engage with the rubric or set goals because of the cognitive load required to orient to the tool. Rather than choosing a targeted domain to improve upon, trainees felt frustrated by the multitude of competencies as if being pulled in multiple directions at once (a phenomena known as curricular overfitting).6 For others, the “M-word” evoked a visceral reaction because it reminded them of milestone-based assessments that were perceived as invalid, arbitrary, or less than beneficial in their graduate medical training. Ultimately, self-motivation and engagement were hindered by a number of systematic factors that led to mistrust (high tension). The far-right end of the curve. Either the competency-based tool was “bad”, or it wasn’t being deployed correctly.
What surprised me most was how much tension could exist within the otherwise innocuous, self-regulated, low-stakes environment of learning to teach – amongst a self-selected group of graduate medical trainees, nonetheless. It begged the question… how much more tension might there be within our CBME assessment frameworks given the summative, serious, and high stakes milieu of GME?
The elephant in the room
After listening to our inaugural clinical educator cohort and debriefing with a few program directors, I want to address the elephant in the room. It’s been 10+ years since CBME was deployed by the United States and Canada in graduate medical training and we still know relatively little about how milestones are experienced by learners. Do trainees grow because of CBME or in spite of it?
According to Mann et al., “the resident voice has been conspicuously absent from discussions regarding the impact of transitioning to CBME, even though trainees are most directly affected by this change.7 ” While our experience may be an isolated one, the signal of mistrust should not be ignored as I suspect the majority of trainees experience at least some level of undesirable difficulty (inappropriately high tension) from our current CBME assessment and feedback structures.2,8 The sources of this tension should be explored and categorized so that mitigating solutions can be further investigated and deployed. A few ideas are listed in Table 1.
If trainee stakeholder voices, experiences, and perspectives are not taken into account, then the gap of mistrust will likely increase. Surveys show Gen Z’s mistrust of higher education is outpacing that of millennials.9 In other words, gone are the days of saying “trust the process.” If learners are to invest in CBME for self-reflective growth, then it’s incumbent on CBME to better understand where our assessment system(s) have fallen short. We must seek out the sources of tension with genuine curiosity and this begins by surveying recent graduates from a variety of institutions.
While results from exemplar institutions are helpful for defining best practices, what can be done to help programs with lesser resources or institutional barriers? What faculty development, coaching, or learner initiatives work for institutions with limited bandwidth?
While CBME theory is sound, its application in GME raises more questions than answers. And we’re missing a key piece of the puzzle – the resident and fellow experience.
ABOUT THE AUTHOR: Cory J. Rohlfsen, MD is an Assistant Professor in the UNMC Division of General Internal Medicine as well as Director of Health Educators and Academic Leaders (HEAL) at University of Nebraska Medical Center.
References
1.Govaerts M. Workplace-Based Assessment and Assessment for Learning: Threats to Validity. J Grad Med Educ 2015;7(2): 265-267.
2. Kinnear B, S Santen, M Kelleher, A Martini, S Ferris, L Edje, E Warm, D Schumacher. How does TIMELESS Training Impact Resident Motivation for Learning, Assessment, and Feedback? Evaluating a Competency-Based Time Variable Training Pilot. Acad Med. 2023; Jan 16: Online ahead of print.
3. Vipler B, A Knehans, D Rausa, P Haidet, JM McCall-Hosenfeld. Transformative Learning in Graduate Medical Education: A Scoping Review. J Grad Med Educ 2021; 13(6): 801-814.
4. Corbett M. From law to folklore: work stress and the Yerkes-Dodson law. J Manag Psychol. 2015; 30: 741- 752.
5. Chen O, JC Castro-Alonso, F Paas, J Sweller. Undesirable Difficulty Effects in the Learning of High-Element Interactivity Materials. Front Psychol. 2018; 13(9):1483.
6. Hoang NS, JN Lau. A Call for Mixed Methods in Competency-Based Medical Education: How We Can Prevent the Overfitting of Curriculum and Assessment. Acad Med. 2018; 93(7): 996-1001.
7. Mann S, AH Truelove, T Beesley, S Howden, R Egan. Resident perceptions of Competency-Based Medical Education. Can Med Educ J. 2020;11(5):e31-e43.
8. Crawford L, N Cofie, L McEwen, D Dagnone, SW Taylor. Perceptions and barriers to competency-based education in Canadian postgraduate medical education. J Eval Clin Pract. 2020;26(4):1124-1131.
9. Fishman R, S Nguyen, L Woodhouse. Varying Degrees: New America’s Sixth Annual Survey on Higher Education. 2022.
10. Wilby KJ, B Paravattil. Cognitive load theory: Implications for assessment in pharmacy education. Research in Social and Administrative Pharmacy. 2021; 17(9): 1645-1649.
11. Deiorio N, M Hammoud. Coaching in Medical Education: A faculty handbook. Accelerating Change in Medical Education Consortium. 2017.
12. Sandars J, D Murdoch-Eaton. Appreciative inquiry in medical education. Medical Teacher. 2017; 39(2): 123-127.
13. Regan L, LR Hopson, MA Gisondi, J Branzetti. Creating a better learning environment: a qualitative study uncovering the experiences of Master Adaptive Learners in residency. BMC Med Educ 2022;22(1):141
14. Daniels VJ, Stach J, Sandhu G. Transitioning to competency-based medical education: impact of educational interventions on internal medicine residents’ understanding of the purpose and process. Can Med Educ J. 2019; 10(4):e96-e98.
15. Ryan MS, AD Blood, YS Park, JM Farnan. Competency-Based Frameworks in Medical School Education Programs: A Thematic Analysis of the Academic Medicine Snapshots, 2022. Acad Med. 2022;97(11S): S63-S70.
16. Image by Benoit De Haas from Pixabay retrieved from: https://pixabay.com/illustrations/tug-of-war-rope-stick-men-game-6947572/ on January 16th. License associated: CCO.
The views and opinions expressed in this post are those of the author(s) and do not necessarily reflect the official policy or position of The Royal College of Physicians and Surgeons of Canada. For more details on our site disclaimers, please see our ‘About’ page