#KeyLIMEpodcast 120: Some Assembly Required: How to Build A Program of Assessment

The Key Literature in Medical Education podcast this week is part 2 of the Live! event that took place at the International Conference on Medical Education where we welcomed guest host Eric Warm from the University of Cincinnati. As the Program Director of the Internal Medicine Program, he states that his “core business” is assessment, which led him to choose this paper by Cees van der Vleuten et al. who he credits for being a rock star of the medical community. Jason Frank added that if there was one thing attendees should take away from ICRE 2016, these tips should be it!

Check out the podcast here and read more below!


KeyLIME Session 120 – Article under review:

Listen to the podcast

View/download the abstract here.

New KeyLIME Podcast Episode Image

van der Vleuten CP, Schuwirth LW, Driessen EW, Govaerts MJ, Heeneman S. 12 Tips for programmatic assessment. Medical Teacher. 2014 Nov 20:1-6. [Epub ahead of print]

Reviewer:  Eric Warm (@CincyIM)


The core function of every medical training program is to assess trainees’ readiness for independent practice. Over the past 30 years, medical educators have developed multiple systems of assessment, often independently from one another. Despite this, there is still much debate about what is best, or how best to deploy the tools of assessment.


To describe the development and implementation of ‘Programmatic of Assessment’ in medical education.

Type of Paper

Non-systematic review

Key Points on Methods

Narrative review of the literature on assessment and systems of assessment

Key Outcomes

The authors describe 12 components required to create high quality programmatic assessment:

  1. Develop a master plan for assessment
  2. Develop examination regulations that promote feedback orientation
  3. Adopt a robust system for collecting information
  4. Assure that every low-stakes assessment provides meaningful feedback for learning
  5. Provide mentoring to learners
  6. Ensure trustworthy decision-making
  7. Organize intermediate decision-making assessments
  8. Encourage and facilitate personalized remediation
  9. Monitor and evaluate the learning effect of the program and adapt
  10. Use the assessment process information for curriculum evaluation
  11. Promote continuous interaction between the stakeholders
  12. Develop a strategy for implementation

Key Conclusions

The authors conclude that no single assessment method is perfect, or tells ‘the truth’ about performance.  Individual assessments of many types should be purposefully chosen so that the whole is greater than the sum of the parts. Importantly, the authors believe that programs of assessment should optimize both the learning and decision making functions of assessment. Implementation of a fully realized program of assessment requires commitment from all stakeholders and is not easy. Although programs may derive some value from implementing portions of programmatic assessment, full value requires full implementation.

Spare Keys – other take home points for clinician educator

Traditional assessment is mostly summative in nature (of learning), but assessment could be much more powerful if it was oriented for learning. A prime purpose for collecting assessment information should be to provide actionable feedback (for learners, programs, and systems).

Shout out

This is a must read for every program director, dean of a medical school, or any medical educator who must assess trainees. Consider using it like a reference guide and a checklist.

Access KeyLIME podcast archives here

Check us out on iTunes