The co-hosts examine a narrative review that looks at Clinical Performance Indicators (CPIs) and the various ways they can be used to enhance competence and ensure safe and effective care. The paper’s authors outline why CPIs are essential, the different levels of each CPI and how they can inform education, as well as the challenges with using CPIs in #meded. Listen here.
KeyLIME Session 227
Smirnova A et al. Defining and Adopting Clinical Performance Measures in Graduate Medical Education: Where Are We Now and Where Are We Going? Academic Medicine. 2019 May;94(5):671-677
Jason R. Frank (@drjfrank)
Regular listeners of KeyLIME will know that #meded (and health professions training more generally) is a field and enterprise widely criticized for unacceptable variation in its product, namely practicing clinicians. Our leaders remind us that this is an age of accountability, and our profession is asked to innovate to improve competence, care, safety, and the systems that produce them.
Enter data. Lots of data. If you have an electronic health record or a digital portfolio or even a great big box of paper with stuff about your trainees, you have data. The tricky part is what to do with it? How can we convert the data all around us into meaningful intelligence that can inform teaching, learning, assessment, curriculum, systems design, accreditation, and more? Enter smart people who think about Clinical Performance Indicators (CPIs).
This paper, Smirnova et al, is a team of heavy hitting #meded authors. (In particular, I encourage you to watch the clever work of Dan J Schumacher from Cincinnati.)
The group set out to describe how CPIs, as measures of the activities, behaviours, and abilities of clinicians, can inform medical education and health care quality improvement.
Key Points on the Methods
This is essentially a narrative review by a learned group. They chose to focus on the implications for residency education, but they suggest their points apply to any phase of a clinician’s career, and any workplace-based learning setting. The paper is essentially framed as a charter and state of the art overview of CPIs for #meded.
Why do we need CPIs? The authors make the case that CPIs are needed to:
• Align #meded with clinical care quality;
• Justify societal investment in #meded via quality markers;
• Connect education to clinical care process indicators as well as patient outcomes;
• Provide feedback loops to teaching, learning, and curriculum;
• Assess learner performance, progression of competence;
• Help evaluate learning environments and accredit programs;
• Attempt to alleviate unacceptable variations in care between training settings;
• Promote patient safety; and,
• Inform #meded research, program evaluation, and CQI.
Levels of CPIs can include :
1. Individual clinician / learner
For each of these levels, CPIs can inform education via:
• feedback loops,
• guides for further learning and reflection,
• personal insights,
• tracking progress
• aggregate clinician performance over time to guide assessment, evaluation, curriculum renewal,
• clinical processes, and patient outcomes,
• characteristics of learning environments
Challenges with CPIs for #meded include:
• need for collaborations across multiple healthcare stakeholders, including educators, patients, learners, health services researchers, economists, digital systems managers, and patient safety/QI leaders;
• need for taxonomies for a lingua franca among training communities;
• need to define CPIs that are clinically relevant, educationally sensitive, and tailored to the learner’s stage of development;
• need to overcome measurement issues, including capturing meaningful data from existing digital systems (e.g. EMRs and registries);
• need to address analysis issues, especially attribution (the effect of a given individual’s behaviours in care) and contribution (the ways a given individual’s abilities can impact), as well as aggregation and nesting;
• need to take into account complexity issues in health care systems, and the interdependence of individual health professionals’ input into care, as well as the contextual influence;
• protecting privacy of people’s data; and,
• linking data sets in useful ways.
The authors conclude that measures of trainee performance are essential ingredients to enhancing competence and ensure safe and effective care. They argue the way forward includes:
1. Developing a common taxonomy of CPIs and relevant patient outcomes by specialty;
2. Defining uniform measures for comparative research;
3. Linking #meded and CPI measures to provide new insights into training systems; and,
4. Developing iterative loops to improve measures, definitions, and measurement systems
Spare Keys – other take home points for clinician educators
1. We do live in an era of accountability, metrics, and digital databases, CPIs are coming, so #meded leaders need to plan for it.
2. CPI papers are also coming to the #meded lit, a torrent is on the way. I highly recommend the work of the authors.
3. What would happen if we correlate entrustment measures to CPIs? Stay tuned.
Access KeyLIME podcast archives here