I was happy to attend the IME2013 conference recently held at the Keck School of Medicine with my colleague Beth. We both had a great time at the conference and in Los Angeles and I’d like to thank everyone who stopped by say hello and chat.
One of the themes of the conference was “cool ideas” and we took that as an opportunity to share our own cool idea – that traditional evaluation practices are dying. Our notion is that the mounting evidence in favor of competency-based education and evaluation means that traditional evaluation forms – delivered once or twice during an experience, are soon going to be replaced.
What we’ve been hearing in our conversations with clients over the last few months is that it’s difficult to measure competence when you only evaluate a student twice during a given experience, and when the only data point is a single question on each of those evaluations that measures “competence” on a scale of 1-5. True competency-based evaluation requires an analysis of the results from multiple points of feedback delivered over the duration of an experience with a qualitative review at the end.
Armed with our cool idea, we asked people at the conference how they were gathering feedback on a regular basis and how they were doing competency-based evaluations, particularly in relation to the ACGME milestones project.
While everyone we spoke to had their own process, what stood out for us was that there is a real lack of resources for handling competency-based of evaluations. Some of the folks we spoke with are doing interesting things by cobbling together solutions on their own, but most of those solutions require a lot of work on the part of the evaluator. We heard from many people how difficult it is to share and integrate the information one individual is capturing with the rest of the evaluation information about a given student or resident.
In the coming months we will be debuting some new reporting tools that allow you to visualize how students and residents are performing on different competencies. We also have a tool we’re working on that will allow students and residents to apply for recognition of competency in one45.
If you have a cool idea about competency-based evaluation that you’d like to share, or if you’d like to hear more about what we heard at the conference or how one45 is planning on supporting this emerging need please let me know – I’d love to keep the conversation going.