Curriculum management is sometimes loved and sometimes hated by medical schools. We at one45 have seen and been a part of many curriculum management efforts in our history. We have a great deal of experience in making the curriculum management process run as smoothly as possible, but it’s never a small amount of work.

Previously, we shared some best practices for building a curriculum map that has value to its users. However, this still begs the question: even if done right, does curriculum management provide a good return on investment (ROI) to a school? In the future, we’d like to be able to present some solid numbers on the ROI of curriculum management. In this post, we’ll try to enumerate the costs and benefits of implementing a curriculum management system.

Costs of Implementing Curriculum Management

Curriculum management at its most basic level is a structured process of planning, mapping, reviewing and adjusting your curriculum so that it stands the best chance of delivering your school’s overall education goals to students.

To do this right, you have to have one curriculum database for your entire curriculum. Many schools do not have this database, and so the first step in curriculum management is to select, gather, create, and input all of the data into a database. From there, a process of tweaking and refining the data needs to take place.

Here are the major costs in implementing a curriculum management system, in (mostly) sequential order:

• Selecting OR creating a curriculum database
• Setting up the curriculum database
• Gathering existing curriculum data from multiple sources (faculty, course directors, etc)
• Creating new curriculum data that may not exist
• Normalizing and inputting this data into the curriculum database
• Establishing mapping and data standards for the curriculum database
• Faculty and staff development to ensure that data follow the standards (for example, when submitting lecture objectives or curriculum content)
• Creating new processes for having faculty submit curriculum data
• Periodic review and updates to the curriculum

For most of the above steps, there will be many sub-costs: faculty and staff time, software costs, technical support costs, etc. This is no small effort!

Benefits of Implementing Curriculum Management

One of the reasons curriculum management is so important in intensive professional programs (like medical education) is that the curriculum in these programs is, frankly, huge. Unlike most undergraduate degree programs, the curriculum of a professional education program is built by multiple departments, with threads, themes, unique experiences, and content areas spanning years, courses and modules. The amount of information that is due to be imparted to students is enormous and ever-changing.

In this environment, stale content, content gaps and redundancies will arise. If left unchecked, these issues cause students to be frustrated, accreditors to frown, and faculty to butt heads. (I once met a med student who mentioned to me that six separate lectures covered the same “introduction to diabetes”!)

Here are the primary benefits to doing curriculum management:

• A (relatively) pain-free accreditation (at least around curriculum management)
• Prevention of curriculum redundancies and associated fallout (student complaints, wasted faculty and staff effort, etc)
• Prevention of curriculum gaps and associated fallout (accreditation issues, clerkship director complaints, poor assessment results, etc)
• Reduced time required to make curriculum interventions (for example, to replace a declining content area with a trending one)
• Reduced time required to answer curriculum questions (accreditation-related, payment related, etc)
• Reduced time required for students and faculty to find relevant curriculum content
• Reduced time required to investigate problems encountered during student assessment – you can see, at a glance, what students were exposed to and what activities they did to give you clues for the next assessment or how to properly adjust the curriculum for next year’s class

For each of these benefits, the dollar value of the benefits would be represented in terms of staff time wasted, additional hires not needed, and any other associated costs.

Concluding Thoughts

Do the costs of curriculum management outweigh the benefits? I think so, especially when it comes to addressing student outcomes. By continually reviewing your curriculum and how students performed against it, you’re able to make sure that you are providing the students with the best educational experience possible. And that alone has tremendous value.

In the future, we’ll work to get some numbers in place for a true cost/benefit analysis. If you’d be willing to share your budget or your expertise, please let us know! We’d love to hear your thoughts on the factors we’ve listed, and anything we may have missed.

Many schools use a lottery system to assign students to clerkships or clinical rotations. In a typical lottery, students are asked to indicate their preferences, and they are randomly matched up with available placements.

While this lottery process may seem straightforward, many schools struggle with some common pitfalls when setting up their lotteries. Here are 3 of 5 typical mistakes we’ve seen, and how you can avoid them:

Mistake #1: Assigning each student a lottery number

The scenario: Student names are drawn at random and each student is assigned a number indicating his or her position in the rotation-picking queue. Student A receives the number 1 and gets to pick her rotation first. The last student to be drawn is last to select his desired rotation. After going through one round of selections, the order is reversed so that the last student now gets to pick his top choice first.

or: Students are each assigned a lottery number and placed into a campus/rotation order for the entire year. In this scenario, the student who gets the last number has no choice but to accept the only rotation schedule that remains available.

How to avoid this: While these two scenarios may appear to be fair, a large portion of your students will be left unhappy with their final rotation schedules. To avoid this, allow your students to select multiple rankings. This way you can provide all students with equal opportunity to receive the rotations of their choice while ensuring a higher number of students will receive their 1st, 2nd, or 3rd choices and so on.

Mistake #2: Trying to plan all 4th year rotations at one time

The scenario: Students doing their last year of clinical experience are sent one all-encompassing survey and asked to rank their core, selective, and elective rotations for the whole year. A student’s experience at a current internship impacts her decision on what electives to take, and she later decides to drop her previously chosen selections.

How to avoid this: Based on our experience, schools that send students surveys in stages and release the results along the way end up with less drop/adds throughout the year, and happier students overall. If the student knows she has received the pediatrics selective rotation of her choice prior to ranking other open slots, she can focus her other selections on the experiences she wants instead of having to rank pediatrics as a priority over and over again.

Mistake #3: Not giving students a “top choice” option

The scenario: A student has a particular pediatric elective in mind for his clinical rotation, but your lottery process does not account for “top preferences”, so he cannot indicate that this elective is his most desired choice.

How to avoid this: Add an option to your lottery workflow that allows students to indicate their top 3 preferences through a year-wide “top choice” ranking option. By doing this, you will increase the student’s chances of getting his desired rotation. One school that recently implemented this new policy saw 189 out 192 students receive a top rotation/location selection. This really helped to increase student satisfaction!

By avoiding these common mistakes in your rotation lottery setup, you can save yourself time, money, and a lot of headaches. In Part 2 of this post, we will share two other common lottery setup mistakes and tips on how to avoid them. Look for the post in the coming weeks!

In the meantime, if you would like some help analyzing how well your institution is handling your rotation selection process and identifying areas where workflows could be improved, get in touch with us. We will create a mock-up of your lottery process in our system and show you the results.

As one45′s Product Manager, I get to speak with a lot of medical education professionals. One thing we all have in common is a focus on continuous improvement. 

I often get to hear about how our clients are improving their processes for measuring learning outcomes, but I don’t usually get an opportunity to share how we are working to improve our processes and outcomes here at one45. In this post, I want to share how we recently applied the principles of continuous improvement to develop our new Form Builder.

Feedback: The Key to Continuous Improvement

As medical educators know, giving and receiving regular feedback is the best method to achieve continuous improvement. If you’re good at giving feedback, it means you’re able to pinpoint a behaviour that is contributing to a positive outcome, or a behaviour that can be changed to improve the outcome. If you’re good at receiving feedback, you’re able to internalize the feedback and adjust your behaviour as a result. Feedback at its best is always geared toward future behaviour, so the sooner you receive feedback, the sooner you can modify your behaviour and get the outcome you want.

Our Continuously Improving Process 

When we decided to release an updated form builder to our clients, we wanted to find a way to incorporate customer feedback as early in our development process as possible. If we could get feedback from our clients during the development process, rather than after the project wrapped up, we believed we could:

• Avoid building a tool that didn’t address an actual customer need
• Incorporate our customers’ great ideas into the product
• Develop the tools faster by getting answers to our questions and validating our assumptions faster and sooner
• Get to know these customers a bit better, and they’d get to know a bit more about our team

What we did

To test our hypothesis, we invited two of our “Super Administrator” users to join us for our development meetings each week over the course of the project. Each week, we reviewed the work done during the previous week; discussed work to be done in the next week; and had an opportunity to go over any problems, questions, or obstacles related to the project.

The meetings were short (only about a half-hour each) but incredibly productive. Each week as we reviewed the previous week’s work we were able to identify areas for improvement, and use “feedback from the field” to resolve our questions about each feature. I’m most proud of our ability to incorporate some of the suggestions for improving form building productivity that really make the tool shine.

The Result

The positive feedback we’ve received from clients about the Form Builder speaks volumes about how well our team executed this project. We owe a huge amount of thanks to the customers who shared their time with us and gave us their valuable feedback along the way. My favorite piece of feedback came after the project, in the form of an email that said: “It was great to be involved with this project. I would definitely be interested in participating in another feedback session!” 

In a previous post, I shared 3 of the 5 top mistakes we see when it comes to setting up lotteries for scheduling students into clerkships or clinical rotations. Now, I’ll share the last two common mistakes we often see. Compared to my previous post, these tips are more advanced and may be most relevant to you if you are already using one45’s lottery tools to create your schedules.

Mistake #4: Planning clerkships and placement locations separately

The scenario: The Office of Education uses a track lottery to place students into their clerkship order for the year. Then, a program coordinator manually compares student requests with staffing requirements in order to find the final placement for a specific clinical site. The program coordinator must repeat this same task again and again throughout the year.

How to avoid this: The Office of Education can run a randomized track lottery to place students into their assigned locations before the academic year starts. Then, the finalized location information can be shared with individual clerkships. This will save valuable administration time and prevent many headaches for program coordinators.

Mistake #5: Overly flexible schedules (that create unnecessary work for administrators!)

The scenario: A school allows students to rank their rotation preferences to create personalized schedules. Rotations have varying block lengths as well as holiday breaks or blackout periods. These irregularities make planning extra difficult.

In the example below, Track 2 shows a student who ended up with two 2-week blocks of “open” time slots in her schedule. Since there aren’t any rotations that are 2 weeks long, these open blocks cannot be filled, which means an administrator must manually adjust her schedule in order to complete it.

In Track 3, Neuro is a 4-week rotation that cannot be split up by a break. Because of the varying rotation lengths, a 2-week “open” period falls on both sides of the Winter Recess. As a result, Neuro cannot be scheduled into this track since it needs to take place over 4 consecutive weeks. Here again, manual administrator work is required to tweak the schedule to make everything fit.

Before – rotations that require manual adjustments to fix the red blocks:

How to avoid this: Use a track lottery process and create pre-determined rotations for your students to rank. This way, each student is guaranteed to have a complete schedule and you can avoid struggling with any availability limitations that may be caused by holiday breaks or blackout periods.

After – three tracks of complete rotations:

I hope these tips come in handy as you are thinking about the best way to set up your program’s rotation lottery. Please get in touch with us if you’d like some one-on-one help with your rotation scheduling process. Also, check out our scheduling and lottery pages for more details on how these tools work.

Imagine a perfect curriculum– one that is tailored to your learning outcomes, with content sequencing that’s just right, teaching types that highlight your institution’s approach to learning, and everything assessed and evaluated in just the right spots. It’s a thing of beauty.

But there’s a problem. A curriculum is just a big learning plan. As new research is published and as practices change, you will be forced to make updates to curriculum content. Abnormal assessment scores will force you to revisit your sequencing and assessment approach. Inevitably, you will be faced with curriculum drift.

How do you prevent curriculum drift from occurring? How do you ensure that your carefully designed curriculum adapts properly to the forces pushing it around? You do it with a structured set of activities designed to assess and adjust your curriculum; in other words, with curriculum management.

Let’s take a look at the activities that make up curriculum management:

1. Curriculum mapping

Curriculum mapping often gets confused with curriculum management, but they’re not the same thing. Curriculum mapping is really just the first step in any curriculum management effort. It’s a process of tagging and linking the structural elements of your curriculum (phases, years, courses, sessions, etc) with each other and with your learning outcomes and content areas.

Curriculum mapping is essential to curriculum management because it transforms your curriculum from a series of documents and experiences into a searchable database that is an accurate representation of your curriculum. Without this database, the other steps in curriculum management are much less valuable.

2. Content & structure analysis

Once your curriculum is mapped, you can start to analyze and assess whether the content is still relevant, and if the structure supports your learning goals. Some common ways of doing this include:

Gap/redundancy analysis
A gap and/or redundancy analysis is a report that looks across your curriculum to find content gaps or redundancies. For example, you may have a redundancy because you’re teaching the same thing about diabetes in several different course areas. You may have a gap if one of your six graduation competencies is under-represented in your fourth year curriculum.

Teaching types analysis
A teaching types analysis is a report that looks across a segment of your curriculum to provide an overview of the teaching types and frequency that your program uses. This report is often a component of an accreditation review, but it can also be used to ensure that your program stays true to its pedagogical foundation, that students are given an opportunity to have different types of learning experiences, and that content is taught in a way most appropriate.

Sequencing analysis
A sequencing analysis is a report that lets you see how topics are sequenced throughout your curriculum. To do a sequencing analysis, learning objectives or content areas are tagged with a list like Bloom’s taxonomy. You then pull a report across the content area to see how the content is sequenced.

Content & structure analysis is essential to curriculum management because it is one of the main ways to prevent curriculum drift. If used on a regular basis, it helps identify missing, inappropriate and stale content so you can replace it with something more appropriate.

3. Student assessment

Student assessment is where the curriculum rubber meets the cold road of reality. Assessment results are the ultimate outcome of your curriculum. Because of this, they provide an important input into a curriculum management process.

Student assessment is essential to curriculum management because assessment results can give you insight into curriculum problems that may not show up in other result areas. Note that not all problems in student assessment are curriculum problems. Sometimes you’ll have issues with particular faculty, individual students, or with other non-curriculum things. But systemic student assessment problems (like missed results in a key content area, or over-performance in another area) can help you find issues that may have otherwise remained hidden.

4. Program evaluation

Program evaluation is essential to curriculum in the same way that student assessment is. Program evaluation provides your staff and faculty with real-time feedback about the curriculum. Not all program evaluation results will provide insight into curriculum problems (for example, a poor teacher evaluation is not necessarily a curriculum management problem). However, if you are diligent about asking curriculum-related questions on your program evaluation forms (for example, asking about content sequencing on a lecture evaluation form), you will have a great source of curriculum feedback to use.

5. Research, review, revise

The final step in curriculum management is the step that makes the whole thing work. You need to take the data you’ve gathered from your content and structure analyses, your student assessment results, and your program evaluation feedback, and decide whether or not curriculum changes are needed. This is frequently the main job of the curriculum committee and is not a task to be taken lightly. Many issues you’ll uncover–such as a deep-seated curriculum issue found through low student assessment scores– will require further research in order to decide how to proceed.

Now that we’ve defined curriculum management in terms of its key components, it should be easier for you to catch and prevent curriculum drift at each of these stages. What is your curriculum committee doing to manage your curriculum and prevent drift? Please share your ideas with us!

Last week, I attended the ICRE conference in Toronto. The ICRE conference is always a great opportunity to catch up with clients and old friends and to learn about how users are using the one45 system.

This year, one45 was once again invited by the Royal College to host a workshop at the ICRE Program Administrators Conference. I get really excited when program administrators approach us wondering how they can take advantage of all that one45 has to offer. Our presentation on “Tips and Tricks for Seasoned Program Administrators” was attended by close to 100 Program Administrators! Additionally, Joane VanBergen, the one45 Super Administrator at the University of Ottawa presented two sessions on “Going paperless with one45” which also received an impressive turn out.

I know that Program Administrators are very busy and it is difficult to find the time to explore the one45 system, so these sessions are a great opportunity to learn more about one45 and share best practices with colleagues. This got me thinking that it would be great to post some of the ideas that came out of these sessions so that if you didn’t attend the ICRE conference this year, you can still learn from how others are using the system.

Here are the top 5 tools and features that require a minimal time investment to set up in your one45 system, but will result in substantial time savings (in no particular order):

1. Attach handouts to your rotations. Do you manually send out your rotation objectives to learners and faculty each block? Eliminate this work by attaching rotation objectives in the one45 system. Learners and faculty can log into their one45 account to view this information at any time, removing the need to email the information. During orientation, explain that this is where the information is stored, and learners/faculty are expected to review this information prior to starting a rotation. You can even audit who has viewed the material! Click here for more information.
2. Use the “custom pattern” tool to send mid-point evaluation forms. Do you send your evaluation forms more often than just the end of the rotation? By default, the one45 system only sends evaluations forms at the end of the rotation (regardless of block length), but you can set up automated midpoint evaluation forms using our custom patterns feature. For more information, click here.
3. Send bulk email reminders to users with incomplete forms. Do you have a low completion rate of forms for your program? Sending out email reminders could help! In one click, you can send a reminder to all users that have incomplete forms. Click here for more information.
4. Set up rotation heads and mentors. Are you manually sending evaluation data to another program? Do you have any faculty that mentor your learners? Instead of manually sharing this information, you can give a faculty access to view evaluations, schedules, logs and grades of a particular learner directly within the faculty’s one45 account. Click here for more information. Also, you can eliminate the manual work of sending evaluations that your learners have completed on an off site rotation using the rotation head tool. Click here to learn more.
5. Send faculty feedback to them directly in the one45 system. Are you manually creating reports on each of your faculty each year? This important process can be set up in the one45 system with ease! Our releasable reports tool allows you to disseminate learner feedback to the To Do box of the faculty. It also has additional benefit of allowing you to track who has viewed their data and you can require the faculty to comment on the feedback that they’ve received. You can prove at accreditation that you have sent this information! Click here for more information.

Following the success of our ICRE workshops, we are considering a full-day one45 User Conference in 2015 to share tips like these and more. Would you be interested in joining? If so, please drop me a line to pre-register!

Everyone has probably witnessed disrespectful or disruptive behavior in the workplace at some point in their careers. Institutionally, however, we usually turn our heads and ignore the problem. In a survey quoted by A. Lorris Betz in his Leadership Plenary talk at this year’s AAMC Conference, 70% of participants indicated that they observe disruptive behavior each month, and 11% said they observe it every day.

Bad behavior is a symptom of a larger cultural disease. With a culture of disrespect where disruptive behavior is tolerated, a viscous cycle exists where the resulting hostile work environments lead to more bad behavior, dissatisfied patients, and an increase in malpractice suits. Worse, students and residents will take their cues from their faculty role models, and emulate those bad behaviors.

Changing workplace culture is a big undertaking, and it requires both leadership and a lot of hard work. Once you’ve decided to make a change, a great place to start is with a measurable goal – how will you know when the initiatives you put in place have succeeded? Once you have defined your goal, get started measuring how well (or not) you’re currently doing. You can do this by adding relevant questions to existing score cards, or assessment forms. For example, here are some questions you could ask in order to measure whether the change you want is happening:

• In the past 30 days, have you been a victim of harassment on the job?
• My department or agency works hard to create a workplace that prevents harassment. (scale of 1-5)
• I am satisfied with the way in which my department or agency responds to matters related to harassment and discrimination. (scale of 1-5)
• I am satisfied with the way in which my work unit responds to matters related to harassment and discrimination. (scale of 1-5)

Initially, the scores you collect might be quite a bit lower than you thought, but that’s okay – once you start measuring the problem, and seeing the results, you’ll have a baseline to judge the effectiveness of your efforts. Here are a few tips to make this process go smoother:

1. Make the data you are collecting public – the data should be anonymized, but you need to be transparent about how things are going. Quality improvement is a team effort, and your faculty, students, and staff will be much more likely to get on board if you’re not seeking to place blame, but trying to get everyone working together to improve results.
2. Share the results - doing this on a regular basis can be a great motivator for everyone as they see the measurements move up or down as you try new things to implement change. If you are consistent with reporting results, your team will also see how committed you are to making this a lasting change, and will be more likely to participate in the process.
3. Build in a little bit of competition – break down scores by categories such as department, or site, and share them. The people at the campus hospital aren’t going to feel good about the folks at the city hospital posting better results week after week. Use this opportunity to drive the cultural changes you want.

Creating lasting cultural change to eliminate disruptive behavior takes time, but getting started is as easy as coming up with a measurable goal and using tools like one45’s assessment and evaluation forms to start collecting feedback about how you’re doing. As you start collecting data and sharing the results with your colleagues, you will be able to measure the effectiveness of your change initiatives and get buy-in from everyone on the team. No one wants to work in a hostile environment, so make the most of the assessment tools available to you and start making a change today.

Today, we are excited to announce the official release of our calendar synchronization feature.

one45 users can now sync events from their one45 calendars to their mobile devices and calendar software. We officially support iOS (including iPhone and iPad), Android Calendar, Google Calendar, Apple Calendar, and Outlook for Windows. If you are running Exchange, you can also use Outlook for Macintosh through the web interface. If you’re using another tool to manage your schedule, and it supports common online calendaring standards, you should also be all set.

Once the calendar synchronization feature is enabled at your institution, you can set up your mobile device or calendar software to sync events from one45 calendars. The one-time setup on your device only takes moments and your one45 calendar feed will then synchronize and update automatically, so you always know where you need to be.

We’d like to thank everyone who gave us feedback on the project along the way, especially our great group of beta users. If you would like to get your one45 calendar on your mobile device, contact the one45 administrator at your institution to enable the calendar synchronization feature at your institution today!

So your institution has decided to purchase one45, and you’ve been selected as the one45 “Super Administrator”. What happens now?

Our clients are often surprised to learn that preparation for their one45 implementation actually begins long before any official implementation kickoff. That’s because we thoroughly document your program’s specific requirements throughout the sales process. When it’s time to implement one45, our Sales & Implementation team snaps to action and uses the information to devise a fully customized training plan for your program.

As your school’s one45 Super Administrator, you will participate in one-on-one training sessions that designed just for you. You’ll also receive a training manual that outlines the specifics involved in each training session and a list of required deliverables we’ll need from you to set up your system prior to training.

Once we’ve nailed down the dates for your deliverables, we schedule a “Kick-off Call” to introduce you to your dedicated Implementation Specialist, review the deliverables, and answer any questions you may have. We also provide links to templates to further help you gather information in appropriate formats for upload.

Here is an example of the items we would need if you were implementing our CurricMap tool:

• Your desired one45 URL
• Your school logo in vector file format
• Program Contact information
• IT Contact information
• Implementation Policy
• Mapping lists
• Learner lists
• Curriculum roles
• Buildings and rooms
• Curriculum schedule
• Mappings and objectives

These deliverables are due two weeks prior to the start of your training. During these 2 weeks, your Implementation Specialist will take the data you have provided, and prepare your one45 system so that we can train you within the context of your program.

Before you know it, the 2 weeks will be over and it will time to start training! Here is an example of our CurricMap training schedule:

By the end of your interactive training sessions, you will feel confident using one45 to meet the goals and requirements of your institution.

Here’s what some of your peers have said about their training experience:

“Central Michigan University is very pleased with the overall implementation of one45. They had provided us with an accurate calendar for implementation and were very flexible with the times to meet as we are all have very busy days… Special thanks to Tamara, as she was great in training and implementing our data into one45 and overall a great person to work with.”

-Thomas I, CMU College of Medicine

 

“Thank you to Wayne, one45 implementation Specialist extraordinaire, and the one45 staff that work with you to make one45 implementation as smooth as possible for even the most complex of scenarios. After only completing implementation in the last couple of weeks, our systems are already much easier to manage and report on and will be invaluable not only in everyday student admin management but also for accreditation reports.”

-Jodie K, The University of Queensland

Implementing any new software system can be a daunting task, but as you can see, we’ve got you covered every step of the way. If you have questions about an upcoming implementation, or if you’re curious about how to get the most value out of one45, contact us at any time. We’re always happy to help.

We are excited today to reveal a new reporting tool for curriculum administrators. The one45 gap and redundancy analysis report gives curriculum administrators greater insight into their curriculum by allowing them to see where competencies, instructional methods, assessment methods, (or anything else that can be captured in a mapping list) are covered in the curriculum.

Curriculum administrators can use the report to look for gaps in coverage as well as areas where particular topics are covered too much. With this report, you can see exactly which sessions and/or courses in your curriculum are associated with a particular assessment method, education method, or competency. You can also use the report to gauge whether topics are clustered together, or spread out in the curriculum.

To run this report, simply log into one45 and look for the new “Gap/Redundancy Analysis” report under Schedules > Curriculum > Print Views.

Give it a try today and let us know what you think!

This October, one45 would like to welcome our administrators to our home city of Vancouver BC with the first annual one45 Administrator Meetup.

The one45 Administrator Meetup is a one-day conference that brings together one45 administrators from all over North America. We’ve scheduled the event to coincide with the ICRE 2015 conference, also happening in Vancouver, to give attendees a chance to attend both conferences and reduce travel costs.

What attendees will get from the meetup:

• Receive one-on-one help from one45 experts to implement or optimize your processes
• Get a sneak peek into our product roadmap and learn about new features
• Receive comprehensive training on one45 (sessions for basic and advanced users)
• Learn valuable tips and tricks on how to maximize your usage of the one45 system
• Interact with top administrators at round table discussions, key note presentations, and a networking lunch

Check out the Program & Registration details here:
https://www.eventbrite.com/e/2015-administrator-meetup-tickets-8093005397#… Read More

I’ve read two interesting and practical articles about entrustable professional activities (EPAs) recently. The first article gives some great insights into how EPAs relate to each other, to training levels, and competency milestones. The second article which I came across from the KeyLime Podcast (episode 91) shares the experiences of a few Canadian Family Medicine Programs as they implemented EPAs.

There is much that the authors of the two articles agree upon. For example, both articles share a similar outlook on how competency milestones influence different levels of entrustment decisions. For example, to reach an oversight-only level of entrustment for a given EPA, a learner needs to have achieved the third milestone of a Medical Knowledge competency.

One of the challenges of EPAs is thinking about the information that someone making a summative entrustment decision needs to see. Clearly defined rules such as competency attainment at a given milestone provide some insight into some of what we need to do in one45.

I thought that the article by Ten Cate et al. did a great job at illustrating some of the other relationships between EPAs and the competency framework. Of particular interest to me was the concept that EPAs can be nested within one another, so that the EPAs a medical student is working to achieve fit into a broader set of EPAs for an intern, junior resident, and so on. The article also lays out how levels of entrustment might be restricted by level of training, which was a new concept for me.

I also learned something new from the article by Schultz et al. about how an EPA can by linked to phases of a clinical encounter. In their work, levels of entrustment are described, not just for the EPAs as a whole, but at each step of a clinical encounter, from hypothesis and history to treatment and follow-up. I haven’t seen this additional level of detail for entrustment described elsewhere, and it will be interesting to see what other creative additions people make to EPAs as they start to integrate them into their assessment practice.

Are there any other rules for entrustment not described in either of these articles that you’ve seen come up in your own practice, or elsewhere in the literature? Join the discussion in our user community: http://forum.one45.com/ 

With the new year now well underway for most of us, I am using this opportunity to start a fresh discussion about how programs like yours are using one45 for competency-based assessment. I’ve had the opportunity to talk to administrators and directors of a number of programs over the last few months and you’ve shared some of the ways that you are using one45 for competency-based assessment. I’ve also fielded a number of questions asking me about what other programs are doing, so I wanted to share what I’ve learned, and I encourage you to share some of your own experiences.

Here are a few of the things I have learned from my conversations:

1. Most programs incorporate competency-based questions on some, if not most of the assessment forms they use in one45.
2. Some programs use the competency reports (“spider graph”) feature in one45 to visualize competencies and to generate reports on learner competence across forms.
3. Some programs use the procedure log tools in one45 to track competence. A few use the procedure log tools for daily assessments like field notes or mini-cex. Competency grids and other advanced log features allow you to quantify exposure to and progress toward competence.
4. While some programs use one45 for daily evaluations or mini-cex, others use paper forms. In some cases, the paper forms are manually transcribed into one45, and in other cases the information stays on paper.
5. There are a few programs that have experimented with other tools, including SurveyMonkey and Sharepoint for implementing daily assessments.

The biggest pain points I’ve heard about are similar to those that often get brought up at conferences when discussing the challenges of implementing competency-based assessment.

1. It’s a lot of work to collect daily (or even weekly) feedback about all of your learners, and form-completion compliance is a huge problem.
2. Keeping track of whether your learners are being assessed often enough is difficult, and ensuring that learners are getting the right assessments is even trickier.
3. Faculty aren’t trained on how to do competency-based assessment, the rubrics are new, and time for faculty development is short or non-existent.
4. If you’ve managed to get past all the barriers described above, making sense of the wave of information you’ve collected, and reporting on competence effectively is hard.

I believe that the key challenge is to remove the barriers that prevent faculty from giving feedback. The two biggest barriers are ready access to the necessary evaluation forms, and the time it takes to fill out those forms which is why we’re building a mobile assessments app. With the app, faculty and learners will have access to request and provide assessments from, and optimized for the device they carry with them everywhere – their phone.

Your feedback

If you have any feedback, or want to share some of the work that you’re doing, I’d love to hear from you.

1. How is your program incorporating competency-based assessment into your workflows?
2. Are there any best practices you can share based on your experience?
3. Are there any workflows that you’ve tried that worked really well and are any that didn’t go well that others should avoid?

I’ve posted a copy of this message on our user forum and if you’re not already a member, you can sign up here. Please share this document with your program administrators, and other colleagues who are working on CBE at your institution – I would love to hear from them as well!

We are pleased to announce a new way for filling out assessments and surveys for our WebEval system.

The one45 mobile app allows faculty, staff and learners to complete their “Forms to complete” assessments from their phone or tablet devices.

Designed from the ground up with mobile device usage in mind, the one45 app gives people the freedom to complete evaluations from anywhere, and will make it easier than ever to get their work done and share feedback with their colleagues.

Our team is very excited by the release of our first native app and we think your faculty and learners are going to love it.

The app is available as a free download from the Apple iOS and Google Play stores.

Programs using DOPS, Mini-CEX, Field Notes, Daily Cards or other workplace-based assessments in their practice are going to love this update. We’ve made a number of changes to our “self-send” tools that make it easier to launch new assessments, and select the correct person to evaluate.

Your learners and faculty can initiate a new assessment directly from their To Do page. When you are setting up an assessment, one45 guides you through choosing the relevant rotation or clerkship, and will suggest the person you’re most likely to assess or be assessed by. We’ve even included a small picture of the person you’re choosing to make sure you’re picking the right person.

Before you launch the new assessment we now give you a warning if the person you’re assessing, or being assessed by isn’t in the schedule to help avoid choosing the wrong rotation, dates, or person.

We’re pleased to be releasing these great new updates on our web-app today, and we are already working on bringing the same great features to our mobile app, which will make it even easier to initiate these assessments. For more information you can check out our updated support documentation.

Good news! one45 users can now view their curriculum, rotation, and academic session events right in the mobile app. To access this new feature, users will need to update to the latest version of the one45 mobile app in their app store.

In the itinerary view, you can immediately see what’s next in your day, or use the drop down menu to browse events that are coming up in the next weeks and months, as well as those that have occured in the past three months.

To view event details, such as its location, description, facilitators, and group, simply tap on an event.

We hope you enjoy this update. Please visit our support site for more information.

one45 administrators, we’re very excited about our new support portal, and we hope you like it too.

Here are a few features that makes our new Support Portal awesome:

• Easier to use both from our end and your end
• Provides you the ability to screen record within the ticket
• Allows inline screenshots so you no longer have to click on a URL to view images
• Allows us to support you more efficiently by allowing more flexibility and automation

To access the new Support Portal, simply click the Support tab in one45, as you have done in the past.

The post Our new and improved Support Portal is here appeared first on one45 software.

This fall Anesthesiology and Otolaryngology programs across Canada will start transitioning to Competency by Design (CBD) – a new competency based approach for measuring and tracking resident performance. The CBD transition means changes in assessment workflows to frequent, formative assessments. The CBD transition also means changes to performance review by linking assessment data to entrustable professional activities (EPAs), competencies, and milestones.

We are very close to releasing our new evaluation workflow tools geared toward programs using one45 for formative and work-place based assessments. The upcoming update gives you the option to release the results of formative assessments available to learners immediately after the forms are submitted, even if you are using those assessments as part of a head evaluation scenario, where an assessment is fed into an end-of-clerkship or end-of-rotation assessment filled out by a program director.

For example, if you have a daily assessment or mini-cex form that your faculty use to assess learners on a daily or weekly basis, you probably want the learners to see the results of those formative assessments right away. You may also want those assessments to feed into an end of clerkship or end of rotation assessment filled out by a program director. The new workflow options give you the flexibility to release feedback right away, and incorporate it into summative assessments later.

More updates are coming before the start of the next academic year. We will be releasing new tools that will allow you to link questions on your forms to entrustable professional activities, competencies, and milestones; and new reporting options that will allow you to measure learner performance against those activities, competencies, and milestones longitudinally regardless of the form or rotation the data came from.

If you’re incorporating more formative feedback into your evaluation workflow, we will be offering two webinars to help you start getting ready for the new academic year. Register here:

Wednesday, May 31st, 2017
Preparing your Program for Competency Based Education in one45

Thursday, June 8th, 2017
Competency Based Education in one45 – Q & A with Product Manager, Jason Ladicos

And also check out this Quick Reference Sheet for preparing your Program for CBD.

The post New competency and workplace-based assessment tools to help you prepare for the new academic year appeared first on one45 software.

Exciting news! Registration is now open for one45’s 2018 User Conference!

The one45 User Conference is a one-day conference that brings together one45 administrators from all over North America. We’ve scheduled the event to coincide with the ICRE 2018 conference, also happening in Halifax, to give you a chance to attend both conferences and reduce travel costs.

What you’ll get from the user conference

• Receive one-on-one help from one45 experts to implement or optimize your processes
• Get a sneak peek into our product roadmap and learn about new features
• Receive comprehensive training on one45 (sessions for basic and advanced users)
• Learn valuable tips and tricks on how to maximize your usage of the one45 system
• Interact with top administrators at round table discussions, keynote presentations, and a networking lunch

Don’t miss out on the Early Bird Discount. Register today and don’t forget to tell your colleagues.

The post Registration is open for our 2018 User Conference! appeared first on one45 software.

Good news! one45 users, that have access to our Grades/Marks module, now have a brand new flexible reporting option.

With the introduction of the new Grades/Marks report you’re able to report across multiple gradesheets, customize the columns displayed, report on inactive gradesheets, and save the report for future repeat use.

Please share this information with your teams that use the Grades/Marks feature so they can take advantage of this new report option!

Please contact us if you have any questions.

The post Introducing the New Grades/Marks Reporting Option! appeared first on one45 software.