Picture yourself near the end of a project lifecycle. Over the last several weeks you attended weekly meetings, had coffee with subject matter experts, adjusted timelines, submitted several drafts, made revisions based on feedback, and finally, your project is ready for an audience. Are you done or did you forget pilot testing!
Once everyone who is close to the content has finished their review and submitted comments, it is time to get fresh eyes on the training before it goes live. Piloting a training is a great way to get that perspective. If you really want a training to be great, you should get it in front of a pilot group.
Earlier this week we ran a few pilot tests in California. As I nervously walked the floor the night before, my husband asked me why I was pacing a hole in the carpet over a simple pilot. Witness accounts differ in what happened next, by I insist that I took a deep breath calmly attempted to explain the value of pilot testing. While I was definitely not yelling in his direction about how critical yet vulnerable a pilot is, I realized that many people may not understand why piloting a training is so important. As the designer who will not be facilitating this course, this may be my only opportunity to see training in action. Pilot testing training is a scary, wonderful, and in my opinion, necessary portion of the development of a training. Let’s take a closer look at why I believe training pilots should be included in all training development lifecycles.
Pilot testing identifies what went right.
As the instructional designer, most of my work is complete by the time I get to a pilot. When I observe a pilot, I get to sit on the sidelines and watch as people interact with the course I have spent several weeks creating. I get to see what happens when the facilitator hands out a worksheet and gives a minor set of instructions as per the facilitator guide. I watch as people pick up their pen and start working or stand up and start on a small group activity and their wheels start turning. As activities are debriefed, I listen as people tell stories about their experiences relevant to the materials, and ask insightful open-ended questions. I see eyes widen as realizations are made while enthusiasm grows around the content, I see the vision I had for the training come to life and I make a mental note to use these strategies again in a future training.
Pilot testing identifies what went wrong.
Pilot participants should know that they are there as a part of the development process and that their feedback is expected. I never approach a pilot thinking I won’t get feedback. I want my training to be solid, but feedback is what separates pilot testing from live training. No matter how great the training, some amazing pilot participant will mention an idea that improves it. Pilot feedback does not mean the training is bad, it means there is an opportunity. Along the same lines, sometimes pilot tests can fall flat. This is also not a failure. A failure would be not piloting a bad training and releasing it just to hit a deadline. It is better to test and fail than to not test at all.
Piloting doesn’t need to be complicated. A small group of participants who are not subject matter experts (SMEs) should take the training with a facilitator who ideally is not the instructional designer of the course. It is better to have the ID observing the room as the course is facilitated. The ID knows how the course should look, observing how it does work helps them understand what adjustments are needed to complete the course.
Pilot Test Debriefs
All pilot testing should wrap up with at least four debrief questions. Pilot testing debriefs do not need to be complicated, they just need to generate a conversation in the room where participants feel comfortable giving candid feedback. The following four debrief questions work for most situations.
- What went well?
- What are some areas of opportunity?
- What is your biggest take away?
- What was missing?
You may need to create customized debrief questions for your pilot. If that is the case, make sure you keep questions open-ended. The value of pilots is the candid feedback from participants which comes from thoughtful open-ended questions.
How do you use pilot testing in your project lifecycle? What have you learned? Let’s talk about it in the comments below!
