Moments ago, I cleared security at London’s Heathrow Airport. As I was re-packing my bag with my laptop and iPad, I noticed this little machine.
I tapped the super-smiley button.
I wonder what they do with the feedback. I’m guessing they use it like many training professionals use similar feedback. If the percent of super smiley feedback is high, they probably advertise it internally, and perhaps even externally to demonstrate a job well-done.
Level 1 Feedback Limitations
The limitations of the “smile sheet” evaluation form are many. All they can really tell us is whether someone enjoyed their experience or not. Low scores should be a concern, but high scores don’t necessarily mean value was added. This sort of quantitative feedback can’t tell us why someone might give a low score. In the example at Heathrow Airport, I could hit the grumpy face, but it doesn’t help any of their supervisory or training staff improve anything. Did I hit the grumpy face because I had a terrible interaction with the security staff? Did I hit the grumpy face because I was pulled aside for random, extra screening? Did I hit the grumpy face because a group of other passengers claimed to be rushing to their airplane which was departing in 10 minutes and therefore they were allowed to cut the lengthy security line while the rest of us waited patiently?
The Question Matters
I know many organizations – large and small – that measure training success by post-training evaluation scores. I understand the reason – as training professionals we want some type of metric that can demonstrate our value. But the minute someone starts asking “tough” questions like: “what value is a 4.3 out of 5 adding to our company’s bottom line?”, the power of the smile sheet metric could quickly lose its luster.
I wonder if the Heathrow staff would get more useful data if they changed their question. Some ideas that came to mind include:
- How did today’s security experience compare to previous experiences?
- Will your flight be safer because of today’s security check?
- Were you treated with respect and dignity during today’s security screening?
The list could go on. I understand that in order to have people participate, they’re limited to one question, and it needs to be simple. But “How was your security experience today?” depends on so many variables.
When it comes to post training evaluation forms, I try to limit the number of questions I ask to three per module/topic:
- This session will help me do my job better
- The facilitator was an expert in this topic
- The facilitator presented the topic in a way that kept me curious and interested
Depending on what I want to get out of the feedback, I may also ask a question about whether or not each learning objective was accomplished. At the end of the evaluation form, I’ll also include these two questions:
- I felt engaged and participated throughout the day
- I felt my fellow attendees were engaged and participated throughout the day
Again, these types of “Level 1” evaluation forms are just taking a snapshot of participants’ subjective feelings of how things went, and including blank text boxes for attendees to write additional comments can add some clues as to why they gave certain scores, but ultimately the value of training initiatives should be measured by specific behavior changes or performance improvements. Those types of measurements require additional feedback down the road – from attendees and ideally their supervisors.
Nonetheless, evaluation forms like this can begin to offer a hint of the value that training adds… if the questions are crafted well.
What questions have I missed that you’re asking your attendees? If you create elearning, would you ask anything different in the post-module evaluation?
The Train Like A Champion Blog is published Mondays, Wednesdays and Fridays. If you think someone else might find this interesting, please pass it along. If you don’t want to miss a single, brilliant post, be sure to click “Follow”! And now you can find sporadic, 140-character messages from me on Twitter @flipchartguy.
