This week we had an opportunity to pilot a training program that we’ve been working on for the past two months. We were excited to unveil it before a pilot audience, especially because we had an opportunity to incorporate a board game into the module.
At the end of the pilot session, we realized that we didn’t quite hit the mark in our first draft. Yesterday, the Endurance Learning leadership team came together via Slack to debrief the experience.
Today’s post is a sort of “fish bowl”, an opportunity to take a look into the conversation that took place as we de-briefed this session.
Tim Waxenfelter (Chief Operating Officer, Endurance Learning, and moderator for this debrief): Thank you both for joining in today’s debrief. I know Heather is joining from Chicago and Brian is somewhere over the western part of the country, joining from an Alaska Airlines flight. You went to Chicago to observe a new training program in action. Watching and getting feedback on something you built is always hard. For today’s chat, let’s talk about what you were trying to achieve and what lessons you learned.
Heather Snyder (Director of Instructional Design, Endurance Learning): Yeah, we went to Chicago to observe a training that included a board game and some other activities.
Brian Washburn (Chief Executive Officer, Endurance Learning): We really wanted to test how that game worked. We knew we had plenty of good content.
Tim: Testing a game. Sounds like a good idea. Going into the session, did you have any reservations? Any anxieties?
Brian: Game design is always hard. We’d tested it internally, but I was anxious on how well it would land among busy professionals. We had spent two months with the content and game design so it obviously made sense to us. Would it make sense to anyone else? And to be honest, I know that I always want to make a session fun and engaging, and sometimes too much time is spent there and not enough time is spent on the content. I was anxious as to whether we were providing enough content.
Heather: Building an effective training game is a really tricky business. And if you don’t get it right, if there’s one thing that doesn’t work in the game, the participants will notice it immediately. You run the risk of having participants focused on the rules or on winning or on the one thing that didn’t work, and suddenly the who purpose of the training takes a back seat to the game.
Tim: When we tested the game internally, we had a lot of discussion about first impressions that participants may have. Did you see a reaction when they first got started?
Heather: I think the participants were a bit overwhelmed. The thing about game design in the training context is that the game can’t be an all-day, Monopoly-like affair. There are other things that also need to be covered, discussions to have, etc. You can’t have them spend the whole training session just trying to figure out how to play the game. On the other hand you can’t over-simplify or it’s kind of a crappy game.
Tim: So did people get lost in the rules of the game?
Brian: That comment did come up when we debriefed with the pilot participants afterwards. People said they felt they were spending too much time wondering if they were following the rules and not enough time actually engaging with the content.
Heather: Our aspiration was that participants would learn content as they progressed through the game. There were a number of trivia questions about key topics that were embedded in the game. In hindsight, they design as it currently stands is probably a better post-training knowledge check than it is a learning experience. They needed more content before we had them play the game.
Tim: Ok, so it sounds like the game didn’t land as you expected. Now what? Do you think you’ll stick with the game?
Brian: Ha! I actually suggested to Heather after the session that we chuck out the game and start with someone different.
Heather: Yes he did. And that would certainly be the easy thing to do. I think the first reaction when something is hard is to pull it. I’m going to caution against that – in general, and here in this case. I think it’s important for us to first re-visit why we chose a game as a way to accomplish our objectives in the first place. Was it just a fun idea, or was it a good idea? Either way, there’s going to be a lot of work – you either start fresh or you retrofit the game, but you don’t just have a knee-jerk reaction.
Tim: So if you stick with the game, what do you think you need to do?
Brian: The first thing we must do is re-visit the game mechanics we chose to incorporate. For example: points. We intended for people to strategically allocate and accumulate points in different categories. What happened, however, is that people just took whichever route would give them the highest overall score, regardless of category.
Heather: I’d like to work backwards through the course, taking is apart bit by bit while re-visiting the objectives, figure out why we started down the path we did, and then see if it all still makes sense. Working backward, we may find some better clues as to how to adjust the game.
Tim: It sounds like you’re getting at the part of instructional design that is more art than science.
Heather: Yes! There really isn’t a formula for what we need to do right now.
Brian: I think the science is: does what we’re doing meet the objectives? I think the art might be: are we willing to let go of something we’ve worked hard on if we’re not currently meeting those objectives?
Tim: Thanks for talking through something that may be a bit painful to discuss, the wounds still fresh – a training program you’ve worked hard on and that didn’t quite land as you’d hoped. Any final thoughts?
Brian: Game design. Gamification. Fun ideas. Hard work. May or may not always be appropriate.
Heather: Game design is a huge undertaking. You have to be ready for the challenge and let content and objectives lead your design.
Tim: We’ll leave it there. Thanks to both of you again. Good luck with the re-writes!
