Have you ever wished you could get more data from your elearning than just who logged in, who completed a course and whether someone passed or failed?
Using xAPI with any digital learning program – whether an elearning module, a video, a digital form or checklist – can help you capture all sorts of data about your learning program and about your learners. What are people typing into an open text field? Are people using the “help” or “need a hint” feature?
Earlier this month, I had a chance to talk with Megan Torrance and Matt Kliewer from Torrance Learning and who specialize on building learning programs with xAPI. As part of this conversation, I also wanted to speak with someone who uses xAPI with their learners. Wendy Morgan, Senior Learning Strategist at the Frank Porter Graham Child Development Institute at the University of North Carolina, Chapel Hill has engaged Torrance Learning in building out learning programs equipped with xAPI. In today’s podcast, Wendy shares her experiences – from a user perspective – of why she wanted to integrate xAPI into her programs and the value it brings to her, her organizations and her learners.
Give the podcast a listen to hear more about xAPI from both a developer and an end-user perspective. Stay for a crazy competitive game of Kahoot to close out the conversation!
Introduction
Brian Washburn: Welcome, everyone, to another episode of Train Like You Listen, a weekly podcast about all things learning and development in bite-sized chunks. I’m Brian Washburn, I’m your host. And I’m also the Co-founder of this company called Endurance Learning. Today I am joined by three guests, so it’s a little bit of a supersized version of Train Like You Listen because we do these podcasts in bite-sized chunks, but today we have three guests.
We have Megan Torrance, who started a whole company called Torrance Learning. We have Matt Kliewer who works with Megan at Torrance Learning; he’s a Web Developer on a Learning Engineering Team. And then we have Wendy Morgan, who is the Senior Learning Strategist at the Frank Porter Graham Child Development Institute at the University of North Carolina Chapel Hill. Go Tar Heels.
I wanted to do something a little different today with today’s podcast. Not just talking about a concept– and today we are going to talk about a concept, which is xAPI- but I also wanted to look at that concept through two different lenses: the people who build out the concept, and that’s going to be the Torrance Learning team, and then the person who’s actually using that concept—Wendy at the University of North Carolina.
And so before we get to all of that though, today’s podcast is brought to you by Soapbox, which is an online tool that you can use for five or ten minutes. You can take care of about 50 or 60% of the work when it comes to developing a live, instructor-led training. You basically go to www.soapboxify.com, you tell the computer how long your presentation is, how many people will attend, whether it’s in-person or virtual, what your learning objectives are, and then Soapbox will instantly generate a training plan for you with clusters of training activities that are designed to help you accomplish all of your wildest learning outcomes. If you want to get more information, go to www.soapboxify.com.
That was a long introduction, but let’s get right to our guests. And as we always do, we start with our guests introducing themselves in exactly six words. And so, Megan, you’ve been with us before, so we’ll go ahead and start with you. We’ll give Matt and Wendy a little bit of time to digest this, but how would you introduce yourself to the audience using exactly six words?
6-Word Biography
Megan Torrance: So, Brian, we are recording this right after the Fall conference season has ended. So I’ve been to Dev Learn, I’ve been to Learning 2022. So, my six words, “I am so exhausted. Oooo, shiny!”
Brian Washburn: Yes. (Chuckles) That sounds like the conference season in a nutshell. Matt, how would you introduce yourself in six words?
Matt Kliewer: So I’ve always struggled with these types of things, but I said, “Staring into space and sometimes absorbing things” because it makes me think of this Super 8 video of me when I was a kid on a big wheel, and my sisters were just spinning around in a circle around me. And I think I still do that. I just stare and go, “Oh, what’s going by?” And that’s what I put.
Brian Washburn: Ad you absorb it, which is—
Matt Kliewer: I absorb it sometimes.
Brian Washburn: Sometimes. (Chuckles) Wendy, how would you introduce yourself in six words?
Wendy Morgan: Well, I’m in a really unique position where I support all of these really impressive research projects at the Frank Porter Graham Child Development Institute at UNC. And I also take all this really impressive knowledge that Torrance Learning has and try to put it into use. So I always feel a little bit like I’m kind of just sort of treading water and pretending like I know what I’m doing. So, my little six-word bio would be, “Still learning with lots of help.”
Brian Washburn: That’s perfect. And that’s exactly– I love that we kind of ended the introductions on that because that helps us get right into the questions. And one of the things that sounds like you’ve gotten some help with is this idea of incorporating xAPI into some of the learning programs that you’re working with. And so I’m going to toss this one, this first question, over to the Torrance Learning team. If you could just explain to the people who are listening who may not be familiar with what xAPI is, what is it, and what does it measure, how does it work?
What is xAPI and What Does it Measure?
Megan Torrance: Sure thing. And, Brian, Matt, and I will do what we usually do – is I give the non-technical answer, and then Matt can drive deep and make it– get the technical answer for people who are listening. And making the assumption that most of the people listening are in the learning and development space, I can relate xAPI to SCORM, right? So SCORM is really the data standard, right? Or the communication standard between eLearning courses and learning management systems. It keeps track of about half a dozen interesting things. It’s relatively shallow data, but it is absolutely consistent globally.
So any course authoring package, any LMS, any course developer on the planet can make training and eLearning that goes into any LMS, right? Which is pretty awesome. But we can only run reports on some very basic stuff. Did you complete it? How did you score?
Brian Washburn: Mhm.
Megan Torrance: Where did you last leave off, right? So we don’t have a lot of depth about any of that. xAPI is different. xAPI lets us track data about any kind of learning or performance experience. Doesn’t have to be an eLearning, doesn’t have to be in an LMS—anything we can digitize. Like SCORM, it still is– it’s just a data standard. It doesn’t do anything itself, but we can track all sorts of different kind of information. Really anything we want, we can track. And it lets us do like some extreme navel gazing on looking into one particular learning experience and seeing every click and every interaction if we want. Or we can level it up and look out and say, “Well, how does this learning experience and this learning experience and this on-the-job performance, how are they all related?” That’s the high level.
Brian Washburn: Yeah.
Megan Torrance: The parentheses and brackets way of describing it.
How Do You Start Using xAPI?
Brian Washburn: And then if we wanted to get more granular, if we didn’t want to stare into space, but we wanted to gaze at our navel, Matt.
Matt Kliewer: (Chuckles)
Brian Washburn: So how does it work? Like when– you don’t just go into Storyline and just click a button that says, “Use xAPI,” do you? Or what do you do in order to get it to work?
Matt Kliewer: So you can do that, but one thing it does need that is a little bit different than SCORM or other previous tracking standards is it does require a specific database called a Learning Record Store. Or it’s a service that has a database underneath. So, with SCORM, you’d put your course in your LMS and everything was fine. With xAPI, it can be that way, you just have to make sure that your LMS can support xAPI, and it will use a Learning Record Store to store all that data. So–
Brian Washburn: How would you know if your LMS supports it?
Matt Kliewer: So, you know, you can look at their documentation, and if it says, “We support xAPI,” then you’re good to go. But you might need to ask some detailed questions about exactly how it works there. There’s some details that we might get into that are– some trickiness to that. But if they say they support xAPI packages, you can just, you know, in your authoring tools, just say publish for xAPI and upload. That’s a good starting point.
Examples of Data xAPI Can Collect
Brian Washburn: And what are some examples, just kind of– and I know that we’re going to talk to Wendy about this in just a second– but what are some examples– you know, Megan, you mentioned you can kind of track every click, if you wanted to– what are some examples of things that people do want to track more than did somebody log in? Did somebody complete a course? Did they pass or fail?
Megan Torrance: Within an eLearning context, there’s absolutely like what did somebody do on an interaction page, right? And if you think about it in an eLearning space, right? A non-quiz, a non-scored interaction page, how many attempts did it take to get something correct? If there was a hint button, did they click the hint button? What order did they click on things? If you had a text box and people entered text, what did they enter? And so all those are eLearning kinds of things that we can do.
We can also use this to keep track of videos–when and how did people watch a video? When did they leave? Checklists or rubrics–how did people score on a variety of different topics and things. How did somebody interact in a virtual classroom environment? All sorts of different things that can be tracked using xAPI.
Brian Washburn: I love this. And so, I’m going to turn our attention to Wendy here for a few minutes. And the first question I have for you, Wendy, is: is this something that you even knew about when you approached Torrance Learning to work with them? Did they talk you into it? What– how did you find out about this, and what kind of caught your eye when it came to xAPI?
Deciding to Use xAPI
Wendy Morgan: Yeah, I knew about xAPI before I even took the role that I’m in now, which I took about seven years ago. And I in part took the role because it was a really ripe environment for xAPI to be necessary and also very useful. Before then I was working for myself and handling, in particular, one healthcare communication client, and they were having some struggles with trying to figure out the right LMS environment and what they should measure and how. And while I was helping support them as an instructional design consultant, because that’s what I was, I learned about xAPI. And in my former life, I was actually trained as a researcher, so that really spoke to me because SCORM was not doing it.
So, I really loved learning about xAPI. I took some– I participated in some webinars and tried to talk them into getting this done. And they– it was not a priority for them at the time. And then this position came about and even in the interviews I mentioned that I would like to get an LRS xAPI environment up and running for them because it sounded like what they needed. They were interested. And so I had buy-in before I even was employed.
Brian Washburn: I love that from the moment of the interview. So people knew what they were getting into. And you work at a research institute, so I’d love to hear a little bit more about the value that you’ve found when you are working on learning programs at the Frank Porter Graham Child Development Institute. What are some of things–
Wendy Morgan: You can call it FPG. (Chuckling)
Brian Washburn: (Chuckling) You don’t want the whole long–
Wendy Morgan: It might be, it just might get long.
Brian Washburn: It might be the last time I also mention it here, so. (Chuckling)
Wendy Morgan: Oh, okay. Sorry.
Brian Washburn: No, no, no, that’s totally fine. What is some of the value that you’ve gotten out of using xAPI in– you know, you had this vision when you came in from the moment of the interview. What’s some of the value that you’ve gotten out of it since you’ve started to incorporate it into some of the things you’re doing?
What Practical Value Can You Get Out of Using xAPI?
Wendy Morgan: Well, I will say that it definitely allows me to utilize strategy because you have to measure whether things work. And if you can’t measure the pieces of your strategy that you feel are very important to the outcomes, then you don’t know what actually resulted in the outcomes. And that’s the research piece of me that I kind of already had coming into it.
So everything that Megan mentioned in terms of what we can measure, we do that. We have designed absolutely intricate Storyline lessons for some projects that we measure– oh my gosh. I think in one we had something like 150 different variables in every single lesson and very, very detailed data reports coming out of that. And those were all used as part of a larger blended learning strategy that was extremely successful, FYI. And then, we’ve also just collected some texts and from simple prompts like how would you respond to this situation? And then we utilize that. We’ll follow up with the people and say, “Now here’s what you said. Why did you say that?” Kind of workshop it.
But the reason that the eLearning piece works is because they can be on their– you know, independent and think about it. It’s not in front of everybody in a room and all of a sudden. So, you know, it’s just helpful. So we have found an amazing amount of use from it. I feel like it’s important to explain why the environment was right for xAPI because in the workplace that I’m in, I support many different projects with all different funding sources and different audiences and different goals that don’t share data and that need to– they need to create training for external community audiences sometimes that are unknown.
So creating a learning platform that everyone on all of these projects would have to sign in and create a sign in, that would be just disastrous for the project goals because people wouldn’t do it. It would be a huge hurdle or barrier to their learners. So we get to collect data that is fairly seamless. It doesn’t intrude into the learner experience. We can create virtual simulations and then we can host those on websites that are already known and visited. So it’s crucial for us.
And I think one of the pieces that I didn’t really understand the value of as much when I began, but I have certainly begun to understand the value as I’ve come along, is that underneath all of this is a strategy. So we have a strategy for what we’re trying to achieve what are the learning– so what are the learning objectives, basically, and then what is the most ideal learner experience for that? And then how do we operationalize that? And no matter what the operational definition is, we can measure it.
Brian Washburn: That is so– such a fascinating answer, and I’m going to come back to you towards the end because it sounds like you knew about this and you had a vision to implement it. And I’m going to come back to you and ask for those who might just be getting started, what advice might you have for them. But before I get there, I do want to turn back to the Torrance Learning team. And we heard from Wendy in terms of some of the arguments for xAPI, and I don’t know if you have some other thoughts or you’ve seen other use cases for it. I’d also love to hear from you if you think that it should be used all the time or are there any arguments against incorporating something like this?
Are There Times When You Should Not Use xAPI?
Megan Torrance: So I’m a huge fan of xAPI, so I pretty much want to use it for everything. I’ll just kind of put that, I’m very, very biased in this case.
Brian Washburn: Totally.
Megan Torrance: There was a joke at one point that I had an xAPI golden hammer, and I was just kinda like looking, you know– roaming around the Earth looking for nails to hit with my xAPI hammer. But you know, I think there’s– it’s not super hard, but it’s not inconsequential to implement and to use. And so you need to be ready. If your business sponsor is not interested in getting more data, if your organization is not interested in paying for that learning record store that Matt mentioned, right? You probably already have software that could create learning experiences that use xAPI. But it’s that database, that learning record store that you need to have and you want to be paying for a real, live account when you run it. And most of them have free trial accounts, but you want to be paying for one when you actually start pushing this out to actual learners.
So you need to be ready to use that data, a use case to have more data. And so those not everybody’s ready for it, right? And that’s totally fair. When would I use it though. Man, If I need more data, right? And we’re kind of very used to, in the L&D space, we’ve only ever had SCORM. So we’ve gotten used to only having shallow data. So we’re not used to wanting or pushing for more data because we don’t know it could exist. And so we get caught in this cycle. So when you’re ready for more data, absolutely. When you’re ready to connect multiple learning experiences into one data set, absolutely. That’s a key place.
xAPI also supports collecting and storing and then transmitting data when you’re not connected to the internet. Matt built a beautiful project a number of years ago that stores data on a USB drive until you get to a place that has WIFI. And also does some really detailed bookmarking or progress tracking that SCORM just can’t handle most of the time. So there’s lots of use cases for it and a few when you might not be quite ready yet.
Can You Go Too Far When Using xAPI?
Brian Washburn: Is there a danger when you introduce this concept to people that they’re like, “I want everything,” and then they don’t actually use a lot of it, right? It’s almost like– I mean, you’re coming from conference season- it’s almost like when people are like, “Ooh, can I get the slides?” And people actually do some– sometimes people use the slides, but sometimes they don’t. And so is there a similar danger here with xAPI that people are like, “I want to measure everything,” and then they have too much, it’s overwhelming? Or they don’t ever actually use it?
Megan Torrance: You want to jump in on this, Wendy?
Wendy Morgan: Yeah. I think that’s exactly where a strategy comes in, right? Like, what data do you need? Why do you need it? How are you going to use it? Just get that.
Brian Washburn: That’s a perfect answer. Yeah, I think that a lot of times people oftentimes are exposed to something they’re like, “Oh, do it all.” And the idea of having strategy is a great way to stay grounded, I think, and have your North Star.
Megan Torrance: Well, and there’s a few ways to look at it too, right? So during the development of a project, right? When I’m in early release, UX testing, all that, I want all the data. I want to know, do people see that hint button, right? I want all that data. Once it gets released, that’s too much data, right? But I’m also, you know– I agree a hundred percent with having a strategy. I also love the serendipity of finding things in the data you didn’t know were there. But if you have too much data, you can’t find them. So there’s this tension back and forth, and it’s kind of– you inadvertently, Brian stumbled on.
Brian Washburn: Mhm.
Megan Torrance: I think the fundamental question is how much data is too much?
Brian Washburn: Yeah. And Matt, I’ll throw that over to you too, if you have anything to add here in terms of how much data’s too much? Or, you know, are there strong arguments that we haven’t talked about either for or maybe not using xAPI in certain?
Matt Kliewer: Yeah, I agree with everything that they’ve said so far. Especially, you know, if you– different LRS vendors will have different pricing tiers and you can start to pay a little bit more if you start to send millions versus the thousands that you actually need. And if you’re sending millions that you don’t need, there could be a cost there. And then, you know, there’s just the cost of just spending the time sifting through things you don’t need and eliminating all those things. So, it can be tempting to just say, “I’ll send everything and I’ll just sift through it later.” But it can also tend to just grow and grow and grow. And then people who are trying to analyze it don’t know what they’re looking for or looking at, so then you just kind of maybe wasted some time doing that, so.
Brian Washburn: And it’s interesting that, you know– I’m just looking at the chat that people are kind of entering as this conversation’s going as well. And there’s a couple of comments just about, caution, it sounds like, and feel free to expand upon it, that the default settings either in Storyline or in an LMS that generally support xAPI may not be as useful as being more strategic in the way that it’s being used. But, Megan and Wendy, I don’t know if you have anything else to add to that.
Megan Torrance: Yeah, I think, you know, out of the box– and Storyline’s xAPI support has gotten better. But, you know, you really need to be ready to use JavaScript or a JavaScript authoring– you know, a statement maker to be able to get better data, right? And to really customize the data you get out of xAPI. With LMSs, many of them do support an xAPI package launch and completion. But if you’re sending them, for example, Wendy’s 150 variables, many of the standard LMSs xAPI support doesn’t know what to do with that data.
Brian Washburn: Mhm.
Megan Torrance: So you really need that learning record store.
Brian Washburn: The last question that I’m going to end on here is really again kind of geared towards Wendy in terms of what advice you might have to offer anyone else out there who knows that they need to develop some sort of learning initiative, but they’ve never heard of, or they’re not familiar with, xAPI until maybe they stumbled upon this podcast.
Advice For Those Just Learning About xAPI
Wendy Morgan: Well, since they have stumbled upon this podcast, I would say learn a little bit about xAPI, but I’m going to start to sound like a broken record here, and it all depends on what your strategy is. So it’s going to go back to: what are you trying to achieve and what do you need in order to do that?
Brian Washburn: I love it. And we’ll leave the last word there. I’m sure that there’s a lot more. I mean, Megan has been doing the whole conference circuit, and I’m sure this has been one of the things that you’ve been talking about. So, if you want to know more information, I think that we have kind of the experts in the field, or some of the experts in the field, at Torrance Learning. Obviously Wendy has been using it kind of on the user end as well. So, please do kind of take a look at the show notes here because we’ll have linked to people’s LinkedIn profiles if you’re interested in more information, I’m sure that people would be willing to talk to you.
xAPI Trivia Challenge
Now before we wrap up, however, we’d like to have a little bit of fun and kind of see what our guests know when it comes to some just general maybe eLearning or xAPI trivia. So we’re going to play a round of Kahoot to end our time here together, and it looks like everyone is logged in. Are you able to see my screen?
Matt Kliewer: Yep.
Brian Washburn: All right, perfect. And for those who have never played Kahoot before or have never used it, you can always go to get Kahoot.com, which is how you set it up. It’s basically a gaming platform that we can use. There are free accounts, there are paid accounts, and the idea here is that, you’ll ask the question and people will answer. The faster you answer, the more points you get, as long as you’re correct. If you get an incorrect answer, you get zero points. So that’s kind of how it works here. And maybe we’ll show the video portion of this podcast when it’s up on the website. Or maybe people will just have to listen in, especially if people have subscribed to the podcast. So I’ll narrate it as well.
So we have five questions here. We have Megan, we have Wendy, we have Matt who are logged in and ready to go. We’ll have a few multiple-choice questions; there’s going to be a few true or false questions as well—give you a heads-up. And, if you’ve never played this before, you’re going to see some stuff on the screen and then on your phone or on your device. I believe what you’re going to see is just a color that corresponds with the choice and a shape that corresponds with the choice. You don’t actually, on your phone, see the actual tech that goes along with the answer. So be ready for it. After the first one, I’m sure that you will get the hang of it.
So I’m going to go ahead and hit start here. Go ahead and get into xAPI trivia with the first question in 3, 2,1, here we go, and it’s double the points, by the way, for this first question. What does the API in xAPI stand for? Is it accelerated process index? Is it application programming interface? Is it algorithm for inference? Or is it some Latin acronym? And it looks like everybody got this one correct. So the first answer, the API in xAPI stands for Application Programming Interface. Everybody got it correct.
We’ll see who answered the fastest because that person will get more points here. And right now on our scoreboard, we have Megan at the top with 1,486 points, Matt with 1392, and Wendy at 1108. There are four more questions to makeup all that ground.
This next question is two or false. Here we go. According to Skill Counter, which was the research that we took a look at, eLearning courses produce 86% fewer greenhouse gases compared to instructor-led training. Is that true or false? And we had two people say true, we had one person say false. The answer is true. According to this research from Skill Counter, eLearning courses actually do produce 86% fewer greenhouse gases compared to ILP. I’m assuming that it takes into account all the travel and transportation that is involved getting to ILP.
Let’s take a look at our leaderboard. We have a new leader—Matt has propelled to the top with two correct answers, Wendy, not far behind, and Megan is ready to make up the difference. And keep in mind that there will be one more question at the end that will be double the point too, so it’s still anyone’s game.
Our third question, probably the most difficult question. Who is the author of Michael Allen’s Guide to eLearning? Is it Megan Torrance, Julie Dirkson, Cammy Bean, or Michael Allen? Four giants in the field, in the industry. We are waiting for one more person to answer the correct answer to who is the author of Michael Allen’s Guide to eLearning is Michael Allen. (Chuckles)
(SIGHS OF RELIEF)
Brian Washburn: It was not a trick question. Oh, we got three correct answers there. And Wendy is getting the little icon saying that she is on fire with three correct answers, even though Matt also has three correct answers in a row. And Megan is starting to make up some ground here. We have our fourth question.
Megan Torrance: Watch out.
Brian Washburn: I know. If you answer faster, you get more points as long as you’re correct. Our fourth question. True-false. According to Josh Berson, the corporate learning market is valued at 360 billion dollars. That’s a big number. Is that true or false? We have all three people that said that was true, and that is correct. So, that’s a lot of money that is involved in the corporate training space. And there’s also a question of is it being put to good use? And I think that xAPI is one of the ways that you can actually start to get some data around that stuff.
So we have a final question coming here, and this will be double the points. It’ll be a tough question. I don’t even remember what this question is, but let’s take a look at the final question. Double the points, anybody’s game. In what year was the term eLearning first coined? Was it 1972, 1986, 1999, or 2010? Third person answered. The correct answer, and nobody got it correct, was 1999.
(ALL LAUGHING)
Brian Washburn: This one surprised me, too. When I googled this and learned this, it was in 1999. Before that, it was computer-based training or web-based training. But in 1999, Elliot Maisy used that term at a conference, and that’s the first recorded instance, I guess, of the term eLearning being used in mass or pop culture. So we have that, the answer.
So let’s take a look and see how we did here. With the podium, we have in third place, at a respectable bronze medal showing, we have Wendy with– who’s the kind of the unicorn with the eye on her head. We have Megan in second place, and the winner, and before we actually started recording, I said that there would be a prize. We will reward the winner with this lovely deck of playing cards. The What’s Your Formula? Elements of Amazing Learning Experiences deck of playing cards. So, Matt, stick around for a moment after we’re done recording and I will get your mailing information to send you your prize.
Matt Kliewer: Oooo.
Brian Washburn: Well, thank you everyone for indulging me in a little fun and kind of nonsense, at the end of this podcast with the Game of Kahoot. Thank you everyone else for listening to today’s podcast about xAPI and the perspective from both those who create it and those who are using it and developing that strategy. If you’re interested in learning more about a broad range of learning and development strategies, you can pick up a copy of the book that this periodic table and these playing cards are based on: What’s Your Formula? Combine Learning Elements for Impactful Training. And until next time, happy training everyone.
This week’s podcast is sponsored by Soapbox. Sign up today for a free demo below.
