Uh oh. Houston – we have a problem…
…my learning game didn’t prepare me for this!
We have recent experimental evidence that narrative educational games lead to poorer learning and take longer to complete than simply displaying the lesson content in a slide presentation.
I read this when it first came out – in an ASTD article – that someone in my online network brought to my attention. And it was no big deal – as it was what I had expected – it concluded what I already knew.
And – I had heard this “controversy” before.
But I reacted a bit differently when I saw an online (albeit, a somewhat mild) attack on Dr. Ruth C. Clark about her recent ASTD article:
April 30, 2012 – by Ruth Colvin Clark
Read the Article above and the Comments and her replies.
I won’t pinpoint my source – one of a couple of attacks that I read online – I’m sure that you can find it – out there in cyberspace. It is currently in the “echo chamber” of RT and Re-Postings.
And maybe “attack” is too strong a word.
But Ruth has been a respected Practitioner bringing and speaking publicly about the Research – in Instruction/ Training/ Learning – for decades (and decades). So I really dislike her name being dragged through the cyber-mud by some whose credentials don’t hold the proverbial candle to her shining star – in the biz.
And – to expect her to dispute and disprove “every other study under the sun” that “proves” that Games Teach – as one criticism demands – is a bit much. Really. And is somewhat telling in itself.
I didn’t check them out myself – “all those other studies” – and who was behind them and what their methods were. And if they were a “vested” interested party or not. But I know that the studies Ruth would review and trust – went through that kind of scrutiny.
About Ruth Colvin Clark
Ruth Clark is a specialist in instructional design and technical training, determined to bridge the gap between academic research and practitioner application in instructional methods. She holds a doctorate in the field and is president of her own company, Clark Training & Consulting. Her books and articles focus on various aspects of training and e-learning.
My Next Step
So – as I have known Ruth for almost 30 years – not really well, but well enough that she answers my phone calls and emails – an NSPI/ISPI tradition – I wanted to do something.
For I also knew that this entire area of hype – Games in Instruction/Learning/Training and Gamification – is another one of those Myths that will take a while – if it can be killed off at all. It might not because as you can imagine there BIG BUCKS involved and it’s just so darn INTUITIVE. Think of Learning Styles (designing for, and not I have a preference for) and the use of MBTI for job selection for future success. They will probably never be beaten back into the bushes – as there are too many vested interests pushing their research with the big bucks that see a sale born every minute.
And again – it’s just so intuitive – isn’t it?
Like Turning Away From the Skid.
Intuitively seductive – or is that – seductively intuitive? Whatever. And – more importantly than intuitively – not correct.
But I digress.
So – I emailed some colleagues who I know know the Research – the Evidence for and against stuff in the Learning Space – for their response. I knew that they know Ruth. And her work. And her focus on the Evidence.
One thing I expected back was some words about “Popularity Is Not Evidence” – because that is often the reason for things being valid in some people’s minds and writing. It is not! But it is what guides many. Too many.
Voices of Wisdom From My Crowd
One respondent was Dr. Richard E. Clark – an APA Fellow – who researches the research at USC’s Center for Cognitive Technologies for other Research Institutes and organizations such as the U.S. Army – who wrote me:
I agree with Ruth but the True Believers in Games out there may never accept the evidence.
I’ve attached my own review and it may be more extensive than Ruth’s. I’m also attaching some slides I used in a debate at AERA in 2011 on this topic. Yet none of this evidence convinces the ideologically committed gamesters. They variously argue that “the research did not study a game, I define “game” differently” or “I’d need to see the game that was tested, it was probably poorly designed” or … it goes on and on.
What I find interesting is that in my review I checked all of the internal research conducted by the military (arguably the biggest buyer of “serious” or training games in the US) and their own research reports indicates that games are not as effective or as efficient at training than other less expensive options (I’ve cited all of the military reviews that are available). Yet the military trainers manage to ignore their own studies.
There are some topics where so much money and ideology and personal expectations are at risk that reason and evidence is simply ignored.
This is why we need a commitment to EPB.
Regards,
Dick
Here is what APA Fellow Dick Clark sent me.
AERA_Games_Debate_Clark 3-20-2011-1 and SeriousGamesResearchET_June07
Dick can be reached at:
Dick Clark – Richard E. Clark, EdD
Center for Cognitive Technology - Rossier School of Education – University of Southern California
clark@usc.edu
And his Research Center – with many PDF papers, etc. – may be found at: www.cogtech.usc.edu
Another Level Another Voice
Jeanne Farrington, EdD wrote in her Performance Improvement Quarterly column - Volume 24, Number 2 / 2011:
From the Research: Myths Worth Dispelling - Seriously, the Game Is Up
Given the average cost of creating a serious simulation game, which can
start with low-fidelity games at $20,000 to $50,000 (Derryberry, 2008), but
easily reach $1 million or more (Clark, 2007; Derryberry, 2008, Sitzmann, in
press), those considering including simulation games in a training program
should have a strong rationale for doing so.
And…
Simulations, which are a hallmark of serious games (Tobias & Fletcher,
2007), provide practice outside the actual performance environment. Embedding
simulations in games can provide varying levels of fidelity to the
intended environment, making the practice more or less realistic.
Simulations are often used when the regular environment is not available,
or if the consequences of error are too high or too costly for people to
practice there. Serious games can also include a wider range of practice
options and can provide choices to learners during game play that might not
be practical in other settings (Aldrich, 2009). In addition, learners can be
afforded nearly unlimited opportunities for practice via serious games.
Although the study did not include controls for time on learning tasks,
Sitzmann (in press) found that those who had unlimited access to games
retained more than those whose practice time was limited.
And…
Although it may seem obvious that apparent similarity between a game
and real life is sufficient, it is not. The key to transfer with simulations is to
ensure that the cognitive processes learners use during game play are similar
to the tasks they are learning to perform (Tobias &Fletcher, 2007). The best
way to ensure this similarity of tasks is through cognitive task analysis (see, for
example, Chipman, Schraagen, & Shalin, 2000).
Dr. Farrington can be reached via her web site at: http://www.jfarrington.com/jeanne.html
It’s the quote in this last slide (above) from what Dick Clark sent me – that really connects with what Ruth was saying in her article…which was…
Despite the uncontested popularity of commercial games and a lot of hype in the training community, the reality is that there is scarce credible evidence on how and when to best use games to improve instructional outcomes and motivation. At this stage, I recommend games to implement drill and practice exercises for tasks that require immediate and accurate responses. Hopefully we will cultivate a more refined approach to categorize the features of games that best match various instructional goals, similar to the Bloom’s Taxonomy of learning objectives. If you are determined to gamify, I recommend testing a prototype version to evaluate its effectiveness and efficiency compared to a more traditional approach.
Let’s Not Play Games With Games in Learning
It doesn’t help when someone writes something along the lines of this (I changed it so don’t bother Googling it):
Reason #1: Per Neuro-Science – playing games causes the brain to release dopamine. And Dopamine creates Pleasure. And Pleasure is good for learning.
What!?!
Tell or suggest that – or the reverse – to a Marine after they’ve just finished up Boot Camp.
But – check with your insurance agent first.
The Bottom Line – Is The Bottom Line
Do Games teach? They can.
Learning objectives can be mastered – hopefully those objectives where authentic enough and not too generic/general. That they align very closely with the Performance Objectives – and don’t include a lot of extraneous things to learn/master that have nothing to do with terminal, on-the-job performance requirements – for that would simply provide for Cognitive Overload of the Learner – and that’s not a good thing.
And if they teach to the objectives they teach it both effectively AND efficiently – compared to other alternatives – or did it “have to be games” – and if so, “why” was that?
Was there a less costly or time consuming approach – and not just first costs – but the life cycle costs – that would include post development & release costs such as costs for administration and maintenance – over the entire life cycle?
That’s what a Serious Business professional is concerned about. Value over the life cycle. Not the cool factor.
And exactly what a Serious Games professional should be all about too.
Good stewardship of Shareholder Equity in the L&D space in not just a good practice.
It one of the key measures of your worth as a professional.
You’d think so if that was all your own money.
Treat the shareholders’ equity – the money – as if it were your own – and as if you were a serious business professional and not some fangirl or fanboy in love with this shiny game technology regardless of plain old business sense. That would be cool.
Use Games in Learning?
Just because you can – doesn’t mean you should.
Reset?
As always – it depends.
# # #
Like this:
Be the first to like this post.
