From an exchange on Google + this morning – expanded a bit.
I was taught to turn the 4 levels (which Kirkpatrick never intended originally as “levels” back in the 50s) of Kirkpatrick’s model upside down: if there aren’t the Results expected, check Transfer – if there was none/little Transfer check Mastery in the Training – if there was none/little, check Reactions – and don’t ask 1st about what people liked/didn’t (room temperature, instructor likability, the fun quotient, etc.) – ask about perceived authenticity and relevance – and then if those were OK – then go to those hygiene factors: the teacher was a real jerk, the room was freezing cold/too hot, etc.
And if the learner wouldn’t know – as w/ new hires versus experienced-in-the-job learners – wait on those L1 evaluations for a while as they won’t be able to assess THAT until after they are in the job for a while. Smiles Testing practices have really screwed up a good approach – as well as view of these as levels in a 1-2-3-4 progression vs. 4-3-2-1 (or mix in a 5 for Phillips if you are so inclined – 4 was always ultimately ROI (or RONA, etc.) for me so I never found L5 necessary).
Here is a prior post on reversing those Evaluations:
In an Enterprise Learning Context – which “I believe” varies from an “Educational Learning Context” or a “Personal Learning Context” where the learning objectives may not always be able to strongly reflect, be directly aligned to the “on-the-job performance objectives” – with more exception for some Personal and some exception for Educational Contexts (Trade Schools should align better than 5th grade Math learning objectives – with any eventual on-the-job performance objectives – but people learning Fly Fishing should be able to see the direct link).
Back to the Fun Part…
I’m not suggesting that fun cannot be/should not be part of the design.
But that should be secondary to authenticity (again, in an Enterprise Learning Context).
I’ve designed/developed intense performance simulations (5 rounds of 5-part simulations over 8 days) for Learners preparing for and running Product Management Team Meetings and then documenting the Product Plan updates) that learners found fun – when that was never the intent. What the experienced (versus the newbies) saw was the authenticity in the grind. The job wasn’t fun (of course there were parts that were more enjoyable than others) – but it was kinda fun to practice doing something very akin to the real job – as they recognized it – in the somewhat safe environment of the Training – where mistakes of millions of dollars were simulations and not real.
And as Thiagi says – all the learning happens in the debrief (w/ structured and shared reflection) – we found that a common praise of that Training (I co-delivered the 8-day course 31 times back in the late 80s into the early 90s including 5 times in The Netherlands) – as grinding as it was.
A poem – written by one attendee (shared here) – spoke to the grind of the 8-days – and the learning. The class always scored very high with most – the more experience in the job the better – and newbies (a week or two on the job) scored it lower as they didn’t appreciate yet what the job really entailed. Especially if they hadn’t read the 100-page “Year in the Life” Product Manager Novel we created as an “advanced organizer” that was a prerequisite – that those who read found helpful to better prepare them for what lay ahead (a job chock-full of ambiguity and constant change) which my clients started to use to help those applying for the job self-select in or out of such a job environment.
Those interested in more about that overall design (it won an ISPI award in 1990 when ISPI was NSPI) by searching on “NS 1251” and “AT&T Network Systems” and “Product Management” here on my web site.
What was universally liked about the course was it’s authenticity – reflecting the need to produce a Product Plan in the mix of uncertainty of ever-changing data from cross-functional team members with different personal and organizational agendas and goals – and facts – under time constraints.
Back to the Digression…
Truly authentic – truly a grind – and fun for those who didn’t take the Learning session so dang serious that they could learn from their mistakes – and had to be defensive about their numbers being right when wrong. Again my client (told me years into this) that the reason he was able to drag other executives to the “Final Product Team Meetings” was that they saw the session as the best, authentic in-basket exercise and selection process for tagging those who showed real promise for advancing in their Product Management world – of over 1200 folks (and growing).
It was fun for some, not all (who stressed and felt a need to “be perfect” in a world without perfection – or where perfection was temporary as “that too would change.” Like the weather.
Fun in Training/Learning – in my opinion – and I’m still looking for the data – is not central to learning (as some claim) and may or may not be appropriate. Training someone to conduct Marketing events at college bars – should be fun. Training someone to diagnose ER situations (in some back room not in the lobby) may not be appropriate.
More about NS 1251…on Video…
Here is a video clip from a session delivered in The Netherlands. The classroom scenes start at the 9 minute mark and go on until about the 11 minute mark – the rest is a lot of time spent touring The NL in 1991 in typical American fashion: 3 countries in 2 days, etc. :)
A fun video about the Sales Management Process (setting sales goals using Muppets) – this was the only intentionally fun thing in the training – which had been created before the Curriculum Architecture effort and development of the NS1251 Course. 15 minutes.
Make it “fun” if appropriate to the Context.
But beware of “unintended consequences.”
# # #