Formative (Predictive) & Summative (Actual) Evaluations of L&D Stuff
1- If the ROI will be negative – Don’t Bother.
2- If you aren’t willing to ensure that it will Transfer – Why Bother?
3- If it is not both Effective and relatively Efficient for the learner – It’s a Bother.
Start with Predicting the Future and Then Measuring After the Fact
To improve future predictions. Because as we all should have learned by now, from that world renown philosopher, Yogi Berra:
“It’s tough to make predictions, especially about the future.”
And you’ll only get better with Practice & Feedback – because Making Predictions has its own Learning Curve. That’s why you predict … and then seek feedback. And apply what you have learned from the Feedback.
Think ROI – As If You Owned the Enterprise
And would you spend (invest) if it were all your money and … future losses or gains?
With the caveat that: you probably can’t afford it all.
Roughing out ROI – because it isn’t that exacting – is normally used to compare multiple opportunities if you can’t afford to address them all. And most enterprises have some “hurdle rate” over some “payback period” that has to be met for consideration.
Investing $10,000 – to get back $20,000 in Return – is different if it only takes 6 months to get that $20k back – than if it takes 4 years or more to get that $20k back. Comprenda?
Talk to people in your Finance organization about that – the ROI – the hurdle rate and the payback period to get yourself in sync – a.k.a.: aligned. And have them come in to share that with others – or do it yourself.
And beware – those figures might not have a long shelf life.
Don’t be a LEARNING CHAMPION first – be a BUSINESS CHAMPION first.
Be a LEARNING CHAMPION second. IMO anyway.
For Transfer: Prepare for the Worst and Hope for the Best
Prepare the Learner – with enough Practice and Feedback for the Context back at the ranch – so to speak. For performing in that context – or those contexts – that might include supervisors/ managers and peers – let alone whoever else is in the performance mix.
But don’t just make up Worst Case Scenarios that have no basis in reality. And don’t let your Hopes soar above the lofty clouds. Be real. Be authentic.
And just because “resistance is futile” is a fun phrase – and not everyone will resist – doesn’t mean you’ll be lucky. Again. Or this time.
This is why my preference is to conduct both Analysis and Design with a Team of Master Performers via a FGP – Facilitated Group Process – as they “get real” – as they live it – and they will help you and your evetual Performance Support and other L&D Content “get real” too. In fact – they’ll insist on it.
Because they dislike (hate) non-real stuff. And once they hear about Performance Support they strive to do most of what’s needed via that approach – and they’ll know when it can stand on its own – or when it might need a little L&D Content – with a little or a lot of Practice and Feedback. They’ll know better than anyone else with an opinion.
Effective & Efficient Learning
BTW – go for Effective 1st and let go Efficiency is you don’t have time and money to get it all. IMO.
Effectiveness and Efficiency in the Learning – not in the achievement of the Terminal Performance required.
That means if your L&D could have been reduced by 10% and still be as effective in getting 90% of the Learners to achieve 90% of the Learning Objectives – or whatever your hurdle rates are – but that would have taken another round of Pilot Tests to insure that what you cut wasn’t too much – wasn’t muscle and/or bone – let it go.
Unless you’ve got the time and money to get it to its absolute most efficient – without messing up effectiveness – let it go.
IMO it is the Accuracy, Completeness and Appropriateness of the Content/Experience – in First/Initial Learning and any Spaced Learning deemed necessary.
Pilot Testing – or whatever you call it – is critical to determining whether your stuff is Accurate enough, Complete enough and Appropriate enough – to meet the authentic Performance Competence requirements. Where the Learners are able to:
Perform Tasks to Produce Outputs to Stakeholder Requirements
The Analysis Team of Master Performers and Other SMEs can tell you what that is. And then the Design Team (of the same people or a subset) will help you design the stuff needed – Performance Support and L&D Content – that is not overkill-in-the-extreme.
And Pilot Testing with other Master Performers – and Novice Performers (or whoever can represent the targeted Learners) can help you determine the Accuracy, Completeness and Appropriateness of the Content/Experience – which the Novice Performers cannot do.
And to judge “does it work – does it cause Learning” is something that the Novice Performs can help you measure – whereas the Master Performers cannot.
It takes both group.
What Happened to Reaction/Happiness/Smiling Faces All Around?
You know – the Level 1 that all too often is the beginning – middle – end of L&D Evaluation? Reaction via Smiley Sheets?
Let it go. Just do it.
I’m pretty sure that if Learners learned actually how to perform back at the ranch – so to speak – they’ll be happy enough – if not smiling, while they perform all the ranch duties that you covered.
The rest – they can ask for some help from the other cowboys and cowgirls. Yahoo!
# # #