Some clients do not like the idea of a Pilot-Test. They think that it can be “just a good excuse” for poor performance.

I’ve told any of my clients that expressed doubts about this that I would only plan on a 10% update in the next and last Phase of an MCD project (the ADDIE-like level of the PACT Processes for T&D/ Learning/ Knowledge Management. Unless of course the project context is more problematic and less predictable for a wide variety of reasons. But that happens less than 20% of the time I’d bet.

Sometimes we’ve had to relabel it – from “Pilot-Test” to something else. Usually to something from the client’s own world – using whatever label they actually use when they “test an entire product or system in some real world contexts” – and that usually stops the questioning about “why do this?” That helps everyone “get” why we too wish to do a “full destructive test” on our creation.

Pilot-Testing is a formal Phase in the MCD “Process” – a set of concepts. models, methods, tools, templates and techniques – reduced to practice, repeatable practice/application, predictable applications.

When we Pilot-Test we are basically looking to see if the Learners can do the Performance articulated in the Performance Models from Phase 2 – and that’s exactly what we would measure for in a level 2 and level 3 evaluation.

The 5th Phase of MCD has 5 Sub-Phases – whereas all of the other PACT Process of the 3 levels of design ( CAD) has 4 Sub-Phases – “5 has 5” I remind Project Planners I have developed.

And whether or not you need to “game your situation” and call what I call the “Pilot-Test” something else – something that resonates internally about what is this and why would we want to do this? – and/or simply use the first and or first few “deliveries” or “accesses” of this instructional content – are local decisions.

I have a structured report template for my PACT Project Managers to use when they get to this point. Of course, it depends on many other things things being structured/ architected/ engineered way up-stream in the PACT MCD Process. Just as with any well-engineered process. Templates can save time and money if they are not “over-engineered” and are “flexible enough” for the inevitable real-world project and client differences that are likely to face.

When all the piece-parts of your modular design have been put together, bench/alpha tested, they are ready for a systems test, a full destructive test, unless you rather go easy on yourselves at this stage and not find issues/opportunities for immediate improvement before making your content “generally available.”

It’s something that I’ve heard and read Bob Mager hammering on over and over again. Pilot Test it! Field Test it. Market Test it! Trial it! Whatever you need to call it – call it that. And then do that test!

It’s good practice – but as with any best practice it needs to be situationally adapted. Don’t let that stop you. Adapt it as needed!

And –  is it necessary 100% of the time? Always?

No. It is not needed 100% of the time.

When deciding if this is worthy of not – do a little Risk Analysis – and use the approach, calculations and language that your Enterprise and/or Industry uses when doing similar efforts on other products and processes.

Never do a step in a project just because someone before laid that out in some sequence. Think through what it accomplishes at what cost.

You wouldn’t need to Pilot-Test a new batch of policy procedure modules when prior efforts were successfully tested. Trust what everyone has learned and do “more of the same” efforts differently, for less cost, unless of course, there is great risk to be mitigated/managed.

# # #