Feb 15 2022

Work in Progress: Worked Examples for Intangible Outputs

Something last week triggered me to reflect on Worked Examples for Intangible Outputs.

The Output Focus

I was thinking back to the training and certification I had done for General Motors University (1995-2000) on my ISD Methods (PACT) using Teams for both Analysis and Design – and how I had learned before that engagement to explain the Output Focus to my own staff at SWI (1982-1997) and then at CADDI (1997-2002), and to ISD staff that I had trained at Amoco, AT&T, Eli Lilly, and Hewlett-Packard to name just a few.

I was training them on how to conduct Analysis and Design using my Facilitated Group Process – my PACT Processes.

The Output is the first thing I try to define during Instructional Analysis. And then the Stakeholder Measures for it.

And that in my experience (IMX) most Master Performers and Other Subject Matter Experts did not automatically think about their Outputs – but they more readily thought about their Tasks (Activities/Behaviors) … so if your questions about “the Output” weren’t resonating with the assembled Master Performers and Other Subject Matter Experts … you should do with what I learned to do and that was to switch to the next column: “Key Tasks.”

This next graphic – is modified from a consulting engagement from 1986 … and was for a Curriculum Architecture Design effort.

What I learned was that if I switched over to the Tasks, I’d soon list a Task with the name of an Output in it. Like magic. And the Analysis Team would collectively say, “Ahhhhh.”

Or something like that. Then, they got it. And we began the analysis of every Area of Performance in the Output column.

Areas of Performance has been my label for what is sometimes called Major Duties, or Key Results Areas, or Accomplishments – since the early 1980s – unless my client had a preference. It’s a segmentation, divide-and-Conquor mechanism for creating a framework before one dives deep into the Analysis of an entire Job or Process/Workstream/WorkFlow.

ADDIE is such a framework (although I adapted that a long time ago as it was missing some elements (Project Planning & Kick-Off) – and didn’t have the emphasis that I desired (Pilot Testing) – and then seemed to include ongoing deployments and evaluation and maintenance (which I truncated off to other Processes/Workstreams/WorkFlows).

Here’s an example of a Worked Example and the Questions I use to develop such – adapted/modified from a consulting project I did back in the mid-1980s. (I’ve been doing this for a long time.)

Here’s a Worked Example Example

From a blank Performance Model Chart to a completed Performance Model Chart…

But Wait – There’s More!

One mistake that ISD Analysts, Designers, and Developers all too often make is to focus exclusively on the development of Instruction (both Job Aids and/or Training) regarding the Major Outputs – and would lose sight of the need to develop Instruction on some of the interim Outputs that were in the Process/ Workstream/ WorkFlow “path” on the way to the terminal Output for the Phase or Segment of the Performance.

This framework/segmentation scheme for the Areas of Performance of a Sales Rep – was also modified from that 1986 project.

This segmentation – of Areas of Performance (AoPs) – is an effort that comes before documenting the Performance Requirements and any Current State Gaps on a Performance Model chart – might inadvertently focus the Analysis Team on just the terminal Output – for example, on: The Territory Plan.

Interim Outputs

But there are interim Outputs in this AoP (a.k.a.: Major Duties, Key Results Areas, Accomplishments, etc.).

And some of those interim Outputs might be tangible (physical, kickable) while others may be intangible, such as Thoughts, Decisions, Agreements, etc. (Cognitive Outputs).

Those need to be determined too – and addressed in the Learning Experience Design that gets Developed into an authentic Learning Experience that reflects the Performance Requirements from Back-on-the-Job for the Target Audience(s).

In my approach to ISD/LXD, I would be cognizant of such interim Outputs – but I wouldn’t go after them at a detailed level in my Analysis Phase, or even in my Design Phase. I would defer getting those details once I was in my Development Phase.

Here’s my adaptation of ADDIE (from the late 1980s).

The reason for deferment (until JIT – Just In Time) is that IMX my clients would pare back some of the Performance we were to focus on as we proceeded through our project – leaving some to continue as Informal Learning (Un-Structured OJT) and our focus would narrow onto just the “High Stakes Performance.”

So I’d want to avoid Boiling the Ocean for a Cup of Tea (as I learned from the TQM movement of the late 1970s) and avoid the Analysis Paralysis that so many of my peers got caught up in while trying to capture EVERYTHING at an EXTREMELY DETAILED LEVEL – way – upfront. Which would then turn out to be a wasted effort – and expense.

I learned to defer that detailed Analysis to Just in Time – which for me was during Development.

ISD (via my ADDIE-like approach) is not a waterfall approach.

While many of my 19 books address this – the first 4 of these latest 5 do.

Thanks to Rummler & Gilbert

I first learned this kind of Performance-Based Thinking back in August 1979 at Wickes Lumber in my first job out of college – and this next graphic is from a binder of Praxis Workshop materials – which is one source for that kind of thinking.

Praxis was the consulting firm of the late Geary A. Rummler and the late Tom Gilbert, back in the 1970s.

More Recently

Last summer (2021) Mirjam Neelen asked me to do a video for her and Paul Kirschner’s session on “Worked Examples for Effective Learning Transfer” for the LDC – the L&D Conference put on by LDA – the L&D Accelerator professional group.

They also had Dick Clark (Richard E. Clark, Ed.D.) do one.

Here is Mirjam summarizing the session at the point where she introduces my and Dick’s videos.

This video of Mirjam is 4:53 minutes in length. She discusses how she deals with performance that’s not clear-cut and is somewhat vague.

Here is my video with Mirjam on “Worked Examples.”

It is 18:10 minutes in length.

Here is Dick Clark’s video with Mirjam on “Worked Examples.” She saved the best for last. :)

It is 18:35 minutes in length.

Worked Examples of Intangible Outputs

The secret is in making the intangible, such as Thoughts, Decisions, Agreements, etc., tangible.

That’s often as simple as writing them down -once you’ve uncovered them – when doing the detailed Analysis of Outputs & Tasks (both Behavioral & Cognitive Tasks).

Then you can ask, about the Outputs, “How would you know a Good One from a Bad One?”

And then you use your model of Stakeholders and standard Key Measures – for probing for any additional thoughts beyond whatever comes as a result of the simple starter question: “How would you know a Good One from a Bad One?”

“How would you know a Good Thought, Decision, Agreement, from a Bad One?”

And then you would determine the Behavioral Tasks – as an anchor during Analysis – and if you follow my approach (but only if appropriate to your effort) – follow that during the Development Phase by determining the Cognitive Tasks – that need to occur before, during, and after the Behavioral Tasks in the Process/Workstream/WorkFlow – or however your client refers to such.

And build THEM BOTH – the Cognitive & Behavioral Tasks – into your Instruction for each Worthy Output – and include them both in your Job Aids/Training, Resources/Courses, Performance Support/Learning Experiences, or however you and your client refers to such.

Another Video – With a Worked Example

See a Worked Example of a Lesson Map – starting at the 21-minute mark…

Books and More Books

As I wrote – 4 of my last 5 books address this – but the book on the 3 Ds of ThoughtFlow Analysis probably goes the deepest – as it addresses my approach to CTA – Cognitive Task Analysis.