Sep 8 2021

Robust Enough in Instructional Systems Design and Learning Experience Design

Robustness

ro·bust·ness /rōˈbəstnəs/
noun

the quality or condition of being strong and in good condition.
“the overall robustness of national and international financial systems”
the ability to withstand or overcome adverse conditions or rigorous testing.
“we can examine the robustness of our results”

My definition of “robustness” is “fit for use and misuse.”

Robustness – Back in 1992 and 1994 at Ford

Back in 1992 I did a Curriculum Architecture Design for the Ford Motor Company and their Ford Design Institute Engineers. The project, Robust Engineering: Engineering Curriculum Design where “robustness” of automotive designs was the focus.

This was my 41st performance-based CAD project, and the architecture defined the requirements for both Instruction and Information (company procedures and documentation) for the various Target Audiences across Engineering at Ford.

In this project, I used my Facilitated Group Process (FGP) and worked with an Analysis Team composed of a number of Ford engineering upper-level managers, plus a few deans and professors from several universities that Ford hired their engineers from. First we collectively built the Performance Models detailing the Key Outputs, Output Measures, and Tasks that followed their Engineering Design Phases. Then we systematically derived the enabling Knowledge/Skill Matrices using my 17 Categories of Enabling Knowledge/Skills. And then finally we configured and detailed the modular designs for a series of ten courses.

I no longer have the documentation for this project, digitized let alone on paper (this was 1992). But I recall the Analysis Team liking my approach at first – but then felt it was too much as we detailed the Instructional and Informational content. I think they felt that the Instructors/Facilitators wouldn’t need as much guidance as what we were producing – and I think that they also didn’t see the need for all of the Application Exercises following the Information and Demonstrations – that I used in Lesson Mapping. They saw this as mainly a series of Lectures. Old school as it were.

I’m not sure what they really put into place as I was immediately off to some other client work, and seldom had time to look back and reach out to see how it all went. They did hire me again, two years later to do another Curriculum Architecture Design for Engineering on Project Strategy Development. That was my 52nd performance-based CAD effort and was an expansion of the work done for the Ford Design Institute in 1992, incorporating many other current vendors’ courseware products originally built for Ford, into this CAD design for a broader set of Ford Engineers.

Robustness – Further Back in 1981 at Motorola

I was a Training Project Supervisor, an employee, at the Motorola Training & Education Center (MTEC) when I first heard the following story from my clients, who were the leaders of Motorola’s Manufacturing, Purchasing, and Materials organizations.

It seems that warranty claims were going through the roof, for police radios, both the handheld radios and those installed in cars.

These were returned with two issues. The handheld radios were often missing the antennas, which had broken off. And the car radios’ knobs on the face were broken off, and they had broken mounts.

So the warrantee people went to investigate as these were incurring huge costs, and the Motorola approach to warrantee claims was, pay them, fix it, whatever it takes. But the recent spike in claims was causing that policy to be questioned.

Using, I assume, the best approaches for Problem Solving, popular in Training programs back-in-the-day, these investigators probably defined the problems, and asked others” what was new or different about this issue, and when did THAT start?”

Long story short, the investigators found that the antennas were breaking off when police used the handheld radio as a club, when they’d whack a perp (perpetrator) in the heat of the moment. They often didn’t have time to holster the radio to then grab and unholster their club. So, they used what they already had in hand. And so it wasn’t an issue of the antennas breaking off of the radios, as much as it was an issue of the radios breaking off of the antennas. They redesigned the radio/antenna and at the end of the manufacturing line there was a big squat cement block, and every once in a while someone would take a radio by the antenna and whack that cement block, to see if the radio would fall off. I was told this story on one of my factory floor tours as I visited one of the 30 manufacturing sites across North America, when we came up to the end of the line and my guide was prompted to tell me this little story – which then led him to telling me the next story.

Long story number two, also short. The investigators uncovered that due to police budgets across the USA being cut, officers seldom rode around on their patrols with a shift partner, as they were used to doing. The only time they’d be buddied up was if their patrol car’s radio wasn’t working. They’d still have their holstered hand radios, of course. So, it was determined was that some police, who had challenging areas to patrol, did not want to go solo, their remedy was to kick the radio until it no longer worked. Then they’d be buddied up with another officer to make their rounds. I cannot recall, at this late date, what Motorola’s response was, if anything, to this discovery, 40 years ago.

The first issue they re-designed for robustness. I cannot recall what, if anything, they did in the second instance.

Robustness in ISD & LXD

What I learned 40 years ago from these 2 Motorola stories, and then learned even more from my Ford experience, was applied was to my approach to Instructional Systems Design efforts at Analysis, Design, Development, and Pilot Test, was to look for the requirements for use, and abuse or misuse.

One would apply these to your LXD – Learning Experience Design efforts as well.

What are/were some of the Use Cases and MisUse Cases for the Outputs and Tasks that my Training (Instruction) was to convey to the Target Audience(s)? Did everyone have the same Performance Requirements – and was everyone at the same starting point, prior-knowledge/skill-wise? Or in what ways did they vary?

What were some of the Use Cases and MisUse Cases for the Job Aids (Instruction) and would it survive the Learning/Use Context? Or in what ways did they vary?

In Analysis I needed to be robust to Target Audience Job Responsibilities Variance – and to understand and document the Target Audience and Performance data, and in the assumptions that I would be making later in Design. And I had better test those assumptions with my client and stakeholders in my Gate Review Meetings at the end of certain phases in my MCD – Modular Curriculum Development approach to performance-based Instructional Development.

In Design I needed to be robust to Target Audiences’ Prior Knowledge – as some members might already know the Content – and either let them skip, or test out, or make them an integral part of the Training effort, as honor their adult learner need to contribute what they know, and to help others.

In Development I needed to be robust to Performance Variations – in the details provided by the Master Performers and Other Subject Matter Experts and determine that while there may indeed be more than one right way to perform Tasks to get to the desired Output, we’d need to settle on one initial set of Tasks to start with, and then look for when and how to introduce variations that might be better in different situations.

In Pilot Test I needed to be robust to Feedback – regarding errors of facts, omissions and overkill, to deal with my goal of getting my Instructional Content to be as Accurate and Complete as possible, while also making it as Appropriate as possible to avoid Extraneous Details that cause Cognitive Overload.

And in my final Phase, Revision & Release, I might be redoing or expanding on some Analysis, or Design, or Development efforts, and maybe even doing some re-Pilot Testing of my Instructional Content (Job Aids and/or Training), to see if it was now Robust enough, and ready for Deployment and/or making it Accessible.

Robust Enough

Robust Enough isn’t always 100% perfectly accurate, complete and appropriate – unless that’s what the Risks and Rewards at stake in the Target Audience’s’ Performance Context dictate is truly needed.

Then – that’s what you do. And Test and Test and Test….