Are Development Ratios for ISD Efforts Meaningful?

You know the claims of 40:1 or 20:1 or 400:1? Is there meaning?

I don’t think so.

Just last week a Internet-Friend in India asked me: “What’s the basis of your development ratios for instructor-led content?”

Short answer: Detailed Project Plans…I used detailed project plans of tasks covering both customer and supplier roles, organized in 6 phases – and then, as necessary, make a phase-by-phase “adjustment for perceptions of ease or difficulty” regarding any insights we collectively have into the politics, accessibility of resources, client/stakeholder attitudes, etc. Then I could add up all the hours and develop a “ratio” of the total hours of everyone compared to the hours of content produced.

Medium Answer: But if I needed to quote a figure off the top of my head, to give them a budgetary figure…it’d be $40k per day for a full effort…including up-front project planning and kick-off, analysis, design, development/acquisition, pilot-testing and then revision & release.

Back in the 1990s we used a $40,000/day for 3-day programs as our “standard-educated-guesstimate” – for mostly skill-level, group-paced content with fairly sophisticated “simulation exercises” to be built – and adjusted from there.

That was the “off-the-top-of-my-head-response” to any client’s questions of “what is this going to cost me?” 1 day might be $50k. 5 days might be $180k. Those are “ballpark numbers” that I felt comfortable with. Those are always full blown ISD efforts, 6 phases including Pilot-Testing.

And I’d done enough less-than-full-blown ISD efforts on low hanging fruit for a lot less. And a couple of less-than-full-blown ISD efforts on high hanging fruit that (unfortunately, but predictably) led to Rework City.

Using a common/standard process approach…ISD is a process in my world…a “lean” process…makes it easy to ball park most of the 6 Phases in my PACT Processes for Modular Curriculum Development. All except Phase 4- Development/Acquisition.

Long Answer: The problem I have always had with development ratios is that no two groups seem to “count time” in the same way nor for the same set of roles/resources.

I had clients in the 1980s who claimed small ratios – “proving” how fast/efficient they were in comparison with other published ratios (40:1) – but they only counted their staff time and not the SMEs, learners in Pilot-Test sessions, etc. And they didn’t count project planning (because they did so little of it, etc.).

Some didn’t count analysis or design time – just development time. So Development Ratios have always been pretty meaningless to me. And it’s an average anyway…and you’d be darn lucky to hit those numbers exactly two projects in a row. The interesting question is the deviation around the norm.

As an equity-owner/partner in 2 consulting firms I had to plan and predict “all of the time” that would be consumed by myself and my staff to either determine a fixed fee amount where I would cover my costs and make a profit (without gouging) – or – bid it “time and expense” and be sure I was going to be very close to the upfront estimates when the dust settled at the end of the project and the final invoice went out to the clients whose expectations I had set!

And not only did we estimate and then accumulate/count all of our staff time as we progressed, I estimated the time for the client and their teams so that I could share that in the Kick-Off meeting with my Project Steering Team, so that they could make the business decision to do one of 4 things in the Gate Review Meeting concluding Phase 1- Project Planning and Kick-Off:

  1. STOP THE EFFORT AS IT MADE NO BUSINESS SENSE
  2. DEFER THE EFFORT UNTIL IT WAS MORE TIMELY
  3. CHANGE THE PLAN TO BE MORE APPROPRIATE
  4. BLESS THE PROJECT AND COMMIT THE RESOURCES

Again, I do detail planning by phases. See a prior Blog of mine on Detailed Project Planning: especially the last section of the plan…the “Task/Time charts.” See the second graphic above. See that there are columns for both the supplier’s time and the customer’s time.

It’s only fair that you count EVERY MINUTE that is spent on the effort, compared to doing nothing at all.

Check out this next graphic of a planning template we would use after the detailed planning efforts to discuss adjustments to the standard hours/days we used for standard efforts. Not all efforts were standard.

I would sit down with my PACT Project Manager (PPM) and review the background picked up on in conversations as we conducted all “planning input interviews” – if this was a new client/situation. If this was a familiar client then we would already know what parts of the effort would be easier or harder than “normal.”

This is how adjustments were made to the detailed plan’s total amount of time…from the “standard-educated-guesstimates” that were informed by dozens and dozens of projects that were always tracked “actuals-to-plan.”

lean-ISD
And in my approach to ISD – the wildest phase and hardest to guess up-front with great accuracy in an ADDIE-like MCD effort is Phase 4 “Development/Acquisition.”

Everything else is fairly predictable, phase-by-phase…but still can be affected by the client or the culture of the Enterprise, etc. – which is why we considered adjustments to any of the phases. But mostly those adjustments were for P4.

The types of media to be developed were factored in. If there was a video to be produced, that added to the development time. If sophisticated “simulation exercises” (both “not equipment-based” or “equipment based” simulations) that had to be factored in.

If the content was mostly “Awareness level vs. Knowledge level vs. Skill level – that was also factored in.

And to make that guess up-front prior to analysis and design we would speculate on whether we might have a “Lesson from Hell” or two in the mix once we got to P4.

That led to a different set of “uncovering questions” in the upfront Client Interview Guide we always used – with appropriate variation based on what we knew for sure or what we felt we needed to bring out for our client’s sake and thinking. Was there controversy about “the process performance ITSELF” or on “how this should be trained” or “in availability of SMEs/Master Performers, etc. – that would cause us the hellishness (use “Lesson from Hades” if Hell is too strong a word)?

Who Cares About Development Ratios Anyway?
My clients didn’t care about Development Ratios, they wanted to know the “cost” to them – which I provided – both our costs (out-of-pocket) AND their internal costs (time in this effort…which is one half of the old “lost opportunity costs” – the other half being what didn’t get done while they were helping on an ISD effort).

If everyone’s time is of equal value (and where would that be???) then Development Ratios might mean something. But I staffed many people typically on each project’s development effort (P4) once we had the design approved, and I paid my project planning analysts/designers more than my developers or graphic artists or word processing people…so the ratio did nothing for me.

Sometimes the Standard Ratio, in my case “dollars to output” versus “time to output” Is Really Overkill. If I thought I could walk into a client’s office and start with a blank Event Map – finish that – and then jump quickly into Lesson Maps and knock off the design (with analysis built into the design steps) in a couple hours, without all of the Phase 1 Project Steering Team efforts, careful documentation and review of analysis data before using it for design that too would be carefully documented and reviewed – then all bets were off.

If this was a real important set of content – then we’d probably have issues of having gone so fast that we would have rework later – and if this was low hanging fruit of little consequence then we could go even faster. And if this effort was “more of the same” – say a batch of 25 additional Product Knowledge sets-of-content, after having done 100 so far – then it may be of higher consequence – but much less risky (for rework). The development Ratios for those first dozen Product Knowledge instructional products would be very different than the last dozen in a series of a couple of hundred.

So what meaning or value is there in using Development Ratios?

None that I have come across.

# # #

Advertisements

One comment on “Are Development Ratios for ISD Efforts Meaningful?

  1. Pingback: How to Estimate Training Time and Costs | Ridge View Media – Jenise Cook – Instructional Design and e-Learning Development

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s