As the Analysis Framework for Ideal Performance and Gap Analysis

Introduction to AoPs

When modeling performance, it is critical to establish good Areas of Performance (AoPs) up-front in the process. The AoPs create a configuration—an organizing scheme for the performance data that will be captured on Performance Model (PM) charts.

There are many means/paths to the ends of AoP creation. You, the analyst, can create them, or you can facilitate an Analysis Team in creating them. It depends on your desire to have your customer really own the AoPs versus you the supplier owning the configuration. As long as I can use it for my downstream needs, I would rather that my customers own the configuration of their performances!

The example below is a set of AoPs for an account representative (salesperson) for the American Boat & Canoe Company (the ABC Company).

Each AoP is then defined further via the Performance Model charts.

Here is another example.

A Performance Model (PM) chart is a device that captures specified data in a specific format about human performance within business processes. The example chart below is one of many pages of PM charts articulating the ideal and gap performance of ABC account representatives.

One of several pages for AoP #1

AoP Definition and Examples

AoPs are the segments of the total performance. They create a framework for the performance (or the piece parts of the performance) that when all added up, represent the whole job (whether it’s a job, a role, a process, or a departmental function—whatever scope the PM effort is charged to address).

AoPs are more process oriented than content oriented. They reflect major output and task sets.

Some good examples of AoPs are shown below. The first set is linear, and the second set is nonlinear. Both sets are appropriate given the specific performance they were intended to capture.

Some examples of nonlinear AoPs are also illustrated – above.

AoPs need to provide a logical structure for looking at the work performance. A set of AoPs created by one analyst would certainly vary from AoPs created by another analyst for a similar performance—the variation between the background and experience of the facilitator and the team members pretty much guarantees it. There are probably approximately 88 good ways to segment performance . . . and 88,000 bad ways. It’s not an exact science, and it’s rarely easy, but there are ways to make this work reliably with a group.

Creating the AoP Structure/Framework

Creating the AoP framework for analyzing performance is like dividing an elephant apart to study it further. An elephant is too big to analyze all at once. As the saying goes, “Don’t bite off more than you can chew.”We could break the elephant down into thirds: front, middle, and rear. Or we could divide it by top, middle, and bottom. Or we might divide it into proper scientific classes of anatomy. Or we could focus in on just its front end, or just its legs and feet, or just its skeleton.

What’s correct depends entirely on your terminal objectives that caused your need to analyze the performance elephant in the first place. Our terminal objective is usually to design, develop, and deploy performance-based T&D within our PACT Processes, lean-ISD methodologies.

It’s also critical that your customers perceive the AoPs as appropriate and complete in representing their performance.

How do you judge the appropriateness and completeness of a set of AoPs? As almost always, it depends. It depends on the scope and terminal objectives of your project. It depends upon the Risks and/or Rewards inheirant in the Current State situation.

To challenge your first cut at a set of AoPs, ask yourself the following:

• Does it directly conflict with or contradict any other prevailing/established models of the process or performance? Will this politically sell? (We want it to!)
• Do all the tasks and outputs of the process/performer fit within this AoP framework? Are there any missing AoPs? (We don’t want any!)
• Does this AoP framework minimize all overlaps and gaps? Will the same outputs or tasks fit into multiple AoPs? (We want it to be very clean and clear, with no overlaps and gaps, if possible.)

But what about “voice of the supplier” (VOS)? Do we get a say in the appropriateness or completeness of the AoPs? Yes, we do!The approach we take in our Analysis Team meetings is to ask the team numerous front-end questions as we waste a page or two of flip chart paper attempting to create some semblance of order out of the many things performers do in their jobs. That gives us “voice of the customer” (VOC).

As the T&D supplier, we have to judge the set of AoPs we are to work with in terms of how well this AoP performance framework will allow us to further analyze the performance targeted by our project.

Will the AoP framework assist us in specifying ideal performance, conducting a gap analysis, and then deriving all of the enabling knowledge and skills? Or will it hinder us? And finally, will it help us later in our post-analysis efforts to configure and sequence instructional content, detail the design, develop the instruction, pilot-test and update, and then finally deploy performance-based training and development (T&D) that has process performance impact for the good? Will we end up with a positive ROI and EVA when the dust settles?

If the AoP framework will help us, cool. If it will hinder us, then start over, even if the Analysis Team loves the non-useful AoP configuration! This may be difficult to sell, but if we ISD suppliers can’t use it as a means to our ends, what good is it?

Utility of the Analysis Team

If we want our Performance Models to be good, we need to ensure that the framework we start with will serve us well and not lead us inadvertently into a blind alley or deep ditch. We always take our time at the front end of our Analysis Team meetings to warm up to the group and create this framework with their assistance.

The Analysis Team is there to help us help them, not to impede us. If we can’t do our job with the models that they demand we use, they hinder our ability to serve them. We must often sell them on helping us meet our needs as well as meeting their needs. This may take some time at the front end of your Analysis Team meeting.

Get your Analysis Team master performers and subject matter experts engaged in this PM process! They usually do want to play, even if they are resistant at the start. This is their world we are trying to model, and it usually is really important to them to get it right. They simply don’t often know what we are looking for as we start, and it feels awkward to everyone (including you, the facilitator-in-charge).

Going slow to go fast is our motto.

Pre-Analysis Meeting Preparation

Ideally, you would have had some familiarity with the performance you are to model prior to attempting to facilitate the Analysis Team meeting. You may have had the opportunity to

• Read job descriptions (which usually won’t tell you much, but it is sort of a “due diligence” task you should complete).
• Observe some of the performers in the work environment(s).

• Interview performers at varying levels of skills and experience and at different ends of the work process and organizational hierarchy. You might talk with one or more

–    Master performers

–    Average performers

–    Rookie performers

–    Supervisors/managers/executives

–    Customers

–    Suppliers

You may build a straw model of AoPs for sharing with the Analysis Team to jump-start the meeting. Or you may decide to keep the straw model in your back pocket and not share it with the Analysis Team because you perceived a potential issue with sharing/imposing it on them and think it best to use it only if the group gets stuck. (Too often the use of a straw model of AoPs inhibits the Analysis Team at the early stages of the meeting, and they aren’t willing to challenge a model that they may find suspect, for whatever reason.)

You may present your straw model, ask for challenges/suggestions/comments, and then move on assuming acceptance when/if everyone is quiet. Maybe they are deliberately thinking through the underlying complexities of your straw model as it compares and contrasts with their other models for how they look at or think about their performance. Give them time. Go slow to go fast . . . later.

Keep in mind that everyone has at least one mental model for what they do in their jobs. They may have gotten theirs from the organization they work for, which may have articulated a model of some sort. They may have been through an extensive process mapping exercise, and the process map now represents how they view their world of performance.

Or maybe a consultant gave them one many (or a few) moons ago, and it’s how they now think and converse about their world. It may exist. Do you know if one already exists or not?

If they’ve got one, great! Latch on to any prevailing models first, and try to make them work for you. That is the politically astute thing to do.

But, if they’ve got a model or framework for their world and you feel it doesn’t work for you, for your needs downstream in our PACT Processes, you’ve got an issue (or a problem or opportunity, depending on how you like to look at your half-filled glass).

You may have to sell them on why you need something different, and then explain what you’re looking for, how you will know in the end if it’s any good for your needs or not, and what it is that is salvageable about their model (if that is true).

Be good. Be lucky. Be whatever it takes.

But don’t allow yourself to get saddled with an inadequate set of AoPs to start your analysis effort. That’ll be the slow kiss of death for your overall effort.

You’ll see and feel the real pain of poor AoPs when you later

• Define the ideal performance and existing shortfalls/gaps.
• Derive the enabling knowledge and skills.
• Assess existing T&D for its fit and appropriateness.
• Read out your analysis data to your Project Steering Team.
• Use the AoPs to configure the Curriculum Path’s AoP-derived Modules and Events in the Design Phase.

Hopefully you find a set of client data that works from the get-go or enables you to create a straw model that will help and not hinder you with the Analysis Team in the analysis meeting.

Otherwise, you’ll need to build your own set of AoPs with the Analysis Team looking over your shoulder as you waste a page or two in the slow start of your analysis meeting. Remember that their role is to judge your work as you do it, to build the quality in rather than attempting to inspect the quality in later. That’s why they are in the room with you—to help guide your activities and outputs and to reduce your later rework.

Either pay now or pay later; and paying later always costs more!

Analysis Team Meeting Tips for Getting Started

We start the effort with the Analysis Team by declaring that, “We are going to waste a page or two (of flip chart paper) just to get started.” The first part of the process, developing AoPs, is the most critical and the least procedural—it may/will feel very messy to the group. They may not be sure any of this is going to work until you are well into the Performance Model charts. You will feel their need to speed up and get to the charts—but don’t give in.

It is important as a facilitator to announce your intentions to the Analysis Team, to think out loud, etc. Allow no surprises, help the team understand what you’re looking for, and anticipate the next steps. They’ll like it better if they don’t always feel like you’re making it up as you go, as if you’re leading them and you don’t know where you’re going or how you’re going to get there.

Always tell them what you know about their performance and how, where, and from whom you received your information. Then attempt to place a stake in the ground. Ask them for a big task-set, a major portion of their performance.

Write the first input you get on your doublewide flip chart easel paper, squarely in the middle of the page (unless it’s obvious that it will end up near the front- or back-end of your AoPs). This is the “stake in the ground.”

Then ask what happens upstream before this, and backward chain additional potential AoPs (and be sure the team understands that they are potential AoPs, not necessarily the final version at all). Exhaust that until you bump into the beginning of the performance, and then go back to the stake and work downstream.

Once you have defined end-to-end performance, probe for other things they do that aren’t yet captured on the chart—sometimes things like maintenance duties, housekeeping, handling emergencies, etc., don’t come out if the group is focusing on a linear work process.

Lead the group—this isn’t a pure facilitator role or open brainstorming approach where the group contributes input and you simply scribe it. Instead you will need to propose ideas, rework ideas, probe for clarification, and rationalize changes so the group agrees with what is emerging on the flip chart page. You may often be forced to capture seemingly contradictory and/or irrelevant suggestions and then integrate them in a way that doesn’t alienate the participants but that, even more importantly, doesn’t pollute the AoP framework. (Remember, you and the rest of the group will be living with these AoPs for the rest of the meeting as well as downstream in the project.)

The key to analysis is recognizing patterns—try to look for and understand the following:

• The heart of their performance

–    What is the “core” performance—what are they on the payroll for?

−    For lumberjacks and lumberjills it would be “sawing down trees”

−    For product managers it would be “managing a product through its life cycle”

−    For sales reps it would be “selling to customers”

• The prevailing cycles of performance

–    Is there a part of the job that continually repeats itself?

−    For lumberjacks and lumber jills it would be “planning and sawing down trees” and then doing it again, unless they have to drag the trees off to the sawmill.

−    For product managers it would be “managing a product though its life cycle” and then doing it with other products simultaneously, each at various stages of their life cycles.

−    For sales reps it would be “preparing for and making sales calls on customers” and then attempting to sell to the next customer. Also one hopes there is the “paperwork” of closing the sale portion of the job.

For example, if you were analyzing a TMC store manager role, you might have made “Financial Management” the AoP and lumped payables, banking, and tax returns (which have different cycles) together. If you were analyzing the bookkeeper role, you would probably use separate AoPs because you would have a different level of focus and detail on these different areas.Some jobs have multiple cycles—managing a store has daily cycles like opening and closing the store, monthly cycles like closing the books, and annual cycles like taking the complete store inventory. Work performances with vastly different cycles usually need to be kept in separate AoPs. But sometimes it can be better to cluster performance of like kinds and separate cycles at the output level.

• The prevailing linear flow to their performance

–    Is there a dominant flow to their job performance?

−    For lumberjacks and lumberjills it could be something like, “find marked tree, plan tree felling, make first cuts, make secondary cuts, fell tree, saw per job order, etc.”

−    For product managers it could be “develop product concept for management approval, assemble cross-functional product team, develop full-blown business case for approval, conduct marketing research, design product and manufacturing process, ramp up and implement the manufacturing process, develop sales channel and sales support system, develop service channel and system, manage product through its life cycle, prepare and plan for product discontinuance”

−    For sales reps it could be “portfolio and account planning, territory planning, sales call preparation, sales call conduct, sales call follow-up, sales administration”

• Any cycles within cycles within each major AoP?

–    If any one (or a few) of the potential AoPs is at the heart of the performance, then there are probably AoPs within.

−    For sales reps it could be that “sales call preparation and sales call conduct” are the heart, repeating over and over again. We may need to break these out further.

Sometimes the physical layout of the workplace can provide clues, especially in production work. Often the work process makes its way from one end of the plant to the other with clearly defined “handoffs” designating the boundaries between AoPs.  In the same manner, you may see clues in the organizational structure—separate functions, roles, titles, etc., may indicate different work.

Depending on the types of jobs you have had prior to becoming a “performance modeler,” you may be able to identify similarities between the performance you are analyzing and your own experience. For example, have you ever worked in a fast food restaurant or gas station? If so, you should be able to conceptualize at least some of the customer service elements of a bank teller or customer service rep or store manager or airline counter clerk role.

One last general consideration is the overall nature of the work you are analyzing. Try to get an overall sense of the job, such as whether it is a job with many routine repetitive tasks or if it is one in which the performer has to first plan, then execute a string of complex, time-consuming tasks. Is it a job with many little miscellaneous tasks and fire fighting, or is the performer worrying about the “big picture?” What is the time frame of the job—is it managing projects that span months or managing a shift every day or managing a job every hour (like a print shop)?

What is the primary accomplishment? Do the AoPs reflect this? It will be easier for reviewers if the AoPs are in proportion to the real emphasis of the job rather than having one giant AoP that is 90 percent of the job and equal billing given to a bunch of peripheral tasks.

If you’re going to study multiple jobs in one analysis meeting, you will want to find the similar/shared AoPs and make sure they are common. Sometimes Analysis Teams resist this because they more readily see the surface (and sometimes unnecessary) differences and not the underlying similarities. While the Analysis Team owns the content, we own the process to ensure that we later see potentially shareable information. It’s called data configuration control, and we need to affect it throughout our ISD processes.

Compare the following jobs:

In the example jobs above, it is likely that the Security Check AoP is the same performance for a sky cap and gate attendant (you know the drill, “Has anyone unknown to you asked you to put things in your luggage . . . etc.”).

However, the Workstation Setup AoP may be slightly different performances. You would need to do some probing for additional outputs, etc., the gate attendant performs because they have to be prepared to check in passengers in addition to luggage. You won’t know as the analyst, but you will have to facilitate a sound group decision—their first reaction may or may not be right.

How Do You Know When You Are Done?

A good approximate target would be five to nine AoPs for a typical job title or small job family. For large, complex, cross-functional processes, you may end up with more than 20 AoPs.

The number of AoPs isn’t critical; there is no right number. The number and total scope of the performances being analyzed and whether or not they have a repetitive nature dictate what the right number and configuration of the AoPs might be.

If the performance of New Plant Development is to build major production facilities around the world . . .

These AoPs Are Better

Than these AoPs

•   Define Project•   Assemble Project Planning and Oversight Team and Orient•   Plan Project (Macro)•   Assemble Project Working Team and Orient•   Plan Project (Micro Details) •   Recruit and Contract with Additional Internal and External Resources •   Oversee and Manage Project Construction Phases •   Oversee Plant Start-up and Pilot-Testing •   Oversee Plant Transition to New Plant Management Team

•   Plan Project•   Implement Plan•   Transition

The AoPs for the New Plant Development function should not include the following:

• Interpersonal Communications Skills
• Negotiating
• Understanding the Local Culture

These are important and necessary to performance, but they are not AoPs. They are knowledge or skill items that are used to or enable someone to plan projects, oversee plant transitions, etc.

Do not confuse and do not allow the Analysis Team to be confused over the enablers of performance and what performance we are looking for. The difference between data on the AoPs and Performance Model charts versus the data on the Knowledge/Skill Matrices is the most easily misunderstood aspect of the PACT Process Performance Modeling by people unfamiliar with the method.

It certainly helps to have seen it done once or several times by a seasoned PACT Practitioner before attempting to go solo. But then again, there is something to “learning by discovery” when your first AoPs don’t work so smoothly in your follow-on analysis efforts or later in your design efforts. You will find a way to self-correct.

Summary

A good set of AoPs sets the stage for the downstream efficiency and effectiveness of the additional analysis efforts and later design work.

AoPs are a segmentation framework for the performance being analyzed. There are many right answers (sets of AoPs) that will work well for you. But there are even more poor answers (sets of AoPs) that will make it harder, if not impossible, to get out all of the knowledge/skill enablers and then design quality, impactful, performance-based T&D.

One thing we haven’t addressed in this article is how to structure AoPs for processes involving multiple roles. For this we employ an organizing scheme that maps the macroprocesses of their organization within which the AoPs fit (sort of like creating “super AoPs”). Needless to say, this makes the task of defining AoPs even more complex. But, it allows multiple analyses to be integrated and improves the efficiency of the analysis process in a large-scale effort (and management AoPs are also tricky). Stay tuned for future articles or maybe even an advanced workshop covering these topics!

Regardless of the project scale, however, your goal should be the same—create a set of performance-segmenting AoPs that

• Defines the work performance in total with minimum/zero overlaps and gaps
• Is useful for our additional analysis efforts (deriving knowledge/skill enablers)
• Is acceptable to the customer and doesn’t violate their other views of the performance (and is ideally from their world in the first place, already in use, and with some level of management and target audience acceptance)
• Will help us ensure that our later T&D designs reflect an orientation to performance first and content second

PACT Process and lean-ISD are service marks of EPPIC, Inc.

More is available in the Resources section of the http://www.eppic.biz web site – including free articles, presentations and books.

# # #