Adapted from an SWI Quarterly Newsletter Article by the late Mark Graham Brown – Summer 1989

8 Primary Types of Measures or Indices

In every Enterprise, there are eight primary types of measures or indices of performance:
• Quality
• Profitability
• Productivity
• Market Standing
• Quality of Life
• Financial Resources
• Innovation/Technology
• Public Responsibility

In any given Enterprise you may only have four or five of these measures, and the relative importance of these measures may change as the business and outside environment change.

Measures such as Quality of Life and Public Responsibility are difficult to measure objectively but are nonetheless important.

Quality of Life has to do with how well the company treats its employees and how the employees feel about working for the organization.

Public Responsibility relates to issues such as safety and concern for the environment. If a company chooses to ignore some of these softer measures, they often find that their financial (business) measures suffer in the long run.

Building the Hierarchy of Measurement Indices

Once you’ve decided what the overall measures are for your organization, you need to build a measurement hierarchy from the top down. This involves taking the overall measure such as sales revenue or gross margin and breaking it down into its components or sub-measures, then tying those sub-measures to functions, positions, and individual employees. (See diagram below.)

By defining the measurement system in this way, you assure that all of the activity being driven by the measurement system leads to the overall results for the organization. It all sounds like common sense, right? It is, but it is rare to find an organization that has built its measurement system in such a logical fashion.

Measurement systems appear to be built over time by a variety of departments and without any clear overall plan. Many times we see several departments or divisions being evaluated on the same primary measures. When something goes wrong, it’s always the other group that is at fault.

Keeping the Measurement System Simple

All of us seem to enjoy keeping track of how we are doing, measuring our own performance against others, or against some standard. All sports and games include measurement systems for evaluating teams and individual players. What’s different about the measurement systems in most sports is that they are very simple. In team sports, the overall measure is the number of wins and losses. Other measures such as passes completed, runs batted in, etc., are used to evaluate both individual performance and of the entire team.

In most organizations, a given individual is measured on ten or more factors. Many managers have 12-16 annual objectives that they are measured on, along with measures of their ongoing responsibilities, and measures of their behavior. No one would argue that all of the factors being measured are important, it’s just too much information for an individual to keep track of on a regular basis. Imagine playing golf, and not only keeping track of the number of strokes per hole, but also keeping track of the force, distance, direction, velocity, and accuracy each time the ball is hit. We could also measure behavioral dimensions like the use of the proper club, the swing, etc. All of these factors are important to playing a good game of golf, but we only have one bottom line measure: how many strokes are you above or below par
for the course.

The best measurement systems in organizations are based upon no more than four primary measures for any one individual. Rather than develop a series of sub-measures, it is more beneficial to state a series of qualifiers that must be observed. These are like the rules in a game. An example of a rule in an organization is not exceeding your budget. As in sports, it is possible to have many rules in organizations. People can remember these if they are consistent.

The Two Measurement Techniques: Counting and Judging

All measurement systems in both sports and work are based upon systems of counting or judging.

Counting is the more Qbjective of the two and is the most widely used. Number of defects per million, strikeouts, projects completed, sales made, and strokes per hole are all things we can count. Some sports and some jobs cannot be adequately measured by counting. A rating or ranking system must be used to evaluate certain performances.

Diving and gymnastics, for example, are both measured using a Judgement system. Similarly, most aspects of quality in organizations cannot be measured by counting; judgments must be used. When using a judgment type measurement system it is always preferable to use a scale where ratings are based upon absolute criteria, rather than one in which the performance of various individuals is ranked against one another.

Deciding Who Should Do the Measuring

The best measurement systems are based upon both internal and external measures. It’s important to know what your customers think of the quality of your products, for example, as well as what your own QC people have to say about them. Some manufacturers provide very detailed quality data to their suppliers, but in most cases, it is difficult to get any amount of detailed feedback from clients or customers. Self-Measurement is often the easiest way of gathering data, but it is also the least credible data. If it is possible to remove human judgement or counting from the measurement system, it is much easier and more objective than systems that depend upon employees making the judgements or counting.

Automated measurement equipment is extensively used in many modern manufacturing facilities, but the technology is unavailable for measuring many service and professional jobs.

Measuring Scientists and Engineers

Scientists and engineers probably have the most difficult jobs to measure. However, by-being-creative, it is possible to come up with objective ways of measuring the performance of these individuals. Dr. Randy Rooker, a Research Manager at NASA’s Langley Research Center, came up with an index for measuring his analysts that gave them a score for each analysis project they completed. Each project was given a different value, based upon its difficulty and the amount of time to complete it. The completed analysis was then evaluated based upon its accuracy, timeliness, completeness, and originality. Each of these factors was worth a fixed percentage. Dr. Rooker’s measurement was simple, easy to administer, and measured all of the major aspects of the analysts’ performance that he considered important.

Building a Measurement System

Creating a measurement system is difficult to do by selecting one function or department and starting with it. A measurement system should be built from the top-down, to assure that all of the lower level measures contribute to the overall measures of the organization. In other words, you start by identifying the key measures for the President and CEO, and work down in a hierarchical fashion until you reach the level of the individual contributors. Each measure should relate to those above it in a logical manner.

Only by building the system in this fashion can you assure that you are measuring the right things.

Although this is conceptually the ideal way of building the measurement system, it is sometimes difficult to convince top management to start with them. An alternative that we have found workable is to develop and pilot test the system in one function that is particularly difficult to measure, such as Engineering or R&D. Once top management sees that the measurement system is driving the desired performance, it may be more amenable to designing new measurement systems for the remainder of the organization.

***

RIP Mark Graham Brown.

See the Summer 1989 SWI Newsletter – here.

See many of Mark’s books – here.

###