Kanban University

Management Training, Consulting, Conferences, Publishing & Software

Marketing

Defining KPIs in Enterprise Services Planning

by

All KPIs should be fitness criteria metrics. All KPIs should be recognizable by your customers and addressing aspects of how they evaluate the fitness of your product or service. If your customer doesn’t recognize or care about your KPIs then they aren’t “key”, “performance” indicators, they may indicate something else but they aren’t predictors of how well your business is performing or likely to perform in future.

This blog follows my recent posts on Market Segmentation and Fitness for Purpose Score explaining how we define Fitness Criteria Metrics. These metrics enalbe us to evaluate whether our product, service or service delivery is “fit for purpose” in the eyes of a customer from a given market segment. They are effectively the Key Performance Indicators (KPIs) for each market segment. All other metrics should either be guiding an improvement initiative or indicating the general health of your business, business or product unit or service delivery capability. If you can’t place a metric in one of these categories then you don’t need it.

Project Manager and Mom

Read more…

Fitness Criteria Metrics

We left our story of Neeta the busy project manager, mother of 4 kids, having established that she represents a member of two market segments – the “working late ordering food for the team in the office” cluster, and the “feed my children, its an emergency!” cluster. We also determined that the main metrics of concern are: delivery time; quality – both functional quality (the menu and order accuracy) and non-functional quality (hot, tasty, artisan, gourmet or maybe not); predictability (of delivery time, and perhaps of quality too); safety or regulatory concerns, perhaps including trust that organic ingredients were used or not. In our example, these 4 main metrics apply whether Neeta is ordering pizza for the team or whether she is ordering for her family. However, the satisfaction thresholds vary significantly based on the context.

When Neeta orders for the team they are happy to wait an hour to 90 minutes for delivery. If minor errors are made in order accuracy it is unlikely to matter too much. However, there is a threshold. If some of the team are vegetarian then there must be some vegetarian pizzas delivered or we’d consider the order a failure. The menu is important in the sense that the geeks want a more exotic set of choices. They are fussy about their non-functional requirements. They want the pizza hot, tasty, artisan and gourmet. They aren’t too particular about predictability of delivery time or order accuracy. A 30 to 45 window for delivery is probably acceptable and a few minor errors in the order is also acceptable. They do care about health and safety in the restaurant but they only care about the traffic safety of the delivery boy in so much as it doesn’t endanger the quality of the pizza on arrival.

When Neeta orders for her family, the threshold levels are significantly different. The kids are really hungry and impatient and now they know that pizza is coming, they are super-excited about it. Fast delivery is essential. Predictable delivery is essential: the 6 year old is now running his countdown timer on his iPad. The menu was important but only in so far as it offered simple plain cheese pizza with tomato sauce. The kids are so excited that they won’t mind if the pizza is a little cold on arrival, nor will they mind if it got shaken up a bit during transportation. Order accuracy is important to the kids. If it isn’t a plain cheese pizza they will be extremely upset and unlikely to eat it at all. Meanwhile, mommy can worry about safety and regulatory concerns but they may have a preference for a restaurant that promises to use organic ingredients. They have no concept of whether they trust this assertion. The restaurant said it was organic – mommy can worry about whether that is true or not.

So in summary…

Kids

Fast delivery

100% order accuracy

Not concerned too much on non-functional quality

Predictable delivery

Not concerned too much on safety or regulatory concerns

Geeks

Longer delivery acceptable

Some errors in order accuracy acceptable

Extremely fussy about non-functional quality

Wider tolerance for unpredictable delivery

Not too concerned about safety or regulatory issues

In 4 of these 5 categories we have significantly different fitness evaluation thresholds for these two segments.

If we are to successfully serve both segments, driving improvements so that we have high levels of customer satisfaction in both segments, we must use the higher thresholds with each metric as our benchmark. Alternatively, we need to segregate our service delivery by segment. We can do this by introducing two classes of service, one of each segment. This might work through pricing, for example, would Neeta pay a premium for the guaranteed fast delivery for the kids? Or it might work through capacity allocation and demand shaping based on it. We might, for example, refuse to take large commercial destination orders during peak times for domestic orders. In this example, we trade off educating our corporate clients to order earlier in the day, versus the risk that they will go elsewhere. We do this because we value the domestic market.

There is no formula for this. No right or wrong. We make choices for our business in terms of which segments we wish to serve and how well we wish to serve them. Choices come with consequences. We need to be prepared to live with these consequences and be willing to be accountable for them.

The metrics and threshold values we’ve developed for each of our segments should become the KPIs for our business and specifically in this case, the pizza delivery service. We should put in place the mechanisms, instrumentation and customer feedback, to measure these metrics. We can use the results at Service Delivery Reviews, Operations Reviews and Strategy Reviews to determine how well we are serving our markets, where we need to make improvements and which segments we wish to serve.

In my next post in this series I will take a look at the other types of metrics: those which guide improvements; and general health indicators. My experience working with clients in 2015 is that most existing KPIs are in fact merely general health indicators. As a consequence these businesses are optimizing for the wrong things and customer satisfaction and their ability to survive and thrive in the market is impaired. All KPIs should be based on threshold values for fitness criteria metrics derived from analysis of the market segments you choose to serve.

Read Part 2: Your KPIs Probably Aren’t! But What Are They?

Market Segmentation for Enterprise Services Planning

by

I realized after posting my article on Fitness For Purpose Score that it isn’t reasonable to expect readers to know the background and context that stimulated it. It isn’t reasonable that I assume readers are up-to-date with speeches I’ve given ove the last two years covering Evolutionary Change, Fitness for Purpose and Enterprise Services Planning. So I felt some explanation of how we do market segmentation for ESP was in order to provide better context for Fitness For Purpose Score.

How do we know whether a change in our service delivery capability represents an improvement? This is the fair and reasonable question that should drive our decision making about how we manage, how we make decisions, and which changes we choose to invest in, consolidate and amplify. In evolutionary theory, a mutuation survives and thrives if it is “fitter” for its environment [this is actually a gross simplification but it will do for introductory paragraph on a related but different topic of marker segmentation.] So how do we know whether or not a change to our service delivery capability makes it fitter for its environment? What do we mean by “environment” in this context? “Environment” is the market that we deliver into. So “fitness” is determined by whether the market feels our product or service and the way we deliver it, is “fit for purpose.” So to understand “fitness” to enable and drive evolutionary improvements, we first need to understand our market and what defines “fitness for purpose.” To do this we segment the market by customer purpose and the criteria with which they evaluate our “fitness for [that] purpose.” …

At Lean Kanban Inc we create our market segmentation by clustering narratives about our customers. We do this by telling stories about them. The technique is a direct application of Dave Snowden’s technique from his Cynefin Framework. To explain this in our training and in the speeches I linked above, I tell the tale of Neeta, a fictional project manager and mother of 4. Neeta is based on a real woman who works in the Canadian public sector and has considerable Kanban expertise. Neeta needs to order pizza for delivery to her office to feed her team who are working late against a deadline. On another evening the same week, she needs to order pizza for delivery to her home to feed her children who are hungry because she came home late. Neeta doesn’t represent one market segment, she represents two! The reason for this is that the purpose, context and fitness selection criteria are different in each of the two contexts.

Project Manager and Mom

When Neeta orders pizza for her children she needs: fast delivery – ideally within 20 minutes; she needs order accuracy – the kids only like plain cheese pizza; the non-functional quality doesn’t matter too much, the kids will eat cold pizza so long as it is cheese pizza; she needs a simple menu and predictable service; she wants delivery when promised because the kids need their expectations set and they are unforgiving; she also cares that the restaurant is clean and can be trusted to follow health and safety regulations; she may care whether or not they use organic ingredients because she is feeding her family.

When Neeta orders  pizza for her office her need are similar but some of the criteria vary and the threshold values are different: she needs delivery in up to 90 minutes; order accuracy is important but if one or two mistakes are made it won’t make a big difference; however, the non-functional quality matters, hot, tasty, pizza with gourmet flavors and exotic ingredients are required for these discerning geeks; it doesn’t matter if delivery isn’t as predictable as it might be, so long as they show up eventually – the team are busy; and yes, she still cares whether the restaurant meets health and safety legislation standards but organic ingredients probably isn’t so much of a concern.

In other words, Neeta decides whether she likes the pizza service and whether she will use it again, based on two different sets of criteria, depending on her context. This may lead her to use different service providers for each purpose, if one provider can’t meet both sets of her needs. As a result Neeta represents two segments, not one.

Pizza Boy

How would you know that Neeta represents two segments and not just one? Traditional demographic profiling wouldn’t give you this insight! Well perhaps she uses different credit cards or payment mechanisms depending on context? And the delivery address is different. So there are some obvious clues. However, the people in the business who know Neeta’s story are the person who took her telephone order, and the delivery boy who delivered the pizzas. It is these frontline staff who understand the customers best.

If you are to cluster customer narratives to determine segmentation, you need to bring frontline staff into the story telling sessions. You need to listen for context, purpose and selection criteria and create segments based on affinity of these aspects of the market. Give each cluster a nickname. Recognize that an individual customer can appear in multiple segments depending on their context on a specific day and time.

The challenge of this for many companies is that the people who best understand the customer’s context, purpose and selection criteria are often the lowest paid, shortest tenured, highest turnover staff in the business. Foolishly, many companies under value, the value of customer facing staff. Traditional 20th Century service delivery businesses take a transaction view of customer interaction rather than a relationship view. If you value repeat business and you value the insights that will enable your business to evolve and survive in a rapidly changing market then you need to value customer facing people and involve them in your strategic planning.

Once you have the clustered narratives defining your segment, now select the segments you want to serve. This is a key piece of strategic planning. Which businesses do you want to be in? Which don’t you care about? Which do you want to actively discourage? Based on this you will develop the Fitness Criteria Metrics to drive your management decision making and evolutionary improvement.

Designing Fitness Criteria Metrics, choosing their threshold values, and making them your KPIs (Key Performance Indicators) will be the subject of my next post.

Fitness For Purpose Score and Net Fitness Score

by

Regular followers of my work will know that I have expressed dissatisfaction with Net Promoter Score (NPS). Steve Denning in his book Radical Management suggested NPS was “the only metric you’ll ever need.” Steve is a writer for Forbes, an investment magazine. High NPS scores correlate with high stock prices and hence from an investor’s point of view NPS is an important metric. If you are a CEO of a public company, who receives a large portion of your salary as bonuses based on changes in the stock price then NPS is an important metric. However, many of my clients who collect NPS data report to me that it isn’t an actionable metric. NPS merely tells you whether you are winning or losing. It doesn’t tell you what to do!

There are some antidotes to NPS’ failings. The second question asking reviewers to “tell us why you gave the rating in the previous question?” provides the opportunity for short narratives. These micro-narratives can be clustered using a tool such as Sensemaker and useful information can be extracted. There may be actionable information hidden in the clustering of narratives. This advanced use of NPS information is very much still in its infancy and not readily available to many or most businesses.

I’ve decided to introduce a new metric into our own surveys. I call this Fitness For Purpose Score. I am hopeful this will become a key strategic planning tool in Enterprise Services Planning.

Fitness For Purpose Score

It is often true that businesses do not know the purpose with which a customer consumes their product or service. A product or service designed for a specific purpose may get used for something else. Some of the more famous examples, are washing machines used to make lassi yoghurt drinks for Indian restaurants. In evolutionary science this is known as an exaptation: where something designed for one purpose is adapted for use with another purpose. To have actionable metrics for product or service delivery improvement, you need to understand the customer’s purpose for consuming your offering. When you understand this purpose, you can create the appropriate fitness criteria metrics. With Enterprise Services Planning (and Kanban) we use fitness criteria metrics to drive improvements. Fitness criteria metrics are used at all levels to compare capability with expectations. Fitness for Purpose Score is intended to help us understand purpose and whether or not our current capability meets expectations. If it doesn’t we can probe for thresholds to establish new fitness criteria metrics.

This is how our sales and marketing team will be using Fitness For Purpose Score in our own surveys in 2016.

Question 1: What was your purpose [in attending our training class? What did you hope to learn, take away, or do differently after the class?]

Question 2: Please indicate how “fit for purpose” you found [this class]?

  1. Extremely – I got everything I needed and more
  2. Highly – I got everything I needed
  3. Mostly – I got most of what I needed but some of my needs were not met
  4. Partially – some of my purpose was met but significant & important elements were missing
  5. Slightly – I took some value from it but most of what I was looking for was missing
  6. Not at all – I got nothing useful

Question 3: Please state specifically why you gave your rating for question 2

Questions 1 and 3 specifically ask for short narrative answers. These micro-narratives can be clustered. Question 1 will provide clusters of purpose which can be validated against our existing market segmentation and may reveal new segments, while question 3 will provide clusters of actionable information for improvements and possibly new fitness criteria metrics or threshold value for existing metrics. We can decide whether or not to pursue specific clusters and whether we are likely to be able to achieve adequate fitness levels to satisfy our customers during our Strategy Review meeting.

For example, our own product is management training, though we also have an event planning and publishing business. We position and sell our intellectual property as management training and we deliver it as training classes and mentoring. We know that a significant segment exists for software process improvement and for process engineers and coaches who consume our products and services in order to help them in their coaching practice. We know this segment exists but we specifically and intentionally don’t cater to it. We feel it would be a strategic distraction and undermine our overall message that managers need to be accountable, to take responsibility, to make better decisions and to take action where and when necessary to improve service delivery. The return-on-investment in our products and services is realized when existing managers change their behavior as a result of our training. And hence, while we appreciate the patronage of process engineers and coaches, we do not specifically cater for their needs.

Net Fitness Score

I purposefully moved away from the NPS use of an eleven point numerical scale [0 thru 10]. My background in human factors, psychology and user experience design, taught me that humans have problems with categorization beyond 6 categories without a specific taxonomy to guide them. This isn’t a result of Miller’s “Magic Number 7” rather the work of Bousfield W.A. & A.K, and Cohen, B.H. between 1952 and 1966 on clustering. For example, if you ask humans to rate something 1 to 10 they will struggle to create 10 distinct categories in their mind. When asked to devise their own taxonomy, or clusters, as lay people to the domain, they will tend to create no more than 6 categories. Hence, a scale of 0 through 5 is most appropriate for general consumption. I believe the NPS people tried this but discovered that in some cultures, such as Finland, people never give the top score on principle. They always choose one below the best. Hence, the NPS reaction to this was to double the scale using 0 through 10 so that people could give a 9 when they are really giving a 4.5. My feeling on this is that it highlights the issues with numerical scales and undeclared taxonomies. The solution of doubling the scale, however, creates a randomness in the system and generates noise in the data reducing the signal strength, because of the general human issue of modeling categories against the scale. Fixing one problem, the cultural propensity never to give top ranking, creates another problem, a cognitive issue in the general population to struggle with more than 6 undeclared categories. Hence, to avoid both problems, I am declaring the categories with narrative.

Scores of 4 and 5 are intended to indicate that someone is satisfied and the product or service was fit for their purpose.

Score of 3 is intended to indicate a neutral person. They didn’t get everything they needed to be delighted with the service but they got something acceptable for their investment in time and money.

Scores of 2 or below are intended to show dissatisfied customers who felt their purpose was unfulfilled by the product or service. This may be because the product is poor or it may be because the purpose was previously unknown or represents a segment that the business has strategically decided to ignore. Not all dissatisfied customers need to be serviced fully and satisfied: some customers, you simply don’t want – they represents segments you aren’t interested in pursuing.

Net Fitness Score [NFS] = % satisfied customers – % dissatisfied customers

NFS can be improved through better marketing communications that direct the right audience to your business and dissuade the wrong audience. So NFS can be used to drive excellence in marketing as well as used to explore new segments and the fitness criteria metrics that light them up as viable and profitable businesses.