Kanban University

Management Training, Consulting, Conferences, Publishing & Software

Fitness for Purpose

Your KPIs Probably Aren’t! But What Are They?

by

This is the 2nd part of my post following Defining KPIs in Enterprise Services Planning.

Running our classroom exercises with private clients this past 18 months has shown me that most businesses have KPIs which really aren’t Fitness Criteria Metrics. In other words, their KPIs are not indicators of performance and certainly not key indicators or indicators of key performance. What I see as KPIs are really what should be classified as general health indicators and in some cases metrics driving specific improvements.

So what are general health indicators? Read more…

General Health Indicators

If we were to talk about individuals as people general health indicators are their pulse, their weight, their blood pressure, their cholesterol levels. If the individual was an athlete, they wouldn’t be setting goals based on achieving a certain blood pressure or a certain pulse rate but they would have expected ranges for these and they would monitor them. They might also monitor, for example, recovery time – the time it takes an accelerated heart rate to recover to a more normal level after stress.

Whether customers walk through your door is a health indicator. It is like having a pulse. Whether they buy something is also a health indicator and whether you had a positive margin in that sale and free cash flow are also health indicators.

In comparison, whether your customer is satisfied and whether they are likely to place repeat business and recommend you to a friend or colleague is a performance indicator. There are plenty of profitable businesses with lackluster stock price performance while others with growth but negative free cash enjoy elevated P/E ratios. They can always raise capital because their performance in the market is improving.

In Agile software development, velocity is a health indicator – you’d rather have some than not – but it is unlikely to be a performance indicator. The reason for this is that the volume or rate of production is not a fitness criteria for your customers and it is rarely a valued criteria even for internal stakeholders. Customers value things like delivery time, quality, useful functionality. You may believe velocity is a proxy for delivery time but it may not be. You may have a lot of velocity but a chronic, systemic problem with growing work-in-progress and ever longer delivery times. You are like a patient with a pulse but your blood pressure is trending the wrong way, your cholesterol is rising and your ability to get day-to-day tasks completed is being impacted. It could be your liver isn’t performing as it should be? If you set a target for your blood pressure you might be able to hit it by taking drugs – this is known as a symptomic fix which allieviates the symptom but fails to get to the root cause, as a result the problems persist and you end up back where you were again, a few months later. The same is true for companies. Many health indicators can be “vanity metrics” like boasting about your “at rest” heart rate – it isn’t always a fitness indicator!

Sure enough if you don’t have sufficient working capital, if you don’t have margin and profits and free cash, your corporate health is in danger, but as Richard Branson said, “People think companies exist to make profits. Wrong! They make profits in order to exist!” When you change to a mindset like Branson’s you start to see general health indicators for what they are, and you stop calling them “Key Performance Indicators.”

Metrics Which Drive Improvements?

While less common, it isn’t unusual to find a manager with a KPI on their scorecard which should rightly be classified as an Improvement Guiding Metric.

For example, we have poor customer satisfaction and a low Net Promoter Score because our delivery times are too long. In other words, the customer views lead time as a key selection criteria. Lead time is a fitness criteria metric and consequently lead time should be one of our KPIs.

In order to improve lead time, we decide that we want to automate testing because after analysis we discovered a significant portion of lead time is delay waiting for manual testing.

So we create an improvement goal to automate the testing of 85% of our application. We want 85% automated test coverage. So we start measuring and reporting the percentage of code covered by automated tests. We may even have placed this metric on the SMART goals for a manager. Despite all of this, it doesn’t make it a “Key Performance Indicator”. Instead it is a derivative of a KPI. The true KPI is lead time. We believe that by improving automated testing we will improve lead time. We may have causation to determine this or it may simply be a postulation and an experiment. The true KPI remains the customers’ fitness criteria – the lead time. So automated code coverage percentage is an Improvement Guide Metric.

Classifying Your Metrics

In our Enterprise Services Planning workshops, we have a series of exercises on metrics and the specific exercise on classifying existing KPIs is particularly enlightening for clients. They discover that most, if not all of their existing KPIs, are in fact health indicators and some improvement guides. Often the clients are not measuring anything which affects customer satisfaction. It is no wonder then that they aren’t happy with the NPS results.

What About Metrics That Aren’t in Any Of The Categories?

During the exercise, clients often find existing metrics they are measuring and reporting which can’t be classified as Fitness Criteria Metrics, Improvement Guides or General Health Indicators. The reality is these are metrics they probably don’t need and should drop. Often these are legacy metrics – metrics that were guiding earlier improvements, or perhaps fitness criteria that no longer apply because the company doesn’t serve that market segment any longer, or the customers in that segment have changed their opinion about what is acceptable and what represents “fit for purpose.” In some cases, the metrics are simply things the business could see and was capable of measuring. They serve no useful purpose.

To discover if a metric has a useful purpose ask yourself, “What decisions does this metric affect?” “What would we do differently if this metric rose or fell or hit some threshold value?” If you can’t identify an action or outcome you expect as a consequence of reporting a metric then you don’t need it.

When clients complete our Enterprise Services Planning workshop they often find they need far fewer metrics than they were already capturing and reporting but equally they need new instrumentation in order to capture the fitness criteria they’ve discovered from thinking about market segments in a new way and seeing their product or service through the customers’ eyes.

Defining KPIs in Enterprise Services Planning

by

All KPIs should be fitness criteria metrics. All KPIs should be recognizable by your customers and addressing aspects of how they evaluate the fitness of your product or service. If your customer doesn’t recognize or care about your KPIs then they aren’t “key”, “performance” indicators, they may indicate something else but they aren’t predictors of how well your business is performing or likely to perform in future.

This blog follows my recent posts on Market Segmentation and Fitness for Purpose Score explaining how we define Fitness Criteria Metrics. These metrics enalbe us to evaluate whether our product, service or service delivery is “fit for purpose” in the eyes of a customer from a given market segment. They are effectively the Key Performance Indicators (KPIs) for each market segment. All other metrics should either be guiding an improvement initiative or indicating the general health of your business, business or product unit or service delivery capability. If you can’t place a metric in one of these categories then you don’t need it.

Project Manager and Mom

Read more…

Fitness Criteria Metrics

We left our story of Neeta the busy project manager, mother of 4 kids, having established that she represents a member of two market segments – the “working late ordering food for the team in the office” cluster, and the “feed my children, its an emergency!” cluster. We also determined that the main metrics of concern are: delivery time; quality – both functional quality (the menu and order accuracy) and non-functional quality (hot, tasty, artisan, gourmet or maybe not); predictability (of delivery time, and perhaps of quality too); safety or regulatory concerns, perhaps including trust that organic ingredients were used or not. In our example, these 4 main metrics apply whether Neeta is ordering pizza for the team or whether she is ordering for her family. However, the satisfaction thresholds vary significantly based on the context.

When Neeta orders for the team they are happy to wait an hour to 90 minutes for delivery. If minor errors are made in order accuracy it is unlikely to matter too much. However, there is a threshold. If some of the team are vegetarian then there must be some vegetarian pizzas delivered or we’d consider the order a failure. The menu is important in the sense that the geeks want a more exotic set of choices. They are fussy about their non-functional requirements. They want the pizza hot, tasty, artisan and gourmet. They aren’t too particular about predictability of delivery time or order accuracy. A 30 to 45 window for delivery is probably acceptable and a few minor errors in the order is also acceptable. They do care about health and safety in the restaurant but they only care about the traffic safety of the delivery boy in so much as it doesn’t endanger the quality of the pizza on arrival.

When Neeta orders for her family, the threshold levels are significantly different. The kids are really hungry and impatient and now they know that pizza is coming, they are super-excited about it. Fast delivery is essential. Predictable delivery is essential: the 6 year old is now running his countdown timer on his iPad. The menu was important but only in so far as it offered simple plain cheese pizza with tomato sauce. The kids are so excited that they won’t mind if the pizza is a little cold on arrival, nor will they mind if it got shaken up a bit during transportation. Order accuracy is important to the kids. If it isn’t a plain cheese pizza they will be extremely upset and unlikely to eat it at all. Meanwhile, mommy can worry about safety and regulatory concerns but they may have a preference for a restaurant that promises to use organic ingredients. They have no concept of whether they trust this assertion. The restaurant said it was organic – mommy can worry about whether that is true or not.

So in summary…

Kids

Fast delivery

100% order accuracy

Not concerned too much on non-functional quality

Predictable delivery

Not concerned too much on safety or regulatory concerns

Geeks

Longer delivery acceptable

Some errors in order accuracy acceptable

Extremely fussy about non-functional quality

Wider tolerance for unpredictable delivery

Not too concerned about safety or regulatory issues

In 4 of these 5 categories we have significantly different fitness evaluation thresholds for these two segments.

If we are to successfully serve both segments, driving improvements so that we have high levels of customer satisfaction in both segments, we must use the higher thresholds with each metric as our benchmark. Alternatively, we need to segregate our service delivery by segment. We can do this by introducing two classes of service, one of each segment. This might work through pricing, for example, would Neeta pay a premium for the guaranteed fast delivery for the kids? Or it might work through capacity allocation and demand shaping based on it. We might, for example, refuse to take large commercial destination orders during peak times for domestic orders. In this example, we trade off educating our corporate clients to order earlier in the day, versus the risk that they will go elsewhere. We do this because we value the domestic market.

There is no formula for this. No right or wrong. We make choices for our business in terms of which segments we wish to serve and how well we wish to serve them. Choices come with consequences. We need to be prepared to live with these consequences and be willing to be accountable for them.

The metrics and threshold values we’ve developed for each of our segments should become the KPIs for our business and specifically in this case, the pizza delivery service. We should put in place the mechanisms, instrumentation and customer feedback, to measure these metrics. We can use the results at Service Delivery Reviews, Operations Reviews and Strategy Reviews to determine how well we are serving our markets, where we need to make improvements and which segments we wish to serve.

In my next post in this series I will take a look at the other types of metrics: those which guide improvements; and general health indicators. My experience working with clients in 2015 is that most existing KPIs are in fact merely general health indicators. As a consequence these businesses are optimizing for the wrong things and customer satisfaction and their ability to survive and thrive in the market is impaired. All KPIs should be based on threshold values for fitness criteria metrics derived from analysis of the market segments you choose to serve.

Read Part 2: Your KPIs Probably Aren’t! But What Are They?

Kanban – The Alternative Path to Agility

by

The Kanban Method was conceived as an alternative path to agility – as a means to improve responsiveness and adaptability without any significant revolution or reorganization in your way or working or political structure of your business. Lean Kanban University has recently introduced a series of training classes developed and evolved from older, tried and tested curriculum to ease adoption of Kanban and communicate the full scope and scale of what is possible when you fully embrace Kanban as a way to manage your modern professional services business.

This table shows The Kanban Method and the alternative path to agility is a single graphic. From left to right you progress from basic introductory shallow kanban boards to full enterprise scale, Enterprise Services Planning.

kanbaninatable

In September our new training facility, the Anderson School of Management opens in Seattle conveniently located near Seattle Center at 200 First Avenue West. We will be offering the full suite of Kanban and Enterprise Services Planning classes every month. Browse our schedule via our training listings. Email our sales team for summer offers and promotions if you register before August 10th, 2015.

Enterprise Services Planning: Module 1 – Portfolio Management

by

Enterprise Services Planning is a new modular 5-day training curriculum for managing modern businesses involving lots of knowledge work and creative services. If your organization contains people who must think and make decisions for their living then Enterprise Services Planning is the management training framework that will transform your business. While ideally taken together as 5 days of intensive emersion, ESP training is offered in 4 modules.

esp_map_large

Map of the Enterprise Services Planning Framework

Enterprise Services Planning: Module 1: Portfolio Management

Training class for up to 24 attendees

Duration: 2 days

Pre-requisites: Informational level knowledge of kanban systems and their application to knowledge work and creative services workflows

Target Audience

“I am a business leader who needs to understand the dynamics of our environment in order to make better decisions about what to start, when to start them and the likelihood of a successful and desirable outcome for our business and our customers”

“I am a portfolio manager who needs to make decisions about risk and capacity allocation, decide when to start projects and initiatives and manage our portfolio of concurrent projects, initiatives and activities.”

“My job is to manage risk for our organization and advise our leaders and the management team in our PMO”

“I work in the PMO and I want to be more effective in my job. I’m overwhelmed and overburdened and I’m looking for simpler, more powerful ways to make decisions, take actions and work with project stakeholders.”

Curriculum

Day 1 – Fitness for Purpose & Cost of Delay

  • Blizzard Skis case study
  • Defining Fitness for Purpose
  • Defining Fitness Criteria Metrics (KPIs)
  • Classes of Service and alignment with market segments and fitness criteria
  • Qualitative assessment of Cost of Delay
    • Market payoff function
    • Defining cost of delay
    • Cost of Delay function shapes
    • Cost of Delay impact assessment
    • Shelf life

Day 2 – Scheduling, Sequencing, Risk & Strategic Alignment

  • Scheduling work
    • optimal start time
    • window of opportunity
  • Sequencing
  • Portfolio Risk
  • Hedging Risk
  • Risk Profiling
  • Pragmatic Philosophy for Risk Management
  • Aligning Strategy with Capability
  • Implementing a regular Strategy Review

Learning objectives

Understanding evolutionary improvement of service delivery by applying evolutionary theory to development of fitness criteria metrics (or, key performance indicators (KPIs)) by understanding what creates “fit for purpose” service delivery based on customer needs and expectations.

Understanding use of classes of services to serve specific market segments and sources of demand to enable delivery within expectations and against the defined fitness criteria metrtics

Understanding cost of delay as a concept and knowing how to classify it in a qualitative and pragmatic fashion using taxonomies

Understanding how to apply cost of delay and lead time capability sensitivity analysis for scheduling. Learning how to determine earliest start, latest start and optimal start dates for requested work

Understanding how to use market role risk assessment to sequence work in large batch commitments (such as projects)

Understanding how to assess portfolio risk based on strategic contribution and market lifecycle stage

Understanding how to hedge portfolio risk using capacity allocation in kanban systems

Understanding how to develop a multi-dimensional risk profile for portfolio or project level use and how to visualize it and use the visualization to inform scheduling and option selection/discard decisions

Learn the 12 point pragmatic approach to risk assessment

Understand appropriate alignment of service delivery capability with strategy and risk hedging allocation

Understand the purpose of a regular Strategy Review to assess market segments, fitness criteria, risk hedging policies and alignment of strategy, risk management policy and service delivery capability

As an entire outcome attendees will have learned how to select work for a portfolio, how to align a portfolio of work with company strategy, how to insure that strategy is aligned with capability, how to schedule and sequence work within the portfolio, and how to hedge risk across the portfolio

Who should attend?

Portfolio and program managers, project managers, service delivery managers, risk managers, those responsible for corporate governance, product managers, marketing managers and strategic planners, senior executives and those responsible for strategy, risk policies and strategic decision making, management trainers, management and executive coaches, anyone interested in resilience and survivability of their business and those responsible for service delivery to customers.

Applicability

This class is ideally suited to a single corporate for private delivery on premises. Typical scope should be a medium-sized entity or a product or business unit of a larger entity. The class is most suitable for the private sector but is adaptable to public sector environments.

Sales

Enterprise Services Planning classes are currently offered exclusively through David J. Anderson & Associates, Inc.

For open registration classes please consult our training listings If you don’t see a class listing near you please contact our sales department via the link at the bottom of the page

For private classes please contact sales.

Download the module curriculum