What you measure shows what you value! If we’re measuring individual performance, that clearly indicates to each person in the team that we value individual performance. If your team members think that individual performance is a metric of value, they are naturally going to try to optimize for that metric. They are going to make choices, both consciously and subconsciously. And not all of these choices are going to result in the outcomes that we desire.
In knowledge work, things go wrong. If the only use of data is to find out who is responsible and keep people accountable, then don’t expect them to be enthusiastic about capturing and using that data.
There is always a context when issues arise. Charts usually don’t convey that context. If success means staying off a naughty list displayed in the office cafeteria, then don’t assume accurate or untampered data will emerge. The first rule of tracking metrics is that you should avoid setting personal goals without purpose.
Tracking Individual Performance Has a Negative Impact on Your Outcomes
A manager I was consulting back in the day, was willing to improve the quality of their deliverables. They decided that if the number of defects assigned to each team member goes above a certain threshold, that would be an accurate indicator that the team was cutting corners and that the results they provided weren’t meeting the quality level their customers expected. It was the product owner that prioritized the issues and assigned them to the responsible team members, so at first glance, that metric felt reasonable.
They created a dashboard that proudly displayed the number of defects currently associated with each individual, sorted highest to lowest. Think about this for a second – your name, listed in red at the top of the dashboard because you have twenty defects assigned to you. And the day after, you get an email from the technical director saying it’s unacceptable that you have so many open “problems”.
This approach was supposed to improve their quality, but what makes it such a bad idea? The list is quite long.
We want defects reported, we don’t want to hide defects. Are you more or less likely to report bugs in your management tool if it gets your name colored in red at the top of the list? You aren’t. Chances are, you’ll end up with an empty list of issues that still gradually find their ways into production.
Let’s dig deeper here. Does it matter how many defects are detected if the implementation isn’t going live anytime soon? If the solution has to be perfect before release, should you test it more? For 2 more weeks, 6 more weeks, 10 more weeks? If your customers can’t provide early feedback, how would you find the rest of the defects or UX flaws in a timely manner?
Are all defects equal? Would you prevent shipping a fix for a significant feature because you have some small layout defects on a specific resolution/browser combination? Quality is relative, not absolute. Withholding this release for a minor defect reduces quality as a whole.
Does it matter who is responsible for resolving the defects? Would just knowing there is a spike in defects be enough?
The mistake this organization made was to show data without context – neither a priority context nor customer impact context.
Even though the defect counts appeared to go down, the customer’s feedback remained negative. The result of this initiative was that the teams adapted by hiding data related to defects.
Connect Your Metrics Back to the Outcome for the Entire Organization
The truth is, we put the team in that position. By tracking the defect count per person, we basically told them how to behave.
 | Eliyahu GoldrattThe originator of the Theory of Constraints (TOC)"Tell me how you will measure me, and then I will tell you how I will behave. If you measure me in an illogical way, don't complain about illogical behavior." |
Our goal is to try to create benefits for the whole. Often though we set measurements for individuals and don’t always connect that back to the outcome for the entire organization. When the focus is on measuring people, they naturally strive to meet our expectations and they will do what they need to do to achieve that goal.
Flow metrics are a great way to measure the performance of your workflow, but the reason why I never apply them on an individual level is that if I do, it makes people make choices I don’t want them to make. They are having to choose between making themselves look good and doing what’s best for the team. And sometimes, doing what’s best for the team takes a hit on their individual stats.
For example, helping other people will dramatically improve the performance of the team as a whole, but it won’t necessarily make individuals shine. And pitifully, I don’t see many teams keep track of when they help each other out.
Strive not to put people in a position where they have to make choices that are best for themselves but hurt the whole. Every time you put a metric in place, ask yourself, “Does this decision make your team feel threatened?”. Think about, “How can we use this metric to make things better but also what are the actions it could provoke that would hurt our performance?”
There is often very little upside to tracking at an individual level. It has many impacting downsides that mean your data becomes needlessly incomplete or irrelevant. Your efforts will only ever make sense if they translate as outcomes to your organization. Keep your metrics aligned to that goal.
Last but not least, don’t worry too much about the actual numbers. Keep your focus on how the trends move over time to evaluate whether your improvement efforts are paying off.
If you’re interested in learning all about flow metrics and analytics that will enable you to achieve sustainable business outcomes, I’d be thrilled to welcome you to our Sustainable Predictability program.
