Data-Driven Decision Making: The Value of Probabilistic Thinking
More than 50 years ago, Peter Drucker, a management consultant and author, described as the founder of modern management, predicted we would shift to a knowledge society. He said information would change the way people work – that knowledge will become the most important economic resource. Instead of generating value through physical labor with their muscles, people will do it with their minds. Drucker described these people as “knowledge workers”.
So, who is a knowledge worker? Simply described, a knowledge worker is someone whose job requires them to think for a living. Knowledge workers are people who “figure out” what they need to do. People who weigh many factors to determine the best course of action. People whose actions are based on multiple sources of information. These are the people who gather clues, follow up and discover new possibilities.
In knowledge work, we make assumptions about the future. In the most effective manner, we validate these assumptions by delivering results early and often. We learn what brings value and avoid the risk of spending time and money on the wrong things. That is simply because the nature of knowledge work is unpredictable.
People Feel More Comfortable Being Wrong Than Uncertain
When it comes to product management, the big question always seems to be “When will this be done?”. The problem is rooted in the expectation of having a single certain delivery date as an answer. And people tend to provide it. Fact is people feel more comfortable being wrong than uncertain.
The fear of uncertainty is a cultural problem. Professor Geert Hofstede, a Dutch population biologist, conducted one of the most comprehensive studies of how values are influenced by national culture. He defines six dimensions of national culture, one being the uncertainty avoidance index. The index is scaled from 1 to 100 – the higher the index the more uncomfortable to uncertainty you are as a society. Countries exhibiting strong uncertainty avoidance index maintain rigid codes of belief and behaviour and are intolerant of unorthodox ideas.
We’d rather provide a single certain delivery date that’s wrong than remain uncertain.
The Difference Between an Estimate and a Forecast
How can we answer the question “When will this be done”? Can we predict exact future delivery dates, despite the unpredictability of knowledge work? Are we capable of predicting the exact time a feature will take to go through the whole process while coordinating the rest of the work in progress at the same time? Is it possible to predict that there won’t be any additional work coming in between, any dependencies, bottlenecks or external blockers that might cause a delay?
There are two approaches trying to provide the answer to “When will this be done” question – an estimate and a forecast. What’s the difference?
Estimates are predictions based on guesswork, judgment or a gut feeling.
The prediction is delivered as a single value – it could be a date or a number of days for example. An estimate doesn’t involve any probability of its occurrence.
Forecasts, on the other hand, are based on historical performance data. The prediction is communicated as a range of values and the probability of those values occurring.
Forecasting is faster, cheaper and more reliable than estimating.
Increasing Predictability Through Probabilistic Outcomes
When making business decisions, we are constantly confronted with possibilities whose outcomes we don’t know. Getting rid of uncertainty and being in control of everything is impossible. The real challenge is dealing with the uncertain and unknown in an effective way. That is the realm of probability.
Using a probabilistic approach of making future predictions based on past performance data increases the predictability of our outcomes. Making probabilistic forecasts helps us maintain high levels of customer and stakeholder trust, satisfaction and retention. It allows us to define service level agreements with more confidence and deliver value to our customers in a consistent, predictable manner.
Learn To Think Probabilistically To Improve Decision Making
Thinking probabilistically means having a willingness to always ask questions like “What else might happen?”, “What could happen next?”, “What if we’re wrong?” and to look at the full range of possibilities that might come to pass rather than to assume that things will go as planned. Ignoring unwanted facts won’t make them disappear. Knowledge work is complicated and lots of unexpected and surprising things happen.
Start making probabilistic forecast using historical performance data in parallel with your old approach. If you have the data, use the data. If you do not have the data, then get the data and use the data. It will probably give you as good or even better answers and it will take a lot less time. Keep both the approaches and observe the results. At some point, it will become obvious which one works best for you.
Thinking probabilistically increases our predictability and improves decision making. It provides an alternative to get more accurate answers with a lot less effort. Most importantly, it enables us to spend our time on what matters the most – delivering customer value.
Meet the Author
Sonya Siderova is a passionate product manager and a driving force behind Nave, a Kanban analytics suite that helps teams improve their delivery speed through data-driven decision making. When she's not catering to her two little ones, you might find Sonya absorbed in a good heavyweight boxing match or behind a screen crafting a new blog post.
Take your training and consultancy sessions to a whole new level. With Kanban analytics on their existing tools, yo… https://t.co/4G3X9FehF5Follow
Learn how to make accurate, data-driven predictions to stay on track, meet deadlines and keep a high level of custo… https://t.co/zPBlWxv5X7Follow
The Flow Efficiency Chart shows your average flow efficiency, as well as how trends have been moving over time. Ide… https://t.co/Eff9ITFFnrFollow
Learn more about the difference between thin-tailed and fat-tailed distributions and the approaches to evaluate you… https://t.co/oHCaDCa4WXFollow
Last chance to get 60% off! We list our Sustainable Predictability digital course at the lowest price ever! The off… https://t.co/Af5U0kiBIEFollow
Get straight to the essence of your Azure board data and analyze your processes with our immersive data-visualizati… https://t.co/MpB4kgNiCeFollow
Kanban can help you run your business better, make your processes more efficient and empower your team to accomplis… https://t.co/RwXHnb4UcEFollow
A Cycle Time Histogram with a big hump on the left and a very long tail to the right indicates that your cycle time… https://t.co/mFKXLpx4HhFollow
Service level agreements define the responsibilities of a service provider to their customers. Defining SLAs are im… https://t.co/s7HeXDvfkWFollow
Today, we’ll explore the consequences of moving cards backward has on your performance, as well as the most effecti… https://t.co/IZmafKMe9YFollow
Value stream mapping is a visual technique that depicts the lifecycle of your product and finds and eliminates wast… https://t.co/fyJZvdPVCxFollow
Our digital course Sustainable Predictability is listed for $397 until Nov 30th. Take advantage of the 60% discount… https://t.co/8bBbiaWPa3Follow
Start making reliable decisions and eliminating the bottlenecks caused by unclear priorities with a dynamic priorit… https://t.co/hVpa8sCtR9Follow
Take your team to a whole new level with Nave's Kanban analytics for Trello. Picture what's going on behind your da… https://t.co/BhnrABnsPBFollow
In our latest article, we’ll take you through the key steps to reducing the impact that blockers have on your deliv… https://t.co/10L6MoruB4Follow
30% discount on all annual plans until the 30th November! Subscribe now with a coupon code NAVEBLACK20… https://t.co/dnSM2KzS5cFollow
The dotted horizontal lines on the Cycle Time Scatterplot are called percentiles. We use percentiles to define the… https://t.co/nlUcIGRDm3Follow
High pressure over long time periods leads to your team suffering from burnout and its symptoms. Learn more about w… https://t.co/hcYg29OE3YFollow
Successful project managers are effective leaders whose decisions will drive a business forward. Here are the top 5… https://t.co/gDVZffzmDbFollow