Data-Driven Decision Making: The Value of Probabilistic Thinking
More than 50 years ago, Peter Drucker, a management consultant and author, described as the founder of modern management, predicted we would shift to a knowledge society. He said information would change the way people work – that knowledge will become the most important economic resource. Instead of generating value through physical labor with their muscles, people will do it with their minds. Drucker described these people as “knowledge workers”.
So, who is a knowledge worker? Simply described, a knowledge worker is someone whose job requires them to think for a living. Knowledge workers are people who “figure out” what they need to do. People who weigh many factors to determine the best course of action. People whose actions are based on multiple sources of information. These are the people who gather clues, follow up and discover new possibilities.
In knowledge work, we make assumptions about the future. In the most effective manner, we validate these assumptions by delivering results early and often. We learn what brings value and avoid the risk of spending time and money on the wrong things. That is simply because the nature of knowledge work is unpredictable.
People Feel More Comfortable Being Wrong Than Uncertain
When it comes to product management, the big question always seems to be “When will this be done?”. The problem is rooted in the expectation of having a single certain delivery date as an answer. And people tend to provide it. Fact is people feel more comfortable being wrong than uncertain.
The fear of uncertainty is a cultural problem. Professor Geert Hofstede, a Dutch population biologist, conducted one of the most comprehensive studies of how values are influenced by national culture. He defines six dimensions of national culture, one being the uncertainty avoidance index. The index is scaled from 1 to 100 – the higher the index the more uncomfortable to uncertainty you are as a society. Countries exhibiting strong uncertainty avoidance index maintain rigid codes of belief and behaviour and are intolerant of unorthodox ideas.
We’d rather provide a single certain delivery date that’s wrong than remain uncertain.
The Difference Between an Estimate and a Forecast
How can we answer the question “When will this be done”? Can we predict exact future delivery dates, despite the unpredictability of knowledge work? Are we capable of predicting the exact time a feature will take to go through the whole process while coordinating the rest of the work in progress at the same time? Is it possible to predict that there won’t be any additional work coming in between, any dependencies, bottlenecks or external blockers that might cause a delay?
There are two approaches trying to provide the answer to “When will this be done” question – an estimate and a forecast. What’s the difference?
Estimates are predictions based on guesswork, judgment or a gut feeling.
The prediction is delivered as a single value – it could be a date or a number of days for example. An estimate doesn’t involve any probability of its occurrence.
Forecasts, on the other hand, are based on historical performance data. The prediction is communicated as a range of values and the probability of those values occurring.
Forecasting is faster, cheaper and more reliable than estimating.
Increasing Predictability Through Probabilistic Outcomes
When making business decisions, we are constantly confronted with possibilities whose outcomes we don’t know. Getting rid of uncertainty and being in control of everything is impossible. The real challenge is dealing with the uncertain and unknown in an effective way. That is the realm of probability.
Using a probabilistic approach of making future predictions based on past performance data increases the predictability of our outcomes. Making probabilistic forecasts helps us maintain high levels of customer and stakeholder trust, satisfaction and retention. It allows us to define service level agreements with more confidence and deliver value to our customers in a consistent, predictable manner.
Learn To Think Probabilistically To Improve Decision Making
Thinking probabilistically means having a willingness to always ask questions like “What else might happen?”, “What could happen next?”, “What if we’re wrong?” and to look at the full range of possibilities that might come to pass rather than to assume that things will go as planned. Ignoring unwanted facts won’t make them disappear. Knowledge work is complicated and lots of unexpected and surprising things happen.
Start making probabilistic forecast using historical performance data in parallel with your old approach. If you have the data, use the data. If you do not have the data, then get the data and use the data. It will probably give you as good or even better answers and it will take a lot less time. Keep both the approaches and observe the results. At some point, it will become obvious which one works best for you.
Thinking probabilistically increases our predictability and improves decision making. It provides an alternative to get more accurate answers with a lot less effort. Most importantly, it enables us to spend our time on what matters the most – delivering customer value.
Meet the Author
Sonya Siderova is a passionate product manager and a driving force behind Nave, a Kanban analytics suite that helps teams improve their delivery speed through data-driven decision making. When she's not catering to her two little ones, you might find Sonya absorbed in a good heavyweight boxing match or behind a screen crafting a new blog post.
Take your training and consultancy sessions to a whole new level. With Kanban analytics on their existing tools, yo… https://t.co/cFCxoaUeJNFollow
How organizations can be successful in their pursuit of agility by leading with purpose, creating customer-centric… https://t.co/5sHBveVpmjFollow
A chain is as strong as its weakest link and a process is limited by its slowest step – its constraint. The Theory… https://t.co/ub5OtCXWijFollow
A Cycle Time Histogram with a big hump on the left and a very long tail to the right indicates that your cycle time… https://t.co/qVnh8bTFZmFollow
Identifying services and achieving flow of information are the keys to scaling across an enterprise. Learn more abo… https://t.co/EFFDbNjV9JFollow
Businesses with higher levels of team motivation perform better for a simple reason: they appreciate their employee… https://t.co/VnkDTJAIF9Follow
Immersive charts for your Jira projects. Get more done, faster, with Nave analytics on Jira. Start your free 14-day… https://t.co/cyZhw9wQw8Follow
Learn how to efficiently manage blocked work in Kanban to drive evolutionary change and deliver work in a quicker,… https://t.co/NINEr9yPZxFollow
Rely on data, not instincts. Nave Power-Up for Trello builds immersive analytical charts over your Trello boards. T… https://t.co/AfHkWytZeBFollow
High flow efficiency is vital to sustaining a scalable, productive business. Learn how you can calculate, analyze a… https://t.co/TjXc2yOgqvFollow
Follow this guide to quickly and easily set up Nave in Jira. We've broken down the process in three simple steps so… https://t.co/6RsMGcCUFFFollow
Product teams often argue which is better, but the focus should be on how to merge Kanban and Scrum together to del… https://t.co/lVOh9ARV7jFollow
Get straight to the essence of your Azure board data and analyze your processes with our immersive data-visualizati… https://t.co/wzeM5fv1UTFollow
Optimizing workflow efficiency is often at the forefront of project managers’ minds, but eliminating individual's i… https://t.co/QqQ6lcK9dkFollow
Yesterday's webinar was a great success. It was such a pleasure to have so many people passionate about making reli… https://t.co/4hszHx1h5hFollow
Rely on data to build better software. Your Jira project, visualised across enhanced Kanban analytics. Try out our… https://t.co/oc8pksGwkTFollow
Improve your business' performance while working remotely with Kanban. Presented by SquirrelNorth along with guests… https://t.co/FvuqFHClcHFollow
Learn how to make accurate, data-driven estimation to stay on track, meet deadlines and keep a high level of custom… https://t.co/MHXZoGOVY1Follow