Release planning is essential to fulfilling the expectations of both your customers and your teams. However, often the approaches we use to create a release plan are inherently unreliable.
One of the senior managers that I’m currently consulting came to me the other day, and said: “Sonya, we need these 70 items to be delivered in the next release. How should we approach that?”. I quickly looked into their historical data and in a couple of minutes, I figured out that there is an 8% probability of achieving that goal. I came back to him and asked: “There is an 8% chance of that happening, are you willing to make a bet on an 8% outcome?”.
And I hear managers ask this question all the time: “How much work can we finish in the next sprint?” or “How many items can we deliver in the next 60 days?”. Deciding how much work you can commit to in your next release could be challenging if you don’t have the means to make reliable decisions at your disposal.
Still, there is a straightforward technique that will enable you to come up with an accurate release commitment without spending any extra time and effort.
The Flaw of Traditional Approaches to Release Planning
The traditional management approaches to release planning suggest that the team should sit down together, go through the individual work items and estimate the effort needed to complete each item (either in hours or story points). Then, the times are accumulated and a margin is added on top of the total sum, in order to account for any unanticipated events.
The result looks like this: “We will need approximately 15 weeks to deliver.” or “We will need about 8 sprints to complete the work.”
Apart from being tremendously time and effort consuming, the main problem with traditional approaches to release planning is that they are not reliable. And here are just a few reasons why these methods are fragile:
- Estimating the effort time required to complete your assignments is a prediction based on guesswork, intuition or gut feeling. It doesn’t use historical facts to produce accurate results.
- By evaluating the scope of the work, you ignore one of the major sources of inefficiency and delays – the variability in your system. Your effort time doesn’t include the waiting time in your process (which usually represents anything between 60% and 99% of your delivery times).
- That commitment comes as a single value and thinking deterministically about the future is simply not enough. There are always more than one possible outcomes that might happen in the future.
When planning our releases, we are constantly confronted with possibilities whose outcomes we don’t know. Getting rid of uncertainty and being in control of everything is impossible. The real challenge lies in dealing with the uncertain and unknown in an effective way. After all, that is within the realm of probability.
The Value of Probabilistic Thinking
When we talk about delivery predictions, we need to acknowledge that there are many possible scenarios that could occur in the future. We simply cannot predict what will be our outcome, and we can never declare anything with 100% certainty. There is no model that provides a 100% probability of achieving a certain goal. The future is not deterministic. That is exactly why we need to start thinking probabilistically.
How can we answer the question “How much can we put into our next release?” with maximum accuracy, while fully embracing all the uncertainty that the future provides?
One of the easiest, fastest and most reliable approaches that provides the answer to this question is producing a probabilistic forecast.
Probabilistic forecasts come with a range of outcomes and the probability of achieving each of them. They typically look like: “We expect to deliver 370 work items or more in the next 15 weeks and we are 85% certain it will be” or “We will deliver at least 350 work items and there is a 95% probability that we’ll meet our commitment”.
Probabilistic forecasting is among the most accurate methods to tackling release planning and it enables managers to come up with a reliable commitment without having to spend any time and effort.
How to Do Release Planning Without Counting Hours (or Story Points)
You don’t need to evaluate your work items in terms of hours (or story points) in order to come up with a reliable release plan. What you actually need to do is look into your past performance and analyze it to produce probabilistic outcomes. And there are tools that will enable you to perform probabilistic forecasts for your release planning.
When determining the amount of work to put into your next release, we use Monte Carlo simulations to come up with a range of probabilities. The simulation relies on a large number of random trials based on your historical performance data to predict the throughput for a future time frame.
You define the start date and the release date and the simulation will provide a range of possible outcomes and the probability that comes with each of them. It will use the throughput of a random day in the past to simulate how many work items are likely to get done on any day in the future.
In the simulation above, on Sep 25th, you had a throughput of 2 tasks. The simulation takes this number and assumes that this is how many assignments will be completed on March 1st. Then, to project the probable throughput of March 2nd, it takes the throughput of another random day in the past, and so on.
The simulation is repeated tens of thousands of times before the results are presented in the form of a probability distribution with percentiles increasing from right to left. In this simulation, we set the release date as May 1st and stated that we want to start working on it on March 1st. The simulation tells us that there is an 85% probability of completing 44 work items during this time period. The further you go to the left, the greater the certainty of completing the projected outcome.
Using Monte Carlo simulations to forecast the amount of work we can put into our next release is one of the most accurate approaches to release planning, as it takes into account all the variability in your system.
If you’re interested in learning all about the reliable and unreliable approaches to making future predictions, I couldn’t be more thrilled to welcome you to our Sustainable Predictability program.
Strategies for Risk Mitigation
You can commit to project scope, delivery time, or quality – but not to all three. So, my advice would be to pick time and quality and keep the scope flexible.
The main benefit of using Monte Carlo simulations is that the method clearly defines the risk associated with certain outcomes. The percentile lines on the distribution (50%, 70%, 85%, 95%, 98%) communicate the risk you’re managing. The probabilities that the simulation produces quantify that risk in terms of percentages.
The question now is how much risk you’re willing to take? If you plan with less confidence – let’s say you commit to 54 work items on the 50th percentile (which is the same confidence level that comes with flipping a coin) – you really ought to ask yourself whether you want to manage that level of risk.
Think about your release planning as gambling (hint: guess where the name of Monte Carlo comes from). The number of items we could supposedly deliver is higher at the lower confidence level. But, how much risk do you want to take?
During our release planning process, we want to mitigate the risk as much as we can. 50% certainty is very low when it comes to release planning. Ideally, you’d like to work with 70%, 85% or 95% certainty when you’re making your commitments.
If you look into the far most left side bar on the Monte Carlo simulation, you will see that the probability is greater than 99%. Remember, there is no such thing as 100% certainty that something will happen in the future.
Your stakeholders want to be as confident as possible that you will deliver on your commitments, and they want to reduce the risk of failure as much as they can. To help your customers or stakeholders understand that concept easily, communicate your release plan in terms of risk.
If you commit to the number of tasks on the 50th percentile, you will be wrong 1 out of every 2 times. If you commit to the amount of work on the 85th percentile, you will be wrong 1 out of every 7 times. If you go for the 95th percentile, the risk of failure drops down to 1 out of 20 times.
The higher the percentile you select, the more confident you can be when making your commitment as the lower the risk of failure will be. That’s how you manage realistic expectations and plan your release in the most effective manner.
