Relative Measures – to Use or Not to Use?

Recently I had some time off and used part of it to take a closer look at additional ‘agile’ blogs, forums, and community resources. I found a post that came from one of the agility consultants I know. It made me think about writing a blogpost about estimating within agile frameworks like Scrum or Kanban (or outside of them, but still in an agile manner). This topic is quite tightly coupled with one of the estimation techniques – relative estimations. I decided to share my experience as I found various misconceptions, but also a lot of value using those techniques.

What is a relative measure (of estimation)? I found a good description of one of them in Derek Davidson’s article and adjusted it a bit for the use of this article:

“Relative unit of measure is a unit decided within a team to provide relative estimates of size of requirements and effort needed to fulfil them.”

Doesn’t explain much, does it? No worries – if you work on a product in an “agile” manner there is a high chance you use such techniques or have at least heard about them. When you ask yourself what’s a relative measure, think about key messages of stories, t-shirt sizes, and similar topics. They are relative, because they need to be compared against something. Each of them needs a certain point of reference that is sized using the same technique and is not described in time units. Why is that important? Because translating it to time units makes it as absolute as time itself, so the whole idea of being relative is gone. There is no point in such translation, because you can use only time units – adding another level of abstraction only increases unnecessary complexity.

The way relative measures are used is that when you need to estimate something (user stories, features, whole backlog, etc.), you discuss the item and the team votes about the size. Because you have the point of reference this is straightforward – if something is smaller, give one unit less than the reference. If bigger – one unit more. Much bigger – two units more. Units can vary a lot – I personally prefer Story Points (SP) using Fibonacci sequence – 1, 2, 3, 5, 8, 13 and so on. It shows that if we want to estimate really big things – the difference between a smaller unit and a bigger unit also grows, as unknowns also grow (dependencies, risks, etc.). In general – the smaller, the more accurate.

Why just not stick to time units and do as we always did? Well, relative measures solve a bunch of issues you might face when trying to estimate in time:

• Start of a discussion: Everyone must agree on more or less the same estimation, with very different opinions the discussion starts. I experienced this benefit several times. When one person gives 3 SP and the other gives 13 SP – one of them (or both) is not aware of something at this moment, but this is exactly when the knowledge sharing starts. You can focus on business and technical questions rather than “maybe Peter can do this in 2h, but I’ll need a day!”.
• Lower Cost: Generally relative estimates are cheaper and faster to get. It is way easier to say something is bigger than/smaller than/same as X rather than estimating how many hours it will take. You do not need to spend hours during meetings to discuss if this is done in 3 or 5 hours and negotiating between the development team and the Product Owner that the team could possibly do it faster.
• “Aging” resistant: 5 Story Points are always 5 Story Points. In time, the team can do more backlog items of the same size within a Sprint as they gain experience, but comparison how big an item is will remain the same. When estimating in time it is way harder – is an hour today worth more than it was in the past?
• Easier to use for medium- and long-term planning: Having the number of items done within a few Sprints, you can calculate the average and use it to predict what - more or less - can be achieved. If the average per Sprint is 20 SP and you have 5 more Sprints – around 100 SP (or more if there is a growing trend) can be delivered. For example, with a backlog of estimated to around 120 SP, the Product Owner and the development team know they need to change something or are at least aware that the risk of not finishing everything is huge.
• Same for the whole team: Relative measures are used by the whole team, therefore you do not need to solve a problem faced when estimating in time. – but whose time is it? The average time of an average development team member? How to calculate Senior’s time? How much is Junior’s time? With relative measures you know that a team delivers 20 SP during a Sprint and this is all you need to know during the planning. Sometimes general capacity can be also useful to make some adjustments, but you can find a number of approaches how to include that in the planning as well.

As for now we have discussed the mostly positive impact of Story Points (SP) to the development team’s and Product Owner’s life. But are there any drawbacks? As usual – there are a few:

• They need some time: To be more precise and more helpful than simple units of time, it takes a while. Both the development team as well as the Product Owner must work with SPs a bit to get a proper understanding on how to use it effectively.
• They are harder to understand: Since SPs are more “virtual” than time units, it is harder to understand the value behind them. This is especially challenging when it comes to the management – those people are used to ask “When this will be done?”, so understanding what relative measures are about might not be easy.
• They won’t solve instability problems (and some other problems as well): If the team or the type of tasks/backlog items changes rapidly, frequently and from one edge to another, relative measures are useless just like any other type of measurement e.g. units of time. A stable vision and team are prerequisites to any product and project.

Misuse, misconception or misunderstanding can also hinder positive effects of relative measures. One of them is translating relative measure to time units I already mentioned earlier in the article. The other things I observed, experienced, or heard about are:

• Focusing on velocity. This is not what you should aim for. Just like in time-based excel reports (100% of our employees’ time was used – but did we achieve any business success?), some organizations treat velocity too serious and create management reports based on them. Velocity is not going up each Sprint by 15%? That might still be ok if you deliver business value to your customers. Focusing on velocity too much will likely result in e.g. inflation of relative measures. What costed 5 SP, will now cost 8 SP to meet the management expectations – but the value delivered will still be the same (or even smaller).
• Comparing between teams. Teams are not comparable in terms of relative measures. Velocity answers the question “how much was done or can be done during the Sprint”, but the thing is that it applies to the context of a single team. Teams doing 30 SP and 120 SP might be creating the same business value. If the reference is different in both teams, velocity will be also different.
• Weird/misleading measures. Remember the KISS principle – Keep It Stupid Simple. I’ve seen some very weird units used to measure size in a relative way. Fruits, robots or similar units might not help. In fact, it can do a contrary, as the differences are not that obvious. How much bigger is an orange from an apple? What kind of orange (there are at least 5 species of an oranges – varying in size and color)? Ideas about sizes of such units can also vary from one person to another. That increases complexity, which is something we want to avoid.

Relative measures can be a great help not only to estimate how much can be done within short period of time, but can also help with longer term backlog planning, monitoring effectiveness trend of the team, improving knowledge sharing, understanding complex business or technical domain, indicating problems (e.g. when velocity drops rapidly) and lower estimation costs. To use them properly, it is advised to follow some good practices:

• Not translate to time units
• Start with a clear reference
• Use easily understandable units.

At SoftwareONE we are living what we preach. As one of the leading IT service provider we are working in an agile manner to develop the best software & service solutions for our customers. I personally work in the development team for PyraCloud, our proprietary digital platform which provides customers with data-driven, actionable intelligence to manage and optimize their software and cloud spend. In this area we use Scrum to deliver value to our customers frequently and learn from this.

Watch our video to learn more about our digital platform PyraCloud and how it can optimize your software estates from on-premise to cloud