Story Point Paradox

Introduction

My employer has started using Story Points to measure efficiency of the development organization. My boss and I figured out a paradox that demonstrates why Story Points are not a good measure of efficiency the way we are using it. I am documenting the paradox here

Background

For those of you who don;t know, Story Points are a Agile development concept that represent the complexity of a task. The idea is that more complex tasks will take longer to do and less complex tasks will be faster. Whenever you start a new sprint, you have a backlog of user stories in priority. You assign Story Points to each user story. You implement as many stories as you can. At the end of the sprint, you add up all your Story Points, and this gives you a measure of how much you accomplished in that Sprint.

The way we have been doing this is that we start of with high level Feature tickets in JIRA. Then we go through requirements gathering and design process will we have enough clarity to start implementing the feature. Then the Team Leads break down the Feature tickets into implementation tickets, distribute the implementation tickets among the developers. The developers estimate the story points on individual implementation tickets, and the story points get aggregated upto the feature level. So, for example feature A needs a a) new web page, b) new service,  d) change in the backend engine, the TL will create 3 tickets, and they will get estimated accordingly to the complexity of each of those tasks

Paradox

So, what's the paradox? This looks fine at the face of it. Making an engine change is 4 times more complicated than making a new webpage so, the engine change gets 8 SPs whereas web page gets 2 SPs. Isn't that nice?

Ok, Now, let's say we make some improvements, whether it's an architectural change, or a development process improvement. Now, because of this improvement, the effort on doing an engine change goes down dramatically, and making an engine change is as easy as making a web page. The question is should the developer continue to put 8 story points on the engine change, or is this a 2 story point

If you think it's 2 Story point, you have just negated all the work we did to become more efficient. It doesn't matter if we make architectural improvements, or build process improvements or invest in infrastructure. Every time we make things easier for developers, the story points will reduce, and it would appear that we haven't become more efficient even if we did.

If you think it's 8 Story points, then you are making it hard for the developer to put story pints. The complexity of making a web page change or an engine change is same for the developer. But,  the developer has to put 8 points on every engoine change and 2 points on every web page change, simply because engine change has always been 8 points. If we have standardize all the story points, we cannot rely on developers to estimate story points