Metrics. Metrics. Metrics. We love numbers. We measure and put numbers to all kinds of things. We use these numbers to mark our projects as red, yellow and red (of course, the project is always green until there are a few weeks left when someone finally blinks and acknowledges reality and begins to use yellow or, god forbid, red).
Unfortunately, in our headlong rush to create metrics we tend to forget the why of what we are doing. Numbers and statuses become an end unto themselves.
There are a myriad of problems with this. First, what get measured gets done. In our rush to get numbers we need to be very careful because measuring the wrong things will lead to all kinds of behaviors that can be detrimental to long term sustainability. For example, one company I worked for misunderstood the team velocity metric and rewarded teams based on the number of points completed. What happened? Over time the point values for stories increased so that teams would look better but the amount of throughput did not increase. This misuse of story points completely invalidated their even relative gross sizes to a point where they could no longer be used to give accurate information back to the business of what teams were capable over the long term. In other words, the valuable ability to be predictable was lost to service a poorly misunderstood metric.
The next problem is that we tend to measure those things that are easy to measure not necessarily those things that are important. There is an old joke about a drunk man looking for his keys under the street lamp.
I found the following account on Wikipedia:
A policeman sees a drunk man searching for something under a streetlight and asks what the drunk has lost. He says he lost his keys and they both look under the streetlight together. After a few minutes the policeman asks if he is sure he lost them here, and the drunk replies, no, that he lost them in the park. The policeman asks why he is searching here, and the drunk replies, “This is where the light is.”
If we only measure only those things that are easy to measure (usually easy to quantify with numbers) as opposed to those things that really matter, then we are no better than that drunk man looking under the streetlight because the light is better. As he will never find his keys, we may never find the truth by measuring what is easy versus what is important.
I often quote from Deming when discussing measurements: “The most important things cannot be measured,” and, “The most important things are unknown or unknowable.”
There is a very simple example that I use often when explaining this concept.
I ask people, “Do you have children?” “Do you love them?” “How do you go about measuring this love?” “Do you use minutes spent? Money spent? An combined weighted score that takes into account both money and time? Or do you do some regular poll of your children to see how much loved they feel on a Likert scale?”
Obviously, the love a parent has for his or her children is of paramount importance, but this is something very hard to measure.
I once spoke to a group of project managers and explained that we measure way too much. We measure things that are either easy to measure or do not really result in better behavior. You would have thought that I advocated clubbing baby seals! They decided that I was against all measurement. The answer is not that I am against all measures, but that I know that measures are limited in value due to the reasons outlines above (and many other human biases), so we need to measure less and be very careful what we measure.
In software development the primary measure of progress has to be working software that meets the needs of the end users. Of course we can measure other things, but there is no more important measure and all other measures need to be subservient to our ability to produce working software.
Remember to check out my Indiegogo campaign!
