Many organizations rely on metrics: spreadsheets, data, numbers and measurements in order to monitor the health of projects. In a traditional project where value isn’t delivered until the end, it’s probably pretty important to try to know where you are along the way. In Scrum, the regular delivery of valuable shipable, high-quality, working software negates the need for many traditional measurements. Here are a few ideas to keep in mind when thinking about metrics in an agile project.
1. Measurement Paradox. Albert Einstein said “not everything that can be counted counts, and not everything that counts can be counted.” This is the heart of the measurement paradox. Especially with modern technology and ALM systems we have access to so much data! But just because the data exists does not automatically mean we should use it or that it’s valuable. Additionally, as we start to use some of this data, we create all of these isolated measurements – and when we present these isolated measurements to management and stakeholders, they draw system wide conclusions. Hey, that’s not fair! Great examples of isolated measurements in software projects include lines of code, number of test cases, number of hours worked, etc.
2. Hawthorne Effect. When you measure something, you influence it. The Hawthorne effect describes this behavioral psychology concept where people unconsciously change the way they behave in response to being measured. This is not necessarily a positive or negative thing, but it turns out that most traditional metrics have negative Hawthorne effects. For example, measuring test pass/fail percentage almost always causes the percentage to go up. As people think to themselves “higher is better,” they may start writing smaller test cases, passing tests and writing defects anyway, or a number of other things that will affect the passing percentage. In an agile project, try measuring accepted features or cycle time for closing customer complaints – these types of things have positive Hawthorne effects.
3. Measure Up! It turns out that not only do you influence what you measure, you also end up getting only what measure, and loose things that you cannot measure like collaboration, customer service, employee happiness, and creativity. Woah! – those are the most important things, aren’t they? The things that every business strives for, yes? Robert Austin talks a lot about these concepts in his book “Measuring and Managing Performance in Organizations” and comes to a clever conclusion: Measure Up. The idea is to measure one level above from where typical metrics are taken. For example, in software project we often tie defects to the developer that wrote them. Instead, try measuring a level up: perhaps defects per team, or better yet, customer reported defects. These higher level measurements make it easier for individuals and teams to collaborate and work together to solve problems. Besides, when it comes right down to it, your customer doesn’t really care about all the low level metrics.
When Scrum teams are delivering done software every few weeks, there is very little need for traditional predictive metrics. Ensuring you have done potentially shipable software each sprint, looking at a release burndown, and perhaps some measurement of team happiness will provide your organization with more valuable information than any set of isolated metrics ever could.
Also cross-posted here: http://www.centare.com/agile-metrics-101/