At Rally®, metrics are an internal tool we use to track how we measure up against goals set by our partners, by our product and project management teams, but most important by ourselves. We want to write good, solid software free of bugs and unnecessary complexity, so we track open bugs, as well as bugs violating our internal service agreements, and open technical debt.
Back in our early days *do*we didn't want to stay up all night doing releases, so we even had a metric to track when releases were being conducted. Thankfully, that metric is no longer needed!
We want our front page to load in a certain amount of time, and we want to support a given number of users. We have uptime obligations from our various contracts, but as an engineering team we typically aim higher – why accept good when we know we can hit great?
That said, if your metrics are always perfect, you're probably not tracking the right things, or are not being aggressive enough in what you're tracking. Awesome though we may be, we are not perfect. We have downtime. We let bugs slip through the cracks and eclipse time-to-fix commitments. We have hotfixes. We don't hit a perfect 100 percent sprint commit every single time. Things will be in the red, and that's a fact of life.
One of the hardest things to learn is the concept of having “permission to fail” – the idea that not only is failure sometimes expected, it is actually encouraged. We’re trained from an early age that failure can never be a good thing, or even have any positive effects. But some of the best ideas throughout history started in failure, and there’s no reason to expect that the pattern won’t continue. This is hard to internalize when a group is involved, as group decisions and goals tend to be safer.
So rather than thinking of a red metric as a personal affront or a failure, we take a different view. To us, a red metric is an opportunity to get better. Even if it's something we normally do right, it's a place to pay attention. The message to our engineering team is this: If you look over some of our metrics and see something that you think you can personally fix, then you have a chance to fix it (and, ideally, brag about it in your performance review). If you think you have a great idea that could make something better, it's your chance to speak up. It’s not turning failure into success, it is evolving the way we think and operate to achieve greatness.
Typically, our engineering quarterly goals focus heavily on bugs, and that's probably the metric that is potentially the messiest, and easiest to manipulate – two clicks and you can take a fix time-violating critical bug and make it minor where it can theoretically sit until the end of time.
But whether it's bugs or anything else, that's not the right thing to do; if you have a whole bunch of overdue bugs in your project, it doesn't mean that you have a low-quality product, it means that you're overdue for a bug smashing sprint or a bug-a-thon. These focused team or organization-wide efforts are a great way to clean things up. Likewise, it's never great to have to scramble to fix something, but if you *do* fix it, it means that your end users are having a better experience, and faster, than they otherwise would have, and that can only be a good thing.
This isn’t an approach that’s just for developers, either. As an engineering leader, I touch most of the other teams at our company, from sales to customer support to recruiting. They all keep their own sets of metrics, and they all are continually on the lookout for ways to improve them, whether they’re in the red or not.
As the DC Metro happily tells us every 3.5 minutes, "If you see something, say something." And that’s how we roll, too. The message to all our teams is to get comfortable with the color red, because it means you’re pushing hard and we know you’ll fix it. And if you spot something the metrics missed, feel free to let us know. In the end, our organization will be stronger for it, and by giving red a chance, so will yours.
Sam Freund is Senior Director of Development and Chief Pony Officer at Rally Health.SM
SAM FREUND
Rally Health
