How To Measure Things

I made a bit of personal history this past Thursday. I got up early and got to the pool before work, as is my daily habit, and logged another 3,000 yard workout. I've been spending some time with "toys" these days: 3x200 free w/ pull buoy; 3x200 free w/ snorkel; 3x200 kick w/ fins; 3x200 free w/ fins; 3x200 IM on 4:00. I felt great when I was done, as usual. I usually do 2,000 yards in the morning, but I cranked it up to 3,000 for three days to meet a goal. I've logged 520,000 yards for 2008. That's 10,000 yards each and every week, or 2,000 yards for each and every working day. My previous best total was 440,000 yards back in 2006. I've been keeping track since 1994. My average over all that time is 340,000 yards per year.

This isn't an extraordinary total for a really good swimmer. I think high school athletes can put in 6,000 yards per day, and Olympic athletes will do my weekly total in a single day. But it's something special for me.

How did this happen? There are a few important factors:

1. Good health. I've blogged about the bulging disk in my neck that ruined my 2007. It helps that I've had no injuries or sick time.
   
2. Availability of a pool. I've been fortunate for my entire career, because I've always been able to find a pool nearby wherever I've worked. And they've generally been staffed with people who make the opening of the doors and the arrival of a lifeguard as dependable as the rising of the sun.
3. Consistency. Woody Allen said "Eighty percent of success is showing up." That's certainly true here. I've always managed to weave swimming into the day in such a way that it's relatively transparent to my wife.
   
4. Swimming with others. For the longest time my ritual was to swim a mile of freestyle at lunchtime and get out. I never swam competitively, so I didn't know about workouts or intervals. I managed to figure out how to do butterfly on my own so I could swim the individual medley, but that was my only concession to trying anything new. In 2000 I finally worked up the courage to try Masters swimming. I was nervous. I didn't know if I could finish a longer workout, or keep up with the intensity. I found out that swimming with others makes all the difference in the world. The people I swim with are now among my best friends. The social interaction and better workouts have contributed greatly to my enjoyment of swimming.
   
5. Measurement. My tracking system is simple: I have an Excel spreadsheet that acts as the database and does all the lovely plotting for me. I roll everything up by the week. I have "best month" totals. I can tell every week exactly how I'm doing. I derive a great deal of motivation knowing that I'm close to a goal of some kind. Sometimes that's the one thing that gets me out of bed early in the morning.
   

I'm already thinking about how I can top it. Just 10% more each day will do the trick. One more 200 to make my habitual daily total 2200 yards. They add up. I find that the snorkel is helping my lung capacity, so I'll work that in every day. And I'd love to learn how to tolerate butterfly for longer stretches. I can endure 100 yards of fly in relative comfort now. I'd like to work that up to 200 yards at a time in 2009.

I've been thinking a lot about that last bit, measurement. Tracking makes a big difference. I've been working on a star schema for a relational database to house my swimming data. I've learned more about data warehousing from that simple problem than I ever knew before. If it ever reaches a more finished state I may blog about that as well.

But it makes me wonder: How can I use measurement, metrics, and tracking to motivate the other important things in my life? My last blog talked about my frustration over lack of progress in learning Lisp and other things. How can I start tracking my progress through "Practical Common Lisp" to ensure that I end up at the desired destination?

What about measuring software development progress? If I could figure out how to do that, I'd be a wealthy man. Everyone in the software development industry laments the sorry state we're in. A large percentage of projects are never completed or fail to deliver the desired functionality. If we only had enough management, oversight, governance, and measurement!

It's not that those things haven't been tried before. You can't throw a rock without hitting a methodology that promises better results than the scorned "waterfall". Project managers have been around for as long as there have been projects. (You can see them in Egyptian hieroglyphs, holding Gantt charts on papyrus.) Analogies abound: "Software development is like building a building" - hence the love of the title "architect". "Software is like manufacturing" - hence the term "widgets".

Where I work, the conclusion is that if some hasn't worked in the past, more in the future will make things better.

Why hasn't any of this worked? What's so different about software development?

I think my swim tracking has a couple of key features that software lacks.

First, it has a well-defined time scale built in. I know exactly how long it takes me to swim those 2000 yards every morning. I have a metronome in my head for each lap, each 200 yards. (Unfortunately, it's a very slow timepiece.) The week closes every Sunday, and the workouts pile up in a steady drip, drip, drip as the year goes on. I can easily extrapolate to the goal at the end of the year: "Still averaging 10K per week, with two in the bank to anticipate that week's vacation in the summertime."

Second, the metrics are well defined. There's nothing fuzzy to account for, like "quality" or "maintainability" or any other "-ility". It's distance and time and frequency.

Third, the metric are additive. I swim in a pool that's 25 yards long. Laps roll up into sets, sets into workouts, workouts into weekly, monthly, and yearly totals. If I look at the plot of yards aggregated at the end of the month I can clearly see where I'm leading or lagging. I got up early to do 3,000 yard workouts last week because I wanted to cross the 520K line without having to worry about winter weather keeping me out of the water.

Fourth, it's just one individual. I'd imagine that it's harder for a coach to keep track of an entire team. There's an art to cultivating each individual to achieve their best and orchestrating a team to an end of year peak for a big meet.

Manufacturing has employed statistics to drive continuous improvement since Deming and Shewhart showed us out. There are any number of Six Sigma black belts running around applying these techniques.

So why does software development resist such improvements? I believe there's something different about it. Like Knuth said, there's an art to it. The social aspects of doing it in large, far-flung groups don't lend themselves to statistical measure. We've all heard about Dilbert-esque measures that reward those who find bugs ("Look at that! Found another one!"). We've griped about the scourge of unintended consequences caused by bad measurements. Measure success and progress by KLOC or number of classes checked in and you're likely to see your code base expand in size.

I'm going to spend some time seeing if I can extend my success at measurement in swimming to other things. If I make any progress, I'm sure I'll post it here.