Tinkering: prototype your way out of uncertainty

Ten years ago I found myself near Portland talking to Bill, a brilliant colleague who was working at one of HP’s printer divisions. We were both worried about the lack of tools available for the particular set of problems we were working on —figuring out the algorithms that decide which drops to print, and when, in an ink-jet printer.

It turned out we had both been thinking along the same lines: we wanted a set of libraries for image processing, written in C++ for speed, and accessible from Python for ease of prototyping. We were all avid users of the available toolkits for manipulating images (the classic pbm, ImageMagick) but there was nothing out there that could meet our rather special needs.

In an hour we had drawn the broad strokes of what we wanted, and in less than a week we had put together a rather crude, but functional, prototype. We started showing it around to engineers, and building cool stuff with it, and very soon we had an enthusiastic user base in several sites in the US and Europe, and a bunch of great engineers contributing code.

And that was before actually telling our managers what we were up to.

How hard do you think it was to convince them to put actual resources on it once we showed it to them?

It was almost a done deal, despite the fact that we didn’t really know what we were doing, nor where it would lead us: our plan was only a broad vision that we were continously fleshing out. And this was not, mind you, a land of plenty: we were a bunch of printer divisions, all resource-strapped and fighting to deliver printers on schedule. But we do have some rather enlightened managers at HP, and the broad support they saw in the engineering community was enough to convince them to let us keep going.

Rinse and repeat

In about two months we threw it away and started again, almost from scratch. By then we understood the problem, knew of the limitations of the approach, and had a rather clear idea of what we needed to do.

Fast-forward to 2011: after another iteration 3 years further down the road, the piece of software became key for development at all high-end printer design sites. These days we are revisiting it again: it has more than 600k lines of code, but we find that its core —what was built during the initial stages, the architecture we settled upon after the initial months of tinkering— is still perfectly valid.

The moral of the story

Prototype. Tinker. You will not understand your problem until you’ve tested the waters, and the way to do it is by getting your feet wet with actual code. Plan to throw it away, at least twice.

Further reading

• Adapt: Why Success Always Starts with Failure (Tim Harford). Brilliant exposition of how successful innovation actually works.
• Hackers and Painters (Paul Graham). This should be required reading for anybody interested in how actual development looks like.
• Coders at Work (Peter Seibel). Get a glimpse on how some of the best coders around actually get things done.
• The Mythical Man-Month (Frederick P. Brooks). A true classic.

(Disclaimer: I do get a cut from your Amazon purchase.)