Proud to be a Scruffy

Zak Kohane was at C-SHALS last week. Zak is unbelievably impressive in the contributions he makes as a physician, researcher, teacher and leader in the healthcare community. As well as his MD, he also has a PhD in Computer Science from MIT. Zak has been a leader in the PHR world including PCHRI conferences I have attended the last couple of years.

Zak presented his own experiences using and promoting semantic web technologies in getting things done. He asserted that there are two kinds of engineers in the world: scruffies and cleanies.

I had previously heard that the two kinds of engineers are starters and finishers. Some start projects, do amazing imaginative work, and leave the project when it is 90% done, too bored to finish it off and make it usable by real humans. Finishers on the other hand know how to make sure the software is of high-quality, easy to integrate, easy to maintain and easy to use. Good software teams need the right balance of starters and finishers.

However, Zak's classification is different (starters and finishers can be either cleanies or scruffies). A cleanie is one who sees progress proceed in stepwise fashion where each step needs to be well thought through and in some way complete before proceeding to the next step. A cleanie sees that overall success in a field may take a while, but finds satisfaction in the beauty of completing each step even though that work is only a building block and not useful on its own. A scruffy sees the world as a random walk through the problems of the world, providing just enough of an answer to get some basic results and building on that. A scruffy knows the work is not complete, and knows s/he can return to it as needed. A scruffy knows some of the work will be useless because it is scruffy, but will be progressing fast enough that some number of failures are not too discouraging.

I used to inhabit a techie world of databases where, in the 1980s and 1990s, large corporations tried to combine all their data into a "data warehouse". This involved getting everyone to agree on how the data is used: do we think about customers at the level of the single company who buy a product or the various locations that use it; do we think of a product as a particular formulation or the various packages in which we sell it. These are important questions and big political fights would ensue. This was a project for cleanies ... those who understood that collating a unified view of the data in the enterprise was a necessary step to seeing how all the activities worked together and could be improved. By combining the sales, delivery and customer service records for a customer, the company could be much smarter about building a strong relationship... but it required the cleanies to be able to combine all that data first. They are still at it, and will never finish (by the time they do, someone has changed a key piece of data elsewhere which needs to be examined).

Meanwhile, the scruffies decided to get on with things and created "data marts" (notice the plural). These are subsets of the company data, designed to help out with a particular task, and so make it much easier to agree on data combinations. The scruffies got two or three departments in a room for a couple of hours, showed them the benefits of agreeing on how to combine these few items of data for this particular purpose, and got on with it. There are now multiple data marts in every company, many overlapping, and the cleanies scoff that the scruffies are making the problem worse ... but the cleanies are never done, and the scruffies have already produced a decade of good results.

This story is a narrative I could not have told before Zak presented his classification. As he spoke, my understanding of my time working in data management fell into focus. I realized that I am a "recovering cleanie" turned scruffy. My undergrad degree is in Mathematics, which is the ultimate cleanie project (especially for those in the pure math world - I suppose applied math is, by nature, scruffy). I have therefore always loved the idea behind cleanie projects, but have come to see over the years that they generally do not and cannot succeed. There are exceptions, I am sure, but I would like you to name one.

Venture capital is a scruffy industry. Startups are scruffy. They cannot afford to build a perfect marketing campaign before executing it and recognise they could never agree on that perfection. I have learned that those projects which appeal to my cleanie nature do not gain traction in the investor world or the customer world. People have objectives to meet to earn their annual bonus (or keep their job) and they don't have time to wait for the cleanie approach to work - so they choose the scruffy approach which mostly does the job.

Non-profits are scruffy as well. Non-profits can be nothing else ... they respond to failures in society, government, civilization. Advocacy groups might possibly be cleanie - they want to fix the underlying issues so that no-one is poor, sick, uneducated, subject to prejudice, harrassment, abuses. Religious groups are cleanie - they have the ultimate long-term cleanie agenda to fix the world, one soul at a time. Most other successful non-profits, those providing services, educating, urging and organizing action are scruffy: they are working with too few resources, happiest when doing something, even if the methodology or the philosophy is incomplete, inconsistent. Most non-profits are startups and most stay small. Some grow big and grow to have cleanie ambitions and some succeed (perhaps more than in industry). The Bill and Melinda Gates Foundation is cleanie with some scruffy undertones - they want to find a vaccine for malaria, for AIDS, for TB (cleanie projects all) - but they also want to get a bed-net into every home at risk from malaria and to provide condoms to all who need them (scruffy approaches to existing problems). Even one of their goals "to develop health solutions that are effective, affordable, and practical" has a scruffy ring to it.

So here I am, a Venture Capitalist, proud to be a scruffy, in a scruffy industry, funding scruffy companies, producing scruffy products, that customers buy because stuff gets done. I am also a Venture Cyclist, a board member of two non-profits: JCDS, a school with cleanie ambitions (as private schools often must, to meet the high expectations of yuppie parents), and Hazon, a scruffy organization hoping to change the world one bike ride and one meal at a time.

There are certain projects that require a cleanie approach: making sure teachers in a school have no criminal record - this should be a cleanie project - but even this simple and obvious need is confounded in a world of scruffy databases and scruffy (or expedient) laws.

My experience is that scruffy trumps cleanie... and we should learn to live with it, and even love it.