/dev/null

On Sadness

I saw a post this evening on a new blog by beN sky Hawk saying that he just lost his mother. I was reminded of these words:

When darkness falls
  like an overcoat,
Life's movements are draped,
  and only shadows remain.

I stand in the rain
  in front of the old church doors,
Having said goodbye again
  to the dearest of my friends.

It ended so serenely,
  not in a hospital, but a church,
The last words I ever told her:
  "Je t'adore" - I love you.

I do not know how she passed
  from this life to the next;
One never really knows
  how a passing will take place.

I only hope that she was able
  to take my words with her,
To wrap around her shoulders
  when she walked into the light.

Posted at 07:15PM Nov 04, 2003 by Cameron Purdy in Personal  |  Comments[3]

Why does Open Source suck?

Catchy title. If I added something to the title about Microsoft .NET / JDO vs. Hibernate vs. Entity EJB / Indian programmers / full frontal nudity / Marc Fleury to the title, I would probably have the most-read Java blog entry ever. Of course, the last two points together form a scary picture, so let's not even go there.

I've been thinking about writing this entry for a long time. Actually, I've been itching to write it. Before I attract too many flames, I should point out that I was trying to keep the title short, and it should actually be: Why it is easy for the quality of such a large percentage of Open Source projects to suck. The reason why I would like to clarify the title is because Open Source, in and of itself, is actually quite an impressive movement and achievement, with at least several dozen extremely impressive projects, and if you were to cut off the bottom 99% of the projects, the remaining 1% would actually be brilliantly good, and -- coincidentally -- those few good projects probably account for 99% of the "good vibes" that open source is associated with.

Let's start with a brief discussion as to why open source projects may be of higher quality: First, if a project is widely used -- which is more likely with open source since people are generally free to download, use and distribute the software -- the result is testing in more diverse environments and situations, which will tend to increase quality over time. Second, since the source is available, if there is a problem, or if anyone takes the time to review the source just for the heck of it, obvious problems can be found, reported and corrected.

The problem is that the useful projects of any quality (note that I said "isUseful() && isQuality()") are few and far between. And that's what I am going to try to explain here. Forget all the well-known projects of respected quality, like Hibernate and Jetty (both of which I've seen code from and we have customers using successfully and happily, so I feel like I can speak well of them.) Also, for a moment, set aside the big projects like Linux and Apache and JBoss, because they are actually made up of sub-projects, many of which suck (quality-wise), and some of which are gems.

I'm going to start by explaining how many engineers that I have met in my life that I would trust to write mission-critical general-purpose software. The number is definitely less than fifty, and probably less than half that, out of the thousands of software engineers that I have known. These few special engineers posess a combination of technical ability, drive, personal responsibility and communication skills that makes them both individually potent and collectively (with a team) magnificent. Most projects can only attract one of these individuals, at the most. Novell, for example, had one; he wrote Netware. Lotus had two; they (with another side-kick) wrote 1-2-3. Microsoft had one (as a contractor) on the Windows team; he made protected mode Windows possible in his spare time. IBM has a whole bunch somewhere hidden away, but unfortunately they are not allowed to write software that will see the light of day ;-). Sun had two; one wrote and the other documented Java. Borland used to be able to attract these types all the time, but could never keep them. Every financial services company that we work with tries to have one overpaid guru per project as an "anchor". You know the type I am talking about: This is the person that everyone goes to for advice, that can fix any problem in the system, that is the only person who truly understands the architecture, that knows how to get the builds to work, that feels that he/she is personally responsible for getting things done, and who always actually wants to quit and go work somewhere else where they might be able to become a nobody, which for them would be a permament vacation, because they could perform a typical job in their sleep and still do an exemplary job at it.

Now of course you are thinking this type of person is you. Well, maybe it is. However, despite that very slim possibility, I must now point out that 99% of programmers that I have met believe that they too are in this elite group. Maybe more than 99%. I want to believe I am in that group too. There was a study released a few years ago that showed that the more incompetent a person is, the more likely it is that they will believe that they are competent and that others are incompetent. (I guess that should prove that I'm incompetent; however, I choose to question the competency of the study author instead. Obviously, he's an idiot.) What I'm trying to say is that these "project anchors" are really rare. Like finding gold nuggets in a stream or something.

Now let's consider how these few engineers end up being the architects and technical leads for companies: They work their way into that position or they get hired and paid extra to do that job. Golden handcuffs, for example. They get paid extra to stick around and keep some old pile of crap working, because their boss knows three things: First, that this person is the only one who can do it; second, that they don't want to do it; and third, that they could make good money doing something fun elsewhere. This approach is generally what I would refer to as a meritocracy, which is to say that, over time, good engineers get more responsibilities, and generally they get better pay, and they move to more and more critical areas of work.

Now let's consider how a programmer becomes the architect for an open source project: They go to sourceforge and fill out a form to start a project. That's right. They don't even need a driver's license or a credit card. Peruse the projects on sourceforge and you can see that Danny who wrote some HTML pages for his cousin Bob's "Furniture City Superstore" web site has decided to build his own operating system kernel or enterprise messaging system or transparent persistence framework or J2EE application server. I'm not kidding, and you know it's true! Luckily for other people who have to work with software, most of these projects go absof*ckinglutely no-where. Unfortunately, a few of these escape "reality gravity", though, and become "successful open source projects".

Now let's consider how a programmer becomes a contributor to an existing open source project: They download the old code, they do something to it, and they submit their changes back. Cripes! They don't even have to fill out a form! Let me tell you, as someone who has seen some of these submissions in detail, this is a very scary thing that anyone with a keyboard can pretend to be 10 million monkeys and whatever appears to work first gets submitted.

The good news for open source software is that it does appear to have an above-average number of good engineers involved with it. Honestly, it does. The bad news is that it is more than unduly influenced by incompetence and (perhaps with the best of intentions) simple technical inexperience. That's because even the good engineers are typically doing it to learn something new and exciting, so you are getting the "first system effect" from even most of the "good" submitters.

So, why did I bother to write this? Good question. Sometimes I read these rants about how open source automatically obtains magical god-like quality. It's true in a twisted way: The ten open source projects that religious open source advocates will choose to point to probably have better quality than the average non-open-source product. My advice is to look at open source with an open but critical mind, the same way you should look at non-open-source products. Don't get caught up in the religion, because in the end, there's still poison in that kool-aid. Open source can benefit us all -- and I believe that it has benefited and continues to benefit us all. Let's not ruin it by attributing magical religious properties to it.

Use it if it works for you. Contribute if you can and you want to. Report back bugs to help it improve. Ask for useful features. Be thankful for it being there. Just don't lie to yourself and others about attributes that open source does not intrinsically have. It still takes a strong vision, continued focus, great engineers, hard work and long-term dedication to make good software. There simply is no silver bullet.

Posted at 07:03PM Nov 04, 2003 by Cameron Purdy in Technology  |  Comments[44]

Too funny!

Maybe I should add this blog to my list too. Check out this post which should give you at least a little bit of a grin.

Posted at 12:41PM Nov 04, 2003 by Cameron Purdy in Technology  |  Comments[0]