Philosophical Tenets of Open Source

Historical Roots of Open Source

The open-source philosophy has its roots in the era of timesharing at large research universities such as MIT, Stanford, and CMU. In the 1960s and 1970s, these schools had research laboratories whose researchers and projects produced software--this included basic platforms such as operating systems and programming languages as well as higher-level, application-type software. This was necessary because either the research work itself was in the area of operating systems and programming languages or there were no suitable commercial or "professionally" created systems filling these roles.

The computers these labs used were almost exclusively timeshared machines, with a limited number of terminals connected to them. During the day, the machine was heavily used not only for research but also for writing papers and for administrative and clerical purposes. Yes, there was email in the 1960s. Imagine a 1 MIP machine with about a megabyte of memory shared by 100-200 people. A lot of real work was done at night.

In a typical corporate office at that time, doors were normally locked at night and the only people wandering the halls were the janitors, who had keys to every office. Terminals were usually in offices, and if those offices were locked, the terminals were not available for use by anyone without a key. Almost certainly there were fewer terminals than people, and so the physical resources were shared as well as the computational ones. It was assumed that work took place during business hours and that the computational resources either sat idle at night or were running under the control of specific authorized individuals. After all, the computers were expensive and were required for the proper operation of the company.

Imagine if this were also true of university research labs. People could start large computations and head home, experimental operating systems might crash, experimental computations might crash the operating system--in short, things could get wedged easily. Moreover, there could be more people wishing to work than available terminals in unlocked areas and offices.

Suppose it's the 1960s and the terminal controlling an out-of-control computation is behind a locked door and the problem is sufficient to hamper other people working at 2 am. What can you do? Suppose there is no way to kill or suspend the computation except by accessing the controlling terminal? Must everyone go home until the one person returns the next day? Time is too valuable, and there might be 50 people who would have to go home because one person has locked his or her door. Or suppose the operating system has crashed in such a way that even rebooting wouldn't fix it--the permanent context has been changed enough that the operating system software needs to be fixed (perhaps a new peripheral has been added, such as a robot arm). Do you wait until you can talk to the programmer in charge to make the change when you yourself are sufficiently expert to fix things?

No, you institute three changes from normal procedures:

• You do not allow offices to be locked.
• You create mechanisms for attaching a different terminal to an existing session.
• You permit anyone to work on any of the source files.

This would be horrifying by some standards, but it worked just fine and created a culture that had very high productivity. First, rather than having a single or several system administrators (sys admins), such labs developed quite a few homegrown sys admins, perhaps a few dozen for a 200-person lab. These folks could do most anything sys admins do today, and in addition they were pretty darn good system programmers--typically for each part of the operating system or other system software there would be one or two experts who had created it and another three to six who were competent enough to maintain and improve it. People who joined the lab had all the source code available to look at how things were done and, in fact, to learn how to program well. What better way to learn about programming than by studying the working code of master programmers?

The result was that the machines were productively used 24 hours a day, 365 (or 366) days a year, with support. Operating systems, programming languages, and a lot of the software we take for granted today were developed incrementally over a few years' time with the full-time or part-time help of hundreds of programmers.

Social pressure and a kind of hierarchy of merit was used as a filter on who could work on what. Vandalism, bad behavior, and harmful mischievousness rarely, if ever, took place, such was the strength of the code of ethics that developed in such installations.

Here's a story we heard from the old days--it may not be true in its details, but its essence is accurate:

A new faculty member joined one of the "open-source-like" labs and had a terminal in his office. Used to a more traditional environment, he started a job, locked the door, and went home for the night. The job ran away and locked up the system in such a way it needed to be killed from the terminal or the system would need to be rebooted. One of the system programmers had to break open the door, pulling the hinges out of the door jamb. The next morning the new faculty member found his door leaning against the wall, splinters all over the floor, a note on his seat explaining what happened, and another note from the director of the lab informing him that his unrestricted funds account would be charged to have the door and jamb repaired.

This is the culture from which the open-source movement emerged. The attitude is that people will behave well and that there is a common good being promoted. Source code is part of a shared experience that builds and supports the culture.

Warning

Therefore, if as you read the following tenets you feel that they don't match how you work, how you feel, and how you could work, then you will not be able to fit into the culture required to make a successful open-source project. In that case, you should not pursue open-sourcing your project: Your project will not succeed, your project will embarrass your company in the public eye, and you will have mangled your project in such a way that you will have to abandon it.

Harsh, yes, but open source is not for everyone or for every project--at least not yet.

Let's look at some of the tenets of open source.

Everyone Together Is Smarter than Your Group Alone

When a producer puts on a play, audience members have no choice but to assume the role of critic: either you like the play or you don't, and you tell your friends what you think. But if the playwright were to turn to potential audience members at some point while writing the play and ask for help, these potential audience members would probably try their best to help and at some point might assume (in their own minds) a co-author role. It's harder for a co-author to be as harshly critical as a critic.

Inviting people to look at source code puts them in a frame of mind to help out. Not only will you gain a lot of testers, but these testers will have the source code and many of them will be able to locate the source of a problem in the code itself. The sum total of people running the software will encompass a wide variety of experience and expertise, and it is not unreasonable to assume that every type of bug in your code has been seen by someone in the pool at one time or another. It's like having a thousand people proofread a book for spelling and typographic errors.

You will mostly get highly informed and dedicated testers. The fact that the source code is available means that many of them will be at least able to develop their own workarounds to problems, and so they will stick with the software longer. But just as important, you will get a number of people who will want to contribute, often substantially, to the core source code.

A very common reaction to the idea of someone outside the organization contributing to the code base is that you don't want "a bunch of losers messing things up." Our experience is the opposite. First, there is no requirement in open source that every change proposed by someone must be taken back into the main source branch. So, you can select what you take, and you can modify or improve it if you wish. Second, it is your decision whether someone is trusted enough to check things directly into the source tree. You will have developed a relationship with all these people, looked at their work, and worked with them to improve their skills and use the style you want and, in most cases you will wish you could hire them. Third, there will not be that many of them anyway. For Linux, which is the biggest open-source project, there are on the order of 100 people who are permitted to check code into the source tree without supervision. Many of these people "own" parts of the code or modules, and they supervise other contributors. Fourth, almost all of the people who have access to the source code will be able to identify and perhaps isolate problems, and this will make things easier on the developers. In fact, most people will hesitate to mess things up and will simply report problems.

The phrase "innovation happens elsewhere" is based on the observation that not all the smart people work for you. But suppose they do: Suppose that all the people out there are losers when it comes to your project and its software. What will you do if one of your developers becomes ill or leaves your company? Any replacement will have to come from this pool of losers. Do you really believe you are in this position? Better have excellent health benefits, fabulous compensation packages, and bodyguards for your developers.

Open Source Requires an Open Development Process--Starting with the Design

It might be tempting to think that open source is really just about making the source code available for viewing. If you drop code into a public location and continue development on a private copy inside your company, you are not doing open source. You are running a source access program or simply providing extra documentation. To develop the right level of trust, the code out there must be the real source code that your developers are working on. You might have various modules checked out while the code is being worked on, but that must be true on a module-by-module basis not the entire source base.

Moreover, your developers are simply part of the larger pool of developers. They are perhaps better focused, they might work together better, or they might have goals of importance to your company, but they are otherwise in exactly the same boat as all the outside developers.

This means also that many, if not all, of what would otherwise be internal development mailing lists need to be public. All design and implementation decisions that affect the public source code must be done in the open. There can be no distinction between us and them.

In exchange for this, you build incredible loyalty and an intense desire to move the code forward with a shared vision. You get immediate and totally relevant feedback from the market. If you are short of resources, perhaps they will be supplied from the outside. The highly influential book on people management Peopleware (by Tom DeMarco and Timothy Lister) suggests that a good way to build a team is to have an introductory shared project, where the project itself comes to symbolize the group. Perhaps it's getting your newly assembled team together to cook an impromptu meal, but in the end it's working a shared artifact that does the trick. It's the same trick here. But it's no trick, it is part of the social nature of people.

The Business Model Must Reinforce the Open-source Effort

You cannot charge people to use code that they are writing or have written. That means that you cannot simply take the code and charge money directly for it. You need to choose a business model that supports open source. The following are some classic open-source business models:

• Bundle open-source software with perhaps some other software and with support, charging for the bundle and for additional testing and quality.
• Add value in the form of additional modules or surrounding software and sell that additional software bundled with the open-source software.
• Provide a service based on the open-source software, such as a subscription service that updates customers' sites with tested and assured code.
• Sell consulting services that leverage the open-source software.
• Sell ancillary things such as books, T-shirts, and mugs.
• Sell hardware that runs the open-source software particularly well.
• Sell software that uses the open-source software as a platform.

There are two more business models that apply if you own the copyright to all of the source code and expect mostly minimal contributions from outside developers.

• Release the software as open source, but license the software to companies that wish to use it in a proprietary product.
• Sell the newest version, but release the previous version as open source.

Ubiquity, winning over hearts, thousands of eyes looking over the code, better platform security, and getting additional outside help are some of the reasons to do open source. The classic case is that you need to make a platform ubiquitous for some other business reason, and so you use open source as the conduit to ubiquity.

Creating and Involving Outside Developers Requires Internal Resources

Many people think doing open source is a no-brainer: You can add developers to your team and don't need to pay them. However, you do need to attract, foster, and support those outside developers, and that does require internal resources. First and foremost, it requires that your internal developers take the time to participate on the public mailing lists, answer questions from outside developers, and promptly respond to bug fixes and code contributions. Your developers also need to document the system architecture so outside folks can find their way around in the source code. You may even need to rewrite the source code to be more modular, especially if it is currently one big monolithic mess.

Another cost is setting up and maintaining the infrastructure needed for an open-source project: a CVS code archive, a bug database, various mailing lists, and a website for the project. For large projects, these can each require a full-time person.

Open Source Is a Gift Economy

To understand open source, it helps to make a distinction between a commodity economy , to which we are accustomed in a capitalist society, and a gift economy . In a gift economy, gifts are exchanged, forming a bond based on mutual obligation: In the simplest form of gift exchange, when one person gives a gift to another, the receiver becomes obligated to the giver, but not in a purely mercenary way--rather, the recipient becomes very much like a member of the giver's family where mutual obligations are many, varied, and long lasting. A person may give a gift with the realistic expectation that someday a gift of equal or greater use value will be received or that the recipient will pass on a further gift. In an open-source project, the gift of source code is reciprocated by suggestions, bug reports, debugging, hard work, praise, and more source code.

The commodity economy depends on scarcity. Its most famous law is that of "diminishing returns," whose working requires a fixed supply. Scarcity of material or scarcity of competitors creates high profit margins. It works through competition.

The gift economy is an economy of abundance--the gifts exchanged are inexhaustible. Gift economies are embedded within noneconomic institutions such as kinship, marriage, hospitality, artistic patronage, and ritual friendship. A healthy Western family operates on a gift economy. In an open-source project, the status and reputation of individuals depend on the quality of the gifts they contribute.

For open source to succeed in communities that include corporations, the source-code gift-based economy needs to thrive alongside the commodity economy just beyond its boundaries.

The Work-In-Progress Effect

The concept of the gift economy is related to what we call the work-in-progress effect. This effect is one of the primary reasons that writers' workshops in the literary and software-patterns worlds work as well as they do. The act of opening the curtain early when there is still time to affect the outcome seems to elicit a profound response from the audience, and the response is heightened when the invitation is to be, essentially, co-authors. Harsh criticism is replaced by constructive criticism. Responsibility becomes jointly held. The co-authors align in their regard for the work, although their individual opinions about what to do may differ quite a bit. The concern becomes how to make the work the best it can be rather than commenting on the work or the author.

Open source depends on this response.

Open Source Is Not Business as Usual

In summary, deciding to make a project open source means that it will be very different from your normal proprietary project. All decisions and design will need to be done in the open. Schedules will depend on people outside your company whom you cannot control. The community that emerges will take on a life of its own, possibly taking the project in directions you neither anticipated nor desire.

Your business model and processes must be different or else you won't succeed. You don't do open source as an add-on to your usual process. Deciding to do open source is like deciding to get married and start a family: It takes a commitment, and once you start down that path it will change how you live.