4 meta principles for software engineering

A brief review and analysis of maxims and manifestos made by software makers

Photo by jimmy teoh

There is this tendency—if not an outright need—for those involved in the actual practice of writing code to create a guiding philosophy for the choices they're making.

Let’s discover the commonalities among them all.

The earliest example I know of occurs in the Foreword to the Unix Time Sharing System in a Bell Labs technical article in 1978, authored by McIlroy, Pinson, and Tague.

(i) Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features.

(ii) Expect the output of every program to become the input to another, as yet unknown, program. Don't clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don't insist on interactive input.

(iii) Design and build software, even operating systems, to be tried early, ideally within weeks. Don't hesitate to throw away the clumsy parts and rebuild them.

(iv) Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you've finished using them.

Importantly, they called out in the same article about the maxims that they had, “gained currency among the builders and users of the UNIX system to explain and promote its characteristic style.”

They arose doing the work. They did not descend down from on high from a plan or an architecture. They emerged naturally.

These four imperatives have had subsequent re-tellings in various other publications, and I don't presume to say that these four are the canonical actual statements.

The nature of folklore is to tell and retell. The story changing each time.

Sometimes though the original set of maxims or imperatives is too generalized. So the new context require a new set of maxims. These are not meant to say the old set was wrong or incomplete. It's just that the context is sufficiently different so as to need to be more specific.

I think that's what happened for Rob Pike's Rules for Coding C in 1989:

Rule 1. You can’t tell where a program is going to spend its time. Bottlenecks occur in surprising places, so don’t try to second guess and put in a speed hack until you’ve proven that’s where the bottleneck is.

Rule 2. Measure. Don’t tune for speed until you’ve measured, and even then don’t unless one part of the code overwhelms the rest.

Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy algorithms have big constants. Until you know that n is frequently going to be big, don’t get fancy. (Even if n does get big, use Rule 2 first.) For example, binary trees are always faster than splay trees for workaday problems.

Rule 4. Fancy algorithms are buggier than simple ones, and they’re much harder to implement. Use simple algorithms as well as simple data structures.

The following data structures are a complete list for almost all practical programs:
array
linked list
hash table
binary tree
Of course, you must also be prepared to collect these into compound data structures. For instance, a symbol table might be implemented as a hash table containing linked lists of arrays of characters.

Rule 5. Data dominates. If you’ve chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming.

Rule 6. There is no Rule 6.

Again, this explosion of defining and stating the rules largely happened, though, after the fact. C was before these rules, not before them.¹

Even the summary of these rules and other philosophies into things like Eric Raymond's 17 Rules of Unix philosophy tends to be read as though, “this was always the plan.” However, this was not so. They were not an original guiding statement and more of a writing down afterwards the approach being taken in an attempt to pass along the mindset.

The rules were discovered rather than declared.

Time went by.

Over a decade, and then, perhaps there was something in the water in the 2000s, we got the Agile Manifesto in 2001²

We are uncovering better ways of developing
software by doing it and helping others do it.
Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

And the Zen of Python in 2004.

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

Both are considered incredibly important statements that guided the Agile movement and the Python language respectively.

Again, both were not before, but during the work. They were discovered to be good. However, both statements were far less prescriptive than any prior statement, and I'll also add that these statements seem in English to be downright poetic.

A downside to these statements being far more flexible was indeed their flexibility. People can and do have arguments about what is Agile and what code is Pythonic.

In contrast when Adam Wiggins decided to given guidance in 2011 on how to build a web application, he didn't just give 12 rules, but basically wrote a small book called the 12 Factor App.³ I’ll attach just the tables of content here:

The Twelve Factors

I. Codebase
One codebase tracked in revision control, many deploys

II. Dependencies
Explicitly declare and isolate dependencies

III. Config
Store config in the environment

IV. Backing services
Treat backing services as attached resources

V. Build, release, run
Strictly separate build and run stages

VI. Processes
Execute the app as one or more stateless processes

VII. Port binding
Export services via port binding

VIII. Concurrency
Scale out via the process model

IX. Disposability
Maximize robustness with fast startup and graceful shutdown

X. Dev/prod parity
Keep development, staging, and production as similar as possible

XI. Logs
Treat logs as event streams

XII. Admin processes
Run admin/management tasks as one-off processes

Some observations

Good principles for making software seem to:

• come from making software

• urge simplicity

• suggest pragmatism

• tend to brevity

Anything that does not include these tends towards specification rather than philosophy.

Now I’m not encouraging anyone to necessarily make maxims or manifestos for the fun of it—and to be fair, I’m not discouraging it either.

I’m just observing.

1. C proliferated and set expectations for the universe of languages now known as C-like languages in subsequent decades, and I get the sense that this philosophy influenced all related languages profoundly.

2. Which I applaud as a manifesto I progressively detest as a practice

3. I had actually wondered if this was somewhat a reaction to the underspecification in the prior popular set of statements. So I asked.