flag of the United Kingdom

Midnight Programmer

Date:  Mon, 1st-Mar-2004prevnext

Tags: Books, Commentary, Programming

In his Pulitzer Prize-winning novel, "The Soul of a New Machine", Tracy Kidder writes about the "Midnight Programmer", that lonely breed whose twilight existence is responsible for most of the applications we take for granted on our desktop.

In this article, I describe some of my own experiences as a midnight programmer, creating the Shapeshifter CMS while the world sleeps...

I live a strange life in a strange world. Like many programmers, the ones I know anyway, I write the majority of my code between (approximately) 20:30 and 08:00. Yes, I wrote those times in the right order.

I don't know why, or how I ended up in with this curious condition. I spent years working regular office hours under various job titles. But as I started to program professionally, my working day began to drift. It began with working late to complete projects by their deadlines, then I found myself losing track of time as I tackled a particular problem. Now I find that I am simply most productive while the rest of the world sleeps, writing thousands of lines of code sustained only by caffeine, nicotine and pizza.

Perhaps it's the absence of interruptions from colleagues, or the perpetually ringing telephone. Perhaps my neurons fire that little bit more quickly in the absence of sunlight. Perhaps I am naturally more motivated and focused during the hours of darkness. Perhaps being able to play whatever music I want, at the volume of my choosing is a major factor. It could be any or all of these, it could be something else I am entirely unable to comprehend.

I can write code during the day. But I have found that I write better code during the night and I write more of it. I can review my code while the sun shines and not make head nor tail of it. I review the same code while the vampires prowl and it all makes perfect sense.

I have never been able to reconcile myself to the cause of this strange phenomenon, but I have learned to live with it. In fact that's not true, I relish it, I embrace it.

The world is special during the early hours of the morning. I periodically leave the office and go outside to stretch my legs and gather my thoughts. During these brief breaks the whole majesty of the night reveals itself. An uncanny stillness surrounds me. I can't describe it, you'd have to experience it to understand. It's almost as if time itself is sleeping, perhaps dreaming on my behalf. The silence is eerie to the city-dweller in me. It's not true silence, there are a myriad of noises if one listens hard enough, rather it is the complete absence of the sounds of the automobile that cause my senses to shiver.

Replacing the sounds of thousands of internal combustion engines are the whispers of the breeze creeping stealthily through the foliage. A dog barks in the distance, an owl calls and the terrifying screeches and howls of two cats fighting assault my ears. The slightest sounds are amplified by the still night air. Those in the distance, inaudible during the day, are clear within the special acoustics of these early hours yet, despite the cacophony, the world seems strangely at peace.

Occasionally a fox will appear, only to scurry away when he becomes aware of me. Once I was privy to a family of them, albeit a single-parent family - a vixen led two cubs on an exploratory examination of the rubbish by the hamburger stand across the street. They were totally oblivious to my presence, else they had simply dismissed me as inconsequential compared to the potential culinary rewards of their outing.

Later, as the night gives way to dawn, the chorus of birdsong begins. The avian population can comfortably compete with any other sound source, bar that of Concorde at take-off. I can identify patterns of communication, or competition, as whistles and squawks originate first from one side of the lot, then the other. It's an awesome cascade of sound, sometimes melodious, sometimes abrupt, sometimes continuous and sometimes intermittent. It takes my breath away.

During these small hours I experience the whole gamut of emotions - contentment, awe and curiosity, combined with the sheer exhilaration of being alive. It is a time of introspection, self congratulation (if I've written a particularly cunning block of code that evening) or speculation and analysis (if I haven't). I breath deeply of the fresh morning air before returning to the office, invigorated and with renewed motivation.

I once nicknamed my office "The Tardis", not because I managed to cram so much hardware in there (grin) - but because, in the office, time begins to flow differently. Picture the scene, it's late at night (or, more accurately, early in the morning). I look at the clock and "03:30" glows back at me. I write some code, test, debug, test again, then I write some more. Slowly my program takes shape, I tweak things here and there, optimise this block, document that one. I run the test suite, then the benchmark application. Compile and build, watch memory usage, monitor disk access, tune the database. More tests, more benchmarking. Finally I check the new code into the CVS repository, my task complete. I feel as if maybe an hour has passed since I last looked at the clock, so it must now be about 04:30. I glance at my watch - 08:10! What? Where did all those hours go? It's amazing how frequently I have to check my computer see what the date is, or even the day! This sensation of time lost will be recognisable to any programmer. It's an uncanny feeling, disturbing to the novice, comfortably familiar to the seasoned and laughable to the old hands.

Entry into this twilight world is free of charge. It requires only an interesting challenge or an impossible deadline. For myself, The Tardis has all the facilities required for an extended stay: there's the sofa-bed, the filter coffee machine, the DVD player and large screen, the MP3 jukebox, the fridge - fully loaded with Dr. Pepper. For food and other perishables, the 24-hour store is less than five minutes away. All one needs to bring to the table is a perverse amount of persistence, a little endurance and a gritty determination to prevail. Soap, a toothbrush /toothpaste and a can of deodorant are also valuable assets.

The programmers' life is not for everyone. One must be prepared to sacrifice personal relationships and vitamin D. Be prepared to welcome a wan complexion and hooded, bleary eyes. Accept that you will soon be a social outcast, bereft of friends and deleted from the cellphones of your former comrades.

Midnight programming offers its own comforts and rewards however. The thrill of success when your code compiles, the pride when you consider the elegance of your syntax and semantics, the sense of achievement when you shave a few milliseconds off the execution time... best of all is that unique thrill when you see users working with your application, that's simply unbeatable.

I won't be able to sustain this lifestyle indefinitely. The lack of sleep and consumption of toxic chemicals will eventually take its toll. But for now, I am a midnight programmer and proud of it.

Addendum (02-March-2004): Tracy Kidder's "The Soul of a New Machine" is well worth a read for those of you who have any kind of interest in computing. William M. Bulkeley (Wall Street Journal) described it as, "Fascinating... a surprisingly gripping account of people at work." That it is.

You can comment on this entry, or read what others have written (8 comments).