Organizing these techy obsessions (part 1)

Anybody who knows me well is aware that I cycle through short-lived technical obsessions. Maybe I’ll dabble with some software for a few hours or a programming language for a few weeks. Sometimes I latch onto something and keep with it for a long time, but for the most part it’s about the thrill of learning and exploration.

Negative side-effects may include…

Each of these short bursts requires startup time and has a learning curve. Even worse, when I move from one project to the next I often don’t take enough time to organize and backup what I’ve accomplished.

I’m not terribly concerned that I’m not focusing on a single project over the long term. The most important ones eventually stick. But what I am concerned about is not being organized enough to easily make progress on an old project or synthesize previously disparate projects. Enduring the exact same start up process over and over is clearly a waste of time. Plus, anticipation of wasted time deters me from returning to old projects, thereby making the projects suffer from bit rot.

System requirements

Many projects require me to use software. Even when I’m developing my own software, I depend programming tools. It’s inescapable. And every piece of software will only work if its requirements are respected. Those requirements may even include other software packages with their own requirements. Eep!

Package managers

Open source operating systems like Linux address this problem by including package managers. These nifty tools let you say, “I want application X to run on my machine,” and it tries to make that happen by installing all the dependencies and then finally application X. Doing this by hand is a pain, so it’s a good thing package managers exist.

So why am I complaining? I could just use a package manager, right? No, it’s not quite that simple. Before I can install an application from the package manager, the software developers would have to first package up the software for the package manager. And then the maintainers of the package manager have to include the package. The net result is my package manager won’t always let me install the software I need.

Can haz codez, plz?

Usually the next step is to download the source code to /usr/local/src and compile it directly. Of course I have to do the same thing for any dependencies the package manager can’t handle. This isn’t too bad when the software developers document the dependencies properly. Still, it’s cumbersome if there are enough dependencies.

Programmers like myself prefer to build on the latest releases of software because sometimes the newer version will behave differently than the older software. Unless the changed behavior is identified as a regression bug, the newer version better reflects where the technology is going. Who would want their precious new code to be obsolete from day one because it depends on deprecated behaviors? Certainly not me.

The requirement to work with the latest releases poses a few problems for minimizing startup time.

1. Previous projects on the same machine may depend on now obsolete behaviors. Installing a newer version of the software may break old projects. Isolating what broke the old project or making changes until it works may not be easy.
2. Installing the latest release of software can require compiling a lot from source, which can be time consuming.
3. Some software needs to be installed using a specialized package manager, such as gem for Ruby libraries. Package managers affect global state of the system, sometimes in the same directories, so the distinct package managers can clash. There’s no standard way to resolve these collisions—it requires experimentation and research time.

All these problems can be resolved by limiting the scope of these global changes. What I mean by global scope here is each problem affects the state within the operating system, but not beyond that. If I had a different operating system for each project, the potential for these sorts of collisions would be non-existent. I couldn’t afford to buy a new computer every time I move on to a new obsession. It’s also silly to keep repartitioning my hard drives to support more operating systems. Instead I could simulate that effect by installing a new virtual machine on my computer for each project.

In the next post I’ll explore whether virtual machines are a good solution or just another problem.